Radar-only 3D human reconstruction

My Wright State capstone uses millimeter-wave radar to estimate a moving person’s 3D pose and body shape, including during rapid movements.

Illustration of a triangulated body with motion traces — Illustration of the reconstruction task.

From radar frames to a mesh

My PyTorch model takes four consecutive frames from a single radar. A 3D convolutional network extracts features for 22 joints. A skeletal graph shares information between connected joints, and a Mamba-style state-space model tracks how those features change over time. The network estimates SMPL-X pose and shape parameters to produce the body mesh.

Learning movement

I use motion-capture data from the M4Human benchmark to supervise changes in joint position during training. At the start of a recording, the loader repeats the first radar frame to fill the four-frame window. I exclude those repeated transitions from the velocity loss.

Training at Wright State

I scheduled CUDA-enabled PyTorch training on an NVIDIA L40S GPU using Slurm. The S2 split used in my run contains 493,925 training samples and 33,987 validation samples. I ran smoke tests and a three-epoch pilot before starting full-scale training.

M4Human benchmark CVPR benchmark paper