AGORA: Avatars in Geography Optimized for Regression Analysis

abstract

image.png

A synthetic dataset with high realism and highly accurate ground truth, use 4240 commercially-available, high-quality, textured human scans in diverse poses and natural clothing, including 257 scans of children.

We create reference 3D poses and body shapes by fitting the SMPL-X body model (with face and hands) to the 3D scans, taking into account clothing, create around 14K training and 3K test images by rendering between 5 and 15 people per image using either image-based lighting or rendered 3D environments, taking care to make the images physically plausible and photoreal.
Extend smpl-x model better to capture children shape

other dataset

image.png

Method: Obtaining reference data

purchased high-quality textured 3D scans from:

选取了4240个扫描模型，包括350个独立的个体,高精度的3d扫描模型包括顶点和表面，需要将一个smpl-x模型拟合到3D扫描模型上，Fitting a SMPL-X mesh M to a scan S amounts to solving for the optimal parameters (θ, β, ψ) ；如何考虑衣服的影响：同一个人的shape参数一样

progress

image.png

使用虚拟的相机参数，渲染得到多视角的图像，使用 SMPLify-X fitting 算法得到初始化的smpl-x模型参数，为了进一步将两个3d模型进行对齐，采用：use the idea of fitting body shape under clothing:

The naked truth: Estimating body shape under clothing
Detailed, accurate, human shape estimation from clothed 3D scan sequences

代价函数：

image.png

children fitting

购买的扫描模型中包含了257个小孩的3d模型，采用上述过程产生的结果不好，由于smpl-x的表达方式对于小孩并不是很鲁棒：

image.png

为了更好的拟合小孩，所以加入了 $\alpha$ 全局参数，然后一起进行优化，但是结果还是不是很自然，we take the mean infant body template from SMIL [20] and convert it to SMPL-X topology, TC ，将婴儿模型SMIL转换为SMPL-x的拓扑结构，进而让两者进行一个结合来优化。

AGORA Dataset

4240 scans spanning more than 350 unique subjects， all paired with SMPL-X fits

Number_scans	body	hand	Face
3161	$\surd$	$\surd$	$\surd$
1079	$\surd$	X	X
4240

dataset split:

Name	train	validation	Test
Num_scans	2930	259	1052
Num_images	14529	1225	3387

Image-render

Camera: focal lengths 18mm, 28mm and 50mm
render-engine:unreal-engine
Env: a single Windows 10 PC with NVIDIA RTX 2080 graphics hardware
photorealistic:
- image-based lighting using freely available HDR backgrounds
- free and commercial 3D environments obtained from the Unreal Marketplace
- 对于一些坐的扫描模型添加椅子来增加真实感
- other tricks.