MMDetection 之外深入解析 mmpose 的 Top-down Heatmap 人体姿态估计模型库【免费下载链接】mmposeOpenMMLab Pose Estimation Toolbox and Benchmark.项目地址: https://gitcode.com/GitHub_Trending/mm/mmposeTop-down heatmap 是人体姿态估计领域最经典、最成熟的范式先用目标检测器框出每一个人再对单人裁剪图回归关键点热图。本文以 configs/body_2d_keypoint/topdown_heatmap/README.md 为核心骨架结合 mmpose 仓库源码与真实配置文件系统讲解该范式的原理、模型配置、编解码codec、训练评测与 COCO/MPII/CrowdPose 等数据集上的完整结果矩阵读完即可独立复现与改造 Top-down 姿态估计模型。Top-down 两阶段范式与 Heatmap 回归核心思想Top-down 方法将姿态估计任务拆分为两个阶段目标检测阶段利用人体检测器如 Faster R-CNN、RTMDet从整幅图像中检测出每个人体边界框单人姿态估计阶段将每个人体框裁剪出来送入姿态估计网络估计该单人的关键点坐标。与直接回归关键点坐标不同Top-down 姿态估计器输出的不是(x, y)数值而是一组热图heatmap。每个关键点对应一张热图热图上某个位置的值表示该位置是该关键点的似然概率。这一范式源自Simple Baselines for Human Pose Estimation and TrackingXiao et al., ECCV 2018该文证明了在 ResNet 骨干后接几个反卷积层 一层卷积输出热图这一简洁结构即可达到极具竞争力的精度此后成为业界事实标准。在 mmpose 中这一范式对应 mmpose/models/pose_estimators/topdown.py 中的TopdownPoseEstimator它负责串联backbone - (neck) - head并实现训练loss()与推理predict()的标准流程。从配置读懂一个 Top-down Heatmap 模型以 HRNet-w32 为例以 COCO 上最常用的基线配置 td-hm_hrnet-w32_8xb64-210e_coco-256x192.py 为例逐段拆解其含义_base_ [../../../_base_/default_runtime.py] # runtime train_cfg dict(max_epochs210, val_interval10) # optimizer optim_wrapper dict(optimizerdict(typeAdam, lr5e-4)) # learning policy param_scheduler [ dict(typeLinearLR, begin0, end500, start_factor0.001, by_epochFalse), # warm-up dict(typeMultiStepLR, begin0, end210, milestones[170, 200], gamma0.1, by_epochTrue) ] # automatically scaling LR based on the actual training batch size auto_scale_lr dict(base_batch_size512) # hooks default_hooks dict(checkpointdict(save_bestcoco/AP, rulegreater))要点说明训练 210 个 epoch每 10 个 epoch 在验证集上评测一次优化器为 Adam初始学习率5e-4前 500 次迭代线性 warm-up在 epoch 170 与 200 处学习率衰减 10 倍auto_scale_lr以 512 为基准 batch size 自动缩放学习率因此在单卡小 batch 上训练也会保持相对合理的梯度尺度checkpoint 钩子以coco/AP为指标、取更大值为准则保存最优权重。接下来是本文档主题中最关键的编解码配置codec# codec settings codec dict( typeMSRAHeatmap, input_size(192, 256), heatmap_size(48, 64), sigma2)这里input_size为模型输入尺寸宽 192、高 256heatmap_size为输出热图尺寸48×64为输入的 1/4sigma2为高斯核标准差。MSRAHeatmap编码器的实现位于 mmpose/codecs/msra_heatmap.py编码时将关键点坐标除以scale_factor input_size / heatmap_size后在热图空间铺高斯核生成目标热图解码时对预测热图取最大值位置再经亚像素精化refine_keypoints并乘以scale_factor还原回输入图像坐标。模型主体配置model dict( typeTopdownPoseEstimator, data_preprocessordict( typePoseDataPreprocessor, mean[123.675, 116.28, 103.53], std[58.395, 57.12, 57.375], bgr_to_rgbTrue), backbonedict( typeHRNet, in_channels3, extradict( stage1dict(num_modules1, num_branches1, blockBOTTLENECK, num_blocks(4, ), num_channels(64, )), stage2dict(num_modules1, num_branches2, blockBASIC, num_blocks(4, 4), num_channels(32, 64)), stage3dict(num_modules4, num_branches3, blockBASIC, num_blocks(4, 4, 4), num_channels(32, 64, 128)), stage4dict(num_modules3, num_branches4, blockBASIC, num_blocks(4, 4, 4, 4), num_channels(32, 64, 128, 256))), init_cfgdict( typePretrained, checkpointhttps://download.openmmlab.com/mmpose/pretrain_models/hrnet_w32-36af842e.pth), ), headdict( typeHeatmapHead, in_channels32, out_channels17, deconv_out_channelsNone, lossdict(typeKeypointMSELoss, use_target_weightTrue), decodercodec), test_cfgdict(flip_testTrue, flip_modeheatmap, shift_heatmapTrue))关键点TopdownPoseEstimator数据预处理采用 ImageNet 风格的均值/方差归一化bgr_to_rgbTrue表示输入为 BGR 顺序的 OpenCV 读图HRNet-w32 四阶段分别维持 1/2/3/4 条并行分支融合高分辨率与低分辨率特征COCO 的 17 个关键点对应out_channels17HeatmapHead在此配置中deconv_out_channelsNone即不接反卷积直接利用 HRNet 的高分辨率特征输出热图。HeatmapHead实现见 mmpose/models/heads/heatmap_heads/heatmap_head.py其默认结构为若干反卷积层默认(256, 256, 256)、核 4×4、步长 2 1×1 卷积输出层而 Simple Baselines 风格如 ResNet 骨干则会启用这 3 层反卷积将特征上采样至 1/4 分辨率损失函数KeypointMSELoss实现于 mmpose/models/losses/heatmap_loss.pyuse_target_weightTrue表示按各关键点的可见权重加权 MSE从而忽略不可见关键点的影响该文件还提供CombinedTargetMSELossUDP combined target、KeypointOHKMMSELoss在线困难关键点挖掘、FocalHeatmapLoss、AdaptiveWingLoss等替代损失flip_testTrue开启水平翻转 TTA推理时同时前向原始图与水平翻转图将翻转图热图按flip_indices翻回并与原热图取平均shift_heatmapTrue时还会对热图做 1 像素偏移补偿以缓解坐标偏差。该逻辑位于HeatmapHead.predict()与 mmpose/models/utils/tta.py。数据管线pipeline配置train_pipeline [ dict(typeLoadImage), dict(typeGetBBoxCenterScale), dict(typeRandomFlip, directionhorizontal), dict(typeRandomHalfBody), dict(typeRandomBBoxTransform), dict(typeTopdownAffine, input_sizecodec[input_size]), dict(typeGenerateTarget, encodercodec), dict(typePackPoseInputs) ] val_pipeline [ dict(typeLoadImage), dict(typeGetBBoxCenterScale), dict(typeTopdownAffine, input_sizecodec[input_size]), dict(typePackPoseInputs) ]训练管线依次执行读取图像 → 由 bbox 计算中心与尺度 → 水平随机翻转 → 随机半身RandomHalfBody→ 随机 bbox 扰动缩放/旋转/平移→ Topdown 仿射变换裁剪 → 用 codec 生成热图目标 → 打包。GenerateTarget的encodercodec正是把上面定义的MSRAHeatmap编码器接入数据流。验证/测试管线不含任何随机增强仅做仿射裁剪。数据加载器与评测器train_dataloader dict( batch_size64, num_workers2, persistent_workersTrue, samplerdict(typeDefaultSampler, shuffleTrue), datasetdict( typeCocoDataset, data_rootdata/coco/, data_modetopdown, ann_fileannotations/person_keypoints_train2017.json, data_prefixdict(imgtrain2017/), pipelinetrain_pipeline)) val_dataloader dict( batch_size32, num_workers2, persistent_workersTrue, drop_lastFalse, samplerdict(typeDefaultSampler, shuffleFalse, round_upFalse), datasetdict( typeCocoDataset, data_rootdata/coco/, data_modetopdown, ann_fileannotations/person_keypoints_val2017.json, bbox_filedata/coco/person_detection_results/COCO_val2017_detections_AP_H_56_person.json, data_prefixdict(imgval2017/), test_modeTrue, pipelineval_pipeline)) val_evaluator dict( typeCocoMetric, ann_filedata_root annotations/person_keypoints_val2017.json) test_dataloader val_dataloader test_evaluator val_evaluator注意验证集通过bbox_file指定了固定的人体检测框结果即 README 中detector having human AP of 56.4的检测器输出这样不同模型的评测口径一致、可公平对比。评测指标由 mmpose/evaluation/metrics/coco_metric.py 中的CocoMetric计算 AP/AR。编解码器的纵深MSRAHeatmap 与 UDPHeatmapREADME 结果表中的Dark、UDP后缀本质上是更换了编解码器这也是 top-down heatmap 系列最重要的精度提升手段Codec配置文件示例特点MSRAHeatmaptd-hm_hrnet-w32_8xb64-210e_coco-256x192.pySimple Baselines 经典做法标准高斯热图MSRAHeatmapunbiasedTruetd-hm_hrnet-w32_dark-8xb64-210e_coco-256x192.pyDarkPose无偏高斯核 解码时热图调制refine_keypoints_dark缓解量化偏移UDPHeatmaptd-hm_hrnet-w32_udp-8xb64-210e_coco-256x192.pyUnbiased Data Processing消除坐标系量化偏差配合use_udpTrue仿射变换从源码看二者的本质差异MSRAHeatmap的缩放因子是input_size / heatmap_size见 msra_heatmap.pyUDPHeatmap采用(input_size - 1) / (heatmap_size - 1)见 udp_heatmap.py并对齐了坐标系两端从而彻底消除坐标变换的量化误差其heatmap_type还支持combined分类图 x/y 偏移图配合CombinedTargetMSELoss使用。对应地UDP 配置中TopdownAffine需显式加use_udpTrue例如 ViTPose 配置 td-hm_ViTPose-huge_8xb64-210e_coco-256x192.pycodec dict(typeUDPHeatmap, input_size(192, 256), heatmap_size(48, 64), sigma2) ... dict(typeTopdownAffine, input_sizecodec[input_size], use_udpTrue)该 ViTPose 配置同时展示了 Transformer 骨干 HeatmapHead 的另一种典型组合backbone 来自 mmpretrain 的VisionTransformerarchhuge、with_cls_tokenFalse、out_typefeatmaphead 使用deconv_out_channels(256, 256)两层反卷积上采样优化器换为 AdamW 并采用 LayerDecay 学习率分层衰减策略LayerDecayOptimWrapperConstructor见 mmpose/engine/optim_wrappers/layer_decay_optim_wrapper.py。训练、评测与推理仓库提供统一的训练/测试入口详见 tools/train.py 与 tools/test.py# 单卡训练 python tools/train.py configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py # 多卡分布式训练 bash tools/dist_train.sh configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py 8 # 使用训练好的权重评测 python tools/test.py configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py work_dirs/td-hm_hrnet-w32_8xb64-210e_coco-256x192/best_coco_AP_epoch_210.pth # 分布式评测 bash tools/dist_test.sh configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py work_dirs/.../best_coco_AP_epoch_210.pth 8其中.pth路径为训练后保存在work_dirs中的实际权重文件请以训练日志中的真实文件名为准。评测前需确保data/coco已按 docs/zh_cn/user_guides/prepare_datasets.md 的说明组织好数据集与bbox_file检测框文件。推理阶段可通过 demo/topdown_demo_with_mmdet.py 串联 mmdet 检测器与 mmpose 姿态模型实现检测 单人姿态估计的完整 Top-down 流程python demo/topdown_demo_with_mmdet.py \ demo/mmdetection_cfg/rtmdet_m_640-8xb32_coco-person.py \ https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192-81c58e40_20220909.pth \ --input demo/resources/000000000785.jpg \ --output-root vis_results/其中rtmdet_m_640-8xb32_coco-person.py为仅检测 person 类别的检测器配置权重地址来自对应模型的 ckpt 链接也可以改用 demo/inferencer_demo.py 通过Pose2DInferencer见 mmpose/apis/inferencers/pose2d_inferencer.py一行命令完成推理。模型库与基准结果总览COCOval2017检测器 human AP56.4输入 256×192ModelInput SizeAPAR详情与下载ViTPose-h256x1920.7900.840vitpose_coco.mdHRNet-w48UDP256x1920.7680.817hrnet_udp_coco.mdMSPN 4-stg256x1920.7650.826mspn_coco.mdHRNet-w48Dark256x1920.7640.814hrnet_dark_coco.mdHRNet-w48256x1920.7560.809hrnet_coco.mdHRFormer-B256x1920.7540.807hrformer_coco.mdRSN-50-3x256x1920.7500.814rsn_coco.mdCSPNeXt-l256x1920.7500.800cspnext_udp_coco.mdHRNet-w32256x1920.7490.804hrnet_coco.mdSwin-L256x1920.7430.798swin_coco.mdViTPose-s256x1920.7390.792vitpose_coco.mdHRFormer-S256x1920.7380.793hrformer_coco.mdSwin-B256x1920.7370.794swin_coco.mdSEResNet-101256x1920.7340.790seresnet_coco.mdSCNet-101256x1920.7330.789scnet_coco.mdResNet-101Dark256x1920.7330.786resnet_dark_coco.mdCSPNeXt-m256x1920.7320.785cspnext_udp_coco.mdResNetV1d-101256x1920.7320.785resnetv1d_coco.mdSEResNet-50256x1920.7290.784seresnet_coco.mdSCNet-50256x1920.7280.784scnet_coco.mdResNet-101256x1920.7260.783resnet_coco.mdResNeXt-101256x1920.7260.781resnext_coco.mdHourglassNet256x2560.7260.780hourglass_coco.mdResNeSt-101256x1920.7250.781resnest_coco.mdRSN-50256x1920.7240.790rsn_coco.mdSwin-T256x1920.7240.782swin_coco.mdMSPN 1-stg256x1920.7230.788mspn_coco.mdResNetV1d-50256x1920.7220.777resnetv1d_coco.mdResNeSt-50256x1920.7200.775resnest_coco.mdResNet-50256x1920.7180.774resnet_coco.mdResNeXt-50256x1920.7150.771resnext_coco.mdPVT-S256x1920.7140.773pvt_coco.mdCSPNeXt-s256x1920.6970.753cspnext_udp_coco.mdLiteHRNet-30256x1920.6760.736litehrnet_coco.mdCSPNeXt-tiny256x1920.6650.723cspnext_udp_coco.mdMobileNet-v2256x1920.6480.709mobilenetv2_coco.mdLiteHRNet-18256x1920.6420.705litehrnet_coco.mdCPM256x1920.6270.689cpm_coco.mdShuffleNet-v2256x1920.6020.668shufflenetv2_coco.mdShuffleNet-v1256x1920.5870.654shufflenetv1_coco.mdAlexNet256x1920.4480.521alexnet_coco.md从表中可清晰看到三条规律其一模型容量与精度整体正相关ViTPose-h 最高AlexNet 最低其二同样骨干下更换更优编解码UDP/Dark能带来稳定增益其三COCO 上完整逐项指标AP50/AP75/AR50 等可在各子页面中查看例如 hrnet_coco.md 还提供了 HRNet-w32/w48 在 256×192 与 384×288 两种分辨率下的详细结果及 ckpt/log 下载地址。MPIIvalPCKh0.5 / PCKh0.1ModelInput SizePCKh0.5PCKh0.1详情与下载HRNet-w48Dark256x2560.9050.360hrnet_dark_mpii.mdHRNet-w48256x2560.9020.303cspnext_udp_mpii.mdHRNet-w48256x2560.9010.337hrnet_mpii.mdHRNet-w32256x2560.9000.334hrnet_mpii.mdHourglassNet256x2560.8890.317hourglass_mpii.mdResNet-152256x2560.8890.303resnet_mpii.mdResNetV1d-152256x2560.8880.300resnetv1d_mpii.mdSCNet-50256x2560.8880.290scnet_mpii.mdResNeXt-152256x2560.8870.294resnext_mpii.mdSEResNet-50256x2560.8840.292seresnet_mpii.mdResNet-50256x2560.8820.286resnet_mpii.mdResNetV1d-50256x2560.8810.290resnetv1d_mpii.mdCPM368x368*0.8760.285cpm_mpii.mdLiteHRNet-30256x2560.8690.271litehrnet_mpii.mdLiteHRNet-18256x2560.8590.260litehrnet_mpii.mdMobileNet-v2256x2560.8540.234mobilenetv2_mpii.mdShuffleNet-v2256x2560.8280.205shufflenetv2_mpii.mdShuffleNet-v1256x2560.8240.195shufflenetv1_mpii.md注CPM 的输入尺寸为 368×368。CrowdPosetestYOLOv3 人体检测器ModelInput SizeAPAR详情与下载HRNet-w32256x1920.6750.816hrnet_crowdpose.mdCSPNeXt-m256x1920.6620.755cspnext_udp_crowdpose.mdResNet-101256x1920.6470.800resnet_crowdpose.mdHRNet-w32256x1920.6370.785resnet_crowdpose.mdAICvalGT bboxModelInput SizeAPAR详情与下载HRNet-w32256x1920.3230.366hrnet_aic.mdResNet-101256x1920.2940.337resnet_aic.mdJHMDBPCK0.2ModelInput SizePCK(norm. by person size)PCK (norm. by torso size)详情与下载ResNet-50256x25696.080.1resnet_jhmdb.mdCPM368x36889.865.7cpm_jhmdb.mdPoseTrack2018valGT bboxModelInput SizeAP详情与下载HRNet-w48256x19284.6hrnet_posetrack18.mdHRNet-w32256x19283.4hrnet_posetrack18.mdResNet-50256x19281.2resnet_posetrack18.mdHuman-Art检测器 human AP56.2 的验证集ModelInput SizeAPAR详情与下载ViTPose-s256x1920.3810.448vitpose_humanart.mdViTPose-b256x1920.4100.475vitpose_humanart.mdHuman-ArtGT bbox 验证集ModelInput SizeAPAR详情与下载ViTPose-s256x1920.7380.768vitpose_humanart.mdViTPose-b256x1920.7590.790vitpose_humanart.mdHuman-Art 的两组结果对比检测框 vs GT bbox直观反映了 Top-down 范式中检测器质量对姿态精度的决定性影响使用 GT bbox 时 ViTPose-b 可达 AP 0.759而接上检测器后降至 0.410因此在真实应用中优化检测器与姿态模型同等重要。选择合适的模型精度、速度与场景综合 README 结果表可给出如下选型参考基于仓库已公开的评测数据追求极致精度ViTPose-hAP 0.790需较大显存适合离线服务精度/速度均衡HRNet-w48/UDPAP 0.768、CSPNeXt-lAP 0.750与 HRNet-w32AP 0.749是通用场景最常用的选择轻量移动端/边缘部署LiteHRNet-18AP 0.642、MobileNet-v2AP 0.648、ShuffleNet-v2AP 0.602参数量小、推理快稠密人群场景优先参考 CrowdPose 结果HRNet-w32 AP 0.675并结合目标检测阶段的人员漏检情况做取舍。需要说明的是上表数值均以 README 给出的特定检测器如 COCO human AP56.4 的检测框为基准评测更换检测器后 AP 会相应变化实际部署前建议在自己的数据与检测器组合上重新评测。延伸阅读更多数据集基准动物、人脸、手部、全身见 configs/body_2d_keypoint/topdown_heatmap/README.md 同级的 animal_2d_keypoint、face_2d_keypoint、hand_2d_keypoint、wholebody_2d_keypoint 等目录框架级配置说明见 docs/zh_cn/user_guides/configs.md训练与测试流程见 docs/zh_cn/user_guides/train_and_test.md数据准备COCO/MPII 等标注格式与目录组织见 docs/zh_cn/user_guides/prepare_datasets.md推理与部署见 docs/zh_cn/user_guides/inference.md 与 demo/topdown_demo_with_mmdet.py。【免费下载链接】mmposeOpenMMLab Pose Estimation Toolbox and Benchmark.项目地址: https://gitcode.com/GitHub_Trending/mm/mmpose创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考 MMDetection 之外：深入解析 mmpose 的 Top-down Heatmap 人体姿态估计模型库
MMDetection 之外深入解析 mmpose 的 Top-down Heatmap 人体姿态估计模型库【免费下载链接】mmposeOpenMMLab Pose Estimation Toolbox and Benchmark.项目地址: https://gitcode.com/GitHub_Trending/mm/mmposeTop-down heatmap 是人体姿态估计领域最经典、最成熟的范式先用目标检测器框出每一个人再对单人裁剪图回归关键点热图。本文以 configs/body_2d_keypoint/topdown_heatmap/README.md 为核心骨架结合 mmpose 仓库源码与真实配置文件系统讲解该范式的原理、模型配置、编解码codec、训练评测与 COCO/MPII/CrowdPose 等数据集上的完整结果矩阵读完即可独立复现与改造 Top-down 姿态估计模型。Top-down 两阶段范式与 Heatmap 回归核心思想Top-down 方法将姿态估计任务拆分为两个阶段目标检测阶段利用人体检测器如 Faster R-CNN、RTMDet从整幅图像中检测出每个人体边界框单人姿态估计阶段将每个人体框裁剪出来送入姿态估计网络估计该单人的关键点坐标。与直接回归关键点坐标不同Top-down 姿态估计器输出的不是(x, y)数值而是一组热图heatmap。每个关键点对应一张热图热图上某个位置的值表示该位置是该关键点的似然概率。这一范式源自Simple Baselines for Human Pose Estimation and TrackingXiao et al., ECCV 2018该文证明了在 ResNet 骨干后接几个反卷积层 一层卷积输出热图这一简洁结构即可达到极具竞争力的精度此后成为业界事实标准。在 mmpose 中这一范式对应 mmpose/models/pose_estimators/topdown.py 中的TopdownPoseEstimator它负责串联backbone - (neck) - head并实现训练loss()与推理predict()的标准流程。从配置读懂一个 Top-down Heatmap 模型以 HRNet-w32 为例以 COCO 上最常用的基线配置 td-hm_hrnet-w32_8xb64-210e_coco-256x192.py 为例逐段拆解其含义_base_ [../../../_base_/default_runtime.py] # runtime train_cfg dict(max_epochs210, val_interval10) # optimizer optim_wrapper dict(optimizerdict(typeAdam, lr5e-4)) # learning policy param_scheduler [ dict(typeLinearLR, begin0, end500, start_factor0.001, by_epochFalse), # warm-up dict(typeMultiStepLR, begin0, end210, milestones[170, 200], gamma0.1, by_epochTrue) ] # automatically scaling LR based on the actual training batch size auto_scale_lr dict(base_batch_size512) # hooks default_hooks dict(checkpointdict(save_bestcoco/AP, rulegreater))要点说明训练 210 个 epoch每 10 个 epoch 在验证集上评测一次优化器为 Adam初始学习率5e-4前 500 次迭代线性 warm-up在 epoch 170 与 200 处学习率衰减 10 倍auto_scale_lr以 512 为基准 batch size 自动缩放学习率因此在单卡小 batch 上训练也会保持相对合理的梯度尺度checkpoint 钩子以coco/AP为指标、取更大值为准则保存最优权重。接下来是本文档主题中最关键的编解码配置codec# codec settings codec dict( typeMSRAHeatmap, input_size(192, 256), heatmap_size(48, 64), sigma2)这里input_size为模型输入尺寸宽 192、高 256heatmap_size为输出热图尺寸48×64为输入的 1/4sigma2为高斯核标准差。MSRAHeatmap编码器的实现位于 mmpose/codecs/msra_heatmap.py编码时将关键点坐标除以scale_factor input_size / heatmap_size后在热图空间铺高斯核生成目标热图解码时对预测热图取最大值位置再经亚像素精化refine_keypoints并乘以scale_factor还原回输入图像坐标。模型主体配置model dict( typeTopdownPoseEstimator, data_preprocessordict( typePoseDataPreprocessor, mean[123.675, 116.28, 103.53], std[58.395, 57.12, 57.375], bgr_to_rgbTrue), backbonedict( typeHRNet, in_channels3, extradict( stage1dict(num_modules1, num_branches1, blockBOTTLENECK, num_blocks(4, ), num_channels(64, )), stage2dict(num_modules1, num_branches2, blockBASIC, num_blocks(4, 4), num_channels(32, 64)), stage3dict(num_modules4, num_branches3, blockBASIC, num_blocks(4, 4, 4), num_channels(32, 64, 128)), stage4dict(num_modules3, num_branches4, blockBASIC, num_blocks(4, 4, 4, 4), num_channels(32, 64, 128, 256))), init_cfgdict( typePretrained, checkpointhttps://download.openmmlab.com/mmpose/pretrain_models/hrnet_w32-36af842e.pth), ), headdict( typeHeatmapHead, in_channels32, out_channels17, deconv_out_channelsNone, lossdict(typeKeypointMSELoss, use_target_weightTrue), decodercodec), test_cfgdict(flip_testTrue, flip_modeheatmap, shift_heatmapTrue))关键点TopdownPoseEstimator数据预处理采用 ImageNet 风格的均值/方差归一化bgr_to_rgbTrue表示输入为 BGR 顺序的 OpenCV 读图HRNet-w32 四阶段分别维持 1/2/3/4 条并行分支融合高分辨率与低分辨率特征COCO 的 17 个关键点对应out_channels17HeatmapHead在此配置中deconv_out_channelsNone即不接反卷积直接利用 HRNet 的高分辨率特征输出热图。HeatmapHead实现见 mmpose/models/heads/heatmap_heads/heatmap_head.py其默认结构为若干反卷积层默认(256, 256, 256)、核 4×4、步长 2 1×1 卷积输出层而 Simple Baselines 风格如 ResNet 骨干则会启用这 3 层反卷积将特征上采样至 1/4 分辨率损失函数KeypointMSELoss实现于 mmpose/models/losses/heatmap_loss.pyuse_target_weightTrue表示按各关键点的可见权重加权 MSE从而忽略不可见关键点的影响该文件还提供CombinedTargetMSELossUDP combined target、KeypointOHKMMSELoss在线困难关键点挖掘、FocalHeatmapLoss、AdaptiveWingLoss等替代损失flip_testTrue开启水平翻转 TTA推理时同时前向原始图与水平翻转图将翻转图热图按flip_indices翻回并与原热图取平均shift_heatmapTrue时还会对热图做 1 像素偏移补偿以缓解坐标偏差。该逻辑位于HeatmapHead.predict()与 mmpose/models/utils/tta.py。数据管线pipeline配置train_pipeline [ dict(typeLoadImage), dict(typeGetBBoxCenterScale), dict(typeRandomFlip, directionhorizontal), dict(typeRandomHalfBody), dict(typeRandomBBoxTransform), dict(typeTopdownAffine, input_sizecodec[input_size]), dict(typeGenerateTarget, encodercodec), dict(typePackPoseInputs) ] val_pipeline [ dict(typeLoadImage), dict(typeGetBBoxCenterScale), dict(typeTopdownAffine, input_sizecodec[input_size]), dict(typePackPoseInputs) ]训练管线依次执行读取图像 → 由 bbox 计算中心与尺度 → 水平随机翻转 → 随机半身RandomHalfBody→ 随机 bbox 扰动缩放/旋转/平移→ Topdown 仿射变换裁剪 → 用 codec 生成热图目标 → 打包。GenerateTarget的encodercodec正是把上面定义的MSRAHeatmap编码器接入数据流。验证/测试管线不含任何随机增强仅做仿射裁剪。数据加载器与评测器train_dataloader dict( batch_size64, num_workers2, persistent_workersTrue, samplerdict(typeDefaultSampler, shuffleTrue), datasetdict( typeCocoDataset, data_rootdata/coco/, data_modetopdown, ann_fileannotations/person_keypoints_train2017.json, data_prefixdict(imgtrain2017/), pipelinetrain_pipeline)) val_dataloader dict( batch_size32, num_workers2, persistent_workersTrue, drop_lastFalse, samplerdict(typeDefaultSampler, shuffleFalse, round_upFalse), datasetdict( typeCocoDataset, data_rootdata/coco/, data_modetopdown, ann_fileannotations/person_keypoints_val2017.json, bbox_filedata/coco/person_detection_results/COCO_val2017_detections_AP_H_56_person.json, data_prefixdict(imgval2017/), test_modeTrue, pipelineval_pipeline)) val_evaluator dict( typeCocoMetric, ann_filedata_root annotations/person_keypoints_val2017.json) test_dataloader val_dataloader test_evaluator val_evaluator注意验证集通过bbox_file指定了固定的人体检测框结果即 README 中detector having human AP of 56.4的检测器输出这样不同模型的评测口径一致、可公平对比。评测指标由 mmpose/evaluation/metrics/coco_metric.py 中的CocoMetric计算 AP/AR。编解码器的纵深MSRAHeatmap 与 UDPHeatmapREADME 结果表中的Dark、UDP后缀本质上是更换了编解码器这也是 top-down heatmap 系列最重要的精度提升手段Codec配置文件示例特点MSRAHeatmaptd-hm_hrnet-w32_8xb64-210e_coco-256x192.pySimple Baselines 经典做法标准高斯热图MSRAHeatmapunbiasedTruetd-hm_hrnet-w32_dark-8xb64-210e_coco-256x192.pyDarkPose无偏高斯核 解码时热图调制refine_keypoints_dark缓解量化偏移UDPHeatmaptd-hm_hrnet-w32_udp-8xb64-210e_coco-256x192.pyUnbiased Data Processing消除坐标系量化偏差配合use_udpTrue仿射变换从源码看二者的本质差异MSRAHeatmap的缩放因子是input_size / heatmap_size见 msra_heatmap.pyUDPHeatmap采用(input_size - 1) / (heatmap_size - 1)见 udp_heatmap.py并对齐了坐标系两端从而彻底消除坐标变换的量化误差其heatmap_type还支持combined分类图 x/y 偏移图配合CombinedTargetMSELoss使用。对应地UDP 配置中TopdownAffine需显式加use_udpTrue例如 ViTPose 配置 td-hm_ViTPose-huge_8xb64-210e_coco-256x192.pycodec dict(typeUDPHeatmap, input_size(192, 256), heatmap_size(48, 64), sigma2) ... dict(typeTopdownAffine, input_sizecodec[input_size], use_udpTrue)该 ViTPose 配置同时展示了 Transformer 骨干 HeatmapHead 的另一种典型组合backbone 来自 mmpretrain 的VisionTransformerarchhuge、with_cls_tokenFalse、out_typefeatmaphead 使用deconv_out_channels(256, 256)两层反卷积上采样优化器换为 AdamW 并采用 LayerDecay 学习率分层衰减策略LayerDecayOptimWrapperConstructor见 mmpose/engine/optim_wrappers/layer_decay_optim_wrapper.py。训练、评测与推理仓库提供统一的训练/测试入口详见 tools/train.py 与 tools/test.py# 单卡训练 python tools/train.py configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py # 多卡分布式训练 bash tools/dist_train.sh configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py 8 # 使用训练好的权重评测 python tools/test.py configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py work_dirs/td-hm_hrnet-w32_8xb64-210e_coco-256x192/best_coco_AP_epoch_210.pth # 分布式评测 bash tools/dist_test.sh configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py work_dirs/.../best_coco_AP_epoch_210.pth 8其中.pth路径为训练后保存在work_dirs中的实际权重文件请以训练日志中的真实文件名为准。评测前需确保data/coco已按 docs/zh_cn/user_guides/prepare_datasets.md 的说明组织好数据集与bbox_file检测框文件。推理阶段可通过 demo/topdown_demo_with_mmdet.py 串联 mmdet 检测器与 mmpose 姿态模型实现检测 单人姿态估计的完整 Top-down 流程python demo/topdown_demo_with_mmdet.py \ demo/mmdetection_cfg/rtmdet_m_640-8xb32_coco-person.py \ https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192-81c58e40_20220909.pth \ --input demo/resources/000000000785.jpg \ --output-root vis_results/其中rtmdet_m_640-8xb32_coco-person.py为仅检测 person 类别的检测器配置权重地址来自对应模型的 ckpt 链接也可以改用 demo/inferencer_demo.py 通过Pose2DInferencer见 mmpose/apis/inferencers/pose2d_inferencer.py一行命令完成推理。模型库与基准结果总览COCOval2017检测器 human AP56.4输入 256×192ModelInput SizeAPAR详情与下载ViTPose-h256x1920.7900.840vitpose_coco.mdHRNet-w48UDP256x1920.7680.817hrnet_udp_coco.mdMSPN 4-stg256x1920.7650.826mspn_coco.mdHRNet-w48Dark256x1920.7640.814hrnet_dark_coco.mdHRNet-w48256x1920.7560.809hrnet_coco.mdHRFormer-B256x1920.7540.807hrformer_coco.mdRSN-50-3x256x1920.7500.814rsn_coco.mdCSPNeXt-l256x1920.7500.800cspnext_udp_coco.mdHRNet-w32256x1920.7490.804hrnet_coco.mdSwin-L256x1920.7430.798swin_coco.mdViTPose-s256x1920.7390.792vitpose_coco.mdHRFormer-S256x1920.7380.793hrformer_coco.mdSwin-B256x1920.7370.794swin_coco.mdSEResNet-101256x1920.7340.790seresnet_coco.mdSCNet-101256x1920.7330.789scnet_coco.mdResNet-101Dark256x1920.7330.786resnet_dark_coco.mdCSPNeXt-m256x1920.7320.785cspnext_udp_coco.mdResNetV1d-101256x1920.7320.785resnetv1d_coco.mdSEResNet-50256x1920.7290.784seresnet_coco.mdSCNet-50256x1920.7280.784scnet_coco.mdResNet-101256x1920.7260.783resnet_coco.mdResNeXt-101256x1920.7260.781resnext_coco.mdHourglassNet256x2560.7260.780hourglass_coco.mdResNeSt-101256x1920.7250.781resnest_coco.mdRSN-50256x1920.7240.790rsn_coco.mdSwin-T256x1920.7240.782swin_coco.mdMSPN 1-stg256x1920.7230.788mspn_coco.mdResNetV1d-50256x1920.7220.777resnetv1d_coco.mdResNeSt-50256x1920.7200.775resnest_coco.mdResNet-50256x1920.7180.774resnet_coco.mdResNeXt-50256x1920.7150.771resnext_coco.mdPVT-S256x1920.7140.773pvt_coco.mdCSPNeXt-s256x1920.6970.753cspnext_udp_coco.mdLiteHRNet-30256x1920.6760.736litehrnet_coco.mdCSPNeXt-tiny256x1920.6650.723cspnext_udp_coco.mdMobileNet-v2256x1920.6480.709mobilenetv2_coco.mdLiteHRNet-18256x1920.6420.705litehrnet_coco.mdCPM256x1920.6270.689cpm_coco.mdShuffleNet-v2256x1920.6020.668shufflenetv2_coco.mdShuffleNet-v1256x1920.5870.654shufflenetv1_coco.mdAlexNet256x1920.4480.521alexnet_coco.md从表中可清晰看到三条规律其一模型容量与精度整体正相关ViTPose-h 最高AlexNet 最低其二同样骨干下更换更优编解码UDP/Dark能带来稳定增益其三COCO 上完整逐项指标AP50/AP75/AR50 等可在各子页面中查看例如 hrnet_coco.md 还提供了 HRNet-w32/w48 在 256×192 与 384×288 两种分辨率下的详细结果及 ckpt/log 下载地址。MPIIvalPCKh0.5 / PCKh0.1ModelInput SizePCKh0.5PCKh0.1详情与下载HRNet-w48Dark256x2560.9050.360hrnet_dark_mpii.mdHRNet-w48256x2560.9020.303cspnext_udp_mpii.mdHRNet-w48256x2560.9010.337hrnet_mpii.mdHRNet-w32256x2560.9000.334hrnet_mpii.mdHourglassNet256x2560.8890.317hourglass_mpii.mdResNet-152256x2560.8890.303resnet_mpii.mdResNetV1d-152256x2560.8880.300resnetv1d_mpii.mdSCNet-50256x2560.8880.290scnet_mpii.mdResNeXt-152256x2560.8870.294resnext_mpii.mdSEResNet-50256x2560.8840.292seresnet_mpii.mdResNet-50256x2560.8820.286resnet_mpii.mdResNetV1d-50256x2560.8810.290resnetv1d_mpii.mdCPM368x368*0.8760.285cpm_mpii.mdLiteHRNet-30256x2560.8690.271litehrnet_mpii.mdLiteHRNet-18256x2560.8590.260litehrnet_mpii.mdMobileNet-v2256x2560.8540.234mobilenetv2_mpii.mdShuffleNet-v2256x2560.8280.205shufflenetv2_mpii.mdShuffleNet-v1256x2560.8240.195shufflenetv1_mpii.md注CPM 的输入尺寸为 368×368。CrowdPosetestYOLOv3 人体检测器ModelInput SizeAPAR详情与下载HRNet-w32256x1920.6750.816hrnet_crowdpose.mdCSPNeXt-m256x1920.6620.755cspnext_udp_crowdpose.mdResNet-101256x1920.6470.800resnet_crowdpose.mdHRNet-w32256x1920.6370.785resnet_crowdpose.mdAICvalGT bboxModelInput SizeAPAR详情与下载HRNet-w32256x1920.3230.366hrnet_aic.mdResNet-101256x1920.2940.337resnet_aic.mdJHMDBPCK0.2ModelInput SizePCK(norm. by person size)PCK (norm. by torso size)详情与下载ResNet-50256x25696.080.1resnet_jhmdb.mdCPM368x36889.865.7cpm_jhmdb.mdPoseTrack2018valGT bboxModelInput SizeAP详情与下载HRNet-w48256x19284.6hrnet_posetrack18.mdHRNet-w32256x19283.4hrnet_posetrack18.mdResNet-50256x19281.2resnet_posetrack18.mdHuman-Art检测器 human AP56.2 的验证集ModelInput SizeAPAR详情与下载ViTPose-s256x1920.3810.448vitpose_humanart.mdViTPose-b256x1920.4100.475vitpose_humanart.mdHuman-ArtGT bbox 验证集ModelInput SizeAPAR详情与下载ViTPose-s256x1920.7380.768vitpose_humanart.mdViTPose-b256x1920.7590.790vitpose_humanart.mdHuman-Art 的两组结果对比检测框 vs GT bbox直观反映了 Top-down 范式中检测器质量对姿态精度的决定性影响使用 GT bbox 时 ViTPose-b 可达 AP 0.759而接上检测器后降至 0.410因此在真实应用中优化检测器与姿态模型同等重要。选择合适的模型精度、速度与场景综合 README 结果表可给出如下选型参考基于仓库已公开的评测数据追求极致精度ViTPose-hAP 0.790需较大显存适合离线服务精度/速度均衡HRNet-w48/UDPAP 0.768、CSPNeXt-lAP 0.750与 HRNet-w32AP 0.749是通用场景最常用的选择轻量移动端/边缘部署LiteHRNet-18AP 0.642、MobileNet-v2AP 0.648、ShuffleNet-v2AP 0.602参数量小、推理快稠密人群场景优先参考 CrowdPose 结果HRNet-w32 AP 0.675并结合目标检测阶段的人员漏检情况做取舍。需要说明的是上表数值均以 README 给出的特定检测器如 COCO human AP56.4 的检测框为基准评测更换检测器后 AP 会相应变化实际部署前建议在自己的数据与检测器组合上重新评测。延伸阅读更多数据集基准动物、人脸、手部、全身见 configs/body_2d_keypoint/topdown_heatmap/README.md 同级的 animal_2d_keypoint、face_2d_keypoint、hand_2d_keypoint、wholebody_2d_keypoint 等目录框架级配置说明见 docs/zh_cn/user_guides/configs.md训练与测试流程见 docs/zh_cn/user_guides/train_and_test.md数据准备COCO/MPII 等标注格式与目录组织见 docs/zh_cn/user_guides/prepare_datasets.md推理与部署见 docs/zh_cn/user_guides/inference.md 与 demo/topdown_demo_with_mmdet.py。【免费下载链接】mmposeOpenMMLab Pose Estimation Toolbox and Benchmark.项目地址: https://gitcode.com/GitHub_Trending/mm/mmpose创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考