Code for "MEBOW: Monocular Estimation of Body Orientation In the Wild", CVPR 2020
This is an official pytorch implementation of MEBOW: Monocular Estimation of Body Orientation In the Wild.
In this work, we present COCO-MEBOW (Monocular Estimation of Body Orientation in the Wild), a new large-scale dataset for orientation estimation from a single in-the-wild image. Based on COCO-MEBOW, we established a simple baseline model for human body orientation estimation. This repo provides the code.
pip install -r requirements.txt
Note that instructions like # COCOAPI=/path/to/install/cocoapi indicate that you should pick a path where you’d like to have the software cloned and then set an environment variable (COCOAPI in this case) accordingly.
# COCOAPI=/path/to/clone/cocoapi
git clone https://github.com/cocodataset/cocoapi.git $COCOAPI
cd $COCOAPI/PythonAPI
# Install into global site-packages
make install
# Alternatively, if you do not have permissions or prefer
# not to install the COCO API into global site-packages
python3 setup.py install --user
Init output(training model output directory) and log(tensorboard log directory) directory:
mkdir output
mkdir log
Your directory tree should look like this:
${HBOE_ROOT}
├── data
├── experiments
├── lib
├── log
├── models
├── output
├── tools
├── README.md
└── requirements.txt
Download pretrained models from the model zoo provided by HRnet(GoogleDrive or OneDrive)
${HBOE_ROOT}
`-- models
`-- pose_hrnet_w32_256x192.pth
models
.
${HBOE_ROOT}
`-- models
`-- model_hboe.pth
python tools/demo.py --cfg experiments/coco/segm-4_lr1e-3.yaml images/demo.jpg
. You may use the path of your own image to replace images/demo.jpg
. For better performance, your image should be a human-cropped image. The ratio of height to width should be around 4:3.For MEBOW dataset, please download images, bboxes and keypoints from COCO download. Please email czw390@psu.edu to get access to the human body orientation annotations. Note: For academic researchers, please use your educational email address. You will directly get access to the annotations via your educational email. For researchers in business companies, please send a formal letter (with the company name and your signature) to promise that you will not use the annotations for commercial purposes. Sorry for the inconvenience.
Put images and all the annotations under {HBOE_ROOT}/data, and make them look like this:
${HBOE_ROOT}
|-- data
`-- |-- coco
`-- |-- annotations
| |-- train_hoe.json
| |-- val_hoe.json
| |-- person_keypoints_train2017.json
| `-- person_keypoints_val2017.json
`-- images
|-- train2017
| |-- 000000000009.jpg
| |-- 000000000025.jpg
| |-- 000000000030.jpg
| |-- ...
`-- val2017
|-- 000000000139.jpg
|-- 000000000285.jpg
|-- 000000000632.jpg
|-- ...
For TUD dataset, please download images from the web page of TUD. The page also provides 8-bin orientation annotation. Continuous orientation annotation for TUD dataset can be found from here. We provide our precessed TUD annotation from /g/personal/czw390_psu_edu/EqU8hWh-NgFOoNmIBEgE5RYBn61ZsFudKHCgbEH9-_V9DA?e=PZzshY">here.
Put TUD images and our processed annotation under {HBOE_ROOT}/data, and make them look like this:
${HBOE_ROOT}
|-- data
`-- |-- tud
`-- |-- annot
| |-- train_tud.pkl
| |-- val_tud.pkl
| `-- test_tud.pkl
`-- images
|-- train
|-- validate
`-- test
We also provide the trained HBOE model (MEBOW as training set). (/g/personal/czw390_psu_edu/EoXLPTeNqHlCg7DgVvmRrDgB_DpkEupEUrrGATpUdvF6oQ?e=CQQ2KY">OneDrive)
python tools/train.py --cfg experiments/coco/segm-4_lr1e-3.yaml
python tools/train.py --cfg experiments/tud/lr1e-3.yaml
You should change TEST:MODEL_FILE to your own in “experiments/coco/segm-4_lr1e-3.yaml”. If you want to test with our trained HBOE model, specify TEST:MODEL_FILE with the downloaded model path.
python tools/test.py --cfg experiments/coco/segm-4_lr1e-3.yaml
This repo is based on HRnet.
If you use our dataset or models in your research, please cite with:
@inproceedings{wu2020mebow,
title={MEBOW: Monocular Estimation of Body Orientation In the Wild},
author={Wu, Chenyan and Chen, Yukun and Luo, Jiajia and Su, Che-Chun and Dawane, Anuja and Hanzra, Bikramjot and Deng, Zhuo and Liu, Bilan and Wang, James Z and Kuo, Cheng-hao},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={3451--3461},
year={2020}
}