项目作者: ducha-aiki

项目描述 :
PyTorch implementation of SIFT descriptor
高级语言: Jupyter Notebook
项目地址: git://github.com/ducha-aiki/pytorch-sift.git
创建时间: 2017-04-30T18:29:05Z
项目社区:https://github.com/ducha-aiki/pytorch-sift

开源协议:BSD 2-Clause "Simplified" License

下载


This is an differentiable pytorch implementation of SIFT patch descriptor. It is very slow for describing one patch, but quite fast for batch. It can be used for descriptop-based learning shape of affine feature.

UPD 08/2019 : pytorch-sift is added to kornia and available by kornia.features.SIFTDescriptor

There are different implementations of the SIFT on the web. I tried to match Michal Perdoch implementation, which gives high quality features for image retrieval CVPR2009. However, on planar datasets, it is inferior to vlfeat implementation.
The main difference is gaussian weighting window parameters, so I have made a vlfeat-like version too. MP version weights patch center much more (see image below, left) and additionally crops everything outside the circular region. Right is vlfeat version

Michal Perdoch kernel
vlfeat kernel

  1. descriptor_mp_mode = SIFTNet(patch_size = 65,
  2. sigma_type= 'hesamp',
  3. masktype='CircularGauss')
  4. descriptor_vlfeat_mode = SIFTNet(patch_size = 65,
  5. sigma_type= 'vlfeat',
  6. masktype='Gauss')

Results:

hpatches mathing results

  1. OPENCV-SIFT - mAP
  2. Easy Hard Tough mean
  3. ------- ------- --------- -------
  4. 0.47788 0.20997 0.0967711 0.26154
  5. VLFeat-SIFT - mAP
  6. Easy Hard Tough mean
  7. -------- -------- --------- --------
  8. 0.466584 0.203966 0.0935743 0.254708
  9. PYTORCH-SIFT-VLFEAT-65 - mAP
  10. Easy Hard Tough mean
  11. -------- -------- --------- --------
  12. 0.472563 0.202458 0.0910371 0.255353
  13. NUMPY-SIFT-VLFEAT-65 - mAP
  14. Easy Hard Tough mean
  15. -------- -------- --------- --------
  16. 0.449431 0.197918 0.0905395 0.245963
  17. PYTORCH-SIFT-MP-65 - mAP
  18. Easy Hard Tough mean
  19. -------- -------- --------- --------
  20. 0.430887 0.184834 0.0832707 0.232997
  21. NUMPY-SIFT-MP-65 - mAP
  22. Easy Hard Tough mean
  23. -------- ------- --------- --------
  24. 0.417296 0.18114 0.0820582 0.226832

Speed:

  • 0.00246 s per 65x65 patch - numpy SIFT
  • 0.00028 s per 65x65 patch - C++ SIFT
  • 0.00074 s per 65x65 patch - CPU, 256 patches per batch
  • 0.00038 s per 65x65 patch - GPU (GM940, mobile), 256 patches per batch
  • 0.00038 s per 65x65 patch - GPU (GM940, mobile), 256 patches per batch

If you use this code for academic purposes, please cite the following paper:

  1. @InProceedings{AffNet2018,
  2. title = {Repeatability Is Not Enough: Learning Affine Regions via Discriminability},
  3. author = {Dmytro Mishkin, Filip Radenovic, Jiri Matas},
  4. booktitle = {Proceedings of ECCV},
  5. year = 2018,
  6. month = sep
  7. }