DeepTag: A General Framework for Fiducial Marker Design and Detection

Zhuming Zhang    Yongtao Hu    Guoxing Yu    Jingwen Dai

Guangdong Virtual Reality Technology Co., Ltd. (aka. Ximmerse), Guangzhou

IEEE Transactions on Pattern Analysis and Machine Intelligence (IEEE TPAMI 2023)

Left: a general marker design supported by DeepTag. Right: top & bottom rows show detection results of existing methods and DeepTag respectively.
DeepTag successfully detects all markers while original methods either miss some or totally fail.

Abstract

A fiducial marker system usually consists of markers, a detection algorithm, and a coding system. The appearance of markers and the detection robustness are generally limited by the existing detection algorithms, which are hand-crafted with traditional low-level image processing techniques. Furthermore, a sophisticatedly designed coding system is required to overcome the shortcomings of both markers and detection algorithms. To improve the flexibility and robustness in various applications, we propose a general deep learning based framework, DeepTag, for fiducial marker design and detection. DeepTag not only supports detection of a wide variety of existing marker families, but also makes it possible to design new marker families with customized local patterns. Moreover, we propose an effective procedure to synthesize training data on the fly without manual annotations. Thus, DeepTag can easily adapt to existing and newly-designed marker families. To validate DeepTag and existing methods, beside existing datasets, we further collect a new large and challenging dataset where markers are placed in different view distances and angles. Experiments show that DeepTag well supports different marker families and greatly outperforms the existing methods in terms of both detection robustness and pose accuracy. Both code and dataset are available.

Paper

• Preprint (arXiv) (PDF, 6.6 MB)

Code

• DeepTag - PyTorch (w/ pre-trained models, hosted on GitHub)

Dataset

• Parameters of used camera (TXT)
• DeepTag dataset (104.0G): readme | data (via TeraBox) | data (via Baidu Drive) (password: 2fym)
• DeepTag dataset - noise part (38.5G): readme | data (via TeraBox) | data (via Baidu Drive) (password: 4i8l)
• DeepTag dataset - motion blur part (3.9G): readme | data (via TeraBox) | data (via Baidu Drive) (password: eade)

Bibtex

@article{zhang2023deeptag,
  title={{DeepTag: A General Framework for Fiducial Marker Design and Detection}},
  author={Zhang, Zhuming and Hu, Yongtao and Yu, Guoxing and Dai, Jingwen},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  volume={45},
  number={3},
  pages={2931-2944},
  year={2023},
  publisher={IEEE}
}