Registration of a generic 3D hand model on 2D radiographs using x-ray simulation and CMA-ES

Written by: Tianci Wen and Radu P. Mihail, and Franck P. Vidal

Published on: April, 2021

---

Summary

Radiographs of the hand are useful in diagnosing and staging diseases such as rheumatoid arthritis (RA) and other musculoskeletal diseases. Radiographs are projections of the 3D anatomy, with the useful information such as pose and pathology becoming lost in the process. We propose a 3D hand pose recovery method for radiographs of hands using a novel hybrid image registration method. Our pose recovery pipeline consists of aligning a simulated X-ray (digitally reconstructed radiograph) of an articulated phantom mesh model to a real hand radiograph using Covariance Matrix Adaptation Evolution Strategy. Early results demonstrate that our approach works well. Further inquiry is required to evaluate the applicability of our registration approach to other articulated musculoskeletal anatomy.

Publications

• Wen, T., Mihail, Radu P., & Vidal, F. P. (2021). 3D-2D registration using x-ray simulation and CMA-ES. International Conference on the Applications of Evolutionary Computation, evo*2021 (pp. 458–468). doi:10.1007/978-3-030-72699-7_29
• Wen, T., Mihail, R. P., Al-Maliki, S., Létang, J.-M., & Vidal, F. P. (2019). Tam, G. K. L., & Roberts, J. C. (Eds.). Registration of 3D triangular models to 2D X-ray projections using black-box optimisation and X-ray simulation. Computer Graphics and Visual Computing (CGVC) (pp. 105–113). The Eurographics Association. doi:10.2312/cgvc.20191265

Citations

@inproceedings{Wen2021EvoSTAR,
    title     = "{3D-2D} Registration using X-ray Simulation and {CMA-ES}", 
    author    = {Tianci Wen and Mihail, Radu P. and Franck P. Vidal},
    year      = 2021,
    doi       = "10.1007/978-3-030-72699-7_29",
    pages     = "458--468",
    booktitle = "International Conference on the Applications of 
        Evolutionary Computation, evo*2021",
}

 @inproceedings {Wen2019CGVC,
    booktitle = {Computer Graphics and Visual Computing (CGVC)},
    editor    = {G. K. L. Tam and J. C. Roberts},
    title     = {Registration of {3D} triangular models to {2D} {X-ray} projections using 
        black-box optimisation and {X-ray} simulation},
    author    = {T. Wen and R. P. Mihail and S. Al-Maliki and {J.-M.} L\'etang and 
        F.\,P. Vidal},
    year      = 2019,
    publisher = {The Eurographics Association},
    pages     = {105-113},
    isbn      = {978-3-03868-096-3},
    doi       = {10.2312/cgvc.20191265},
}