27641 Geometric Deep Learning Model for Detection of Knee Osteoarthritis


In the last decade, artificial neural networks a.k.a. Deep Learning (DL) led to an unprecedented performance in a range of problems from various disciplines, most notably in computer vision, speech recognition, and natural language processing. The present huge success of DL stems from the availability of large-scale datasets and affordable high computational power. However, the DL research has focused so far only on data that lie in Euclidean domains, i.e., on regular grids. While this is true for classical one-dimensional signals and image/video data, in many domains the data do not live on regular lattices. For example, point cloud data that are encountered in computer vision and in autonomous driving are defined on graphs and not on Euclidean grids. Similarly, social network data, as well as many instances of data in computer graphics, recommender systems, biology, and physics live on graphs and manifolds. The direct application of existing DL approaches in these fields is not possible, for multiple reasons (Fig. 2). An emerging field of machine learning, named Geometric Deep Learning aims to bridge this gap by extending DL techniques to graph/manifold structured data (see Fig. 1).  


Figure 1:  The “5G” of Geometric Deep Learning: different types of geometric structures where the generalized design                                               Figure 2:  Geometric deep learning on graphs and manifolds. The non-Euclidean nature of data 
                                                         of geometric deep learning can be applied to. [3]                                                                                                                  makes the definition of basic operations (such as convolution) rather elusive. [2]

Recently, geometric deep learning has achieved impressive results in medical imaging data processing, imaging genetics, and spatiotemporal anatomical representations. One of the possible future applications is to predict the risk of osteoarthritis (OA). Osteoarthritis is a chronic disease in which the cartilage in the knee joint becomes inflamed and damaged, resulting in pain, swelling, stiffness, and reduced mobility (Figure 3). It is of great significance to construct a satisfactory 3D model of the knee joint that meets the requirements of biomechanical analysis and motion simulation. Existing literature indicates that there are ongoing efforts to develop MRI imaging methods to further optimize cartilage assessment, with the aim of improving clinical evaluation and providing reproducible and detailed information that physicians can use in follow-up studies to analyze the effects of new surgical or drug therapies. With recent developments in geometric deep learning, this problem can be directly addressed by analyzing 3D knee joint models, providing more accurate results. 

Figure 3: Clear difference between the healthy knee and the knee with cartilage damage. [4]


The main goal of this thesis is to develop a novel geometric deep learning (GDL) prediction model for the risk of OA progression by using knee images in patients. With the available annotated data, it is known which of the patients developed arthritis throughout their life and which didn’t, and an efficient machine learning model can use this information to learn subtle patterns on the MRI scans. The developed model could then be used for predicting whether a patient has a future osteoarthritis risk or not. All the models for OA prediction in the available literature are focusing on classical deep learning-based models, where the data resides in Euclidean domains. In this work, a student should develop a 3D prediction model of the knee, based on geometric deep learning. In order to cope with the massive amounts of data points generated in the acquisition process, efficient 3D representation methods are needed. After getting familiar with the provided dataset and the state-of-the-art GDL techniques and their applications in medical imaging, the student should build a novel geometric deep learning model for predicting the osteoarthritis risk. At the beginning of the semester, a minicourse will be organized to familiarize the student with the topic, and the necessary literature and code will be provided.


  1. R. Mahum, S.U. Rehman, T. Meraj, et al. A novel hybrid approach based on deep CNN features to detect knee osteoarthritis. Sensors-Basel, 2021L21(18). (2021)
  2. M. Bronstein et al. Geometric deep learning on graphs and manifolds. SIAM 2018. (2018)
  3. M. Bronstein et al. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv:2104.13478 (2021).
  4. Osteoarthritis – Healthdirect (https://www.healthdirect.gov.au/osteoarthritis)