Methods for Image Data Analysis
The MIDA group focuses on development of general methods for image data analysis. Our aim is to devise generally applicable methods, which work well independent of the particular application and types of images used. We therefore strive for robust methods which are performing well under varying conditions. Also aiming for practically useful methods, we essentially always collaborate with other groups, including Social Robotics Lab, Quantitative Microscopy and MedIP - Medical Image Processing.
- Explainable Artificial Intelligence
- Image Registration
- Representation Learning and Image Translation
- Image Similarity and Distance Measures
Explainable Artificial Intelligence
Interpretation of classification behaviour of deep neural network models
Deep convolutional neural networks demonstrate state-of-the-art performance in many image analysis tasks, however, their opacity does not allow to infer how they arrive at a decision. We are aiming at detection of oral cancer at an early stage, and it is particularly important to develop a reliable algorithm. In our workflow, trained deep convolutional neural networks are used to differentiate cytological images into normal and abnormal classes. We examine methods that could elevate understanding of the deep learning classification properties and enable interpretation of data classification. Furthermore, we would like to increase understanding of the premalignant state by exploring and visualizing what parts of cell images are considered as most important for the task.
- N. Koriakina, N. Sladoje, E. Wetzer, J. Lindblad. Uncovering hidden reasoning of convolutional neural networks in biomedical image classification by using attribution methods. 4th NEUBIAS Conference, Bordeaux, France, March 2020.
- N. Koriakina, N. Sladoje, E. Bengtsson, E. Darai Ramqvist, J-M. Hirsch, C. Runow Stark, J. Lindblad. Visualization of convolutional neural network class activations in automated oral cancer detection for interpretation of malignancy associated changes. 3rd NEUBIAS Conference, Luxembourg, Feb. 2019.Poster
Trustworthy AI-based decision support in cancer diagnostics
To reach successful implementation of AI-based decision support in healthcare it is important to have trust in the system outputs. One reason for lack of trust is the lack of interpretability of the complex non-linear decision making process. A way to build trust is thus to improve humans’ understanding of the process, which drives research within the field of Explainable AI. For a successful implementation of AI in healthcare and life sciences, it is imperative to acknowledge the need for cooperation of human experts and AI-based decision making systems: Deep learning methods, and AI systems, should not replace, but rather augment clinicians and researchers. This project aims to facilitate understandable, reliable and trustworthy utilization of AI in healthcare, empowering the human medical professionals to interpret and interact with the AI-based decision support system.
- J. Lu, N. Sladoje, C. Runow Stark, E. Darai Ramqvist, J-M. Hirsch, J. Lindblad. A Deep Learning based Pipeline for Efficient Oral Cancer Screening on Whole Slide Images. In Proc. of the Intern. Conf. of Image Analysis and Recognition, ICIAR 2020, Lecture Notes in Computer Science - LNCS 12132, pp 249-261, Springer 2020. [Online]
In this project we study general-purpose registration methods based on distances incorporating both intensity and spatial information. Thus far, we have developed methods that increase convergence regions when using gradient-based optimization, while improving the accuracy, as well as having similar or shorter run-time than comparable intensity-based methods. Through our work on the CoMIR representations, we have shown that these methods can be extended from mono-modal to multi-modal scenarios. Our current research in this project aims to find paths towards greater generality in applicability, increased computational efficiency, and methods to overcome the idiosyncratic challenges of various applications (such as local discontinuities, missing structures, noise and other corruption).
- J. Öfverstedt, J. Lindblad, and N. Sladoje. INSPIRE: Intensity and Spatial Information-Based Deformable Image Registration. Submitted [Preprint]
- J. Öfverstedt, J. Lindblad, N. Sladoje. Fast and Robust Symmetric Image Registration Based on Distances Combining Intensity and Spatial Information. IEEE Trans. on Image Processing, Vol.28(7), pp. 3584 - 3597, 2019. [Online]
Representation Learning and Image Translation
Robust learning of geometric equivariances
The project builds on, and extends recent works on Geometric deep learning and aims at combining it with Manifold learning, to produce truly learned equivariances without the need for engineered solutions and maximize benefits of shared weights (parameters to learn). A decrease of the numbers of parameters to learn leads to increased performance, generalizability and reliability (robustness) of the network. An additional gain is in reducing a risk that the augmented data incorporates artefacts not present in the original data. A typical example is textured data, where interpolation performed in augmentation by rotation and scaling unavoidably affects the original texture and may lead to non-reliable results. Reliable texture-based classification is, on the other hand, in many cases of high importance in biomedical applications.
- N. Pielawski, E. Wetzer, J. Öfverstedt, J. Lu, C. Wählby, J. Lindblad, N. Sladoje. CoMIR: Contrastive Multimodal Image Representation for Registration. In Proc. of NeurIPS 2020 [Online]
- T. Majtner, B. Bajic, J. Lindblad, N. Sladoje, V. Blanes-Vidal, E. S. Nadimi. On the Effectiveness of Generative Adversarial Networks as HEp-2 Image Augmentation Tool. In Proceedings of the Scandinavian Conference on Image Analysis, SCIA2019, Norrköping, Sweden, Lecture Notes in Computer Science, LNCS-11482, pp. 439-451, Springer 2019. [Online]
Multi-layer object representations for texture analysis
Texture features such as local binary patterns have shown to provide complementary information to that of plain intensity images in learning algorithms. We investigate methods on the fusion of texture and intensity sources, as well as the problems connected to the fact that many texture descriptors are unordered sets and require suitable (dis-)similarity measures in order to be processes by for example convolutional neural networks. We develop strategies to integrate more complex texture features into learning methods and evaluate their performance on various biomedical images. Such hybrid object representations show promising results in, e.g., detection and segmentation of high resolution transmission electron microscope (TEM) images, taking it one step closer to automation of pathological diagnostics.
- E. Wetzer, J. Gay, H. Harlin, J. Lindblad, and N. Sladoje. When texture matters: Texture-focused CNNs outperform general data augmentation and pretraining in Oral Cancer Detection. In Proceedings of the 17th IEEE International Symposium on Biomedical Imaging (ISBI), IEEE, pp. 517-521, Iowa City, USA, April 2020. doi:10.1109/ISBI45749.2020.9098424.
- B. Bajic, T. Majtner, J. Lindblad, N. Sladoje. Generalized Deep Learning Framework for HEp-2 Cell Recognition Using Local Binary Pattern Maps. IET Image Processing, Vol. 14, No. 6, pp. 1201-1208, 2020. [Online]
- E. Wetzer, J. Lindblad, I.-M. Sintorn, K. Hultenby, N. Sladoje. Towards automated multiscale imaging and analysis in TEM: Glomerulus detection by fusion of CNN and LBP maps. In Proc. of the ECCV 2018, Workshop on BioImage Computing, Lecture Notes in Computer Science, LNCS-11134, pp. 465-475, Munich, Germany, Sept. 2018. [Online]
Image Similarity and Distance Measures
Stochastic Distance Transform
Distance transforms (DTs) are based on minimal distances and are thus noise sensitive; a single noisy point can change the distance substantially. The Stochastic Distance Transform (SDT) is a distance transform based on stochastic modelling of the binary images using the theory of discrete random sets. In this project, we explore theoretical properties of the SDT, efficient algorithms that enable the computation of these distance transforms, and how the accuracy of the resulting distances are substantially improved by adopting the SDT in favor of classical DTs.
- J. Öfverstedt, J. Lindblad, and N. Sladoje. Stochastic Distance Transform: Theory, Algorithms and Applications. Journal of Mathematical Imaging and Vision 62(5), 751-769, 2020. [Online]
- J. Öfverstedt, J. Lindblad, N. Sladoje. Stochastic Distance Transform. In Proceedings of the 21th International Conference on Discrete Geometry for Computer Imagery, DGCI2019, Paris, France, Lecture Notes in Computer Science, LNCS-11414, pp. 75-86, Springer 2019. [Online]
Combining Shape and Intensity Information
Similarity (or distance) measures between images are fundamental components in image analysis, and are used in many tasks such as
- template matching,
- image registration,
- objective functions for training various types of Neural Networks.
We study measures which combine image intensity and spatial information efficiently and aim to demonstrate that they lead to practical, robust, high performance methods for these and other common tasks.
- J. Öfverstedt, N. Sladoje, and J. Lindblad. Distance Between Vector-valued Fuzzy Sets based on Intersection Decomposition with Applications in Object Detection. In Proc. of the 13th International Symposium on Mathematical Morphology, ISMM2017, Fontainebleau, France, Lecture Notes in Computer Science, LNCS-10225, pp. 395-407, Springer 2017. [Online]
- N. Sladoje and J. Lindblad. Distance Between Vector-valued Representations of Objects in Images with Application in Object Detection and Classification. In Proc. of the 18th International Workshop on Combinatorial Image Analysis, IWCIA2017, Plovdiv, Bulgaria, Lecture Notes in Computer Science, LNCS-10256, pp. 243-255, Springer 2017. [Online]
Karl Bengtsson Bernander
Marc Fraile Fabrega