A new AI model that is capable of simultaneous analysis of different skin images can potentially improve the diagnostic accuracy of both dermatology and non-dermatology professionals.
An international research team led by AI and machine learning experts from Monash University developed a multimodal foundation model, called PanDerm, designed as a clinical decision support tool in dermatology. It can process multiple skin images at once and provide diagnostic probability assessments.
The AI model was trained on more than two million skin images of four types: close-up photos, dermoscopic images, pathology slides, and total body photographs, which were sourced from 11 institutions in different countries.
It was trained to perform a wide range of clinical tasks, including total-body skin cancer detection and risk assessment, cancer recurrence and metastasis prediction, skin type assessment, mole counting, lesion change tracking, differential diagnosis of various skin conditions, and lesion segmentation.
The PanDerm team involved researchers and doctors from Alfred Health, the University of Queensland, Princess Alexandra Hospital in Brisbane, Royal Prince Alfred Hospital, NSW Health Pathology, the University of Florence in Italy, Medical University of Vienna in Austria, NVIDIA AI Technology Centre in Singapore, and Hospital General Universitario de Alicante in Spain.
FINDINGS
The researchers conducted diagnostic performance validations and three reader studies to evaluate their model, the findings of which have been published in Nature Medicine.
Among notable findings was that the model outperformed clinicians in detecting early-stage melanoma, the most aggressive type of skin cancer, by 10%.
It was also found that the AI helped raise the accuracy of dermatologists in diagnosing skin cancer from dermoscopic images by 11% points to 80%.
Another significant finding was that PanDerm enhanced the ability of non-dermatologists to identify and differentiate skin conditions, such as inflammatory dermatoses and pigmentary disorders, based on photos by 16.5%. These include generalists who routinely perform initial skin assessments: general practitioners, general medicine practitioners, and nursing and clinical trial assistants.
Interestingly, the model was also found to surpass existing models (such as SwAVDerm, SL-Imagenet, and DINOv2) in performing various clinical tasks related to the assessment of skin cancer and other skin conditions, even when trained with only 10% of labelled data. Tasks include risk stratification of lesions, phenotype assessment, detection of lesion changes and malignancy, multi-class cancer diagnosis, lesion segmentation, and metastasis prediction and prognosis.
“Given limited specialist access in primary care settings where most skin conditions are initially evaluated, these findings indicate PanDerm’s potential to address dermatological expertise gaps across healthcare settings through both its technical capabilities and clinical applications. Importantly, across both human-AI collaboration studies, PanDerm alone performed equivalently to clinicians with PanDerm assistance in skin cancer diagnosis and even outperformed human-AI collaboration in differential diagnosis,” the authors said.
“This phenomenon probably stems from clinicians’ selective incorporation of AI recommendations rather than blind adherence, representing a balanced clinical implementation in which practitioners maintain their diagnostic autonomy while still benefiting from AI support,” they explained.
WHY IT MATTERS
Assessing skin conditions in clinical practice involves numerous tasks – from risk assessments and image analysis to monitoring lesions and predicting outcomes. While AI-powered clinical support tools are widely available for dermatology, they remain limited to single, isolated tasks.
“The absence of integrated AI solutions capable of supporting these various workflows currently hampers the practical impact of AI in dermatology,” the researchers said.
“Previous AI models have struggled to integrate and process various data types and imaging methods, reducing their usefulness to doctors in different real-world settings,” Monash University associate professor Zongyuan Ge, one of the study’s lead co-authors, was quoted as saying in a media release.
The PanDerm team’s study, according to H. Peter Soyer, another lead co-author, has revealed the potential of a new multimodal foundation model to support skin disease care in low-resource settings.
“The strength of PanDerm lies in its ability to support existing clinical workflows. It could be particularly valuable in busy or resource-limited settings, or in primary care where access to dermatologists may be limited. We have seen that the tool was also able to perform strongly even when trained on only a small amount of labelled data, a key advantage in diverse medical settings where standard annotated data is often limited,” said the professor and the director of the Dermatology Research Centre at the University of Queensland.
Additionally, PanDerm could be indispensable in detecting the deadly and invasive melanoma early. “This kind of assistance could support earlier diagnosis and more consistent monitoring for patients at risk of melanoma,” said Victoria Mar, one of the study’s lead co-authors and a professor and director of the Victorian Melanoma Service at Alfred Health.
The research team plans to conduct more clinical evaluations of their dermatology foundation model with a focus on ensuring equitable performance across different patient populations and healthcare settings.
THE LARGER TREND
It is also in Australia where the world’s first AI-driven pop-up skin care clinic was set up to detect skin cancers such as melanoma early. Roughly two of three Australians would be diagnosed with some form of skin cancer in their lifetime. Based on government statistics, around 400,000 cases are reported each year. The nurse-led pop-up clinic by health charity Skin Check Champions aims to bring that number down by half and increase skin screening by a quarter with the support of AI.
Outside Australia, recently, South Korea has approved its first locally developed AI-powered smartphone application for skin cancer diagnosis. The canofyMD SCAI by LifeSemantics received regulatory approval from the Ministry of Food and Drug Safety in June last year.
A new AI model that is capable of simultaneous analysis of different skin images can potentially improve the diagnostic accuracy of both dermatology and non-dermatology professionals.
An international research team led by AI and machine learning experts from Monash University developed a multimodal foundation model, called PanDerm, designed as a clinical decision support tool in dermatology. It can process multiple skin images at once and provide diagnostic probability assessments.
The AI model was trained on more than two million skin images of four types: close-up photos, dermoscopic images, pathology slides, and total body photographs, which were sourced from 11 institutions in different countries.
It was trained to perform a wide range of clinical tasks, including total-body skin cancer detection and risk assessment, cancer recurrence and metastasis prediction, skin type assessment, mole counting, lesion change tracking, differential diagnosis of various skin conditions, and lesion segmentation.
The PanDerm team involved researchers and doctors from Alfred Health, the University of Queensland, Princess Alexandra Hospital in Brisbane, Royal Prince Alfred Hospital, NSW Health Pathology, the University of Florence in Italy, Medical University of Vienna in Austria, NVIDIA AI Technology Centre in Singapore, and Hospital General Universitario de Alicante in Spain.
FINDINGS
The researchers conducted diagnostic performance validations and three reader studies to evaluate their model, the findings of which have been published in Nature Medicine.
Among notable findings was that the model outperformed clinicians in detecting early-stage melanoma, the most aggressive type of skin cancer, by 10%.
It was also found that the AI helped raise the accuracy of dermatologists in diagnosing skin cancer from dermoscopic images by 11% points to 80%.
Another significant finding was that PanDerm enhanced the ability of non-dermatologists to identify and differentiate skin conditions, such as inflammatory dermatoses and pigmentary disorders, based on photos by 16.5%. These include generalists who routinely perform initial skin assessments: general practitioners, general medicine practitioners, and nursing and clinical trial assistants.
Interestingly, the model was also found to surpass existing models (such as SwAVDerm, SL-Imagenet, and DINOv2) in performing various clinical tasks related to the assessment of skin cancer and other skin conditions, even when trained with only 10% of labelled data. Tasks include risk stratification of lesions, phenotype assessment, detection of lesion changes and malignancy, multi-class cancer diagnosis, lesion segmentation, and metastasis prediction and prognosis.
“Given limited specialist access in primary care settings where most skin conditions are initially evaluated, these findings indicate PanDerm’s potential to address dermatological expertise gaps across healthcare settings through both its technical capabilities and clinical applications. Importantly, across both human-AI collaboration studies, PanDerm alone performed equivalently to clinicians with PanDerm assistance in skin cancer diagnosis and even outperformed human-AI collaboration in differential diagnosis,” the authors said.
“This phenomenon probably stems from clinicians’ selective incorporation of AI recommendations rather than blind adherence, representing a balanced clinical implementation in which practitioners maintain their diagnostic autonomy while still benefiting from AI support,” they explained.
WHY IT MATTERS
Assessing skin conditions in clinical practice involves numerous tasks – from risk assessments and image analysis to monitoring lesions and predicting outcomes. While AI-powered clinical support tools are widely available for dermatology, they remain limited to single, isolated tasks.
“The absence of integrated AI solutions capable of supporting these various workflows currently hampers the practical impact of AI in dermatology,” the researchers said.
“Previous AI models have struggled to integrate and process various data types and imaging methods, reducing their usefulness to doctors in different real-world settings,” Monash University associate professor Zongyuan Ge, one of the study’s lead co-authors, was quoted as saying in a media release.
The PanDerm team’s study, according to H. Peter Soyer, another lead co-author, has revealed the potential of a new multimodal foundation model to support skin disease care in low-resource settings.
“The strength of PanDerm lies in its ability to support existing clinical workflows. It could be particularly valuable in busy or resource-limited settings, or in primary care where access to dermatologists may be limited. We have seen that the tool was also able to perform strongly even when trained on only a small amount of labelled data, a key advantage in diverse medical settings where standard annotated data is often limited,” said the professor and the director of the Dermatology Research Centre at the University of Queensland.
Additionally, PanDerm could be indispensable in detecting the deadly and invasive melanoma early. “This kind of assistance could support earlier diagnosis and more consistent monitoring for patients at risk of melanoma,” said Victoria Mar, one of the study’s lead co-authors and a professor and director of the Victorian Melanoma Service at Alfred Health.
The research team plans to conduct more clinical evaluations of their dermatology foundation model with a focus on ensuring equitable performance across different patient populations and healthcare settings.
THE LARGER TREND
It is also in Australia where the world’s first AI-driven pop-up skin care clinic was set up to detect skin cancers such as melanoma early. Roughly two of three Australians would be diagnosed with some form of skin cancer in their lifetime. Based on government statistics, around 400,000 cases are reported each year. The nurse-led pop-up clinic by health charity Skin Check Champions aims to bring that number down by half and increase skin screening by a quarter with the support of AI.
Outside Australia, recently, South Korea has approved its first locally developed AI-powered smartphone application for skin cancer diagnosis. The canofyMD SCAI by LifeSemantics received regulatory approval from the Ministry of Food and Drug Safety in June last year.
