Advertisement

Artificial intelligence in the medical physics community: An international survey

Published:January 13, 2021DOI:https://doi.org/10.1016/j.ejmp.2020.11.037

      Highlights

      • Medical Physicists recognize the value and implications of AI in diagnostic and therapy.
      • Medical Physicists want to be more involved in Artificial Intelligence projects and training.
      • Artificial intelligence should be included in medical physics training.

      Purpose

      To assess current perceptions, practices and education needs pertaining to artificial intelligence (AI) in the medical physics field.

      Methods

      A web-based survey was distributed to the European Federation of Organizations for Medical Physics (EFOMP) through social media and email membership list. The survey included questions about education, personal knowledge, needs, research and professionalism around AI in medical physics. Demographics information were also collected. Responses were stratified and analysed by gender, type of institution and years of experience in medical physics. Statistical significance (p<0.05) was assessed using paired t-test.

      Results

      219 people from 31 countries took part in the survey. 81% (n = 177) of participants agreed that AI will improve the daily work of Medical Physics Experts (MPEs) and 88% (n = 193) of respondents expressed the need for MPEs of specific training on AI. The average level of AI knowledge among participants was 2.3 ± 1.0 (mean ± standard deviation) in a 1-to-5 scale and 96% (n = 210) of participants showed interest in improving their AI skills. A significantly lower AI knowledge was observed for female participants (2.0 ± 1.0), compared to male responders (2.4 ± 1.0). 64% of participants indicated that they are not involved in AI projects. The percentage of female leading AI projects was significantly lower than the male counterparts (3% vs 19%).

      Conclusions

      AI was perceived as a positive resource to support MPEs in their daily tasks. Participants demonstrated a strong interest in improving their current AI-related skills, enhancing the need for dedicated training for MPEs.

      Keywords

      1. Introduction

      In the last decade, widespread applications of Artificial Intelligence (AI) have found their ways into our daily life. AI is fundamentally changing various aspects of human existence, ranging in difficulty from the lock system for our mobile phones, to major decision-making tasks for space ships, and healthcare is not an exception to this influence [

      Hamet P, Tremblay J. Artificial intelligence in medicine. Metabolism 69 (2017) S36–S40, insights Into the Future of Medicine: Technologies, Concepts, and Integration.https://doi.org/10.1016/j.metabol.2017.01.011.

      ,
      • Hosny A.
      • Parmar C.
      • Quackenbush J.
      • Schwartz L.H.
      • Aerts H.J.W.L.
      Artificial intelligence in radiology.
      ].
      AI can revolutionize clinical practices, helping to optimise a wide spectrum of clinical tasks such as predicting scheduled hospital attendance [
      • Nelson A.
      • Herron D.
      • Rees G.
      • Nachev P.
      Predicting scheduled hospital attendance with artificial intelligence.
      ], automatic disease detection in diagnostic images [
      • Bi W.L.
      • Hosny A.
      • Schabath M.B.
      • Giger M.L.
      • Birkbak N.J.
      • Mehrtash A.
      • Allison T.
      • Arnaout O.
      • Abbosh C.
      • Dunn I.F.
      • Mak R.H.
      • Tamimi R.M.
      • Tempany C.M.
      • Swanton C.
      • Hoffmann U.
      • Schwartz L.H.
      • Gillies R.J.
      • Huang R.Y.
      • Aerts H.J.W.L.
      Artificial intelligence in cancer imaging: clinical challenges and applications.
      ,
      • Agarwal R.
      • Diaz O.
      • Yap M.H.
      • Lladó X.
      • Martí R.
      Deep learning for mass detection in full field digital mammograms.
      ], and many more. Therefore, it is not surprising that, over the past years, AI is one of the primary and most rapidly growing topics discussed in scientific sessions and exhibition floors of major medical conferences like the European Congress of Radiology (ECR), the American Association of Physicists in Medicine (AAPM) annual meeting, the Radiological Society of North America (RSNA) annual meeting, the annual congress of the European Association of Nuclear Medicine (EANM) or the Annual Meeting of the European Society of Medical Imaging and Informatics (EuSoMII).
      Such a rapid change in healthcare is already affecting the future of medical physics, by introducing new and previously impossible opportunities, as well as pitfalls [
      • Thrall J.H.
      • Li X.
      • Li Q.
      • Cruz C.
      • Do S.
      • Dreyer K.
      • Brink J.
      Artificial intelligence and machine learning in radiology: opportunities, challenges, pitfalls, and criteria for success.
      ,
      • Xing L.
      • Krupinski E.A.
      • Cai J.
      Artificial intelligence will soon change the landscape of medical physics research and practice.
      ,

      G. Mahadevaiah, P. RV, I. Bermejo, D. Jaffray, A. Dekker, L. Wee, Artificial intelligence-based clinical decision support in modern medical physics: Selection, acceptance, commissioning, and quality assurance, Med Phys 47(5) (2020) e228–e235.https://doi.org/10.1002/mp.13562.

      ]. In order to keep up with this change, it is essential to start working on the way medical physicists will re-shape and adapt their roles to this new technological implementation in healthcare provisions [
      • Kortesniemi M.
      • Tsapaki V.
      • Trianni A.
      • Russo P.
      • Maas A.
      • Källman H.-E.
      • Brambilla M.
      • Damilakis J.
      The European Federation of Organisations for Medical Physics (EFOMP) White Paper: Big data and deep learning in medical imaging and in relation to medical physics profession.
      ,
      • Cui S.
      • Tseng H.-H.
      • Pakela J.
      • Ten Haken R.K.
      • El Naqa I.
      Introduction to machine and deep learning for medical physicists.
      ].
      Although explicit tasks differ per country, medical physicists play a major role in bridging the gap between applied and life sciences, requiring a highly multidisciplinary work approach. Their primary professional value comes from their ability to interface with other specialties across the hospital. Medical Physicists combine the knowledge from diverse disciplines to support a decision that merges the context, the data, the clinical information and other available knowledge to answer the clinical questions or to allow for safe implementation of technical innovations within healthcare; all this while keeping patient safety as the primary focus.
      According to Annex 1 of the European Radiation Protection guidelines number 174 [

      Putten Wvd, Caruana CJ, Evans S, Guibelalde E, Christofides S. Radiation Protection No 174. European Guidelines on Medical Physics Expert, European Commission; 2014. doi:10.2833/18393.

      ], the Medical Physics Expert (MPE) must contribute to maintaining and improving the quality, safety, and cost-effectiveness of healthcare services through patient-oriented activities. This requires expert action, involvement regarding the specification, selection, commissioning, quality assurance (QA) and quality control (QC), installation design, surveillance and optimised clinical use of medical devices, including AI software. In some cases, medical physicists are also involved in in–house development of AI software. Frequent questions that a MPE involved in implementation of AI-based software is facing include: (i) what is a suitable QA or QC test; (ii) how should it be performed; (iii) how can we support other stakeholders and ensure that the software is safely integrated in the clinical workflow; (iv) how can we perform surveillance on this tool over time; (v) how can we collaborate with industry to improve these tools?.
      Medical physicists in modern medicine should be ready and be able to address all of the challenging questions listed above through a multidisciplinary approach, by keeping their knowledge up-to-date [
      • Geleijns J.
      • Breatnach É.
      • Cantera A.C.
      • Damilakis J.
      • Dendy P.
      • Evans A.
      • Faulkner K.
      • Padovani R.
      • Van Der Putten W.
      • Schad L.
      • Wirestam R.
      • Eudaldo T.
      Core curriculum for medical physicists in radiology.
      ,
      • Eriksen J.G.
      • Beavis A.W.
      • Coffey M.A.
      • Leer J.W.H.
      • Magrini S.M.
      • Benstead K.
      • Boelling T.
      • Hjälm-Eriksson M.
      • Kantor G.
      • Maciejewski B.
      • Mezeckis M.
      • Oliveira A.
      • Thirion P.
      • Vitek P.
      • Olsen D.R.
      • Eudaldo T.
      • Enghardt W.
      • François P.
      • Garibaldi C.
      • Heijmen B.
      • Josipovic M.
      • Major T.
      • Nikoletopoulos S.
      • Rijnders A.
      • Waligorski M.
      • Wasilewska-Radwanska M.
      • Mullaney L.
      • Boejen A.
      • Vaandering A.
      • Vandevelde G.
      • Verfaillie C.
      • Pötter R.
      The updated ESTRO core curricula 2011 for clinicians, medical physicists and RTTs in radiotherapy/radiation oncology.
      ,
      • Del Guerra A.
      • Bardies M.
      • Belcari N.
      • Caruana C.J.
      • Christofides S.
      • Erba P.
      • Gori C.
      • Lassmann M.
      • Lonsdale M.N.
      • Sattler B.
      • Waddington W.
      Curriculum for education and training of Medical Physicists in Nuclear Medicine: Recommendations from the EANM Physics Committee, the EANM Dosimetry Committee and EFOMP.
      ]. Currently, there is a shortage if not a total lack of courses or workshops on AI-related methods that approach an education program focused purely on the medical physicists needs.
      For this reason, in mid-2019 a working group on AI was established under the umbrella of the scientific committee of the European Federation of Medical Physics (EFOMP). The main objective of the workgroup is to build an educational and professional curriculum for medical physicists working in diagnostic imaging and radiotherapy
      https://www.efomp.org/index.php?r=news/view&id=110
      . Eventually, the proposed curriculum could be leveraged to provide an educational platform around AI through the European School of Medical Physics Expert (ESMPE) course modules, which are annually organised to provide basic and advanced up-to-date training for medical physicists at an international level.
      In order to build a meaningful and representative curriculum, the level of knowledge and experience on AI in diagnostic and therapy among European MPEs needed to be assessed. For this reason, an international survey among MPE was set-up.
      The aim of this manuscript is to report on the results of the survey and to suggest guidelines for the future implementation of education programs on AI focused on MPE needs.

      2. Methods and materials

      A 25-questions flash survey entitled ‘Artificial intelligence in medical physics’ was run over a month and a half from 12/02/2020 to 31/03/2020, as part of the aforementioned EFOMP working group in AI. Responders were recruited through EFOMP e-mail lists and social media (e.g. Twitter, LinkedIn) and through its National Members Organisations (NMO). The survey was designed and distributed using Google Forms (Google LLC, Mountain View, CA, USA). Several types of answers were used in this survey depending on the question: yes-no, 4 or 5 points scale, multiple choice, checkboxes, as well as short and long open answers. Responses were recorded anonymously (no name or email address was required), although other personal information (e.g. age, gender) was required to further investigate answers of different subgroups.
      The results of the survey were stored in a CSV file and further analysed using Python 3.7.4 [
      • Van Rossum G.
      • Drake F.L.
      Python 3 Reference Manual.
      ]. The SciPy 1.5.0 library was used to perform further statistical analysis using Welch’s two sample t-test [

      Jones E, Oliphant T, Peterson P. SciPy: open source scientific tools for Python; 2001. http://www.scipy.org.

      ]. Welch’s two sample t-test is intended for analysis of the statistical significance between two independent groups of unequal sizes [
      • Ruxton G.D.
      The unequal variance t-test is an underused alternative to Student’s t-test and the Mann-Whitney U test.
      ,
      • Derrick B.
      • White P.
      Why Welch’s test is Type I error robust.
      ]. For ordinal data, mean and standard deviation are reported.
      In this work, the results regarding three subgroups were investigated: Male–Female, Public–Private sector and Less than or equal to-More than 10 years experience. Also, responses from French and non-French participants were analysed since preliminarily investigation of the data showed unbalance towards French respondents. Therefore, the scale answers of the participants were first converted to a numerical scale. Our null hypothesis corresponds to no difference in the results achieved among each group (e.g., gender, place of work, years of experience as MPE). If the p value 0.05, the null hypothesis (no difference between the different groups) is rejected.
      The survey was organised into three main sections. The first part of the survey (8 questions) addressed the individual knowledge, thoughts and involvement of the responders in AI algorithms to support their daily work. The second part of the survey (10 questions) investigated the needs, preferred learning format and content desired for a dedicated AI training course. This information is currently being used for the design of an ESMPE course module. The final part (7 questions) corresponded to demographic questions to investigate how responses vary between subgroups.
      As part of the study design, an initial survey was designed and tested in line with good practice by a subgroup of the AI EFOMP working group [
      • Kelley K.
      • Clark B.
      • Brown V.
      • Sitzia J.
      Good practice in the conduct and reporting of survey research.
      ]. Following on from the pilot study, the final questions for the survey were defined and the survey published. The complete survey questions are shown in Table A.1.

      3. Results

      A total of 219 responders from 31 countries (93%, n = 203 of European countries; 7%, n = 16 of non-European countries) participated in the survey. 37% of participants worked in France (n = 82), 11% (n = 25) from Spain, 7% (n = 15) from Greece, 6% (n = 14) from Ireland and 5% (n = 12) from the United Kingdom. The rest of the represented countries had less than 10 participants (<5%). Of the 219 participants, 69% (n = 150) were male, 29% (n = 64) were female and 2% (n = 5) preferred not to say. It was also noticed that from the 82 French participants, 37% were female. Responders aged between 24 and 79 years (42 ± 10 years old) at the time of the survey completion.
      A total of 76% (n = 166) of the responders work primarily in the public sector (“Public healthcare” n = 149, “University” n = 15 and “Government” n = 2), whereas the remaining 24% (n = 53) have a primary job in the private sector of healthcare (“Private Healthcare” n = 50, “Private company” n = 3). Regarding their experience, 41% (n = 90) of the responders have 10 years experience or less as medical physicists, while the 59% (n = 129) of them have more than 10 years experience.
      Fig. 1 shows the responses of the participants with respect to their knowledge and skill levels in AI, ranging from 1 (none) to 5 (expert). The mean AI knowledge for all participants is 2.3±1.0 (mean±standard deviation). Significant difference (p< 0.05) is observed between genders. The average AI knowledge for men is 2.4±1.0, while female shows an average AI knowledge of 2.0±1.0 (t-value = 2.45; p = 0.016). Similarly, the average AI knowledge between French (2.04±0.89) and non-French participants (2.5±1.1) was found to be significantly different (t-value = −3.21; p = 0.002). For the other two groups (place of work and experience), no significant difference is found between each pair. MPEs working within a public sector show an average rate of 2.3±1.0, whereas a private sector has a mean rate of 2.3±1.0 (t-value = 0.36, p> 0.05). Based on the participant experience, responders with less than or equal to 10 years experience as MPE show a mean AI knowledge rate value of 2.2±0.9, whereas those with more than 10 years experience have an average rate value of 2.4±1.1 (t-value = −0.76, p> 0.05).
      Figure thumbnail gr1
      Fig. 1Percentage of responses to the AI knowledge rate. Responses for all participants and different groups (gender, place of work and experience) are presented.
      AI knowledge of participants was obtained as part of their education (i.e. MSc, PhD) in 26% of the cases (n = 58), from reading the literature and books in 23% (n = 52), by attending face to face courses in 20% (n = 44) and via online courses in 16% (n = 36). In contrast, only 4% of participants acquired their knowledge by attending dedicated conferences or workshops (n = 10). Additionally, responders highlighted several online platforms where they have acquired basic AI knowledge: Coursera, UDEMY, FUN MOOC, edX, Stanford University, MIT XPro, among others.
      In total, 88% (n = 193) of responders agree that the MPEs need specific training on AI (10%, n = 22 were not sure and 2%, n = 4 did not agree), while 80% of responders (n = 175) consider that it is important that AI becomes a part of the MPE curriculum. On the other hand, 3% (n = 7) do not think AI should be a part of the curriculum, and 17% (n = 37) were not sure about their standpoint on this topic. From all 219 responders, 96% (n = 210) were interested in improving their knowledge of AI, while 4% (n = 9) were not interested in this matter.
      A total of 90% (n = 198) of responders expressed their desire to participate in an AI school for MPEs. While looking at the different learning methods, 40% (n = 87) of responders preferred a fully online course with pre-recorded lectures, 39% (n = 85) selected a combination of an online and a face to face sessions, whereas only 21% (n = 47) favoured traditional face to face course only. At the same time, 90% (n = 198) of responders found it useful to have an online platform with AI material to learn at their own pace.
      From all 219 participants, 81% (n = 177) agreed that AI will improve the daily work of MPEs (answers “Yes” and “Definitely Yes” in Fig. 2). There was no statistical difference in responses on this question between Male–Female (p = 0.96), Public–Private (p = 0.33) and French-Non-French (p = 0.79) subgroups of the participants. However, a significant difference was observed in the responses depending on their years experience (p = 0.05).
      Figure thumbnail gr2
      Fig. 2Responses to the question “Can AI improve daily work of MPE?”. Responses for all participants and different groups (gender, place of work and experience) are presented.
      The areas where AI will help MPEs most (more than 100 votes) were identified as: (1) data analysis, (2) image quality, (3) optimisation, (4) quality assurance, (5) predictive modelling, (6) diagnosis, (7) radiomics and (8) dosimetry.
      Responders were encouraged to select the AI topics they found most important to learn. The top ten items include: basics of AI (78% of total responders, n = 170), QC applications (75%, n = 164), integration of AI in clinical workflow (65%, n = 143), techniques to evaluate AI tools (63%, n = 138), medical image analysis (62%, n = 135), basic programming (60%, n = 131), advanced programming (44%, n = 96), data curation and pre-processing (41%, n = 89), commissioning (37%, n = 83), and ethics and legal aspects (37%, n = 80).
      Fig. 3 illustrates the level of involvement of the MPE in AI projects. In general, 64% of participants are not involved in AI projects, 28% are involved to some extent and 9% are not interested in participating in AI projects. A significant difference was observed in the involvement of male and female MPEs in AI projects. While 19% of male participants were involved in a project leading role, only 3% of female participants had equivalent positions (p = 0.009). Further investigation showed that 19% of non-French participants lead AI projects. However, only 5% of French participants are in charge of AI projects (p = 0.001). Also, a 13% of French participants showed no interest in AI, in contrast to the 6% of non-French participants who had the same opinion. Remaining comparisons did not reach a significant level (p>0.05).
      Figure thumbnail gr3
      Fig. 3Responses to the question “Are you currently involved in AI projects?”. Responses for all participants and different groups (gender, place of work and experience) are presented.
      When looking at applications of AI that are being developed by the responders institutions, these were (1) diagnosis definition, (2) optimisation of clinical/ therapeutic treatment, (3) data preparation and data mining, and (4) PACS optimisation, among others.
      During the survey, a text box was provided to all participants to express their thoughts about AI. Several participants raised the need for documents and/or scientific articles to address AI challenges specifically in medical physics and to provide guidance to explain AI to other colleagues (e.g. technologists, dosimetrists, etc.). Also, short and flexible courses specific to MPEs at different levels should be available to allow proper balance between both personal and professional life.

      4. Discussion

      This paper discusses the results of the international survey performed by the EFOMP working group on AI, intended to assess the general AI knowledge level among MPEs, as a help-tool for revise current medical physics curricula and for designing teaching courses specific for MPEs. After quantitative analyses of the data received within the runtime of the survey (219 replies), several interesting observations were made.
      As shown in Fig. 1 and Fig. 2, AI is currently a growing sector, where having ”deep knowledge” is no longer easy, where few people can concretely answer specific questions or have a clearer knowledge of the requirements and didactic needs. From Fig. 3 it is evident the growing interest in being involved in AI projects or being responsible for the AI project, which further support the analyses obtained from this ”flash” survey.
      These needs can be solved by organising the promotion of the AI discipline for the medical physicist through a coordinated curriculum across Europe, which will enable coherent and homogeneous knowledge and skills across the Europe. The definition of a common curriculum for the medical physicist is necessary to cover the needs in the different disciplines, in which today the medical physicist is called to be responsible for the activities.
      First, there was no significant difference (p> 0.05) in the average level of AI-related knowledge (Fig. 1) between the early career (10 year of work experience) and experienced MPEs (> 10 years). Such a result is partially unexpected, as we would expect younger physicists to have been trained in emerging topic as AI. At the same time, it may just indicate that experienced MPEs are actively involved in continual education courses, not necessarily medical physics specific, plausibly to keep up with the rapidly changing healthcare field. When looking at the fraction of MPEs that had no work experienced with AI (Fig. 1), this value was lower for the early career participants (20%, compared to 26% for experienced MPEs), what can be explained by higher percentage of early career MPEs that followed AI-related courses as a part of their university education or training.
      A key message extracted from the survey is that although 81% of participants think AI will benefit their daily work as MPE, the majority of the responders (64%) are not involved in AI projects but they would like to. In fact, the vast majority of the participants (96%; n = 210) shows predisposition to improve their current knowledge of AI. This directly implies that there is a strong need (88% of responses) for AI-specific training in the medical physics field to address current clinical challenges. Such knowledge should include basic AI understanding, methods to properly integrate and evaluate AI tools or programming skills for the development of specific tasks (e.g. QC). These findings are in line with the nature of our working group and 90% (n = 198) of the participants showed interest in participating in the AI school developed within the ESMPE scheme. This is also in line with the literature, where several papers have been published in the last years in an attempt to cover such demand on AI aspects in medical physics [

      G. Mahadevaiah, P. RV, I. Bermejo, D. Jaffray, A. Dekker, L. Wee, Artificial intelligence-based clinical decision support in modern medical physics: Selection, acceptance, commissioning, and quality assurance, Med Phys 47(5) (2020) e228–e235.https://doi.org/10.1002/mp.13562.

      ,
      • Kortesniemi M.
      • Tsapaki V.
      • Trianni A.
      • Russo P.
      • Maas A.
      • Källman H.-E.
      • Brambilla M.
      • Damilakis J.
      The European Federation of Organisations for Medical Physics (EFOMP) White Paper: Big data and deep learning in medical imaging and in relation to medical physics profession.
      ,
      • Cui S.
      • Tseng H.-H.
      • Pakela J.
      • Ten Haken R.K.
      • El Naqa I.
      Introduction to machine and deep learning for medical physicists.
      ].
      Another interesting trend that was observed in the results is related to gender distribution and role that MPEs play in AI-related research projects (Fig. 3). Based on our results, approximately 28% of all MPEs are involved in AI-related research, from which 14% play a major or project-leading role (19% of male, 3% of female). Such a misbalance between male and female MPEs that play project leading may be related to several biases present in the study. In particular, this may be related to lower average knowledge rate (Fig. 1) of female survey participants (2.0±1.0), when compared to their male counterparts (2.4±1.0), which will ultimately introduce further imbalances into gender comparisons. Similar trend was observed by Pinto dos Santos et al. [
      • Pinto dos Santos D.
      • Giese D.
      • Brodehl S.
      • Chon S.H.
      • Staab W.
      • Kleinert R.
      • Maintz D.
      • Baeßler B.
      Medical students’ attitude towards artificial intelligence: a multicentre survey.
      ], as they observed that male medical students had significantly more expertise in the use of technology than female. At the same time, these results can also point at underlying gender inequalities related to promotions of female MPE to project-leading roles, thus requiring further attention in follow-up studies.
      The results observed for the MPEs working in the public and private sector did not show significant differences. It has to be noted that in many cases, the MPEs indicated they work in both the public and private sector. In such cases, public dedication was considered their primary workplace.
      In the literature, it is observed that other healthcare professionals (radiologists, surgeons, medical students, etc.) have been interviewed with respect to the role of AI in their current or future professions [
      • Pinto dos Santos D.
      • Giese D.
      • Brodehl S.
      • Chon S.H.
      • Staab W.
      • Kleinert R.
      • Maintz D.
      • Baeßler B.
      Medical students’ attitude towards artificial intelligence: a multicentre survey.
      ,
      • Gong B.
      • Nugent J.P.
      • Guest W.
      • Parker W.
      • Chang P.J.
      • Khosa F.
      • Nicolaou S.
      Influence of artificial intelligence on canadian medical students’ preference for radiology specialty: a national survey study.
      ,
      • Laï M.
      • Brian M.
      • Mamzer M.
      Perceptions of artificial intelligence in healthcare: findings from a qualitative survey study among actors in France.
      ,
      • van Hoek J.
      • Huber A.
      • Leichtle A.
      • Härmä K.
      • Hilt D.
      • von Tengg-Kobligk H.
      • Heverhagen J.
      • Poellinger A.
      A survey on the future of radiology among radiologists, medical students and surgeons: Students and surgeons tend to be more skeptical about artificial intelligence and radiologists may fear that other disciplines take over.
      ]. In general, healthcare professionals see AI as a technological disrupting in medicine which will improve patient care. The majority have a positive feeling towards the use of AI, although this might differ between specialities [
      • Laï M.
      • Brian M.
      • Mamzer M.
      Perceptions of artificial intelligence in healthcare: findings from a qualitative survey study among actors in France.
      ,
      • van Hoek J.
      • Huber A.
      • Leichtle A.
      • Härmä K.
      • Hilt D.
      • von Tengg-Kobligk H.
      • Heverhagen J.
      • Poellinger A.
      A survey on the future of radiology among radiologists, medical students and surgeons: Students and surgeons tend to be more skeptical about artificial intelligence and radiologists may fear that other disciplines take over.
      ]. Similarly, our study shows that the majority of the MPEs (81% of participants) are favourable to the use of AI in the daily clinic.
      Regarding professional training, several studies have demonstrated that healthcare professions (physicians, radiologists and medical students) need specific AI education programs to better understand AI tools and their use in healthcare, especially during the first years of specialisation [
      • Pinto dos Santos D.
      • Giese D.
      • Brodehl S.
      • Chon S.H.
      • Staab W.
      • Kleinert R.
      • Maintz D.
      • Baeßler B.
      Medical students’ attitude towards artificial intelligence: a multicentre survey.
      ,
      • Gong B.
      • Nugent J.P.
      • Guest W.
      • Parker W.
      • Chang P.J.
      • Khosa F.
      • Nicolaou S.
      Influence of artificial intelligence on canadian medical students’ preference for radiology specialty: a national survey study.
      ,
      • Laï M.
      • Brian M.
      • Mamzer M.
      Perceptions of artificial intelligence in healthcare: findings from a qualitative survey study among actors in France.
      ,
      • van Hoek J.
      • Huber A.
      • Leichtle A.
      • Härmä K.
      • Hilt D.
      • von Tengg-Kobligk H.
      • Heverhagen J.
      • Poellinger A.
      A survey on the future of radiology among radiologists, medical students and surgeons: Students and surgeons tend to be more skeptical about artificial intelligence and radiologists may fear that other disciplines take over.
      ,
      • Kobayashi Y.
      • Ishibashi M.
      • Kobayashi H.
      How will ”democratization of artificial intelligence” change the future of radiologists?.
      ]. In the same line, our results suggested that MPEs also want to improve their understanding and implementation of AI technology (i.e. continuous learning).
      It is important to mention several limitations of this work that may have affected results of the survey. Although this work is, to the best of our knowledge, the largest MPE-dedicated international survey of AI in healthcare (219 reactions from 31 countries), country-wise distribution of participant was highly unbalanced. Despite using comparable methods for dissemination of the survey within the EU, several major EU countries had little to no participants (e.g. Italy: 3%, n  = 6; the Netherlands: 3%, n  = 6; Sweden: 0%, n  = 0; Germany: 0%, n  = 0). This obviously could have introduced additional biases into the results. Therefore, conclusions extracted from this work do not directly represent the general standpoint of MPEs within all EU countries, but only as a minority of them.
      Thirty-seven (37%) of all participants work in France (n = 82), obviously introducing a certain degree of unbalance to the study. Such an uneven distribution between reactions to the survey is expected to be related to the way the survey was promoted between NMOs. Further comparison between responses from the French (n  = 82; 37% female) and non-French (n  = 137; 25% female) cohorts revealed additional statistically significant trends. First, French responses followed similar trends as those previously discussed within gender-specific comparisons of the responses (e.g. lower general AI knowledge and fewer project leaders within the participants). A plausible explanation for such a difference may be a higher percentage of female participants within the French cohort, compared to non-French participants (37% vs 25%). Additionally, although France have been described in the literature to stay on the forefront of AI introduction within various areas of the healthcare sector [
      • Laï M.
      • Brian M.
      • Mamzer M.
      Perceptions of artificial intelligence in healthcare: findings from a qualitative survey study among actors in France.
      ], we observed a significantly larger number of participants showing no interest in participating in AI projects (13% for French participants vs 6% for non-French participants).
      Also, it was noticed that a small percentage of the 219 participants (9%) had no interested in participating in AI projects (Fig. 3). Thus the conclusions extracted from this work can be biased towards medical physics with a special interest in AI.
      Furthermore, another potential limitation is the uneven distribution between some of the groups. For example, the number of women participated in the survey (n = 64) was low (less than half) compared to the male participants (n = 150). However, this has been addressed, at least from the statistical point of view, by using Welch’s two sample t-test.
      In conclusions, similarly to other healthcare professionals [
      • Pinto dos Santos D.
      • Giese D.
      • Brodehl S.
      • Chon S.H.
      • Staab W.
      • Kleinert R.
      • Maintz D.
      • Baeßler B.
      Medical students’ attitude towards artificial intelligence: a multicentre survey.
      ,
      • Laï M.
      • Brian M.
      • Mamzer M.
      Perceptions of artificial intelligence in healthcare: findings from a qualitative survey study among actors in France.
      ,
      • van Hoek J.
      • Huber A.
      • Leichtle A.
      • Härmä K.
      • Hilt D.
      • von Tengg-Kobligk H.
      • Heverhagen J.
      • Poellinger A.
      A survey on the future of radiology among radiologists, medical students and surgeons: Students and surgeons tend to be more skeptical about artificial intelligence and radiologists may fear that other disciplines take over.
      ], AI is well perceived as a powerful support tool for MPE daily tasks by the majority of the participants in the survey. MPEs have made clear their desire to be more involved in AI projects and improve their current training. Such AI-specific training should be integrated in the educational and professional curriculum and AI courses specific for MPEs should be organised.

      Acknowledgments

      This work has been developed as part of the EFOMP Working Group on Artificial Intelligence.

      Appendix A. Questions

      Table A.1List of questions used in the survey.
      1. How would you rate your knowledge of AI?
      2. If you have some knowledge of AI, how did you get it?
      3. If followed an online course on AI, please provide the name of the platform.
      4. Can AI improve the daily work of MPEs?
      5.Check the areas you think AI will replace or assist medical physicists.
      6. Are you currently involved in AI projects?
      7. In case you participate in AI projects, what is your degree of participation?
      8. Is your hospital/institution already using AI or going to do so in the next 12 months? If so, for what application?
      9. Do medical physicists need specific training on AI?
      10. Should AI be part of the MPE curriculum?
      11. Are you interested in improving your knowledge of AI?
      12. Which topics would you like to learn? (multiple choice)
      13. Would you be interested in participating into an AI school if organised as part of the European School of Medical Physics (ESMP)?
      14. Which teaching format do you prefer for an AI course?
      15. If the school is eventually organised, which format would you prefer?
      16. In case you could attend, would you take an exam at the end of the school?
      17. Would you be interested in using an online platform with AI material to learn on your own pace?
      18. We leave you some space to add any comment/thought about AI and the working group
      19. What is your gender?
      20. What is your age?
      21. In which country do you currently work?
      22. What is your highest level of education?
      23. Where do you work?
      24. How many years of experience as medical physicist do you have?
      25. What is your main speciality?

      References

      1. Hamet P, Tremblay J. Artificial intelligence in medicine. Metabolism 69 (2017) S36–S40, insights Into the Future of Medicine: Technologies, Concepts, and Integration.https://doi.org/10.1016/j.metabol.2017.01.011.

        • Hosny A.
        • Parmar C.
        • Quackenbush J.
        • Schwartz L.H.
        • Aerts H.J.W.L.
        Artificial intelligence in radiology.
        Nat Rev Cancer. 2018; 18: 500-510https://doi.org/10.1038/s41568-018-0016-5
        • Nelson A.
        • Herron D.
        • Rees G.
        • Nachev P.
        Predicting scheduled hospital attendance with artificial intelligence.
        npj Digital Med. 2019; 2: 26https://doi.org/10.1038/s41746-019-0103-3
        • Bi W.L.
        • Hosny A.
        • Schabath M.B.
        • Giger M.L.
        • Birkbak N.J.
        • Mehrtash A.
        • Allison T.
        • Arnaout O.
        • Abbosh C.
        • Dunn I.F.
        • Mak R.H.
        • Tamimi R.M.
        • Tempany C.M.
        • Swanton C.
        • Hoffmann U.
        • Schwartz L.H.
        • Gillies R.J.
        • Huang R.Y.
        • Aerts H.J.W.L.
        Artificial intelligence in cancer imaging: clinical challenges and applications.
        CA Cancer J Clin. 2019; 69: 127-157https://doi.org/10.3322/caac.21552
        • Agarwal R.
        • Diaz O.
        • Yap M.H.
        • Lladó X.
        • Martí R.
        Deep learning for mass detection in full field digital mammograms.
        Comput Biol Med. 2020; 121 (103774)https://doi.org/10.1016/j.compbiomed.2020.103774
        • Thrall J.H.
        • Li X.
        • Li Q.
        • Cruz C.
        • Do S.
        • Dreyer K.
        • Brink J.
        Artificial intelligence and machine learning in radiology: opportunities, challenges, pitfalls, and criteria for success.
        J Am College Radiol. 2018; 15: 504-508https://doi.org/10.1016/j.jacr.2017.12.026
        • Xing L.
        • Krupinski E.A.
        • Cai J.
        Artificial intelligence will soon change the landscape of medical physics research and practice.
        Med Phys. 2018; 45: 1791-1793https://doi.org/10.1002/mp.12831
      2. G. Mahadevaiah, P. RV, I. Bermejo, D. Jaffray, A. Dekker, L. Wee, Artificial intelligence-based clinical decision support in modern medical physics: Selection, acceptance, commissioning, and quality assurance, Med Phys 47(5) (2020) e228–e235.https://doi.org/10.1002/mp.13562.

        • Kortesniemi M.
        • Tsapaki V.
        • Trianni A.
        • Russo P.
        • Maas A.
        • Källman H.-E.
        • Brambilla M.
        • Damilakis J.
        The European Federation of Organisations for Medical Physics (EFOMP) White Paper: Big data and deep learning in medical imaging and in relation to medical physics profession.
        Phys Med: Eur J Med Phys. 2018; 56: 90-93https://doi.org/10.1016/j.ejmp.2018.11.005
        • Cui S.
        • Tseng H.-H.
        • Pakela J.
        • Ten Haken R.K.
        • El Naqa I.
        Introduction to machine and deep learning for medical physicists.
        Med Phys. 2020; 47: e127-e147https://doi.org/10.1002/mp.14140
      3. Putten Wvd, Caruana CJ, Evans S, Guibelalde E, Christofides S. Radiation Protection No 174. European Guidelines on Medical Physics Expert, European Commission; 2014. doi:10.2833/18393.

        • Geleijns J.
        • Breatnach É.
        • Cantera A.C.
        • Damilakis J.
        • Dendy P.
        • Evans A.
        • Faulkner K.
        • Padovani R.
        • Van Der Putten W.
        • Schad L.
        • Wirestam R.
        • Eudaldo T.
        Core curriculum for medical physicists in radiology.
        Recommendations from an EFOMP/ESR working group, Insights Imaging. 2012; 3: 197-200https://doi.org/10.1007/s13244-012-0170-3
        • Eriksen J.G.
        • Beavis A.W.
        • Coffey M.A.
        • Leer J.W.H.
        • Magrini S.M.
        • Benstead K.
        • Boelling T.
        • Hjälm-Eriksson M.
        • Kantor G.
        • Maciejewski B.
        • Mezeckis M.
        • Oliveira A.
        • Thirion P.
        • Vitek P.
        • Olsen D.R.
        • Eudaldo T.
        • Enghardt W.
        • François P.
        • Garibaldi C.
        • Heijmen B.
        • Josipovic M.
        • Major T.
        • Nikoletopoulos S.
        • Rijnders A.
        • Waligorski M.
        • Wasilewska-Radwanska M.
        • Mullaney L.
        • Boejen A.
        • Vaandering A.
        • Vandevelde G.
        • Verfaillie C.
        • Pötter R.
        The updated ESTRO core curricula 2011 for clinicians, medical physicists and RTTs in radiotherapy/radiation oncology.
        Radiother Oncol. 2012; 103: 103-108https://doi.org/10.1016/j.radonc.2012.02.007
        • Del Guerra A.
        • Bardies M.
        • Belcari N.
        • Caruana C.J.
        • Christofides S.
        • Erba P.
        • Gori C.
        • Lassmann M.
        • Lonsdale M.N.
        • Sattler B.
        • Waddington W.
        Curriculum for education and training of Medical Physicists in Nuclear Medicine: Recommendations from the EANM Physics Committee, the EANM Dosimetry Committee and EFOMP.
        Physica Med. 2013; 29: 139-162https://doi.org/10.1016/j.ejmp.2012.06.004
        • Van Rossum G.
        • Drake F.L.
        Python 3 Reference Manual.
        CreateSpace, Scotts Valley, CA2009
      4. Jones E, Oliphant T, Peterson P. SciPy: open source scientific tools for Python; 2001. http://www.scipy.org.

        • Ruxton G.D.
        The unequal variance t-test is an underused alternative to Student’s t-test and the Mann-Whitney U test.
        Behav Ecol. 2006; 17: 688-690https://doi.org/10.1093/beheco/ark016
        • Derrick B.
        • White P.
        Why Welch’s test is Type I error robust.
        Quant Methods Psychol. 2016; 12: 30-38https://doi.org/10.20982/tqmp.12.1.p030
        • Kelley K.
        • Clark B.
        • Brown V.
        • Sitzia J.
        Good practice in the conduct and reporting of survey research.
        Int J Quality Health Care. 2003; 15: 261-266https://doi.org/10.1093/intqhc/mzg031
        • Pinto dos Santos D.
        • Giese D.
        • Brodehl S.
        • Chon S.H.
        • Staab W.
        • Kleinert R.
        • Maintz D.
        • Baeßler B.
        Medical students’ attitude towards artificial intelligence: a multicentre survey.
        Eur Radiol. 2019; 29: 1640-1646https://doi.org/10.1007/s00330-018-5601-1
        • Gong B.
        • Nugent J.P.
        • Guest W.
        • Parker W.
        • Chang P.J.
        • Khosa F.
        • Nicolaou S.
        Influence of artificial intelligence on canadian medical students’ preference for radiology specialty: a national survey study.
        Acad Radiol. 2019; 26: 566-577
        • Laï M.
        • Brian M.
        • Mamzer M.
        Perceptions of artificial intelligence in healthcare: findings from a qualitative survey study among actors in France.
        J Transl Med. 2020; 18: 14https://doi.org/10.1186/s12967-019-02204-y
        • van Hoek J.
        • Huber A.
        • Leichtle A.
        • Härmä K.
        • Hilt D.
        • von Tengg-Kobligk H.
        • Heverhagen J.
        • Poellinger A.
        A survey on the future of radiology among radiologists, medical students and surgeons: Students and surgeons tend to be more skeptical about artificial intelligence and radiologists may fear that other disciplines take over.
        Eur J Radiol. 2020; 121 (108742)https://doi.org/10.1016/j.ejrad.2019.108742
        • Kobayashi Y.
        • Ishibashi M.
        • Kobayashi H.
        How will ”democratization of artificial intelligence” change the future of radiologists?.
        Japanese J Radiol. 2019; 37: 9-14