Advertisement

Expanding the medical physicist curricular and professional programme to include Artificial Intelligence

Published:March 31, 2021DOI:https://doi.org/10.1016/j.ejmp.2021.01.069

      Highlights

      • We developed a Medical Physicists Curricular Program regarding AI.
      • It can be considered as a supplement to all sub-speciality curricula of EFOMP.
      • It is recommended to be utilised by national training and regulatory bodies.
      • It can also be implemented via the European School of Medical Physics Expert modules.

      Abstract

      Purpose

      To provide a guideline curriculum related to Artificial Intelligence (AI), for the education and training of European Medical Physicists (MPs).

      Materials and methods

      The proposed curriculum consists of two levels: Basic (introducing MPs to the pillars of knowledge, development and applications of AI, in the context of medical imaging and radiation therapy) and Advanced. Both are common to the subspecialties (diagnostic and interventional radiology, nuclear medicine, and radiation oncology). The learning outcomes of the training are presented as knowledge, skills and competences (KSC approach).

      Results

      For the Basic section, KSCs were stratified in four subsections: (1) Medical imaging analysis and AI Basics; (2) Implementation of AI applications in clinical practice; (3) Big data and enterprise imaging, and (4) Quality, Regulatory and Ethical Issues of AI processes. For the Advanced section instead, a common block was proposed to be further elaborated by each subspecialty core curriculum. The learning outcomes were also translated into a syllabus of a more traditional format, including practical applications.

      Conclusions

      This AI curriculum is the first attempt to create a guideline expanding the current educational framework for Medical Physicists in Europe. It should be considered as a document to top the sub-specialties’ curriculums and adapted by national training and regulatory bodies. The proposed educational program can be implemented via the European School of Medical Physics Expert (ESMPE) course modules and – to some extent – also by the national competent EFOMP organizations, to reach widely the medical physicist community in Europe.

      Keywords

      1. Introduction

      Over the years, the amount of information gathered in the healthcare enterprise has dramatically increased. The data explosion reached clinical and biomedical disciplines, including imaging, laboratory, genetic, physiological, pharmaceutical, and structured clinical data, as well as unstructured text-based records, and the related health information generated with it. Clinical data are increasingly appended by digital health-related data from other sources too, like home-based or wearable healthcare electronic devices for vital signs or physical activity data retrieval [
      • Kortesniemi M.
      • Tshapaki V.
      • Trianni A.
      • Russo P.
      • Maas A.
      • Källman H.E.
      • et al.
      The European Federation of Organisations for Medical Physics (EFOMP) white paper: big data and deep learning in medical imaging and in relation to medical physics profession.
      ,
      • Shilo S.
      • Rossman H.
      • Segal E.
      Axes of a revolution: challenges and promises of big data in healthcare.
      ].
      This data volume increases demands for large storage space in PACS, databases and hardware for their processing, analysis, archiving, and management. Due to the growing amounts of 3D imaging exams performed in modern radiology departments, around 50% of radiologists’ working time is spent on reviewing CT exams alone, and the rates of radiologists’ burnout are concerningly escalating [
      • Bhatia N.
      • Trivedi H.
      • Safdar N.
      • Heilbrun M.E.
      Artificial Intelligence in Quality Improvement: Reviewing uses of artificial intelligence in noninterpretative processes from clinical decision support to education and feedback.
      ]. This data-intensive outburst and the crushing imaging reporting workload cannot be handled by just means of human power. Hence, there is an apparent urgency for robust and reliable software that can support clinicians, medical physicists and healthcare professionals in their tasks and Artificial Intelligence (AI) has been identified as a potential partner that can play a crucial role in this journey [

      European Society of Radiology (ESR). Impact of artificial intelligence on radiology: a EuroAIM survey among members of the European Society of Radiology. Insights Imaging 2019;10:105. https://doi.org/10.1186/s13244-019-0798-3.

      ].
      In parallel to the aforementioned changes to medical imaging, the evolution of computer hardware, even for desktop machines, has increased the number of users and developers of machine learning (ML) applications. High performance computing facilities based on computer clusters and cloud infrastructures are becoming more common. Computer science applications have flourished taking advantage of these technological developments to provide the tools that can transform current healthcare processes to be more precise and efficient. Using this computational power, AI systems can mine the huge amount of digital data generated in healthcare today to build systems, that can take decisions, make recommendations, make diagnosis or even predict patient’s outcome. Computing power will be a crucial factor in the evolution of AI-based applications together with the development of new architectures to optimize the current computational limitations towards quantum computing [
      • Darwish A.
      • Hassaniem A.E.
      • Das S.
      A survey of swarm and evolutionary computing approaches for deep learning.
      ].
      Current medicine aims at personalizing patient protocols. It also strives to be objective and quantitative to reach consistent and comparative results, expanding to parameters capable of describing treatment efficacy and care outcome. Therapeutic approaches to the individual patient demand the usage of various types of data, ranging from simple biochemical tests to more complex ones like histology or PET/MR data, for example.
      Working at the boundary between imaging science, informatics, and medicine, Medical Physicists (MPs) can play an important role in the implementation of AI in healthcare. Furthermore, for several specific tasks of the MP profession, such as quality control and planning in radiotherapy treatments, an integration process is underway with AI tools. As such, it is of paramount importance that MPs will broaden their active involvement in the healthcare community to ensure that their interdisciplinary knowledge brings maximal impact in this technological shift.
      The European Federation of Organisations for Medical Physics (EFOMP) shared its position on this topic, by publishing a white paper on Big Data and Deep Learning in medical imaging, in relation to the medical physics profession. Medical physicists must be prepared to face this transformational technology by updating their training and education programs [
      • Kortesniemi M.
      • Tshapaki V.
      • Trianni A.
      • Russo P.
      • Maas A.
      • Källman H.E.
      • et al.
      The European Federation of Organisations for Medical Physics (EFOMP) white paper: big data and deep learning in medical imaging and in relation to medical physics profession.
      ,
      • Xing L.
      • Krupinski E.A.
      • Cai J.
      Artificial intelligence will soon change the landscape of medical physics research and practice.
      ,
      • Peeken J.C.
      • Bernhofer M.
      • Wiestler B.
      • Goldberg T.
      • Cremers D.
      • Rost B.
      • et al.
      Radiomics in radiooncology – challenging the medical physicist.
      ]. However, there is currently a lack of courses and workshops that are really tailored to the MPs needs. Within the aims of EFOMP, this kind of education could be imparted within the European School of Medical Physics Expert (ESMPE) course modules, which are organized regularly to provide basic and advanced training for MPs at an international level.
      To be up to the task in this scenario, EFOMP established in April 2019 a new working group (WG) on AI in Medical Physicists [

      EFOMP working groups website;2020 https://www.efomp.org/index.php?r=pages&id=working-groups [accessed in October 2020].

      ]. The WG is formed by several MPs and experts from around EU from both diagnostic and therapeutic disciplines. The group has the objective of structuring an AI Curricular and Professional Program for MPs as well as to support the preparation of an ESMPE AI module.
      The aim of this paper is to report on the WG’s formulation of the European Curricular and Professional Program for MPs on AI.

      2. Material and methods

      2.1 The current role of medical physicists and future perspectives

      To understand the rationale and challenges for building a European Curricular and Professional Program on AI for MPs, we started from the role that the MPs play today and what would like to claim for the years to come.
      The overall goal of the MPs is to contribute to patient safety and secure the quality of care, as established in the 2013/59/Euratom directive [

      European Guidelines on Medical Physics Expert. European Commission Radiation Protection n° 174. Guibelande E, Christofides S, Caruana CJ, Evans S, van der Putten W. Luxembourg: Publications office of the European Union 2014. ISBN: 978-92-79-35786-2.

      ]. Along with the introduction of more complex imaging modalities, computational methods and software applications in the medical field, there has also been a need to increase the skills of the MP to keep up with the technological expansion [
      • Kortesniemi M.
      • Tshapaki V.
      • Trianni A.
      • Russo P.
      • Maas A.
      • Källman H.E.
      • et al.
      The European Federation of Organisations for Medical Physics (EFOMP) white paper: big data and deep learning in medical imaging and in relation to medical physics profession.
      ,

      Samei E, Grist TM. Why physics in medicine? Physica Medica (2019) 64:319-22 doi: S1120-1797(19)30104-8 [pii].

      ,
      • Samei E.
      • Pawlicki T.
      • Bourland D.
      • Chin E.
      • Das S.
      • Fox M.
      • et al.
      Redefining and reinvigorating the role of physics in clinical medicine: a report from the AAPM Medical Physics 3.0 Ad Hoc Committee.
      ,

      Sensakovic WF, Mahesh M. Role of the Medical Physicist in the Health Care Artificial Intelligence Revolution. J Am Coll Radiol (2019) 16:393-4 doi: S1546-1440(18)31164-5.

      ,

      El Naqa I, Das S. The role of machine and deep learning in modern medical physics. Med Phys 2020;47(5):e125–3126.

      ]. MPs’ expertise and tasks have focused on overall concepts of quality (including quality management (QM), quality assurance (QA), and quality control (QC) tests), safety (radiation dosimetry and safety, and electromagnetic safety issues), optimisation, as well as on research, development and evaluation of new analytical techniques.
      To be successful in the provision of medical physics expertise on these general task areas, it is imperative that MPs working in the hospital environment are capable of combining thorough knowledge, practical skills and academic competence in a multi-professional medical community. This is also clearly stated in the EU RP174- which in turn is based on the Euratom Basic Safety Standards (BSS) Directive [

      Council of the European Union. (2013). Council Directive 2013/59/Euratom laying down basic safety standards for protection against the dangers arising from exposure to ionising radiation, and repealing Directives 89/618/Euratom, 90/641/Euratom, 96/29/Euratom, 97/43/Euratom and 2003/122/Euratom. Official Journal L-13 of 17.01.2014.

      ]- presenting European Guidelines on Medical Physics Expert and strengthening the importance of education and training as the foundations of the MP profession [

      European Guidelines on Medical Physics Expert. European Commission Radiation Protection n° 174. Guibelande E, Christofides S, Caruana CJ, Evans S, van der Putten W. Luxembourg: Publications office of the European Union 2014. ISBN: 978-92-79-35786-2.

      ].
      While the focus of these two documents describing the role of MPs is on radiation protection and dosimetry, the EFOMP statement policy 16 [
      • Caruana C.J.
      • Tsapaki V.
      • Damilakis J.
      • Brambilla M.
      • Martin G.M.
      • Dimov A.
      • et al.
      EFOMP policy statement 16: the role and competences of medical physicists and medical physics experts under 2013/59/EURATOM.
      ], elaborated on the BSS by expanding the role of the MP behind radioprotection and dosimetry, states that the MP needs “to ensure that medical radiological equipment (including radiation sources and software devices) in hospital is judiciously chosen according to the desired clinical outcomes, properly calibrated, quality controlled and very importantly used effectively, safely and efficiently for the good of the patient”.
      Indeed, in some countries, with the transposition of the BSS at national level states, the follow up of software tools that have an impact on the information content of an exam is already a legal task of the MPs [

      Federale Overheidsdienst Binnenlandse Zaken en Federaal Agentschap voor Nucleaire Controle. Koninklijk besluit betreffende de bescherming tegen ioniserende stralingen tijdens diergeneeskundige blootstellingen 2020/2000245. BELGISCH STAATSBLAD — 20.02.2020.

      ].
      It is in this spirit that the WG developed a curricular program for MPs, considering that the tasks of the “MPs 3.0” go beyond quality control, radiation protection and dosimetry. Moreover, they expand to clinical implementation, functional verification and validation, risk assessment, performance monitoring and optimisation, cost-effectiveness and up to a certain point, the research and development of AI-software as a Medical Device [

      El Naqa I, Das S. The role of machine and deep learning in modern medical physics. Med Phys 2020;47(5):e125–3126.

      ,

      Huq MA, Fraass BA, Dunscombe PB, Gibbons JP, Ibbott GS, Mundt AJ, et al. Summary of the report of Task Group 100 of the AAPM: Application of risk analysis methods to radiation therapy quality management. 2016;43(7):4209-62. https://doi.org/10.1118/1.4947547.

      ].
      To build a meaningful and representative curriculum, the level of knowledge and experience on AI in diagnostic and therapy applications among European MPs needed to be assessed. For this reason, as a preliminary step to develop the training content, an international survey among MPs was set. The survey was divided into three groups of questions: on the individual knowledge, thoughts and involvement of the responders with AI applications; on the needs, preferred learning format and content for the AI training course (which could be used to the design of an ESMPE module); and on demographic questions to investigate how responses vary between subgroups. The results of the survey recently reported [
      • Diaz O.
      • Guidi G.
      • Ivashchenko O.
      • Colgan N.
      • Zanca F.
      Artificial intelligence in the medical physics community: an international survey.
      ], have been used for building the curriculum.

      2.2 Training program framework

      The method we used to develop the AI curriculum for MPs roots on previously published guidelines and EFOMP curricula [
      • Del Guerra A.
      • Bardies M.
      • Belcari N.
      • Caruana C.J.
      • Christofides S.
      • Erba P.
      • et al.
      Curriculum for education and training of medical physicists in nuclear medicine. Recomendations from the EANM Physics Committee, the EANM Dosimetry Committee and EFOMP.
      ,

      Geleijns J, Breatnach E, Calzado Cantera A, Damilakis J, Dendy P, Evans A, et al. Core curriculum for medical physicists in radiology. Recommendations from EFOMP/ESR working group. Insights Imaging 2012;3:197-200.

      ,
      • Eriksen J.G.
      • Beavis A.W.
      • Coffey M.A.
      • Leer J.W.
      • Magrini S.M.
      • Benstead K.
      • et al.
      The updated ESTRO core curricula 2011 for clinicians, medical physicists and RTTs in radiotherapy/radiation oncology.
      ] but expands on the new challenges and topics that have been brought to the profession by the AI arrival. In this sense, the proposed curriculum is to be considered as an addendum to the previously published EFOMP core curricula for MPs in Nuclear Medicine (NM) (in 2013 [
      • Del Guerra A.
      • Bardies M.
      • Belcari N.
      • Caruana C.J.
      • Christofides S.
      • Erba P.
      • et al.
      Curriculum for education and training of medical physicists in nuclear medicine. Recomendations from the EANM Physics Committee, the EANM Dosimetry Committee and EFOMP.
      ]), Radiology (in 2011 [

      Geleijns J, Breatnach E, Calzado Cantera A, Damilakis J, Dendy P, Evans A, et al. Core curriculum for medical physicists in radiology. Recommendations from EFOMP/ESR working group. Insights Imaging 2012;3:197-200.

      ]), and Radiotherapy [RT] (in 2011 [
      • Eriksen J.G.
      • Beavis A.W.
      • Coffey M.A.
      • Leer J.W.
      • Magrini S.M.
      • Benstead K.
      • et al.
      The updated ESTRO core curricula 2011 for clinicians, medical physicists and RTTs in radiotherapy/radiation oncology.
      ]).
      We also aimed at developing a curriculum which is in line with the Bologna Declaration [

      The Bologna declaration of 19 June 1999: Joint declaration of the European Ministers of Education. https://www.eurashe.eu/library/modernising-phe/Bologna_1999_Bologna-Declaration.pdf [accessed in November 2020].

      ], and the recommendations from the European Parliament and the council of 23 April 2008, on the establishment of the European qualifications framework for lifelong learning [

      The European Qualifications Framework for lifelong learning (EQF). European Commission, 2008. doi:10.2755/14352.

      ].
      As the depth of the MPs involvement with AI technologies will strongly depend on the speed with which they will be deployed in their countries and institutions, we have subdivided the curriculum in two levels, Basic and Advanced. The aim of this Basic training level is to introduce MPs to the pillars of knowledge, development and applications of AI, in the context of medical imaging and radiotherapy. The Advanced level instead aims at building on the knowledge acquired in the Basic and comprises common topics to the three current main subspecialties: (1) Diagnostic and Interventional Radiology, (2) Nuclear Medicine, and (3) Radiation Oncology. The lack of recent core curriculum documents per subspecialty was a limitation during the preparation of the Advanced level training [
      • Del Guerra A.
      • Bardies M.
      • Belcari N.
      • Caruana C.J.
      • Christofides S.
      • Erba P.
      • et al.
      Curriculum for education and training of medical physicists in nuclear medicine. Recomendations from the EANM Physics Committee, the EANM Dosimetry Committee and EFOMP.
      ,

      Geleijns J, Breatnach E, Calzado Cantera A, Damilakis J, Dendy P, Evans A, et al. Core curriculum for medical physicists in radiology. Recommendations from EFOMP/ESR working group. Insights Imaging 2012;3:197-200.

      ,
      • Eriksen J.G.
      • Beavis A.W.
      • Coffey M.A.
      • Leer J.W.
      • Magrini S.M.
      • Benstead K.
      • et al.
      The updated ESTRO core curricula 2011 for clinicians, medical physicists and RTTs in radiotherapy/radiation oncology.
      ,

      The Bologna declaration of 19 June 1999: Joint declaration of the European Ministers of Education. https://www.eurashe.eu/library/modernising-phe/Bologna_1999_Bologna-Declaration.pdf [accessed in November 2020].

      ]. The Working Group approach was to create a common Advanced curriculum that could be further defined, in the future EFOMP core curricula for the three main subspecialties. Note that a fourth subspeciality for Advanced, “Others”, is proposed to comprise modalities and subjects not covered by the core curricula but for which the role of MPs is starting to increase, such as biophysics, physiological signal measurements and cardiology, to cover fields which will have AI applications in the future. However, this fourth category will not be expanded in this curriculum to cohere with the core curricula of EFOMP.
      Note that the advanced knowledge and competences required to develop in-house AI-software are considered out of scope for the present curriculum. These would correspond to a third “High level” and it is advised for MPs interested in this type of in-depth training, to look for the specific courses and training programs available outside of EFOMP.
      As a common methodology for both levels, the learning outcomes (LOs) of the training program were established, identifying the current and potential tasks that MPs can be involved with respect to AI in their work settings. For these definitions we followed the approach presented in the most recently published EFOMP curriculum for nuclear medicine [
      • Del Guerra A.
      • Bardies M.
      • Belcari N.
      • Caruana C.J.
      • Christofides S.
      • Erba P.
      • et al.
      Curriculum for education and training of medical physicists in nuclear medicine. Recomendations from the EANM Physics Committee, the EANM Dosimetry Committee and EFOMP.
      ]. The LOs were also translated into a syllabus of a more traditional format, including topics and potential practicals and resources (see Appendix).
      The learning outcomes are statements that describe three aspects: (1) Knowledge, (2) Skills and (3) Competence (KSC), that the students should acquire when successfully completing training at each one of the parts.
      In brief, these three aspects, in relation to the LOs were defined as:
      • Knowledge: Facts, principles theories and practices
      • Skills: Ability to use knowledge and know-how to complete tasks and solve problems (cognitive skills (logical, intuitive and creative thinking) and practical skills (use of methods, materials, tools and instruments))
      • Competence: Responsibility, management and autonomy.

      3. Results

      The general overview of the curriculum and its two levels are described in detail in the following sections. In Fig. 1 (Basic) and 2 (Advanced) an overview of the levels that were used to define the curriculum framework are reported.
      Figure thumbnail gr1
      Fig. 1Overview of the Medical Physicist curriculum framework for Artificial Intelligence: Basic level.
      Note that in Fig. 2 (Advanced) we also list a possible stratification of sub-specialties and associated modalities, as we consider they should be addressed when revising the current core curricula.
      Figure thumbnail gr2
      Fig. 2Overview of the Medical Physicist curriculum framework for Artificial Intelligence: Advanced level (subspecialties).

      3.1 Basic level

      For the Basic level, LOs expressed as KSCs were stratified into four subsections that correspond to (1) Medical imaging analysis and AI basics (part II in Appendix), (2) Implementation of AI applications in clinical practice (part III in Appendix), (3) Big data and enterprise imaging (part IV in Appendix) and (4) Quality, Regulatory and Ethical Issues of AI processes (part V in Appendix). Note that the list of practical applications (which would serve as input for a possible ESMPE school module) is reported together with the syllabus, in the same Appendix.
      The list of the LOs for the Basic part of the curriculum is given in Table 1, Table 2, Table 3, Table 4, which correspond to the four subsections as defined above.
      Table 1LOs for Basic Level: Medical imaging analysis and AI introduction.
      Knowledge

      List and comprehensive learning of:
      • basic AI terms and definition (including neural network parameters and machine learning (ML) problem types), differences between AI, machine learning, deep learning, and traditional Computed Aided Diagnosis (CAD)
      • image theory and image quantification principles, and image transformations (segmentation, registration, reconstruction)
      • basic statistics for model validation (sensitivity, specificity, accuracy etc., F1 score, Receiver Operating Characteristic (ROC) methods, confidence intervals (CI), etc).
      • popular tools, open data sets, available test sets, available languages and frameworks
      • high level process for AI applications development, validation and clinical implementation (use case definition, data access and curation, training/testing strategies, IT infrastructure requirements)
      Skills
      • use properly and accurately AI related terminology, distinguish between ML, DL and CAD applications and identify if one application is AI-based or not
      • perform tasks related to image processing by managing tools for image segmentation, Region of Interest/Volume of Interest (ROI/VOI) extraction, labelling, archiving, etc.. Assess the variability associated to different segmentation and VOI extractions methods (with manual, analytic algorithms and AI processes)
      • formulate, carry out and interpret simple multilayer neural network performance (hyperparameter optimization, train/testing strategies, generalization aspects, presentation of results, interpretability and explainability)
      • manage simple classification tasks, including strategies of train/test set divisions for validation purposes
      • use techniques for data retrieval, anonymization and labelling and be able to create datasets (including images as required) and check their suitability, as input for multilayer neural networks, applying data curation criteria
      Competences
      • identify the activities in own institution where MP could play a role in relation to AI
      • take responsibility for reinforcing an accurate use of AI related terminology among peers
      • be responsible of supporting in image processing tasks, such as image segmentation, labelling, in datasets to be used for AI validation and testing, etc.
      • contribute to educate AI basics to MPs and other MDs professionals.
      Table 2LOs for Basic Level: Implementation of AI applications.
      Knowledge

      List and comprehensive learning of:
      • open datasets and previous AI challenges, synthetic dataset characteristics and requirements, existing and available (pre-)trained networks and available software and computational platforms for data annotation and curation
      • basic IT infrastructure requirements for AI applications (GPU, High Performance Computing (HPC), external scientific computing services, cloud computing, hybrid computing solutions)
      • medical imaging data readiness levels (MIDaR) and annotation, labelling and segmentation methodologies for clinical applications
      • the principles of AI software as medical device management.
      Know how to:
      • evaluate clinical needs and preparation of technical specifications for tender purposes
      • deal with AI commercial solutions during procurement, commissioning, integration in clinical workflow and QA
      • process and critically analyze manufacturer specifications from AI applications, white papers, and other documentation
      • perform data retrieval, anonymization and labelling in clinical practice in a simple case study
      • develop prospective and/or retrospective validation studies for AI step by step also using case examples, in a multidisciplinary team
      Skills
      • study and extract relevant technical information related to AI software applications in scientific papers, white papers, reports and manufacturers documentation
      • prepare written documents and reports to share in a multidisciplinary team during procurement (including tender procedures), commissioning and/or integration on clinical workflow
      • support and advise on data curation for local validation and QA of AI commercial applications
      • validate and test (and potentially create) synthetic datasets for AI-validation and in silico trials
      • participate and provide support during procurement, commissioning and/or integration in clinical workflow of AI applications, including the implementation of related QA programs
      • perform data retrieval, anonymization and labelling in clinical practice in a simple case study
      • participate as a MP member in a multidisciplinary team in development of prospective and/or retrospective validation studies blueprint (data retrieval, anonymization and labelling in clinical practice) in a simple study for AI step by step using case examples
      • evaluate critically MIDaR for AI validation databases in a simple clinical study.
      Competences
      • master the definition of technical specifications and requirements for the procurement and commissioning of commercial AI software applied to medical data
      • communicate:
        • relevant technical information provided by the AI manufacturer and write reports or other useful materials to be used during commissioning and acceptance.
        • results to peers (in the form of notebooks, notes, resumes, reports, poster, article, oral presentation) at institutional, national and international meetings and for research journals.
      • oversee, where appropriate as a member of a multidisciplinary team:
        • infrastructures for display and reading of images and for the reporting and archiving of findings for AI clinical validation.
        • the acceptance, commissioning into clinical use, and in-house validation of commercial AI software applied to medical data, including data access, selection and curation, handling multi-dimensional datasets
        • performance assessment and tracking (QA) of AI software
        • the potential role of phantom studies in the implementation, validation, QA program and/or follow up performance of AI software
      • advise on potential pitfalls of in-house (including combining databases from different machines or vendors) and multi-centre validation of AI software
      • propose medical physics sessions as part of education and training of AI applications to medical data and take responsibility on the training of other MPs and other professionals in relation to AI medical applications
      • read and analyse critically international literature and participate/organise networks for research, benchmark and development of AI applications to medical data within own scientific community.
      Table 3LOs for Basic level: big data and enterprise imaging (EI).
      Knowledge

      List and comprehensive learning of:
      • basic principles, existing standards and platforms, data mining techniques and radiomics and imaging biobanks for enterprise imaging
      • describe the main requirements for transparent reporting on multivariable prediction models (TRIPOD statement and checklist

        Collins GS, Reitsma JB, Altman DG, Moons KGM. TRIPOD statement.Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 2014;350. g7594.

        )
      • key performance indicators definition for big data information analysis
      Know how:
      • to benchmark, validate and curate data coming from different sources in the big data enterprise solution
      • to define and personalize dashboards in the enterprise solution interface, in function of the user (MPs, radiologists, oncologists, administrative staff, etc.)
      • to process and critically analyse manufacturer specifications white papers, and other relevant documentation for EI solutions
      Skills
      • critically read vendor and other related documentation and identify potential pitfalls and solutions during implementation phase and the integration in the clinical workflow in the EI context
      • critically revise data mining techniques performance and output
      • define the requirements, construct and validate (consistency check of) datasets as input for radiomic studies
      • check the robustness of radiomic features and the potential role of harmonization methods based on AI
      • be familiar with standardization initiatives for biomarkers and radiomics [25]
      Competences
      • communicate relevant technical information provided by the AI manufacturer and write reports or other useful materials to be used during commissioning and acceptance.
      • in a multidisciplinary team, oversee:
        • the acceptance and commissioning into clinical use of EI software applied to medical data
        • in-house validation studies, including the data access, selection and curation, handling multi-dimensional datasets working in a multidisciplinary team
      • communicate clearly results to peers at local and international meetings and for research journals on radiomics studies or AI study (in the form of notebooks, notes, resumes, reports, poster, article, oral presentation).
      • advise on possibilities and potential pitfalls in in-house (including combining databases from different machines or vendors) and multi-centre validation of radiomics features
      • take responsibility for the training of other MPs and other professionals in relation to radiomics-based AI medical applications and propose medical physics sessions as part of education and training of radiomics applications to medical data
      • critically analyse international literature within a given area of research about radiomics applied to medical data and participate/organise networks for research and development of these applications within own scientific community.
      Table 4LOs for Basic Level: Quality, Regulatory and Ethical Issues of AI processes.
      Knowledge
      • list legal, ethical aspects and challenges at national and international level about the use of AI applications
      • explain the process leading to CE marking:
        • terms used for European (CE) and US (FDA) AI software approval process and regulation
        • the current regulation for AI software as a medical device for commercial and in-house use software
        • the meaning of ‘acceptability criteria’ as applied to AI software as medical devices
      • describe the main aspects of national and international law (including General Data Protection Regulation, GDPR) about privacy related to the use of medical data in AI processes
      Skills
      • evaluate if a specific AI software is GDPR compliant
      • evaluate and participate in the preparation of documents for ethical committees’ approval
      • organize data flow and analyze data safety, security and privacy regulations in AI applications and infrastructure (including clinical trials, in a multidisciplinary team)
      • design and test physical and technical methods for quality control of data input to AI systems in standardise QA programs, including clinical trials (in a multidisciplinary team)
      • autonomously acquire, critically assess and extract relevant information from AI manufacturers user/technical specifications, literature and other documentation related to their clinical evidence generation and compare with results in literature. Communicate such information to peers in the form of reports or other documentation.
      • promote and implement the basic safety standards, guideline defined in the national and international legislation applicable to AI via technical and clinical audit to observe, manage and mitigate AI use correlated potential risks.
      Competences
      • take responsibility for AI device management including evaluation of intended use, compliance to national and international regulations, risk analysis, compliance with GDPR, and HIPAA (Health Insurance Portability and Accountability Act) when dealing with countries outside EU, in a multi-disciplinary team
      • take responsibility for assessing post-market surveillance of a CE marked device integrated in clinical workflow
      • take initiative, when needed and relevant, to further testing FDA/CE market tools in real/different clinical context other than the one used for regulatory process
      • use the methodologies of Health Technology Assessment (HTA) to carry out a HTA of an AI device in conjunction with other healthcare professionals.
      • organize and manage quality control teams in own areas of Medical Physics AI applications

      3.2 Advanced level

      The learning outcomes for this common Advanced curriculum, shared by all subspecialties are presented in Table 5.
      Table 5LOs for Advanced Level (shared by all subspecialties).
      Knowledge

      Identify, describe, define:
      • advanced methods for QA of AI tools (including image quality assessment using model observers and other metrics related to the clinical task) in patient and phantom (physical and virtual) images
      • different uses of AI for image reconstruction, registration, noise and artifact reduction, filtering and transformation
      • AI methods for object/anatomical region/organs recognition, segmentation, and localization
      • the use of AI tools for optimization (dose monitoring, organ dose, organ recognition, patient positioning, contrast dose, automatic positioning of simulated radiation beams or radiopharmaceuticals on virtual phantoms)
      • the use of AI tools for management (workflow, hanging protocols, appropriateness and utilization, acquisition time)
      • the use of AI tools for big data mining (enterprise imaging platforms and governance)
      • AI applications for image harmonization for multi-centre studies, extraction of image biomarkers and building predictive/prognostic models
      Skills
      • perform calibration or model adjustment of the equipment, software, and QA programs according to the clinical and research use of AI tools
      • develop and implement advanced image quality metrics framework in patient and phantom QA studies
      • use software suitable for AI basic processes and relative libraries
      • evaluation of a simple multilayer neural network performance including generalization aspects, presentation, interpretation and explanation of results
      • take corrective measures of QA and model including interruption of AI system if hazard for patients and/or workers could appear.
      Competences:
      • critically evaluate and improve the performed QA and QC programs, in cooperation with other healthcare professionals when required
      • coordinate and/or participate in a research project on the use of AI for image restoration/reconstruction, automated diagnosis, disease or tissue characterization or outcome prediction
      • project optimization study for effective use of AI reconstruction methods
      • implement a pipeline to extract image biomarkers and building predictive/prognostic models using AI
      • to automate the patient dose registry or archive managements and to prevent potential overexposures
      • plan processes regarding AI adverse events, such as mismatch and misdiagnosis or mistreatment in use of AI model for trials in radiology, radiotherapy, molecular imaging
      • support for the correct management of processes for obtaining patients’ informed consent for diagnostic and therapeutic procedures, where AI is used for trials or clinical practice (i.e. outcome prediction)
      • support and advise on regulatory roadmap for in-house or commercial use of AI applications (intended use, risk categorization, verification and validation, clinical evidence, post-market surveillance)
      • support vendors in their regulatory process for an AI medical device in terms of data curation and clinical evidence generation, in the context of a partnership project
      Note that the skill set of any MP subspecialty needs to incorporate the QA aspects related to AI to help maintain acceptable quality and safety of diagnostics or care. We cannot emphasize enough how challenging it is to include the AI scope to the existing KSC set, taking into account that AI methods can be applied in basically every phase of the diagnostic or therapy process in healthcare. Therefore, current QA methods need to be improved, and in many cases new QA metrics or methods are required to encompass the performance and safety effects of AI [
      • Orlhac F.
      • Frouin F.
      • Nioche C.
      • Ayache N.
      • Buvat I.
      Validation of a method to compensate multicenter effects affecting CT radiomics.
      ].
      Translating this to radiology would mean for example, proposing an in-silico QA methodology for assessing and benchmarking AI application in mammography, by means of generating hybrid (real and synthetic) images and through identification of quality metrics that impact the output of AI software tools for lesion detection and characterisation [
      • Badano A.
      • Graff C.G.
      • Badal A.
      • Sharma K.
      • Zeng R.
      • Samuelson F.W.
      • et al.
      Evaluation of digital breast tomosynthesis as replacement of full-field digital mammography using an in silico imaging trial.
      ].
      In a NM context this could be further elaborated by critically judging the relevance of the performed QA and QC programs integrated with molecular imaging and quantitative analysis software in diagnosis, prognosis and therapy, and integrating in them the necessary specific tests the performance tests of AI-support tools in terms of accuracy and safety [

      Park JE, Kim D, Kim HS, Park SY, Kim JY, Cho SJ, et al. Quality of science and reporting of radiomics in oncologic studies: room for improvement according to radiomics quality score and TRIPOD statement. European Radiology 2020;30:523-536. doi: https://doi.org/10.1007/s00330-019-06360-z.

      ,
      • Arabi H.
      • Zaidi H.
      Applications of artificial intelligence and deep learning in molecular imaging and radiotherapy.
      ].
      In RT, knowledge-based treatment planning is an example of the application where the impact of AI developments is obvious and where an updated QA process is needed to decrease outliers in treatment plans [

      Vandewinckele L, Claessens M, Dinkla A, Brouwer C, Crijns W, Verellen D, et al. Overview of artificial intelligence-based applications in radiotherapy: Recommendations for implementation and quality assurance. Radiotherapy and Oncology 2020 (in press). doi: https://doi.org/10.1016/j.radonc.2020.09.008.

      ,
      • Wang M.
      • Zhang Q.
      • Lam S.
      • Cai J.
      • Yang R.
      A review on application of deep learning algorithms in external beam radiotherapy automated treatment planning.
      ].

      4. Discussion

      This curriculum is intended as a guideline to incorporate Artificial Intelligence to the training of MPs in Europe and associated countries under the EFOMP umbrella. During the initial phase of this WG activities, a pilot online international survey with AI-specific questions was prepared and distributed among the MPs community [
      • Diaz O.
      • Guidi G.
      • Ivashchenko O.
      • Colgan N.
      • Zanca F.
      Artificial intelligence in the medical physics community: an international survey.
      ]. In total, 219 participants from 31 countries provided their answers and the majority of them (88%) agreed with the statements “Medical Physicists need specific training on AI” and (80%) with the statement “I strongly believe that AI should be part of the medical physicists curriculum”. In the present curriculum, we included and privileged the topics that obtained a higher score in terms perceived importance and impact of AI in the profession (e.g. basic AI, QC applications, integration of AI in clinical workflow).
      The AI curriculum presented in this publication reflects our present understanding of the current stage of the rapidly evolving AI field and how it is reflected in the MPs profession in terms of KSCs.
      The first level of training (Basic) is intended to give a broad overview of the fields where the role of MP will become relevant and needed for the deployment of AI in healthcare. The justification of this level is to provide common foundations for all MPs enrolling and to cement the general KSCs required to deal with AI technologies in their routine present and future practice, also avoiding biases and potential differences between all EFOMP country members.
      The second level of training (Advanced), focused on the MP sub-specialities [

      European Guidelines on Medical Physics Expert. European Commission Radiation Protection n° 174. Guibelande E, Christofides S, Caruana CJ, Evans S, van der Putten W. Luxembourg: Publications office of the European Union 2014. ISBN: 978-92-79-35786-2.

      ], shares common content across the subspecialties, indicating that similar blocks of knowledge are needed in different fields of medical physics. The lack of update of the EFOMP core curricula for subspecialties [
      • Del Guerra A.
      • Bardies M.
      • Belcari N.
      • Caruana C.J.
      • Christofides S.
      • Erba P.
      • et al.
      Curriculum for education and training of medical physicists in nuclear medicine. Recomendations from the EANM Physics Committee, the EANM Dosimetry Committee and EFOMP.
      ,

      Geleijns J, Breatnach E, Calzado Cantera A, Damilakis J, Dendy P, Evans A, et al. Core curriculum for medical physicists in radiology. Recommendations from EFOMP/ESR working group. Insights Imaging 2012;3:197-200.

      ,
      • Eriksen J.G.
      • Beavis A.W.
      • Coffey M.A.
      • Leer J.W.
      • Magrini S.M.
      • Benstead K.
      • et al.
      The updated ESTRO core curricula 2011 for clinicians, medical physicists and RTTs in radiotherapy/radiation oncology.
      ] represented a challenge in the creation of this document. The contents of this section need to eventually lean on and align with those documents and thus, would potentially need to be reviewed in the future when updated core curricula will be released [

      EFOMP working groups website;2020 https://www.efomp.org/index.php?r=pages&id=working-groups [accessed in October 2020].

      ].
      A potential third level of training (High level), related to the development of in-house AI-based tools by MPs, has been described briefly and in more general terms at the beginning of this document. This WG considered this out of the scope as the number of MPs involved in the development of AI tools themselves is still low, and mostly limited to university hospitals and research activities.
      As we are aware, the field of AI in general and its implementations in medicine are still taking shape because even the basic AI architectures along with data sharing capabilities (also in legal and ethical aspects [

      European Commission. Ethics guidelines for trustworthy AI. Report dated on 8 April 2019, published on https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai. accessed in: 25 Nov. 2020.

      ]) are still in relatively early development. Therefore, our proposed curriculum has been formed as a consensus to reflect the present need for training and education content in this area but many parts in the curriculum should be reviewed regularly to keep it updated. Specifically, we acknowledge that the technology of AI is rapidly evolving and that was the justification to emphasize the main pillars on the Basic training level and to limit the curriculum and the KSCs to more common items in the Advanced level (Appendix).
      Nevertheless, the fast development of the AI field implies that many advanced aspects of AI training for MPs should be resilient and adaptable. It is obvious that the field of AI is driving a major transformation in the present time and also in the future, which makes it challenging to envision how fast some of our tasks as MPs will have to change and adapt. This does not indicate that we are in a position to await how AI unfolds, but rather that MPs should start to own their future roles and involvement in AI deployment in healthcare, and prepare via training for the future ahead.
      The traditional role of MPs includes important aspects of quantitative methods, which serve as a source of reference in the development and implementation of AI in the clinical field. We could call this the metrology of information and Big Data. The automated objective evaluation of the quality of the images that are generated, the traceability of imaging and treatment protocols among equipment, departments and hospitals will need the active engagement of MPs to help safeguard the integrity and robustness of the generated patient data, working side by side with other healthcare colleagues. This is due to the fundamental need of the continuously learning AI systems to have reliable and curated ground truth information regularly brought in to keep the DL networks updated.
      In the future, it is more likely that the current medical physics subspecialities (radiology, nuclear medicine, radiotherapy and potentially new ones) will be increasingly merged together in the context of comprehensive diagnostic-care chains and outcome driven processes in healthcare. In that continuum, the AI methods fit well as comprehensive tools to drive effective medical decision-making but also the intermediate steps along that path must be reliably managed and assured. Otherwise, the outcome is not reliable for safe, cost-effective and high-quality care. Those intermediate steps can also be regarded as checkpoints where MP knowledge and role are pivotal to have reliable, quantitative and objectively determined reference values for the QA and continuous learning labels required by the AI systems. Medical Pysicist training and knowledge is a combination of physics, informatics and medicine, including long experience on clinical environment and practices. This combined information is also needed to build bridges across the clinical and IT professions, and for the understanding of the underlying issues of technology which may induce caveats in patient safety and quality of care if they are not properly accounted for. MP role in the future medicine is therefore even more pronounced, with emerging AI applications, as the gatekeeper of safety and quality and a crucial linchpin in healthcare between radiologists (and other medical professionals using AI-based tools), computer scientists, AI developers and IT professionals.

      5. Conclusions

      This curriculum is the first attempt to create a guideline to extend the education of MPs in Europe in relation to the emergence of AI in healthcare. It is the result of the discussions within an international WG and builds on previous regulations. The WG has made an initial proposal related to the potential responsibilities of MPs with regard to AI applications with an effect on their role. Yet, there is a need for a wider discussion for our profession in this regard, encouraged and led by our MP professional societies.

      Acknowledgements

      The authors would like to acknowledge O. Ivashchenko, C. Zervides, K. Cobaerts, P. van Ooijen, T. Mäkelä, P. Deak, H. de las Heras, W. Niessen, M. Leick, H. Azadbakht and Costantinos Koutsojannis, for their assistance on selected topics.

      Appendix.

      EFOMP AI Working group curriculum for medical physicists
      Note that practical examples were incorporated in the basic level curriculum only, in order to provide concrete cases to demonstrate the main methods and tools in AI which are essential to facilitate deeper understanding of the AI principles, and therefore, to build knowledge.
      On the other hand, practical examples were not explicitly presented in the advanced level curriculum because the advanced level is entirely already related to practical applications, with their development and implementation in the main sub-specialities and within their various modalities. Therefore, advanced level training should already be applied (mostly in practice) learning in all of its specific curriculum topics.
      BASIC LEVEL
      PART I: INTRODUCTION
      • 1.
        The strategic role, competence and education of medical physicist in the era of AI
        • a.
          The philosophy of AI: Hype vs reality, era of changes, opportunities for the future
        • b.
          MP role for the integration of (commercial) AI-based tools in the clinical workflow and in the research context
        • c.
          Proper use of AI terminology
      PART II: TECHNOLOGY: THE PILLARS
      • 2.
        Medical Imaging analysis
        • a.
          Image theory
        • b.
          Image segmentation, registration and visualization
        • c.
          Image quantification (inc. Radiomics’ based approaches)
        • d.
          Image management for RT
        • e.
          Image reconstruction
        • f.
          Predictive models
      • Practical: Image processing tools relevant for AI: Freeware tools applied to automatic 2D/3D segmentation, ROI/VOI extraction and features.
      • 3.
        Basics of machine learning (ML) and deep learning (DL) in imaging
        • a.
          Machine learning in general:
          • i.
            Justification: Ideology, terminology (e.g. underfitting/overfitting) and history of ML evolution
          • ii.
            Formulating a problem as a ML problem
            • 1.
              Types of problems: function approximation, dimensionality reduction, anomaly detection, decision making / prediction, classification, regression
            • 2.
              Rationale behind using ML approach
            • 3.
              The importance of data curation
            • 4.
              Methods for data augmentation
          • iii.
            ML classes: clustering algorithms, support vector machines (SVM), decision trees / random forest learnings, reinforcement learning, and others.
          • iv.
            Why to use Deep Learning?
            • 1.
              Differences between ML, DL and traditional CAD (computer-aided diagnosis)
            • 2.
              Representative power of neural networks. Examples in medical imaging
        • b.
          Deep learning as a subset of ML
          • i.
            An overview of basic NN structure and various architectures with focus on convolutional neural networks (CNN).
          • ii.
            Training a network and the standard terminology: loss function, regularization, underfitting/overfitting, stochastic gradient descent (SGD), batch, epoch, backpropagation, inference/prediction/forward pass, hyperparameter optimization, activation maps, data augmentation, adversarial training
        • c.
          Validating your choice of network and the final results
          • i.
            Terminology and common approaches (validation, train and test set division, class imbalance, cross-validation, conformance to TRIPOD statement)
          • ii.
            Basic statistics: sensitivity, specificity, accuracy (etc.), F1 score, Receiver Operating Characteristic (ROC), Area Under the ROC curve, confidence intervals (CI), statistical significance
      • Practical
        • Hands-on: Jupyter notebook applied to simple classification problems, train-validation-test set division
        • Application of NN and DL models to a simple image segmentation task with a help of transfer learning: leveraging pre-trained models and adapting to new contexts, evaluating the results
        • Demonstration of hyperparameter optimization, the effect of different loss functions etc. on overfitting/underfitting and dying neurons. Visualization of activation, weights using TensorBoard.
      PART III: Implementation of AI applications in clinical practice
      • 4.
        Introduction on how to develop AI applications
        • a.
          Use case definition
        • b.
          Overview state of the art: open datasets, available (pre-)trained networks, previous challenges, available software and computational platforms for data annotation, curation, and AI applications development
        • c.
          Accessing, curating and preparing imaging data: approaches and challenges:
          • i.
            Medical imaging data readiness levels (MIDaR)
          • ii.
            Pre-processing and harmonization methods
          • iii.
            Annotation methodologies
          • iv.
            Labeling and segmentation methodologies
          • v.
            Reinforced learning and data
          • vi.
            Multi-dimensional/-model data
          • vii.
            Synthetic datasets and in-silico trials foundations
        • d.
          What’s already out there? Open datasets, available trained networks (e.g. from previous challenges)
        • e.
          IT infrastructure requirements (on-site/in the cloud): GPU, workstations, external scientific computing services, cloud computing, hybrid computing solutions
        • Practical:
          • Demonstration of data retrieval, anonymization and labeling in clinical practice
          • Based on case studies: clinical study design and validation, ground truth, metric definition, training, validation, test of the dataset, observing validation loss/accuracy during training, overfitting etc., learning rate schedule, overall evaluation of the network performance including generalization aspects, presentation of results. Explainability.
          • Getting started: Develop an AI tool based on a CNN configuration programming in Python and application to a simple simulated data set. Practical exercise on data curation and annotation using freeware software
          • Case examples how to set up IT infrastructures in general
          • Multiprofessional aspects: example of decision-making process of how to determine what data parameters to include in the data sets
          • Hands-on on a public set of data
      • 5.
        Role of the MP: how to deal with AI-based commercial solutions
        • a.
          Procucrement of commercial AI software/applications
        • b.
          Specifications: what information to expect from the AI vendor, white papers and other documentations/resources
        • c.
          Implementation, resourcing, organizational validation, acceptance and commissioning into clinical use, education and training of radiologists and beyond
        • d.
          Integration in clinical workflow: case examples of clinical validation with prospective and/or retrospective validation projects, preferably extending to multi-center examples
        • Practical
          • Case studies: examples of vendor documentation with analysis of relevant/not relevant information
          • Evaluation of a new AI project implementation with possible problems and solutions
          • Experiments on the ACR lab -comprehensive overview of existing AI tools per specialty and related public results/trials/challenges
      PART IV: Big Data and Enterprise Imaging
      • 6.
        What is enterprise imaging (EI)?
        • a.
          Basic principles
        • b.
          Existing standards and platforms
        • c.
          How to define key performance indicators for leveraging big data information
        • d.
          EI in the context of AI
        • Practical:
          • Example case of vendor documentation with analysis of relevant/not relevant information, evaluation of a new AI project implementation with possible problems and solutions - possible integration with PACS/RIS.
      • 7.
        Imaging Biomarkers and Radiomics
        • a.
          Basics
        • b.
          Overview of tools on the market for imaging biomarkers
        • c.
          Standardization initiatives (e.g. IBSI)
        • d.
          Imaging biobanks
        • e.
          Radiomics workflow: data selection, acquisition protocol requirements, VOI segmentation methods, features extraction and selection, internal and external validation
        • f.
          Challenges for multi-centers radiomics studies: features robustness analysis and AI support methods to harmonize data
        • Practical:
          • Coronary artery calcium scoring (due to high relevance in healthcare, as the cardiovascular diseases are most prominent killers around the world population).
      PART V: Quality, Regulatory and Ethical Issues
      • 8.
        Policy and Privacy of data
        • a.
          International Law and data exchange
        • b.
          iCloud (case example)
        • c.
          General data protection regulation (GDPR), and HIPPA when dealing with data from countries outside EU, i.e. USA.
        • d.
          AI process, decision making and ISO 9004 certification standard - Disaster Recovery solution
        • Practical:
          • Hands-on test to assess if a set of data is GDPR compliant
          • Use case: Lessons learned from hospital ethics committee report and legal department
      • 9.
        Quality Assurance and Regulatory Affairs (QARA):
        • a.
          European software approval process and regulation (leading to CE mark)
        • b.
          Differences between European and US (FDA) software approval roads and regulation.
        • c.
          Current regulation for developing AI software as a medical device (MDR), both for commercial and in-house use
        • d.
          Post market surveillance
        • e.
          Further testing of such FDA/CE marked tools in a real clinical context
        • f.
          Example of Health Technology Assessment (HTA) and risk analysis for a commercial AI solution
        • g.
          Legal aspects and challenges in institutional, national and international level
        • Practical:
          • Workflow for software approval preparations (physicist perspective).
          • How to support hospital leadership and legal department for in-house developed software
      • 10.
        Ethical challenges in AI - in general and in healthcare
        • a.
          Use case: Lessons learned from hospital ethics committee report and legal department
        • b.
          Correct information of patients if AI software or predictive methods are used in clinical process
        • c.
          Maintenance and archive of the original data, clinical data, synthetic data, and model used
        • d.
          Use of chatbot and make decision systems based on AI for diagnostic and therapy, or for follow-up (virtual medical examination) and schedule of exams/therapy/surgery. (correct prioritization of the emergency or of the rescheduling)
        • Practical:
          • Diagnostics of incurable neurological disease - the patient perspective: to know or not to know / or tougher: AI prognostics related to population demongraphics: link to healthcare prioritisation.
      ADVANCED LEVEL
      Note: All common advanced items refer specifically to AI applications and not to the conventional processes and corresponding methods. These common advanced AI items are utilised by all medical physics subspecialities (and modalities as applicable).
      Part I. Common to all sub-specialties
      • 1.
        Advanced QA on AI medical devices performance
        • a.
          Implementation of QA protocol for AI testing in clinical practice; identify pitfalls and improvement opportunity
        • b.
          Virtual clinical trials and hybrid data set generation for QA of AI tools
        • c.
          QA on patient data
      • 2.
        AI for Image generation and restoration, including reconstruction, denoising and artifact reduction
        • a.
          Benchmarking with conventional reconstruction methods
          • i.
            Phantom and patient studies as a physical ground-truth reference
          • ii.
            Technical and clinical (observer performance experiment) image quality measurements for clinical validation
          • iii.
            Pitfalls and instabilities in AI-based reconstruction methods
          • iv.
            Commercial and in-house developed methods
      • 3.
        Image processing, including localization, registration, segmentation, quantitative analysis, and classification
        • a.
          Popular tools, approaches and strategies
        • b.
          Ground truth generation and QA (human generated segmentations and/or labels)
        • c.
          Relevant metrics to validate AI performance: DICE, Hausdorff distance, etc.
        • d.
          Accuracy and robustness validation, testing and QA
      • 4.
        AI protocol optimisation and radioprotection
        • a.
          Procedure and treatment optimization
          • i.
            Patient positioning and anatomical verification
          • ii.
            Target and organs at risk (OAR) auto-contouring: validation and verification
          • iii.
            Treatment/protocol guidance and automatic adaptation, dose tracking
          • iv.
            Contrast or radiopharmaceutical use (inc. simulated procedures on virtual phantoms)
        • b.
          Patient dose registry applications archive management (AI automatization) and AI applications for dosimetry and risk management
      • 5.
        AI for workflow optimization and automation
        • a.
          Management
          • i.
            Workflow
          • ii.
            Viewing protocols
          • iii.
            Procedure/protocol appropriateness (justification)
          • iv.
            Procedure duration
          • v.
            Patient informed consent procedure for trials/clinical practice (i.e. outcome prediction)
        • b.
          Big data mining (enterprise imaging platforms and governance)
      • 6.
        Biomarkers and radiomics framework
        • a.
          Extraction methods for biomarkers
        • b.
          QA framework for biomarkers and radiomics (phantom and patient studies); harmonization for multicenter studies
        • c.
          Validation and robustness of texture features
        • d.
          Protocol and image quality effects of texture analysis: traceability and pitfalls
        • e.
          Using biomarkers in predictive/prognostic models
      • 7.
        Regulatory roadmap of AI applications
        • a.
          Steps (checklist) towards clinical/use clearance
          • i.
            In-house AI methods
          • ii.
            Research partnerships with industry (pre-FDA/CE mark process)
        • b.
          Through worked-out examples: Checklist and related requirements
          • i.
            Intended use definition
          • ii.
            Risk analysis and classification
          • iii.
            Validation and testing framework:
            • Definition of imaging/others protocols (input generation QA)
            • Image database requirements (data curation, clinical evidence generation and ground truth definition
            • Statistical study power analysis
            • TRIPOD statement compliance
            • Data curation: patient/intended use case cohort description,
            • Robustness of performance and accuracy testing (patient/phantom studies as required)
          • iv.
            Testing generalization (robustness against databases from multiple protocols/vendors/models to generate input data)
          • v.
            Post-market surveillance
            • Risk assessment and incident and accident prevention and reporting
            • FMEA/FMEAC (Failure mode and effect (and critically) analysis) in AI applications in radiology

      References

        • Kortesniemi M.
        • Tshapaki V.
        • Trianni A.
        • Russo P.
        • Maas A.
        • Källman H.E.
        • et al.
        The European Federation of Organisations for Medical Physics (EFOMP) white paper: big data and deep learning in medical imaging and in relation to medical physics profession.
        Editorial Phys Med. 2018; 56: 90-93https://doi.org/10.1016/j.ejmp.2018.11.005
        • Shilo S.
        • Rossman H.
        • Segal E.
        Axes of a revolution: challenges and promises of big data in healthcare.
        Nat Med. 2020; 26: 29-38https://doi.org/10.1038/s41591-019-0727-5
        • Bhatia N.
        • Trivedi H.
        • Safdar N.
        • Heilbrun M.E.
        Artificial Intelligence in Quality Improvement: Reviewing uses of artificial intelligence in noninterpretative processes from clinical decision support to education and feedback.
        J Am Coll Radiol. 2020; 17: 1382-1387https://doi.org/10.1016/j.jacr.2020.08.002
      1. European Society of Radiology (ESR). Impact of artificial intelligence on radiology: a EuroAIM survey among members of the European Society of Radiology. Insights Imaging 2019;10:105. https://doi.org/10.1186/s13244-019-0798-3.

        • Darwish A.
        • Hassaniem A.E.
        • Das S.
        A survey of swarm and evolutionary computing approaches for deep learning.
        Artif Intell Rev. 2020; 53: 1767-1812https://doi.org/10.1007/s10462-019-09719-2
        • Xing L.
        • Krupinski E.A.
        • Cai J.
        Artificial intelligence will soon change the landscape of medical physics research and practice.
        Med Phys. 2018; 45: 1791-1793https://doi.org/10.1002/mp.12831
        • Peeken J.C.
        • Bernhofer M.
        • Wiestler B.
        • Goldberg T.
        • Cremers D.
        • Rost B.
        • et al.
        Radiomics in radiooncology – challenging the medical physicist.
        Physica Med. 2018; 48: 27-36https://doi.org/10.1016/j.ejmp.2018.03.012
      2. EFOMP working groups website;2020 https://www.efomp.org/index.php?r=pages&id=working-groups [accessed in October 2020].

      3. European Guidelines on Medical Physics Expert. European Commission Radiation Protection n° 174. Guibelande E, Christofides S, Caruana CJ, Evans S, van der Putten W. Luxembourg: Publications office of the European Union 2014. ISBN: 978-92-79-35786-2.

      4. Samei E, Grist TM. Why physics in medicine? Physica Medica (2019) 64:319-22 doi: S1120-1797(19)30104-8 [pii].

        • Samei E.
        • Pawlicki T.
        • Bourland D.
        • Chin E.
        • Das S.
        • Fox M.
        • et al.
        Redefining and reinvigorating the role of physics in clinical medicine: a report from the AAPM Medical Physics 3.0 Ad Hoc Committee.
        Med Phys. 2018; https://doi.org/10.1002/mp.13087
      5. Sensakovic WF, Mahesh M. Role of the Medical Physicist in the Health Care Artificial Intelligence Revolution. J Am Coll Radiol (2019) 16:393-4 doi: S1546-1440(18)31164-5.

      6. El Naqa I, Das S. The role of machine and deep learning in modern medical physics. Med Phys 2020;47(5):e125–3126.

      7. Council of the European Union. (2013). Council Directive 2013/59/Euratom laying down basic safety standards for protection against the dangers arising from exposure to ionising radiation, and repealing Directives 89/618/Euratom, 90/641/Euratom, 96/29/Euratom, 97/43/Euratom and 2003/122/Euratom. Official Journal L-13 of 17.01.2014.

        • Caruana C.J.
        • Tsapaki V.
        • Damilakis J.
        • Brambilla M.
        • Martin G.M.
        • Dimov A.
        • et al.
        EFOMP policy statement 16: the role and competences of medical physicists and medical physics experts under 2013/59/EURATOM.
        Phys Med. 2018; 48: 162-168https://doi.org/10.1016/j.ejmp.2018.03.001
      8. Federale Overheidsdienst Binnenlandse Zaken en Federaal Agentschap voor Nucleaire Controle. Koninklijk besluit betreffende de bescherming tegen ioniserende stralingen tijdens diergeneeskundige blootstellingen 2020/2000245. BELGISCH STAATSBLAD — 20.02.2020.

      9. Huq MA, Fraass BA, Dunscombe PB, Gibbons JP, Ibbott GS, Mundt AJ, et al. Summary of the report of Task Group 100 of the AAPM: Application of risk analysis methods to radiation therapy quality management. 2016;43(7):4209-62. https://doi.org/10.1118/1.4947547.

        • Diaz O.
        • Guidi G.
        • Ivashchenko O.
        • Colgan N.
        • Zanca F.
        Artificial intelligence in the medical physics community: an international survey.
        Phys Med. 2021; : 141-146https://doi.org/10.1016/j.ejmp.2020.11.037
        • Del Guerra A.
        • Bardies M.
        • Belcari N.
        • Caruana C.J.
        • Christofides S.
        • Erba P.
        • et al.
        Curriculum for education and training of medical physicists in nuclear medicine. Recomendations from the EANM Physics Committee, the EANM Dosimetry Committee and EFOMP.
        Phys Med. 2013; 29: 139-162https://doi.org/10.1016/j.ejmp.2012.06.004
      10. Geleijns J, Breatnach E, Calzado Cantera A, Damilakis J, Dendy P, Evans A, et al. Core curriculum for medical physicists in radiology. Recommendations from EFOMP/ESR working group. Insights Imaging 2012;3:197-200.

        • Eriksen J.G.
        • Beavis A.W.
        • Coffey M.A.
        • Leer J.W.
        • Magrini S.M.
        • Benstead K.
        • et al.
        The updated ESTRO core curricula 2011 for clinicians, medical physicists and RTTs in radiotherapy/radiation oncology.
        Radiother Oncol. 2012 Apr; 103: 103-108https://doi.org/10.1016/j.radonc.2012.02.007
      11. The Bologna declaration of 19 June 1999: Joint declaration of the European Ministers of Education. https://www.eurashe.eu/library/modernising-phe/Bologna_1999_Bologna-Declaration.pdf [accessed in November 2020].

      12. The European Qualifications Framework for lifelong learning (EQF). European Commission, 2008. doi:10.2755/14352.

      13. Collins GS, Reitsma JB, Altman DG, Moons KGM. TRIPOD statement.Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 2014;350. g7594.

        • Orlhac F.
        • Frouin F.
        • Nioche C.
        • Ayache N.
        • Buvat I.
        Validation of a method to compensate multicenter effects affecting CT radiomics.
        Radiology. 2019; 291: 53-59https://doi.org/10.1148/radiol.2019182023
        • Badano A.
        • Graff C.G.
        • Badal A.
        • Sharma K.
        • Zeng R.
        • Samuelson F.W.
        • et al.
        Evaluation of digital breast tomosynthesis as replacement of full-field digital mammography using an in silico imaging trial.
        JAMA Netw Open. 2017; 1e18574https://doi.org/10.1001/jamanetworkopen.2018.5474
      14. Park JE, Kim D, Kim HS, Park SY, Kim JY, Cho SJ, et al. Quality of science and reporting of radiomics in oncologic studies: room for improvement according to radiomics quality score and TRIPOD statement. European Radiology 2020;30:523-536. doi: https://doi.org/10.1007/s00330-019-06360-z.

        • Arabi H.
        • Zaidi H.
        Applications of artificial intelligence and deep learning in molecular imaging and radiotherapy.
        Eur J Hybrid Imag. 2020; 4https://doi.org/10.1186/s41824-020-00086-8
      15. Vandewinckele L, Claessens M, Dinkla A, Brouwer C, Crijns W, Verellen D, et al. Overview of artificial intelligence-based applications in radiotherapy: Recommendations for implementation and quality assurance. Radiotherapy and Oncology 2020 (in press). doi: https://doi.org/10.1016/j.radonc.2020.09.008.

        • Wang M.
        • Zhang Q.
        • Lam S.
        • Cai J.
        • Yang R.
        A review on application of deep learning algorithms in external beam radiotherapy automated treatment planning.
        Front Oncol. 2020; 10: 5809191https://doi.org/10.3389/fonc.2020.580919
      16. European Commission. Ethics guidelines for trustworthy AI. Report dated on 8 April 2019, published on https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai. accessed in: 25 Nov. 2020.