Artificial intelligence (AI) is a field that enables computer systems to solve problems by adapting to changing circumstances, often by mimicking human reasoning and judgement. Several demographic and healthcare trends are driving the use of AI in medical imaging. The amount of medical imaging data being acquired is steadily increasing (Larson et al., 2011; Smith-Bindman et al., 2008, 2012; Winder et al., 2021). There is also a widespread shortage of healthcare workers (Core Health Indicators in the WHO European Region 2015. Special Focus: Human Resources for Health, 2017) with an ever-increasing workload (Levin et al., 2017), and the number of medical imaging examinations is expected to grow exponentially over the next two decades (Tsao, 2020). Radiologists and radiology technologists are in particularly scarce supply (AAMC Report Reinforces Mounting Physician Shortage, 2021, Clinical Radiology UK Workforce Census 2019 Report, 2019). Finally, the ageing world population (Population Ages 65 and above, n.d.; WHO, n.d.-a) and an increasing global burden of chronic illnesses (WHO, n.d.-b) are expected to compound these problems in the near future.
Broadly speaking, the advantages of AI in medical imaging could potentially include the ability to provide insights that would otherwise not be possible using traditional methods (such as humans looking at images) and to may do so in a faster and automated way (without the need for human interaction). AI-based solutions in medical imaging could improve and accelerate the detection of disease, generate in-depth risk assessment of disease development and progression, and may reduce subjectivity in the interpretation of medical imaging data.
The current state of AI in medical imaging
Over the past few years, the landscape of AI in medical imaging has changed dramatically. Many promising applications have arisen, the field has seen an unprecedented surge in funding, and we have seen positive trends in the adoption of AI solutions by radiologists, as well as their approval by regulatory bodies.
Applications
Although radiology departments provide a plethora of services, the core service provided is the imaging study. Applications of AI in medical imaging can therefore be categorized into those applied either before, during, or after the imaging study.
Before Image Acquisition
Several steps have to take place within the context of a radiology department’s workflow before a patient is undergoing imaging study. AI applications that aim to improve these steps are referred to as “upstream AI” and could potentially increase efficiency and provide more personalized decision making in a radiology department.
Missed medical appointments are common, reduce the efficiency of hospitals, and waste resources (Dantas et al., 2018). Studies from Japan (Kurasawa et al., 2016) and the United Kingdom (Nelson et al., 2019) have shown that AI can be used to predict no-shows with high accuracy. This allows the use of targeted strategies to reduce the likelihood of a patient missing their appointment, including sending automated reminders.
One of the most important decisions made in the radiology department is the exact scan protocol to use on a given patient. While this applies to all imaging modalities, the widest range of choice is seen with magnetic resonance imaging (MRI). This includes choosing the appropriate set of sequences and making decisions about whether or not to administer intravenous contrast agents. Natural language classifiers that interpret the narrative text of the clinician’s scan requests have been used to select appropriate MRI protocols. In one study, a gradient boosting classifier predicted the appropriate MRI brain protocol to use based on the scan request with high accuracy (95 %) (Brown & Marotta, 2018). For musculoskeletal MRI, a deep learning classifier was 83 % accurate in determining the need for a contrast agent (Trivedi et al., 2018). Such applications can substantially improve efficiency by foregoing the time-consuming task of radiologists going through unstructured narrative scan requests written by referring clinicians.
During Image Acquisition
Substantial improvements have recently been made in the use of AI for improving image quality. In a recent survey, radiologists identified the enhancement of image quality as being the most mainstream current use case for AI in medical imaging (Alexander et al., 2020). While earlier attempts at reducing image noise using deep learning techniques were criticized for removing details from the images that jeopardized the visibility of essential features within the images, more recent implementations have made this issue largely obsolete.
In particular, deep learning techniques like generative adversarial networks have shown great potential in image denoising (Wang et al., 2021). Some of these applications target the image reconstruction stage (where the raw sensor data is converted into an interpretable image) providing superior signal-to-noise ratios and reducing image artefacts (Zhu et al., 2018). In lung cancer screening, deep-learning-based image denoising improved both the image quality and the diagnostic accuracy of ultra-low-dose computed tomography (CT) for detecting suspicious lung nodules (Hata et al., 2020; Kerpel et al., 2021). Scans that were 40-60 % acquired faster than standard scans and enhanced with deep-learning-based algorithms were of better image quality than, and similar diagnostic value as, standard scans of the brain (Bash, Wang, et al., 2021; Rudie et al., 2022) and spine (Bash, Johnson, et al., 2021). Similarly, convolutional neural networks can be used to reduce specific CT and MRI artefacts and improve spatial resolution (Hauptmann et al., 2019; K. H. Kim & Park, 2017; Park et al., 2018; Y.Zhang & Yu, 2018).
Reconstruction algorithms based on deep learning have enabled ultra-low-dose computed tomography scans to be acquired while maintaining diagnostic quality. This is of particular benefit in children and pregnant women, where reduction of radiation dose to the absolute minimum is critical. These deep- learning-based CT image reconstruction approaches are associated with lower image noise and better image texture than state-of-the-art alternatives like iterative reconstruction (Higaki et al., 2020; McLeavy et al., 2021; Singh et al., 2020). In positron emission tomography, deep learning can reduce injected tracer dosage by one-third and scan times by up to half while maintaining scan quality (Katsari et al., 2021; Le et al., 2020; Xu et al., 2020).
After Image Acquisition
Radiology technologists and radiologists usually share the task of calling back patients for repeat examinations, but doing so consistently and reliably is exceedingly difficult due to time constraints. Image quality of AI enhanced brain MRI scans has been shown to be equal to or better than conventional scans, even when using acquisition protocols that reduce scan times by 45-60% (Schreiber-Zinaman & Rosenkrantz, 2017).
Prioritizing scan reading on a radiologist’s worklist is often done based on several factors including the type of scan, the referring department, and direct communication with the radiologist about the scan’s urgency. Several approaches have been tested to influence the order in which scans are read to improve efficiency and ensure the most critical scans are seen first. These include assigning different radiologists specific exams based on how quickly they read certain scan types (Wong et al., 2019) and automatically detecting emergent findings on the images and pushing these cases to the “top of the list” (Prevedello et al., 2017; Winkel et al., 2019).
About 70 % of all AI-based solutions in radiology focus on “perception” - a category of functionalities that includes segmentation, feature extraction, as well as detection and classification of pathology (Rezazade Mehrizi et al., 2021). Within this category, the majority of tools extract information from the imaging data with or without quantification as well as draw the user’s attention to potential pathology (Rezazade Mehrizi et al., 2021; van Leeuwen et al., 2021). Over the past few years, some of the most promising applications in this category have included the detection of brain vessel occlusion, brain haemorrhage, lung nodules, pneumothorax and pleural effusions, fractures, and the characterization of breast lesions.
Funding
The total amount of investment in AI-based medical imaging companies amounted to $ 1.17 billion between 2014 and 2019 (Alexander et al., 2020). In the same period, the number of companies in this space tripled, leading to a drop of almost 30 % in the median investment in each company (Alexander et al., 2020). Between 2019 and 2020, private investment in AI companies increased by 9.3 % (D. Zhang et al., 2021). By 2030, investment in AI-based solutions in medical imaging is expected to exceed $3 billion (Tsao, 2020).
Adoption
There have been positive trends in the adoption of AI tools by radiologists and radiology technologists over the past few years. Between 2015 and 2020, AI use in radiology departments went up by 30 %, according to a survey of 1,861 radiologists conducted by the American College of Radiology (ACR) (Allen et al., 2021).
Despite this promising trend, the adoption of AI tools is widely considered to be disproportionately low relative to the amount of funding, the number of companies, and the perceived promise of these tools. The ACR survey provides some insight into why and offers a starting point for developing strategies to improve AI adoption.
Almost three-quarters of radiologists who were not using AI had no plans to do so in the future because they either were not convinced of its benefits or did not think the associated costs were justified (Allen et al., 2021).
Similar results have been found in other studies, with radiologists citing scepticism in the capabilities of AI tools and the fact that relatively few have regulatory approval as reasons for not adopting them in their practice (Alexander et al., 2020).
Regulatory success
Until August 2019, 60 % of available AI-based radiology solutions had no regulatory approval (Rezazade Mehrizi et al., 2021). As of April 2020, a total of 100 AI solutions had a CE mark, a prerequisite for them to be made commercially available as medical devices in Europe (van Leeuwen et al., 2021). As of the time of writing, more than 150 AI solutions have gained FDA clearance (AI Central, n.d.). Several useful databases of approved or cleared AI-based solutions in medical healthcare are currently available (AI Central, n.d., AI for Radiology, n.d., Medical AI Evaluation, n.d., The Medical Futurist, n.d.)
The future of AI in medical imaging
The past few years have seen exponential growth in the interest in AI in medical imaging, both in terms of the amount of research and the amount of money being invested in the field. This interest runs the gamut of the radiology workflow, but “perception” applications - for the quantification of biomarkers and the detection of disease processes - have dominated so far. In the radiology community, trends have shifted from AI being perceived as an unwelcome intruder to increased adoption, albeit with some scepticism and hesitation regarding its value. The first AI solutions in medical imaging were granted regulatory approval, and we have seen the first indications of how such solutions may be reimbursed.
New directions
With increasing acknowledgement that a large proportion of AI’s potential in medical imaging may lie in “upstream” or “non-interpretative” applications, the field is likely to expand its focus in the coming years. This will include more research into applications that improve the efficiency of radiology workflows and provide more personalized patient care (Alexander et al., 2020). AI is likely to become more involved even earlier in the patient management process - i.e. before the clinician decides that a diagnostic image test is necessary. Such applications, essentially clinical decision support systems, have successfully been used for decision-making about treatments in several settings (Bennett & Hauser, 2013; Komorowski et al., 2018), successfully used in treatment decision making (Bennett & Hauser, 2013). In the future, AI solutions may draw clinicians’ attention to the need for further imaging tests based on reviewing the patient’s clinical information, laboratory tests, and prior imaging tests (Makeeva et al., 2019).
The vast majority (77-84 %) of currently available AI solutions in medical imaging target CT, MRI and plain radiographs (Rezazade Mehrizi et al., 2021; van Leeuwen et al., 2021). Nuclear imaging techniques, such as positron emission tomography (PET). provide unique information not readily gained from other modalities. PET has thus far been largely neglected in terms of AI research (Rezazade Mehrizi et al., 2021; van Leeuwen et al., 2021), and is thus a potentially promising avenue for the field’s expansion.
AI research is also expected to undergo a shift in the type of data being used. The typical inpatient receives more than one imaging study during their hospital stay (Shinagare et al., 2014). Despite this, only about 3 % of current AI-based radiology solutions combine data from multiple modalities (Rezazade Mehrizi et al., 2021; van Leeuwen et al., 2021). Combining data from multiple imaging sources may improve the diagnostic capabilities of AI solutions. Furthermore, future AI solutions in radiology are likely to combine imaging information, clinical information, as well as non- imaging diagnostic tests (Huang et al., 2020). By doing this, AI solutions may be able to identify patterns in the data collected during a patient’s hospital stay that may not be readily identifiable by healthcare workers (Rockenbach, 2021). This could ultimately lead to more accurate diagnoses and could help inform better and more personalized treatment decisions.
The expectations for AI-based medical imaging solutions are also likely to shift from the current focus of triage, image enhancement and automation. With increasing algorithmic complexity, data availability, and experience with these tools, this shift may lead to AI solutions reaching specific diagnoses and recommending specific steps in a patient’s management plan. Similar to how the introduction of the first AI tools for image screening and processing around 2018 spurred investment in the field, marketing analyses predict a similar investment boost in the next few years as AI tools providing specific diagnoses and management steps become more widespread (Michoud et al., 2019).
One important criticism of the current, arguably still nascent, landscape of AI in medical imaging is that it is too fragmented. Radiology professionals would likely welcome a more streamlined integration of AI solutions in their daily workflow. This includes seamless integration of these solutions into established radiology workflows, with as much as possible happening “in the background” without user input. Furthermore, the outputs of these solutions could be integrated into available radiological information systems. Consequently, the field could move from the plethora of currently available niche AI solutions, each targeted towards a single very specific application, to broader software suites that perform many different functions for a given imaging modality or body region.
The fragmented investment in the AI in medical imaging market (Alexander et al., 2020) fosters innovation, allowing many players to test out different strategies in this emerging field. However, in the long term, consolidation may increase adoption and stimulate the kind of seamless integration into existing workflows that is needed, allowing fewer companies to offer these solutions at scale (Alexander et al., 2020).
Challenges
Quality and reporting of evidence
In a review of 100 CE-marked AI solutions, 64 % of them had no peer-reviewed scientific evidence for their efficacy (van Leeuwen et al., 2021). Where there was scientific evidence, the level was low, rarely exceeding the demonstration of diagnostic accuracy (van Leeuwen et al., 2021). Another systematic review of the evidence for deep learning algorithms in medical imaging found a generally high diagnostic accuracy, albeit with a high risk of bias across studies (Aggarwal et al., 2021). The main sources of bias include the lack of external validation (D. W. Kim et al., 2019; Liu et al., 2019), insufficiently detailed reporting of results (Liu et al., 2019), retrospective study design (Nagendran et al., 2020), and the inaccessibility of data and code to reviewers and readers (Nagendran et al., 2020).
Overall, studies on AI tools have shown a worrying lack of standardized reporting and adherence to recommended reporting guidelines (Aggarwal et al., 2021; Yusuf et al., 2020). This is despite the fact that several extensions to established reporting guidelines, as well as AI-specific guidelines, are currently available (Shelmerdine et al., 2021). Widespread implementation of these guidelines should be a focus of AI developers in the future.
AI developers should also be cognizant that the currently “acceptable” level of evidence for AI-based solutions is likely to become obsolete in the near future. Both regulators and potential users will likely demand higher levels of evidence for these solutions, akin to the evidence required for new pharmaceutical drugs. In the next few years, will see more of these AI solutions being tested in randomized clinical trials. In the more distant future, it is plausible that such expectations will go beyond providing evidence of the safety, efficacy, or diagnostic performance of these solutions, to the demonstration that they provide added monetary or societal value.
Rising up to the challenge of improving the quality and reporting of evidence for AI-based solutions may pay off in the long run. It could reduce the risk of bias in AI studies, could allow the thorough and transparent assessment of study quality by potential users and regulators, and could facilitate systematic reviews and meta-analyses. These steps may increase the trust in, and uptake of, AI-based solutions and ensure that they offer realistic, sustainable improvements in people’s lives.
Regulation
Several aspects inherent to AI pose challenges to attempts at regulating it like other healthcare interventions. The inner workings of AI solutions are often opaque and difficult to comprehensively describe in a manner traditionally expected by regulatory bodies.
The past few years have shown us that these regulatory challenges are far from intractable. Both the Food and Drug Administration and the European Commission have recently proposed initial regulatory frameworks for AI solutions (Center for Devices & Radiological Health, 2021; European Commission, 2021).
In part as a response to the transparency necessary for regulatory approval, researchers have made substantial progress in making AI’s decision-making more understandable and explainable. This movement towards “interpretable AI” will gain further impetus in the near future as reliance on AI for real-world clinical decision-making increases.
This has many advantages, including making regulatory approval easier, increasing trust in these solutions by users, minimizing biases, and improving the reproducibility of these solutions (Holzinger et al., 2017; Kolyshkina & Simoff, 2021; “Towards Trustable Machine Learning,” 2018; Yoon et al., 2021).
Data privacy
From development and testing to implementation, AI solutions in medical imaging require access to patient data. This has raised concerns about data privacy, which is a multifaceted and highly complex issue (Murdoch, 2021) that is prominently represented in the regulatory pathways of different countries (COCIR, the European Coordination Committee of the Radiological, Electromedical and Healthcare IT Industry, 2020). Suggested solutions to the data privacy question have ranged from those focusing on oversight to more technical approaches.
The patients providing the data have to be made aware that they are doing so, as well as be informed about why and how their data will be used (Lotan et al., 2020), as explicitly stipulated in the EU’s General Data Protection Regulation (GDPR) (General Data Protection Regulation (GDPR) – Official Legal Text, 2016). Considering the fast-paced nature of the development of AI solutions, whether patients can be kept sufficiently informed as these algorithms are continuously retrained has been questioned (Kritikos, 2020). While fully anonymized data is not subject to such strict requirements under the GDPR (What Is Personal Data?, 2021), anonymization is exceedingly difficult to achieve for medical imaging data.
The data privacy issue will have to be approached on several fronts. In addition to legislation governing the use of patient data, it is becoming increasingly clear that everyone involved in the development and use of AI solutions - developers, payers, regulatory bodies, researchers and radiologists - has a role to play in ensuring that the data is protected and used responsibly.
Moreover, the next few years will likely see further research into technical approaches to strengthen data protection. These include better ways to reduce the chances of data being traced back to individuals, methods for keeping sensitive data stored locally even when the algorithm being trained is hosted in some “central” location, data perturbation to minimize the information within a given dataset pertaining to individual patients, and data encryption (G. Kaissis et al., 2021; G. A. Kaissis et al., 2020).
Democratization
If AI in medical imaging is to live up to its potential, the algorithms being developed have to work for everyone. This “democratization” of AI involves ensuring that healthcare providers have the knowledge and skills needed to use AI-based solutions. With a few exceptions (Paranjape et al., 2019), medical student curricula currently include little to no dedicated education about AI (Banerjee et al., 2021; Blease et al., 2022). Surveys from around the world have shown that medical students’ and doctors’ (Ahmed et al., 2022; Bisdas et al., 2021; Collado-Mesa et al., 2018; Kansal et al., 2022; Pinto Dos Santos et al., 2019; Sit et al., 2020) exposure to AI during training was low despite the high demand for more AI education (Kansal et al., 2022; Ooi et al., 2021; Sit et al., 2020). In addition, there are still large differences between genders and countries in the perceived knowledge about AI amongst medical students (Bisdas et al., 2021). There are many reasons for these differences and many challenges associated with the widespread integration of AI education into healthcare training curricula. In the coming years, strategies to tackle these issues should be investigated to ensure that future healthcare providers are equipped with the knowledge and skills they need to work in an environment where AI plays a growing role.
Democratization also involves ensuring that patients of different genders, lifestyles, ethnicities, and geographical locations can benefit from AI-based solutions. For this to happen, these solutions have to be accessible and their performance generalizable. The latter requires the acquisition of diverse data from multiple institutions, preferably from multiple countries, for training AI-based solutions. It also requires the implementation of safeguards to ensure that sources of bias throughout the development process are not propagated to the trained algorithm (Vokinger et al., 2021), an issue that has only recently come to the forefront (Larrazabal et al., 2020; Obermeyer et al., 2019; Seyyed-Kalantari et al., 2021).
Reimbursement
As countries’ policies for regulating AI in healthcare gradually begin to take shape, one important aspect that needs attention is who will pay for these AI solutions, and according to what framework.
Many consider Germany’s 2020 Digital Supply Act a step in the right direction for reimbursement of digital health solutions. Under this policy, digital applications prescribed by physicians are reimbursable by statutory health insurance if they are proven to be safe, be compliant with data privacy statutes, and improve patient care. The UK, on the other hand, has released a guide for potential buyers of AI-based solutions, which serves as a starting point for companies to prepare for reimbursement applications (A Buyer’s Guide to AI in Health and Care, 2020).
Thus far, reimbursement success stories in the digital health space have been few and far between (Brink- mann-Sass et al., 2020; Hassan, 2021). This is in part due to requirements varying greatly by country (COCIR, the European Coordination Committee of the Radiological, Electromedical and Healthcare IT Industry, 2020). In general, providers of digital health solutions will need to provide evidence for the overall value that these solutions bring, including detailed health economics studies showing potential cost savings.
Radiology’s position as a service provider to multiple hospital departments means that AI-based solutions in this space will be expected to show a far-reaching impact (van Duffelen, 2021). Companies will need to show short-term value (e.g. faster/better image reading and reporting) as well as long-term value (e.g. early diagnosis and treatment, disease prevention, reduction in unnecessary follow-up). The coming years will see companies compete to demonstrate such impact, while at the same time experimenting with different pricing models and navigating the evolving bureaucratic reimbursement landscape.
Conclusion
Over the past few years, the field of AI in medical imaging has undergone a rapid but steady transformation. AI can now achieve things in radiology that few people thought possible a mere decade ago. The field is also gradually overcoming one of its most significant perceived hurdles - regulatory approval. In addition, while fear and scepticism dominated radiologists’ perception of the future of AI in their speciality a few years ago, this is no longer the case.
The massive progress and interest in the field of AI in medical imaging is expected to continue into 2022 and beyond. Several exciting transformations await the field - it will likely expand its focus in the coming years to improve radiology workflow efficiency, involve hitherto neglected imaging modalities, combine data from multiple modalities, and provide more concrete diagnostic predictions and management recommendations. Easy-to-use and comprehensive software suites utilizing AI will be incorporated into existing radiology workflows, making radiologists’ and radiographers’ work easier and more efficient.
As in any rapidly growing field, several scientific, regulatory, and economic challenges face AI in medical imaging. But the past few years have shown us that even the most difficult problems can be solved. Developers and users of AI-based solutions need to be aware of these issues so that they can adapt their strategies to changing expectations on a regulatory and societal level. Doing this will allow them to thrive in a fascinating field with the potential to improve virtually every aspect of healthcare.
References
AAMC Report Reinforces Mounting Physician Shortage. (2021). AAMC. https://www.aamc.org/news-insights/press- releases/aamc-report-reinforces-mounting-physician-shortage
A buyer’s guide to AI in health and care. (2020). NHS Transformation Directorate. https://www.nhsx.nhs.uk/ai-lab/ explore-all-resources/adopt-ai/a-buyers-guide-to-ai-in-health- and-care/
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021). Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis. NPJ Digital Medicine, 4(1), 65.
Ahmed, Z., Bhinder, K. K., Tariq, A., Tahir, M. J., Mehmood, Q., Tabassum, M. S., Malik, M., Aslam, S., Asghar, M. S., & Yousaf, Z. (2022). Knowledge, attitude, and practice of artificial intelligence among doctors and medical students in Pakistan: A cross-sectional online survey. Annals of Medicine and Surgery (2012), 76, 103493.
AI Central. (n.d.). Retrieved February 23, 2022, from https://aicentral.acrdsi.org/
AI for Radiology. (n.d.). Retrieved February 23, 2022, from https://grand-challenge.org/aiforradiology/
Alexander, A., Jiang, A., Ferreira, C., & Zurkiya, D. (2020). An Intelligent Future for Medical Imaging: A Market Outlook on Artificial Intelligence for Medical Imaging. Journal of the American College of Radiology: JACR, 17(1 Pt B), 165–170.
Allen, B., Agarwal, S., Coombs, L., Wald, C., & Dreyer, K. (2021). 2020 ACR Data Science Institute Artificial Intelligence Survey. Journal of the American College of Radiology: JACR, 18(8), 1153–1159.
Banerjee, M., Chiew, D., Patel, K. T., Johns, I., Chappell, D., Linton, N., Cole, G. D., Francis, D. P., Szram, J., Ross, J., & Zaman, S. (2021). The impact of artificial intelligence on clinical education: perceptions of postgraduate trainee doctors in London (UK) and recommendations for trainers. BMC Medical Education, 21(1), 429.
Bash, S., Johnson, B., Gibbs, W., Zhang, T., Shankaranarayanan, A., & Tanenbaum, L. N. (2021). Deep Learning Image Processing Enables 40 % Faster Spinal MR Scans Which Match or Exceed Quality of Standard of Care : A Prospective Multicenter Multireader Study. Clinical Neuroradiology. https://doi.org/10.1007/s00062-021-01121-2
Bash, S., Wang, L., Airriess, C., Zaharchuk, G., Gong, E., Shankaranarayanan, A., & Tanenbaum, L. N. (2021). Deep Learning Enables 60 % Accelerated Volumetric Brain MRI While Preserving Quantitative Performance: A Prospective, Multicenter, Multireader Trial. AJNR. American Journal of Neuroradiology, 42(12), 2130–2137.
Bennett, C. C., & Hauser, K. (2013). Artificial intelligence framework for simulating clinical decision-making: a Markov decision process approach. Artificial Intelligence in Medicine, 57(1), 9–19.
Bisdas, S., Topriceanu, C.-C., Zakrzewska, Z., Irimia, A.-V., Shakallis, L., Subhash, J., Casapu, M.-M., Leon-Rojas, J., Pinto Dos Santos, D., Andrews, D. M., Zeicu, C., Bouhuwaish, A. M., Lestari, A. N., Abu-Ismail, L. ’i, Sadiq, A. S., Khamees, A. ’atasim, Mohammed, K. M. G., Williams, E., Omran, A. I.,… Ebrahim, E. H. (2021). Artificial Intelligence in Medicine: A Multinational Multi-Center Survey on the Medical and Dental Students’ Perception. Frontiers in Public Health, 9, 795284.
Blease, C., Kharko, A., Bernstein, M., Bradley, C., Houston, M., Walsh, I., Hägglund, M., DesRoches, C., & Mandl, K. D. (2022). Machine learning in medical education: a survey of the experiences and opinions of medical students in Ireland. BMJ Health & Care Informatics, 29(1). https://doi.org/10.1136/ bmjhci-2021-100480
Brown, A. D., & Marotta, T. R. (2018). Using machine learning for sequence-level automated MRI protocol selection in neuroradiology. Journal of the American Medical Informatics Association: JAMIA, 25(5), 568–571.
Center for Devices, & Radiological Health. (2021, June 22). Digital Health Software Precertification (Pre-Cert) program.
U.S. Food and Drug Administration. https://www.fda.gov/ medical-devices/digital-health-center-excellence/digital-health- software-precertification-pre-cert-program
Clinical radiology UK workforce census 2019 report. (2019). https://www.rcr.ac.uk/publication/clinical-radiology-uk- workforce-census-2019-report
COCIR, the European Coordination Committee of the Radiological, Electromedical and Healthcare IT Industry. (2020). Market Access Pathways for Digital Health Solutions. https://www.cocir.org/fileadmin/Publications_2020/20062_ COCIR_Market_Access_Pathways_Digital_Health.pdf
Collado-Mesa, F., Alvarez, E., & Arheart, K. (2018). The Role of Artificial Intelligence in Diagnostic Radiology: A Survey at a Single Radiology Residency Training Program. Journal of the American College of Radiology: JACR, 15(12), 1753–1757.
Core Health Indicators in the WHO European Region 2015. Special focus: Human resources for health. (2017, August 14). World Health Organization. https://www.euro.who.int/en/ data-and-evidence/evidence-resources/core-health-indicators- in-the-who-european-region/core-health-indicators-in-the- who-european-region-2015.-special-focus-human-resources- for-health
Dantas, L. F., Fleck, J. L., Cyrino Oliveira, F. L., & Hamacher, S. (2018). No-shows in appointment scheduling - a systematic literature review. Health Policy, 122(4), 412–421.
Esses, S. J., Lu, X., Zhao, T., Shanbhogue, K., Dane, B., Bruno, M., & Chandarana, H. (2018). Automated image quality evaluation of T2 -weighted liver MRI utilizing deep learning architecture. Journal of Magnetic Resonance Imaging: JMRI, 47(3), 723–728.
European Commission. (2021). Proposal for a Regulation Of The European Parliament And Of The Council Laying Down
Harmonised Rules On Artificial Intelligence (Artificial Intelligence Act) And Amending Certain Union Legislative Act. https://eur-lex. europa.eu/resource.html?uri=cellar:e0649735-a372-11eb- 9585-01aa75ed71a1.0001.02/DOC_1&format=PDF
General Data Protection Regulation (GDPR) – Official Legal Text. (2016, July 13). General Data Protection Regulation (GDPR). https://gdpr-info.eu/
Hata, A., Yanagawa, M., Yoshida, Y., Miyata, T., Tsubamoto, M., Honda, O., & Tomiyama, N. (2020). Combination of Deep Learning-Based Denoising and Iterative Reconstruction for Ultra-Low-Dose CT of the Chest: Image Quality and Lung-RADS Evaluation. AJR. American Journal of Roentgenology, 215(6), 1321–1328.
Hauptmann, A., Arridge, S., Lucka, F., Muthurangu, V., & Steeden, J. A. (2019). Real-time cardiovascular MR with
spatio-temporal artifact suppression using deep learning-proof of concept in congenital heart disease. Magnetic Resonance in Medicine: Official Journal of the Society of Magnetic Resonance in Medicine / Society of Magnetic Resonance in Medicine, 81(2), 1143–1156.
Higaki, T., Nakamura, Y., Zhou, J., Yu, Z., Nemoto, T., Tatsugami, F., & Awai, K. (2020). Deep Learning Reconstruction at CT: Phantom Study of the Image Characteristics. Academic Radiology, 27(1), 82–87.
Holzinger, A., Biemann, C., Pattichis, C. S., & Kell, D. B. (2017). What do we need to build explainable AI systems for the medical domain? In arXiv [cs.AI]. arXiv. http://arxiv.org/ abs/1712.09923
Hötker, A. M., Da Mutten, R., Tiessen, A., Konukoglu, E., & Donati, O. F. (2021). Improving workflow in prostate MRI: AI- based decision-making on biparametric or multiparametric MRI. Insights into Imaging, 12(1), 112.
Huang, S.-C., Pareek, A., Seyyedi, S., Banerjee, I., & Lungren, M. P. (2020). Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines. NPJ Digital Medicine, 3, 136.
Kaissis, G. A., Makowski, M. R., Rückert, D., & Braren, R. F. (2020). Secure, privacy-preserving and federated machine learning in medical imaging. Nature Machine Intelligence, 2(6), 305–311.
Kaissis, G., Ziller, A., Passerat-Palmbach, J., Ryffel, T., Usynin, D., Trask, A., Lima, I., Mancuso, J., Jungmann, F., Steinborn, M.-M., Saleh, A., Makowski, M., Rueckert, D., & Braren, R. (2021). End-to-end privacy preserving deep learning on multi-institutional medical imaging. Nature Machine Intelligence, 3(6), 473–484.
Kansal, R., Bawa, A., Bansal, A., Trehan, S., Goyal, K., Goyal, N., & Malhotra, K. (2022). Differences in Knowledge and Perspectives on the Usage of Artificial Intelligence Among Doctors and Medical Students of a Developing Country: A Cross- Sectional Study. Cureus, 14(1), e21434.
Katsari, K., Penna, D., Arena, V., Polverari, G., Ianniello, A., Italiano, D., Milani, R., Roncacci, A., Illing, R. O., & Pelosi, E. (2021). Artificial intelligence for reduced dose 18F-FDG PET examinations: a real-world deployment through a standardized framework and business case assessment. EJNMMI Physics, 8(1), 25.
Kerpel, A., Marom, E. M., Green, M., Eifer, M., Konen, E., Mayer, A., & Betancourt Cuellar, S. L. (2021). Ultra-Low Dose Chest CT with Denoising for Lung Nodule Detection. The Israel Medical Association Journal: IMAJ, 23(9), 550–555.
Kim, D. W., Jang, H. Y., Kim, K. W., Shin, Y., & Park, S. H. (2019).
Design Characteristics of Studies Reporting the Performance of Artificial Intelligence Algorithms for Diagnostic Analysis of
Medical Images: Results from Recently Published Papers. Korean Journal of Radiology: Official Journal of the Korean Radiological Society, 20(3), 405–410.
Kim, K. H., & Park, S.-H. (2017). Artificial neural network for suppression of banding artifacts in balanced steady-state free precession MRI. Magnetic Resonance Imaging, 37, 139–146.
Kolyshkina, I., & Simoff, S. (2021). Interpretability of Machine Learning Solutions in Public Healthcare: The CRISP-ML Approach. Frontiers in Big Data, 4, 660206.
Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. C., & Faisal, A. A. (2018). The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care. Nature Medicine, 24(11), 1716–1720.
Kritikos, M. (2020). What if artificial intelligence in medical imaging could accelerate Covid-19 treatment? Think Tank
- European Parliament. https://www.europarl.europa.eu/ thinktank/en/document/EPRS_ATA(2020)656333
Kurasawa, H., Hayashi, K., Fujino, A., Takasugi, K., Haga, T., Waki, K., Noguchi, T., & Ohe, K. (2016). Machine-Learning- Based Prediction of a Missed Scheduled Clinical Appointment by Patients With Diabetes. Journal of Diabetes Science and Technology, 10(3), 730–736.
Larrazabal, A. J., Nieto, N., Peterson, V., Milone, D. H., & Ferrante, E. (2020). Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis. Proceedings of the National Academy of Sciences of the United States of America, 117(23), 12592–12594.
Larson, D. B., Johnson, L. W., Schnell, B. M., Salisbury, S. R., & Forman, H. P. (2011). National trends in CT use in the emergency department: 1995-2007. Radiology, 258(1), 164–173.
Le, V., Frye, S., Botkin, C., Christopher, K., Gulaka, P., Sterkel, B., Frye, R., Muzaffar, R., & Osman, M. (2020). Effect of PET Scan with Count Reduction Using AI-Based Processing Techniques on Image Quality. Journal of Nuclear Medicine: Official Publication, Society of Nuclear Medicine, 61(supplement 1), 3095–3095.
Levin, D. C., Parker, L., & Rao, V. M. (2017). Recent Trends in Imaging Use in Hospital Settings: Implications for Future Planning. Journal of the American College of Radiology: JACR, 14(3), 331–336.
Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., Ledsam, J. R., Schmid, M. K., Balaskas, K., Topol, E.J., Bachmann, L. M., Keane, P. A., & Denniston, A. K. (2019). A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. The Lancet. Digital Health, 1(6), e271–e297.
Lotan, E., Tschider, C., Sodickson, D. K., Caplan, A. L., Bruno, M., Zhang, B., & Lui, Y. W. (2020). Medical Imaging and Privacy in the Era of Artificial Intelligence: Myth, Fallacy, and the Future. Journal of the American College of Radiology: JACR, 17(9), 1159–1162.
Mairhöfer, D., Laufer, M., Simon, P. M., Sieren, M., Bischof, A., Käster, T., Barth, E., Barkhausen, J., & Martinetz, T. (2021). An AI-based Framework for Diagnostic Quality Assessment of Ankle Radiographs. https://openreview.net/pdf?id=bj04hJss_xZ
Makeeva, V., Gichoya, J., Hawkins, C. M., Towbin, A. J., Heilbrun, M., & Prater, A. (2019). The Application of Machine Learning to Quality Improvement Through the Lens of the Radiology Value Network. Journal of the American College of Radiology: JACR, 16(9 Pt B), 1254–1258.
McLeavy, C. M., Chunara, M. H., Gravell, R. J., Rauf, A., Cushnie, A., Staley Talbot, C., & Hawkins, R. M. (2021). The future of CT: deep learning reconstruction. Clinical Radiology, 76(6), 407–415.
Medical AI Evaluation. (n.d.). Retrieved February 23, 2022, from https://ericwu09.github.io/medical-ai-evaluation/
Michoud, L., Tschudi, Y., & Villien, Y. (2019). Artificial Intelligence for Medical Imaging: Market and Technology Report 2020. Yole Développement. https://s3.i-micronews.com/ uploads/2020/01/YDR20059-AI-for-Medical-Imaging_Yole_ sample.pdf
Murdoch, B. (2021). Privacy and artificial intelligence: challenges for protecting health information in a new era. BMC Medical Ethics, 22(1), 122.
Nagendran, M., Chen, Y., Lovejoy, C. A., Gordon, A. C., Komorowski, M., Harvey, H., Topol, E. J., Ioannidis, J. P. A., Collins, G. S., & Maruthappu, M. (2020). Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies. BMJ, 368. https://doi.org/10.1136/bmj.m689
Nelson, A., Herron, D., Rees, G., & Nachev, P. (2019). Predicting scheduled hospital attendance with artificial intelligence. Npj Digital Medicine, 2(1), 26.
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447–453.
Ooi, S. K. G., Makmur, A., Soon, A. Y. Q., Fook-Chong, S., Liew, C., Sia, S. Y., Ting, Y. H., & Lim, C. Y. (2021). Attitudes toward artificial intelligence in radiology with learner needs assessment within radiology residency programmes: a national multi-programme survey. Singapore Medical Journal, 62(3), 126–134.
Paranjape, K., Schinkel, M., Nannan Panday, R., Car, J., & Nanayakkara, P. (2019). Introducing Artificial Intelligence Training in Medical Education. JMIR Medical Education, 5(2), e16048.
Park, J., Hwang, D., Kim, K. Y., Kang, S. K., Kim, Y. K., & Lee, J. S. (2018). Computed tomography super-resolution using deep convolutional neural network. Physics in Medicine and Biology, 63(14), 145011.
Pinto Dos Santos, D., Giese, D., Brodehl, S., Chon, S. H., Staab, W., Kleinert, R., Maintz, D., & Baeßler, B. (2019). Medical students’ attitude towards artificial intelligence: a multicentre survey. European Radiology, 29(4), 1640–1646.
Population ages 65 and above. (n.d.). The World Bank. Retrieved February 23, 2022, from https://data.worldbank.org/ indicator/SP.POP.65UP.TO.ZS
Prevedello, L. M., Erdal, B. S., Ryu, J. L., Little, K. J., Demirer, M., Qian, S., & White, R. D. (2017). Automated Critical Test Findings Identification and Online Notification System Using Artificial Intelligence in Imaging. Radiology, 285(3), 923–931.
Rezazade Mehrizi, M. H., van Ooijen, P., & Homan, M. (2021). Applications of artificial intelligence (AI) in diagnostic radiology: a technography study. European Radiology, 31(4), 1805–1811.
Rockenbach, M. A. B. (2021, June 13). Multimodal AI in healthcare: Closing the gaps. CodeX. https://medium.com/codex/ multimodal-ai-in-healthcare-1f5152e83be2
Rudie, J. D., Gleason, T., Barkovich, M. J., Wilson, D. M., Shankaranarayanan, A., Zhang, T., Wang, L., Gong, E., Zaharchuk, G., & Villanueva-Meyer, J. E. (2022). Clinical Assessment of Deep Learning–based Super-Resolution for 3D Volumetric Brain MRI. Radiology: Artificial Intelligence, e210059.
Schreiber-Zinaman, J., & Rosenkrantz, A. B. (2017). Frequency and reasons for extra sequences in clinical abdominal MRI examinations. Abdominal Radiology (New York), 42(1), 306–311.
Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y., & Ghassemi, M. (2021). Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nature Medicine, 27(12), 2176–2182.
Shelmerdine, S. C., Arthurs, O. J., Denniston, A., & Sebire, N. J. (2021). Review of study reporting guidelines for clinical studies using artificial intelligence in healthcare. BMJ Health & Care Informatics, 28(1). https://doi.org/10.1136/ bmjhci-2021-100385
Shinagare, A. B., Ip, I. K., Abbett, S. K., Hanson, R., Seltzer, S. E., & Khorasani, R. (2014). Inpatient imaging utilization: trends of the past decade. AJR. American Journal of Roentgenology, 202(3), W277–W283.
Singh, R., Digumarthy, S. R., Muse, V. V., Kambadakone, A. R., Blake, M. A., Tabari, A., Hoi, Y., Akino, N., Angel, E., Madan, R., & Kalra, M. K. (2020). Image Quality and Lesion Detection on Deep Learning Reconstruction and Iterative Reconstruction of Submillisievert Chest and Abdominal CT. AJR. American Journal of Roentgenology, 214(3), 566–573.
Sit, C., Srinivasan, R., Amlani, A., Muthuswamy, K., Azam, A., Monzon, L., & Poon, D. S. (2020). Attitudes and perceptions of UK medical students towards artificial intelligence and radiology: a multicentre survey. Insights into Imaging, 11(1), 14.
Smith-Bindman, R., Miglioretti, D. L., Johnson, E., Lee, C., Feigelson, H. S., Flynn, M., Greenlee, R. T., Kruger, R. L., Hornbrook, M. C., Roblin, D., Solberg, L. I., Vanneman, N., Weinmann, S., & Williams, A. E. (2012). Use of diagnostic imaging studies and associated radiation exposure for patients enrolled in large integrated health care systems, 1996-2010. JAMA: The Journal of the American Medical Association, 307(22), 2400–2409.
Smith-Bindman, R., Miglioretti, D. L., & Larson, E. B. (2008). Rising use of diagnostic medical imaging in a large integrated health system. Health Affairs, 27(6), 1491–1502.
The Medical Futurist. (n.d.). The Medical Futurist. Retrieved February 23, 2022, from https://medicalfuturist.com/fda- approved-ai-based-algorithms/
Towards trustable machine learning. (2018). Nature Biomedical Engineering, 2(10), 709–710.
Trivedi, H., Mesterhazy, J., Laguna, B., Vu, T., & Sohn, J. H. (2018). Automatic Determination of the Need for Intravenous Contrast in Musculoskeletal MRI Examinations Using IBM Watson’s Natural Language Processing Algorithm. Journal of Digital Imaging, 31(2), 245–251.
Tsao, D. N. (2020, July 27). AI in medical diagnostics 2020- 2030: Image recognition, players, clinical applications, forecasts: IDTechEx. https://www.idtechex.com/en/research- report/ai-in-medical-diagnostics-2020-2030-image- recognition-players-clinical-applications-forecasts/766
van Duffelen, J. (2021, February 22). Making a case for buying medical imaging AI: How to define the return on investment.
Aidence. https://www.aidence.com/articles/medical-imaging- ai-roi/
van Leeuwen, K. G., Schalekamp, S., Rutten, M. J. C. M., van Ginneken, B., & de Rooij, M. (2021). Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. European Radiology, 31(6), 3797–3804.
Vokinger, K. N., Feuerriegel, S., & Kesselheim, A. S. (2021). Mitigating bias in machine learning for medicine. Communication & Medicine, 1, 25.
Wang, S., Cao, G., Wang, Y., Liao, S., Wang, Q., Shi, J., Li, C., & Shen, D. (2021). Review and Prospect: Artificial Intelligence in Advanced Medical Imaging. Frontiers in Radiology, 1. https://doi. org/10.3389/fradi.2021.781868
What is personal data? (2021, January 1). ICO - Information Commissioner’s Office; ICO. https://ico.org.uk/for-organisations/ guide-to-data-protection/guide-to-the-general-data-protection- regulation-gdpr/what-is-personal-data/what-is-personal-data
WHO. (n.d.-a). Ageing and health. Retrieved February 23, 2022, from https://www.who.int/news-room/fact-sheets/detail/ ageing-and-health
WHO. (n.d.-b). Noncommunicable diseases. Retrieved March 6, 2022, from https://www.who.int/news-room/fact-sheets/detail/ noncommunicable-diseases
Winder, M., Owczarek, A. J., Chudek, J., Pilch-Kowalczyk, J., & Baron, J. (2021). Are We Overdoing It? Changes in Diagnostic Imaging Workload during the Years 2010-2020 including the Impact of the SARS-CoV-2 Pandemic. Healthcare (Basel, Switzerland), 9(11). https://doi.org/10.3390/healthcare9111557
Winkel, D. J., Heye, T., Weikert, T. J., Boll, D. T., & Stieltjes, B. (2019). Evaluation of an AI-Based Detection Software for Acute Findings in Abdominal Computed Tomography Scans: Toward an Automated Work List Prioritization of Routine CT Examinations. Investigative Radiology, 54(1), 55–59.
Wong, T. T., Kazam, J. K., & Rasiej, M. J. (2019). Effect of Analytics-Driven Worklists on Musculoskeletal MRI
Interpretation Times in an Academic Setting. AJR. American Journal of Roentgenology, 1–5.
Xu, F., Pan, B., Zhu, X., Gulaka, P., Xiang, L., Gong, E., Zhang, T., Wang, J., Lin, L., Ma, Y., & Gong, N.-J. (2020). Evaluation of Deep Learning Based PET Image Enhancement Method in Diagnosis of Lymphoma. Journal of Nuclear Medicine: Official Publication, Society of Nuclear Medicine, 61(supplement 1), 431–431.
Yoon, C. H., Torrance, R., & Scheinerman, N. (2021). Machine learning in medicine: should the pursuit of enhanced interpretability be abandoned? Journal of Medical Ethics. https://doi.org/10.1136/medethics-2020-107102
Yusuf, M., Atal, I., Li, J., Smith, P., Ravaud, P., Fergie, M., Callaghan, M., & Selfe, J. (2020). Reporting quality of studies using machine learning models for medical diagnosis: a systematic review. BMJ Open, 10(3), e034568.
Zhang, D., Mishra, S., Brynjolfsson, E., Etchemendy, J., Ganguli, D., Grosz, B., Lyons, T., Manyika, J., Niebles, J. C., Sellitto, M., Shoham, Y., Clark, J., & Perrault, R. (2021). The AI Index 2021 Annual Report. In arXiv [cs.AI]. arXiv. http://arxiv.org/ abs/2103.06312
Zhang, Y., & Yu, H. (2018). Convolutional Neural Network Based Metal Artifact Reduction in X-Ray Computed Tomography. IEEE Transactions on Medical Imaging, 37(6), 1370–1381.
Zhu, B., Liu, J. Z., Cauley, S. F., Rosen, B. R., & Rosen, M. S. (2018). Image reconstruction by domain-transform manifold learning. Nature, 555(7697), 487–492.
