Cancer is a leading cause of global morbidity and mortality, with one in five people on average developing cancer in their lifetimes (The Burden of Cancer, n.d.). Cancer screening targets asymptomatic individuals and aims to identify either early-stage cancer or precancerous conditions. In many cases, this allows for timely intervention and improved treatment outcomes. In general, screening can be thought of as serving either a preventive or early detection role. Preventive screening aims to detect benign conditions that can turn cancerous, which is only possible with some cancers, while early detection approaches aim to detect early-stage cancer. Importantly, screening should not be considered a single test, but a process that includes identifying the target population, conducting diagnostic tests, and planning further work-up including treatment when necessary (World Health Organization. Regional Office for Europe, 2022).
Radiology has long played an essential role in determining the extent of local and distant tumor spread after a cancer diagnosis is made. However, it is also indispensable in the screening pathways of several common cancers. In these cases, medical imaging studies are either the primary screening tool or are used to decide on further work-up after screening using other methods, such as blood tests. Depending on the type of cancer, screening can involve medical imaging techniques such as mammography, computed tomography (CT), magnetic resonance imaging (MRI), or ultrasound. National screening programs using medical imaging have been established for some of the most common cancers. Most of these programs target specific populations at risk of the specific cancer in question, identified using modifiable or non-modifiable risk factors.
Because cancer screening targets healthy people, it is especially essential that a screening program’s benefits outweigh its harms. This has to be carefully established for each program and is sometimes controversial (Lam et al., 2014). However, certain advantages and disadvantages to cancer screening apply to all screening techniques and cancers (Kramer, 2004; World Health Organization. Regional Office for Europe, 2022). Screening can reduce healthcare costs and improve patient quality of life. It also often improves the prognosis and treatment outcomes of people identified as having cancer and can provide reassurance to people in whom cancer is not found. However, sometimes early detection does not change the prognosis, and in these people, screening may instigate unnecessary treatment resulting in reduced health or quality of life. In fact, screening can sometimes detect cancers that would never have led to ill health or death in the person’s lifetime. In addition, false positives and false negatives are inevitable with any screening test. The former leads to overtreatment, with the resulting psychosocial and physical side effects, and the latter to false reassurance and delayed treatment.
Breast cancer is the second leading cause of cancer deaths in women (Bray et al., 2018) and one of the most common cancers worldwide (Sung et al., 2021). Early detection and treatment can improve outcomes, and studies have shown up to 20% lower mortality in screened populations compared with populations not offered screening. Studies estimate that one breast cancer death is prevented on average for every 250 to 414 women screened (Marmot et al., 2013; Tabár et al., 2011). More than 100 countries worldwide have implemented large-scale breast cancer screening programs (Existence of National Screening Program for Breast Cancer, n.d.). The start of screening is recommended between ages 40 and 50 years (Ren et al., 2022) and is almost exclusively done using either mammography, which uses low-dose X-rays to image the breasts, or digital breast tomosynthesis, a similar technique that uses multiple projections to create a series of stacked images of the breast.
The algorithm increased breast cancer detection by 12-27% by triaging mammograms that were assessed as negative after double-reading yet were considered suspicious by the algorithm for further assessment using MRI or ultrasound.
The accuracy of mammography varies considerably and even the most experienced radiologists’ readings have high false positive and false negative rates (Elmore et al., 2009; Lehman et al., 2015). It is estimated that at least one in three women screened will have a false positive mammography result during their lifetime (Castells et al., 2006). Mammography is particularly challenging in dense breasts (Boyd et al., 2007) and in women on hormone replacement therapy (Banks et al., 2006). Mammography screening is also a labor-intensive process. In many European countries, the standard of care is consensus double reading, in which two radiologists consecutively read each case and resolve disagreements by consensus (Giordano et al., 2012). There is unfortunately also a shortage of radiologists and radiographers specifically trained in mammography in many countries (Moran & Warren-Forward, 2012; Rimmer, 2017; Wing & Langelier, 2009).
Systems based on artificial intelligence (AI) have been incorporated at various stages in the breast cancer screening process. In a study of almost 30,000 women in the United States and the United Kingdom who received screening mammograms at intervals of 1–3 years and a follow-up period of up to 39 months, an ensemble of three deep-learning models was compared to histopathology and the interpretations of board-certified radiologists (McKinney et al., 2020). The algorithm had a 1.2–5.7 % higher specificity and a 2.7–9.4 % higher sensitivity compared to the radiologists who performed the first reading. The authors estimated that using the algorithm could render second readings unnecessary in up to 88 % of screening cases while maintaining accuracy, freeing up much-needed resources.
Consistently promising results have been reported in studies using AI-based systems in conjunction with radiologists. A study of almost 16,000 women receiving either digital mammography or digital breast tomosynthesis in Spain estimated that using a deeplearning-based algorithm would result in a 72.5% lower workload compared to double reading while maintaining sensitivity (Raya-Povedano et al., 2021). In this model, the least suspicious examinations would only be read by the algorithm and the top 2 % most suspicious examinations, as judged by the algorithm, would be flagged for further workup regardless of the radiologists’ interpretation. Similarly, a study of 7364 women in Sweden found that a commercially available deep-learning algorithm accurately classified the least suspicious mammograms, and these women underwent no further workup (Dembrower et al., 2020). This was achieved with a false negative rate of 0–2.6 %. The algorithm also increased breast cancer detection by 12–27 % by triaging mammograms that were assessed as negative after double-reading yet were considered suspicious by the algorithm for further assessment using MRI or ultrasound.
Other studies have used AI-based systems as a decision-referral step. In a study of over a million mammograms in Germany, a deep convolutional neural network (CNN) assigned a confidence score to each mammogram (Leibig et al., 2022). Assessments that the algorithm made with high confidence underwent no further workup, while low-confidence assessments were referred to the radiologist. This approach was associated with a 4 % increase in sensitivity and a 0.5 % increase in specificity compared with the assessment of a single radiologist unaided by the algorithm. In this scenario, 63 % of mammograms were automatically triaged by the algorithm, and the improved performance compared to a single radiologist’s reading was consistent across eight screening sites and three device manufacturers.
Lung cancer is the leading cause of cancer deaths worldwide, causing almost 1.8 million deaths in 2020 (Sung et al., 2021). An example of a targeted screening approach, lung cancer screening is recommended based on individual risk. Screening of adults aged 50 to 80 years with a 20-pack-year smoking history using low-dose computed tomography (LDCT) has been recommended in the US since 2013 after initial studies showed a relative reduction in lung cancer mortality of 20% (Lung Cancer: Screening, 2021; National Lung Screening Trial Research Team et al., 2011). A similar screening program is being rolled out in the United Kingdom (NHS England, 2022).
The study found that the use of the algorithm was associated with improved sensitivity to nodules across different levels of experience of the first reader.
In patients who undergo lung cancer screening using LDCT, AI has shown promise for the automatic detection of lung nodules likely to represent malignancy. This is important because the detection of lung nodules by radiologists is burdensome, time-consuming, and prone to error (Al Mohammad et al., 2019; Armato et al., 2009; Gierada et al., 2017; Leader et al., 2005). In a study of almost two thousand patients, a CNN-based algorithm designed to automatically detect lung nodules was tested as a second reader (Katase et al., 2022). The ground truth consisted of nodules identified by two experienced radiologists as being high-risk according to the clinical history as well as the nodule’s morphology. The study found that the use of the algorithm was associated with improved sensitivity to nodules across different levels of experience of the first reader. Although overall sensitivity was lower for ground glass nodules and nodules less than 1 cm in diameter, sensitivity for these was much higher when the algorithm was used compared to when the radiologist interpreted the images alone. False positives included areas of pleural inflammation or peripheral vessels while false negatives were often faint or poorly demarcated ground glass nodules or nodules close to the diaphragm. Importantly, the authors found consistent model performance across a range of CT radiation doses in a phantom study, indicating that their results might be generalizable to other chest CT protocols (Katase et al., 2022). Another study found a sensitivity of 93 % and specificity of 96 % of a CNNbased algorithm for the detection of lung nodules on LDCT compared to the consensus of two radiologists (Chamberlin et al., 2021). False positives in this study included areas of atelectasis, parenchymal changes associated with infection, and osteophytes protruding into the lung fields from thoracic vertebrae.
Beyond the mere identification of lung nodules, some studies have attempted to classify the risk of malignancy of identified nodules. A multi-component algorithm that includes lung segmentation, cancer region detection, and cancer prediction models was tested on 6716 LDCTs and validated on an independent dataset of 1139 LDCTs (Ardila et al., 2019). The algorithm outputs a probability of malignancy based on either single LDCTs or, when available, prior LDCTs from the same patient. Using a ground truth of biopsy-proven lung cancer, the algorithm performed as well as six radiologists when prior LDCTs were available. In the cases without prior LDCTs, the algorithm had an 11 % lower false positive rate and a 5 % lower false negative rate than the radiologists.
An assessment of the lung parenchyma on LDCT beyond the presence of lung nodules is a recent and promising approach to identifying the future risk of lung cancer. One study found that a 3D-CNN algorithm, tested on over 15,000 LDCTs, had an area under the receiver operating characteristic curve (AUC) of 0.86–0.94 (depending on the dataset) for predicting one-year lung cancer (Mikhael et al., 2023). Interestingly, the AUC of the algorithm after excluding cases where visible nodules were present at baseline in the same location as the future cancers was 0.82. The algorithm also showed a lower false positive rate than established scores of malignancy based on nodule morphology when the entire LDCT volume was assessed by the algorithm. These findings suggest that other features beyond the suspicious nodules were contributing to the algorithm’s prediction. Importantly, this means that the algorithm is detecting features in LDCT beyond what radiologists typically consider relevant for predicting lung cancer risk.
The eligibility criteria for lung cancer screening in the US, which come from the Centers for Medicare and Medicaid Services (CMS), miss over half of lung cancer cases (Y. Wang et al., 2015). Although other, more complex, score-based “pre-screening” tools exist, the information they require, such as the number of pack years, is often inaccurate or unavailable (Kinsinger et al., 2017). AI has thus been used to identify more individuals at high risk for lung cancer to include them in screening programs. A study of 5615 individuals found that a combination of plain chest radiographs, age, sex, and current smoking status allows a more targeted selection of patients for screening with LDCT (Lu et al., 2020). The model in this study had an AUC of 0.7 for predicting 12-year incident lung cancer compared to an AUC of 0.63 for the CMS criteria, translating to 30.7 % fewer lung cancer cases being missed with the algorithm. The model also predicted 12-year lung cancer mortality with an AUC of 0.76. The authors do not recommend routine chest radiographs for pre-screening but advocate the use of this model in patients undergoing chest radiographs for other clinical indications.
Colorectal cancer is the third most common cancer in both women and men and is a major cause of cancer death worldwide (Sung et al., 2021). It develops as a cascade of events as intestinal mucosal cells accumulate genetic mutations, transforming first into hyperproliferative mucosa, then a benign adenoma, and, in some cases, an adenocarcinoma (Kuipers et al., 2015). Colorectal cancer screening is primarily preventive - it aims to detect potentially cancerous adenomas so that they can be removed, an approach that reduces the disease’s mortality (Zauber et al., 2012).
A recent proof-of-concept study used a fully automated approach using CNNs for polyp segmentation and distinguishing between benign and premalignant polyps.
Colorectal cancer screening is routinely done by either looking for blood in the stool using highly sensitive assays or by visualizing the lumen of the intestine using optical colonoscopy (Helsingen Lise M. & Kalager Mette, 2022). Optical colonoscopy is an established and reliable method for identifying colorectal adenomas and allows them to be immediately removed. However, its main disadvantages are low patient compliance and the need for sedation (Inadomi et al., 2012; Joseph et al., 2012; OECD, 2012; Stock et al., 2011; Use of Colorectal Cancer Screening Tests, 2023).
A promising emerging alternative to optical colonoscopy is computed tomography colonography. This technique has similar diagnostic accuracy to optical colonoscopy (Pickhardt et al., 2003, 2011, 2018), is preferred by patients (Ristvedt et al., 2003), and has better compliance (Moawad et al., 2010). It also does not require sedation and can pick up clinically relevant findings outside the bowel that are invisible to optical colonoscopy (Smyth et al., 2013). On the other hand, CT colonography requires bowel preparation (like optical colonoscopy), exposes the patient to some ionizing radiation, and does not allow for simultaneous polyp resection. Despite these disadvantages, the American College of Radiology recommends CT colonography for screening patients with average or moderate risk of colorectal cancer (Expert Panel on Gastrointestinal Imaging: et al., 2018).
CT colonography images undergo a series of preparation steps before being interpreted. These include preprocessing to remove artifacts, extracting the colon from the rest of the abdominal structures, 3D reconstruction of the colon, and visualization of the colon lumen. A recent study combined a novel colon segmentation and reconstruction method with polyp detection using a CNN (Alkabbany et al., 2022). The automated colon segmentation showed a more than 90 % overlap with manual expert segmentation in 70% of cases and colon polyps were detected with an AUC of 0.93, a sensitivity of 97 %, and a specificity of 79 %.
Differentiating between benign polyps and those with malignant potential is a challenge in both optical colonoscopy and CT colonography and has been the focus of several studies using AI. Radiomics-based approaches for classifying benign versus premalignant polyps on CT colonography have shown AUCs of up to 0.91 but require manual segmentation of the polyps (Grosu et al., 2021; Song et al., 2014). A recent proof-of-concept study used a fully automated approach using CNNs for polyp segmentation and distinguishing between benign and premalignant polyps (Wesp et al., 2022). The authors trained the CNN on data from 63 patients and tested it on an independent dataset of 59 patients, showing an AUC of up to 0.83 and a sensitivity and specificity of up to 80 % and 69 % respectively. Such AI-based approaches can potentially be used as a second reader to help guide the decision on polyp removal.
Hepatocellular carcinoma (HCC) is one of the most common causes of cancer deaths in the world (Sung et al., 2021). Individuals with liver cirrhosis or chronic hepatitis B or C virus infection are at high risk for developing HCC (Vogel et al., 2022). Screening these patients is associated with a reduction in mortality from HCC (Singal et al., 2022; Zhang et al., 2004). Screening is usually performed using abdominal ultrasound every six months (European Association for the Study of the Liver, 2018; Frenette et al., 2019; Marrero et al., 2018) with or without measuring alpha-fetoprotein levels in the blood (Colli et al., 2006; Tzartzeva et al., 2018). Suspicious lesions identified on ultrasound are further characterized using either CT, MRI, or both.
Deep learning techniques have also been extensively applied in liver imaging using B-mode ultrasound showing promising results for detecting and classifying focal liver lesions as benign or malignant.
The pathogenesis of HCC involves a complex interplay between liver nodules that exist in different stages of chronic liver injury. Regenerative nodules form in response to hepatocyte damage and are commonly seen in cirrhotic livers. Genetic mutations can accumulate over time within these regenerative nodules, converting them to dysplastic ones with a high risk of progressing to HCC as more mutations accumulate (Kudo, 2009). Differentiating between dysplastic and malignant nodules using imaging is challenging (Park et al., 2017). Moreover, the imaging features of HCC sometimes overlap with those of other liver lesions, including hemangiomas, simple liver cysts, and focal nodular hyperplasia (Heiken, 2007).
Using a radiomics approach combining perfusion information and texture analysis in contrast-enhanced ultrasound, a study of 72 patients found a balanced accuracy of 0.84 for distinguishing between benign and malignant liver lesions (Turco et al., 2022). Another study using contrast-enhanced ultrasound found a sensitivity of 94.8 % and specificity of 93.6 % for distinguishing between HCC and focal nodular hyperplasia using a support vector machine learning approach (Huang et al., 2020), with other studies finding similar results (Gatos et al., 2015; Kondo et al., 2017). In a multicenter study investigating the differentiation of 11 different types of focal liver lesions using contrast-enhanced ultrasound and histopathology as a reference, support vector machine learning (AUC = 0.883) outperformed an artificial neural network (AUC = 0.829) and both approaches outperformed an experienced radiologist (AUC = 0.702) (Ta et al., 2018).
Deep learning techniques have also been extensively applied in liver imaging using B-mode ultrasound. These studies have shown promising results for detecting (Brehar et al., 2020; Schmauch et al., 2019; Tiyarattanachai et al., 2022) and classifying focal liver lesions as benign or malignant (Schmauch et al., 2019) or classifying them into specific entities (Hassan et al., 2017; Virmani et al., 2014). Using a deep learning approach, one study found that combining information on patient demographics and laboratory results with B-mode ultrasound images improved the AUC for classifying liver lesions as benign versus malignant from 0.721 (using ultrasound alone) to 0.994 (Sato et al., 2022). Another study of 334 patients found that the detection rate of focal liver lesions on B-mode ultrasound using a CNN was higher for HCC than for other focal liver lesions and the CNN outperformed human experts (with an algorithm detection rate of 100 % compared to 39.1 % for non-radiologists and 69.6 % for radiologists) (Tiyarattanachai et al., 2022).
Prostate cancer is the most common cancer in men in Europe and the United States (Ferlay et al., 2018; Siegel et al., 2021) and is the third most common cancer in the world (Sung et al., 2021). In countries where programs exist, screening is usually based on measuring levels of serum prostate-specific antigen (PSA). Serum PSA has high sensitivity but low specificity for prostate cancer (Merriel et al., 2022). Screening based on PSA alone thus leads to many unnecessary biopsies, with up to 75 % of systematic prostate biopsies - those done without targeting a specific location within the prostate, instead taking multiple biopsies from different parts of the gland - being negative (Ahmed et al., 2017). In addition, PSA screening tends to detect lower-risk and slower-growing cancer that is considered clinically insignificant because it does not threaten patient survival (US Preventive Services Task Force et al., 2018; Welch & Albertsen, 2020). Screening based on serum PSA levels followed by a systematic biopsy is thus overall of questionable benefit. Instead, the ideal approach would detect cancer and simultaneously characterize its clinical significance.
A study using a random forest-based classifier to detect suspicious areas on multiparametric prostate MRI was associated with Shorter reading times and improved specificity.
Multiparametric MRI plays an increasingly important role in the workup of screened prostate cancer cases and includes diffusion-weighted and T2-weighted sequences, with or without a T1-weighted dynamic contrast-enhanced sequence (Walker et al., 2020). False positives and the detection of clinically insignificant prostate cancer can be reduced using MRI, which may help reduce overtreatment (Drost et al., 2019). Studies suggest that MRI before biopsy can reduce the number of unnecessary biopsies by a third (Elwenspoek et al., 2019), and this approach has been included in several guidelines on prostate cancer management (Leitlinienprogramm Onkologie: Prostatakarzinom, n.d., Overview | Prostate Cancer: Diagnosis and Management | Guidance | NICE, n.d.; Mottet et al., 2017). MRI can also help direct targeted biopsies in patients with negative systematic prostate biopsies (Hoeks et al., 2012; Hugosson et al., 2022; Penzkofer et al., 2015; Siddiqui et al., 2015; Sonn et al., 2014). In patients found to have very low- or low-risk prostate cancer, MRI can be useful to actively monitor the disease, an approach that is associated with good long-term outcomes (Klotz et al., 2015). Reading prostate MRIs is challenging, however, and even standardized reporting systems have a steep learning curve and diagnostic performance varies greatly between radiologists and institutions (Kohestani et al., 2019; Muller et al., 2015; Rosenkrantz et al., 2017; Smith et al., 2019; Westphalen et al., 2020).
Segmentation of the entire prostate gland allows the determination of the gland’s volume, which is used for calculating the PSA density (a metric that helps differentiate between benign prostatic hypertrophy and prostate cancer) and radiotherapy planning. Manual prostate segmentation by radiologists is, however, time-consuming and prone to errors (Garvey et al., 2014). Automated segmentation of the prostate gland using AI-based tools is feasible and accurate, and several commercial tools are currently available for this purpose (AI for Radiology, n.d.; Bardis et al., 2021; Belue & Turkbey, 2022; Sanford et al., 2020; Sunoqrot et al., 2022; Turkbey & Haider, 2022; Ushinsky et al., 2021; van Leeuwen et al., 2021; B. Wang et al., 2019).
AI-based approaches have also proven useful for the identification and segmentation of prostate cancer on multiparametric MRI. Algorithms generally classify lesions either into two classes (e.g. clinically significant versus clinically insignificant prostate cancer) or multiple classes using the PI-RADS score (Belue & Turkbey, 2022; Twilt et al., 2021). In a multi-reader, multi-center study, using a random forest-based classifier to detect suspicious areas on multiparametric prostate MRI was associated with shorter reading times (2.7 to 4.4 minutes with the algorithm versus 3.5 to 6.3 minutes without the algorithm depending on reader experience) and improved specificity (71.5 % versus 44.8 %) (Gaur et al., 2018).
Several studies using deep learning approaches have achieved AUCs of up to 0.89 for detecting prostate cancer on multiparametric MRI (Arif et al., 2020; Saha et al., 2021). A commercially available deep-learning-based algorithm improved radiologists’ detection of clinically significant prostate cancer (using the consensus of three experienced radiologists as a reference), increased interreader reliability, and reduced median reading time (Winkel et al., 2021). Similar to the situation in breast cancer, diagnostic accuracy is highest when AI-based tools and radiologists’ interpretations are considered together rather than relying on the assessment of one or the other (Cacciamani et al., 2023).
AI has also been used to classify prostate cancer aggressiveness. In an MRI-based radiomics study, a support vector machine classifier was used to segment areas of prostate cancer, followed by texture analysis and quantitative feature extraction (Giannini et al., 2021). In the same study, another support vector machine classifier used the extracted features to classify tumor aggressiveness using histopathological grading as a reference. Trained on 72 patients’ data, the study found an AUC of 0.81 in a validation dataset of 59 patients (positive predictive value = 81 %, negative predictive value = 71 %). In another study of 107 patients’ multiparametric prostate MRIs, radiologists’ PI-RADS classifications were combined with a likelihood score derived from a random forest classifier, and all suspicious regions identified in this way were biopsied (Litjens et al., 2015). Including the algorithm’s score was associated with a higher probability of detecting prostate cancer (AUC = 0.88 with and 0.81 without the algorithm) and of detecting more aggressive cancers (AUC = 0.87 with and 0.78 without the algorithm). In a study of 417 patients a CNN achieved an AUC of 0.81 for classifying clinically significant prostate cancer using multiparametric MRI with only a slightly lower sensitivity compared to highly experienced radiologists (Cao et al., 2019).
Like with many other applications of AI in radiology, the lack of interpretability of deep learning models of prostate MRI hampers and delays their implementation in clinical practice (Aristidou et al., 2022; Reddy et al., 2020; Reyes et al., 2020; Vayena et al., 2018). A study using a CNN on prostate MRI from 1224 patients and histopathology as a reference found an AUC of 0.89 for distinguishing clinically significant prostate cancer from other prostate changes (Hamm et al., 2023). In addition, they included a voxelwise heat map of areas suspicious of clinically significant prostate cancer and PI-RADS-inspired descriptive explanations of how the CNN came to its conclusion. The algorithm was associated with a reduction in reading time from 85 seconds to 47 seconds and an increase in reading confidence in nonexpert readers.
Medical imaging plays a central role in the screening pathways of several of the most common cancers. Reading screening examinations requires considerable skill and experience, and current demand far exceeds the supply of trained radiologists (AAMC Report Reinforces Mounting Physician Shortage, 2021, Clinical Radiology UK Workforce Census 2019 Report, 2019). The use of AI-based tools for cancer screening holds immense promise for mitigating these issues. The benefits of such approaches have included improved identification of individuals eligible for screening, better diagnostic accuracy, reduced reporting times, and improved radiologists’ confidence in their own diagnostic decisions. The most promising results have been found when AI-based systems and radiologists have made decisions on screening examinations together. Collaborative decision-making between AI-based tools and radiologists can thus pave the way for a transformative era in cancer screening.
AAMC Report Reinforces Mounting Physician Shortage. (2021). AAMC. https://www.aamc.org/news-insights/press- releases/aamc-report-reinforces-mounting-physician-shortage
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021). Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis. NPJ Digital Medicine, 4(1), 65. https://doi.org/10.1038/s41746-021-00438-z
Ahmed, H. U., El-Shater Bosaily, A., Brown, L. C., Gabe, R., Kaplan, R., Parmar, M. K., Collaco-Moraes, Y., Ward, K., Hindley, R. G., Freeman, A., Kirkham, A. P., Oldroyd, R., Parker, C., Emberton, M., & PROMIS study group. (2017). Diagnostic accuracy of multi-parametric MRI and TRUS biopsy in prostate cancer (PROMIS): a paired validating confirmatory study. The Lancet, 389(10071), 815–822. https://doi.org/10.1016/ S0140-6736(16)32401-1
AI for radiology. (n.d.). Retrieved June 4, 2023, from https://grand-challenge.org/aiforradiology/?subspeciality=All& modality=All&ce_under=All&ce_%20class=All&fda_ class=All&sort_by=ce+certification&search=prostate
Alkabbany, I., Ali, A. M., Mohamed, M., Elshazly, S. M., & Farag, A. (2022). An AI-Based Colonic Polyp Classifier for Colorectal Cancer Screening Using Low-Dose Abdominal CT. Sensors, 22, (24). https://doi.org/10.3390/s22249761
Al Mohammad, B., Hillis, S. L., Reed, W., Alakhras, M., & Brennan, P. C. (2019). Radiologist performance in the detection of lung cancer using CT. Clinical Radiology, 74(1), 67–75. https://doi.org/10.1016/j.crad.2018.10.008
Ardila, D., Kiraly, A. P., Bharadwaj, S., Choi, B., Reicher, J. J., Peng, L., Tse, D., Etemadi, M., Ye, W., Corrado, G., Naidich, D. P., & Shetty, S. (2019). End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nature Medicine, 25(6), 954–961. https://doi. org/10.1038/s41591-019-0447-x
Arif, M., Schoots, I. G., Castillo Tovar, J., Bangma, C. H., Krestin, G. P., Roobol, M. J., Niessen, W., & Veenland, J. F. (2020). Clinically significant prostate cancer detection and segmentation in low-risk patients using a convolutional neural network on multi-parametric MRI. European Radiology, 30(12), 6582–6592. https://doi.org/10.1007/s00330-020-07008-z
Aristidou, A., Jena, R., & Topol, E. J. (2022). Bridging the chasm between AI and clinical implementation. The Lancet, 399(10325), 620. https://doi.org/10.1016/S0140-6736(22)00235-5
Armato, S. G., 3rd, Roberts, R. Y., Kocherginsky, M., Aberle, D. R., Kazerooni, E. A., Macmahon, H., van Beek, E. J. R., Yankelevitz, D., McLennan, G., McNitt-Gray, M. F., Meyer, C. R., Reeves, A. P., Caligiuri, P., Quint, L. E., Sundaram, B., Croft, B. Y., & Clarke, L. P. (2009). Assessment of radiologist performance in the detection of lung nodules: dependence on the definition of "truth." Academic Radiology, 16(1), 28–38. https://doi.org/10.1016/j.acra.2008.05.022
Banks, E., Reeves, G., Beral, V., Bull, D., Crossley, B., Simmonds, M., Hilton, E., Bailey, S., Barrett, N., Briers, P., English, R., Jackson, A., Kutt, E., Lavelle, J., Rockall, L., Wallis, M. G., Wilson, M., & Patnick, J. (2006). Hormone replacement therapy and false positive recall in the Million Women Study: patterns of use, hormonal constituents and consistency of effect. Breast Cancer Research: BCR, 8(1), R8. https://doi.org/10.1186/bcr1364
Bardis, M., Houshyar, R., Chantaduly, C., Tran-Harding, K., Ushinsky, A., Chahine, C., Rupasinghe, M., Chow, D., & Chang, P. (2021). Segmentation of the Prostate Transition Zone and Peripheral Zone on MR Images with Deep Learning. Radiology. Imaging Cancer, 3(3), e200024. https://doi.org/10.1148/rycan.2021200024
Belue, M. J., & Turkbey, B. (2022). Tasks for artificial intelligence in prostate MRI. European Radiology Experimental, 6(1), 33. https://doi.org/10.1186/s41747-022-00287-9
Boyd, N. F., Guo, H., Martin, L. J., Sun, L., Stone, J., Fishell, E., Jong, R. A., Hislop, G., Chiarelli, A., Minkin, S., & Yaffe, M. J. (2007). Mammographic density and the risk and detection of breast cancer. The New England Journal of Medicine, 356(3), 227–236. https://doi.org/10.1056/NEJMoa062790
Bray, F., Ferlay, J., Soerjomataram, I., Siegel, R. L., Torre, L. A., & Jemal, A.(2018). Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: A Cancer Journal for Clinicians, 68(6), 394–424. https://doi.org/10.3322/caac.21492
Brehar, R., Mitrea, D.-A., Vancea, F., Marita, T., Nedevschi, S., Lupsor-Platon, M., Rotaru, M., & Badea, R. I. (2020). Comparison of Deep-Learning and Conventional Machine- Learning Methods for the Automatic Recognition of the Hepatocellular Carcinoma Areas from Ultrasound Images. Sensors, 20(11). https://doi.org/10.3390/s20113085
Cacciamani, G. E., Sanford, D. I., Chu, T. N., Kaneko, M., De Castro Abreu, A. L., Duddalwar, V., & Gill, I. S. (2023). Is Artificial Intelligence Replacing Our Radiology Stars? Not Yet! European Urology Open Science, 48, 14–16. https://doi. org/10.1016/j.euros.2022.09.024
Cao, R., Mohammadian Bajgiran, A., Afshari Mirak, S., Shakeri, S., Zhong, X., Enzmann, D., Raman, S., & Sung, K. (2019). Joint Prostate Cancer Detection and Gleason Score Prediction in mp-MRI via FocalNet. IEEE Transactions on Medical Imaging, 38 (11), 2496–2506. https://doi.org/10.1109/ TMI.2019.2901928
Castells, X., Molins, E., & Macià, F. (2006). Cumulative false positive recall rate and association with participant related factors in a population based breast cancer screening programme. Journal of Epidemiology and Community Health, 60(4), 316–321. https://doi.org/10.1136/jech.2005.042119
Chamberlin, J., Kocher, M. R., Waltz, J., Snoddy, M., Stringer, N. F. C., Stephenson, J., Sahbaee, P., Sharma, P., Rapaka, S., Schoepf, U. J., Abadia, A. F., Sperl, J., Hoelzer, P., Mercer, M., Somayaji, N., Aquino, G., & Burt, J. R. (2021). Automated detection of lung nodules and coronary artery calcium using artificial intelligence on low-dose CT scans for lung cancer screening: accuracy and prognostic value. BMC Medicine, 19(1), 55. https://doi.org/10.1186/s12916-021-01928-3
Clinical radiology UK workforce census 2019 report. (2019). https://www.rcr.ac.uk/publication/clinical-radiology-uk- workforce-census-2019-report
Colli, A., Fraquelli, M., Casazza, G., Massironi, S., Colucci, A., Conte, D., & Duca, P. (2006). Accuracy of ultrasonography, spiral CT, magnetic resonance, and alpha-fetoprotein in diagnosing hepatocellular carcinoma: a systematic review. The American Journal of Gastroenterology, 101(3), 513–523. https://doi.org/10.1111/j.1572-0241.2006.00467
Dembrower, K., Wåhlin, E., Liu, Y., Salim, M., Smith, K., Lindholm, P., Eklund, M., & Strand, F. (2020). Effect of artificial intelligence-based triaging of breast cancer screening mammograms on cancer detection and radiologist workload: a retrospective simulation study. The Lancet. Digital Health, 2(9), e468–e474. https://doi.org/10.1016/S2589-7500(20)30185-0
Drost, F.-J. H., Osses, D. F., Nieboer, D., Steyerberg, E. W., Bangma, C. H., Roobol, M. J., & Schoots, I. G.(2019). Prostate MRI, with or without MRI-targeted biopsy, and systematic biopsy for detecting prostate cancer. Cochrane Database of Systematic Reviews, 4(4), CD012663. https://doi.org/10.1002/14651858. CD012663.pub2
Elmore, J. G., Jackson, S. L., Abraham, L., Miglioretti, D. L., Carney, P. A., Geller, B. M., Yankaskas, B. C., Kerlikowske, K., Onega, T., Rosenberg, R. D., Sickles, E. A., & Buist, D. S. M. (2009). Variability in interpretive performance at screening mammography and radiologists’ characteristics associated with accuracy. Radiology, 253(3), 641–651. https://doi.org/10.1148/radiol.2533082308
Elwenspoek, M. M. C., Sheppard, A. L., McInnes, M. D. F., Merriel, S. W. D., Rowe, E. W. J., Bryant, R. J., Donovan, J. L., & Whiting, P. (2019). Comparison of Multiparametric Magnetic Resonance Imaging and Targeted Biopsy With Systematic Biopsy Alone for the Diagnosis of Prostate Cancer: A Systematic Review and Meta-analysis. JAMA Network Open, 2(8), e198427. https:// doi.org/10.1001/jamanetworkopen.2019.8427
European Association for the Study of the Liver. (2018). EASL Clinical Practice Guidelines: Management of hepatocellular carcinoma. Journal of Hepatology, 69(1), 182–236. https://doi.org/10.1016/j.jhep.2018.03.019
Existence of national screening program for breast cancer. (n.d.). Retrieved April 2, 2023, from https://www.who.int/data/gho/data/indicators/indicator-details/GHO/existence-ofnational-screening-program-for-breast-cancer
Expert Panel on Gastrointestinal Imaging:, Moreno, C., Kim, D. H., Bartel, T. B., Cash, B. D., Chang, K. J., Feig, B. W., Fowler, K. J., Garcia, E. M., Kambadakone, A. R., Lambert, D. L., Levy, A. D., Marin, D., Peterson, C. M., Scheirey, C. D., Smith, M. P., Weinstein, S., & Carucci, L. R. (2018). ACR Appropriateness Criteria® Colorectal Cancer Screening. Journal of the American College of Radiology: JACR, 15(5S), S56–S68. https://doi.org/10.1016/j.jacr.2018.03.014
Ferlay, J., Colombet, M., Soerjomataram, I., Dyba, T., Randi,G., Bettio, M., Gavin, A., Visser, O., & Bray, F.(2018). Cancer incidence and mortality patterns in Europe: Estimates for 40 countries and 25 major cancers in 2018. European Journal of Cancer, 103, 356–387. https://doi.org/10.1016/j.ejca.2018.07.005
Freeman, K., Geppert, J., Stinton, C., Todkill, D., Johnson, S., Clarke, A., & Taylor-Phillips, S. (2021). Use of artificial intelligence for image analysis in breast cancer screening programmes: systematic review of test accuracy.BMJ, 374, n1872.https://doi.org/10.1136/bmj.n1872
Frenette, C. T., Isaacson, A. J., Bargellini, I., Saab, S., & Singal, A. G. (2019). A Practical Guideline for Hepatocellular Carcinoma Screening in Patients at Risk. Mayo Clinic Proceedings. Innovations, Quality & Outcomes, 3(3), 302–310. https://doi.org/10.1016/j.mayocpiqo.2019.04.005
Garvey, B., Türkbey, B., Truong, H., Bernardo, M., Periaswamy, S., & Choyke, P. L. (2014). Clinical value of prostate segmentation and volume determination on MRI in benign prostatic hyperplasia. Diagnostic and Interventional Radiology, 20(3), 229–233. https://doi.org/10.5152/dir.2014.13322
Gatos, I., Tsantis, S., Spiliopoulos, S., Skouroliakou, A., Theotokas, I., Zoumpoulis, P., Hazle, J. D., & Kagadis, G. C. (2015). A new automated quantification algorithm for the detection and evaluation of focal liver lesions with contrastenhanced ultrasound. Medical Physics, 42(7), 3948–3959. https://doi.org/10.1118/1.4921753
Gaur, S., Lay, N., Harmon, S. A., Doddakashi, S., Mehralivand, S., Argun, B., Barrett, T., Bednarova, S., Girometti, R., Karaarslan, E., Kural, A. R., Oto, A., Purysko, A. S., Antic, T., Magi-Galluzzi, C., Saglican, Y., Sioletic, S., Warren, A. Y., Bittencourt, L., … Turkbey, B. (2018). Can computeraided diagnosis assist in the identification of prostate cancer on prostate MRI? a multi-center, multi-reader investigation. Oncotarget, 9(73), 33804–33817. https://doi.org/10.18632/ oncotarget.26100
Giannini, V., Mazzetti, S., Defeudis, A., Stranieri, G., Calandri, M., Bollito, E., Bosco, M., Porpiglia, F., Manfredi, M., De Pascale, A., Veltri, A., Russo, F., & Regge, D. (2021). A Fully Automatic Artificial Intelligence System Able to Detect and Characterize Prostate Cancer Using Multiparametric MRI: Multicenter and Multi-Scanner Validation. Frontiers in Oncology, 11, 718155. https://doi.org/10.3389/fonc.2021.718155
Gierada, D. S., Pinsky, P. F., Duan, F., Garg, K., Hart, E. M., Kazerooni, E. A., Nath, H., Watts, J. R., Jr, & Aberle, D. R. (2017). Interval lung cancer after a negative CT screening examination: CT findings and outcomes in National Lung Screening Trial participants. European Radiology, 27(8), 3249–3256. https://doi.org/10.1007/s00330-016-4705-8
Giordano, L., von Karsa, L., Tomatis, M., Majek, O., de Wolf, C., Lancucki, L., Hofvind, S., Nyström, L., Segnan, N., Ponti, A., Eunice Working Group, Van Hal, G., Martens, P., Májek, O., Danes, J., von Euler-Chelpin, M., Aasmaa, A., Anttila, A., Becker, N., … Suonio, E. (2012). Mammographic screening programmes in Europe: organization, coverage and participation. Journal of Medical Screening, 19 Suppl 1, 72–82. https://doi.org/10.1258/jms.2012.012085
Grosu, S., Wesp, P., Graser, A., Maurus, S., Schulz, C., Knösel, T., Cyran, C. C., Ricke, J., Ingrisch, M., & Kazmierczak, P. M. (2021). Machine Learning-based Differentiation of Benign and Premalignant Colorectal Polyps Detected with CT Colonography in an Asymptomatic Screening Population: A Proof-of-Concept Study. Radiology, 299(2), 326–335. https://doi.org/10.1148/radiol.2021202363
Hamm, C. A., Baumgärtner, G. L., Biessmann, F., Beetz, N. L., Hartenstein, A., Savic, L. J., Froböse, K., Dräger, F., Schallenberg, S., Rudolph, M., Baur, A. D. J., Hamm, B., Haas, M., Hofbauer, S., Cash, H., & Penzkofer, T.(2023). Interactive Explainable Deep Learning Model Informs Prostate Cancer Diagnosis at MRI. Radiology, 307(4), e222276. https://doi.org/10.1148/radiol.222276
Hassan, T. M., Elmogy, M., & Sallam, E.-S. (2017). Diagnosis of Focal Liver Diseases Based on Deep Learning Technique for Ultrasound Images. Arabian Journal for Science and Engineering, 42(8), 3127–3140. https://doi.org/10.1007/s13369-016-2387-9
Heiken, J. P. (2007). Distinguishing benign from malignant liver tumours. Cancer Imaging: The Official Publication of the International Cancer Imaging Society, 7 Spec No A(Special issue A), S1–S14. https://doi.org/10.1102/1470-7330.2007.9084
Helsingen Lise M., & Kalager Mette. (2022). Colorectal Cancer Screening — Approach, Evidence, and Future Directions. NEJM Evidence, 1(1), EVIDra2100035. https://doi.org/10.1056/ EVIDra2100035
Hoeks, C. M. A., Schouten, M. G., Bomers, J. G. R., Hoogendoorn, S. P., Hulsbergen-van de Kaa, C. A., Hambrock, T., Vergunst, H., Sedelaar, J. P. M., Fütterer, J. J., & Barentsz, J. O. (2012). Three-Tesla magnetic resonance-guided prostate biopsy in men with increased prostate-specific antigen and repeated, negative, random, systematic, transrectal ultrasound biopsies: detection of clinically significant prostate cancers. European Urology, 62(5), 902–909. https://doi.org/10.1016/j. eururo.2012.01.047
Huang, Q., Pan, F., Li, W., Yuan, F., Hu, H., Huang, J., Yu, J., & Wang, W. (2020). Differential Diagnosis of Atypical Hepatocellular Carcinoma in Contrast-Enhanced Ultrasound Using Spatio-Temporal Diagnostic Semantics. IEEE Journal of Biomedical and Health Informatics, 24(10), 2860–2869. https://doi.org/10.1109/JBHI.2020.2977937
Hugosson, J., Månsson, M., Wallström, J., Axcrona, U., Carlsson, S. V., Egevad, L., Geterud, K., Khatami, A., Kohestani, K., Pihl, C.-G., Socratous, A., Stranne, J., Godtman, R. A., Hellström, M., & GÖTEBORG-2 Trial Investigators. (2022). Prostate Cancer Screening with PSA and MRI Followed by Targeted Biopsy Only.The New England Journal of Medicine, 387(23), 2126–2137. https://doi.org/10.1056/NEJMoa2209454
Inadomi, J. M., Vijan, S., Janz, N. K., Fagerlin, A., Thomas, J. P., Lin, Y. V., Muñoz, R., Lau, C., Somsouk, M., El-Nachef, N., & Hayward, R. A. (2012). Adherence to colorectal cancer screening: a randomized clinical trial of competing strategies. Archives of Internal Medicine, 172(7), 575–582. https://doi. org/10.1001/archinternmed. 2012.332
Joseph, D. A., Jessica B. King, M. P. H., Miller, J. W., & Richardson, L. C. (2012, June 15).Prevalence of Colorectal Cancer Screening Among Adults — Behavioral Risk Factor Surveillance System, United States, 2010. https://www.cdc.gov/MMWr/preview/mmwrhtml/su6102a9.htm
Katase, S., Ichinose, A., Hayashi, M., Watanabe, M., Chin,K., Takeshita, Y., Shiga, H., Tateishi, H., Onozawa, S., Shirakawa, Y., Yamashita, K., Shudo, J., Nakamura, K., Nakanishi, A., Kuroki, K., & Yokoyama, K. (2022). Development and performance evaluation of a deep learning lung nodule detection system. BMC Medical Imaging. 22(1), 203. https://doi.org/10.1186/s12880-022-00938-8
Kinsinger, L. S., Anderson, C., Kim, J., Larson, M., Chan, S. H., King, H. A., Rice, K. L., Slatore, C. G., Tanner, N. T., Pittman, K., Monte, R. J., McNeil, R. B., Grubber, J. M., Kelley, M. J., Provenzale, D., Datta, S. K., Sperber, N. S., Barnes, L. K., Abbott, D. H., … Jackson, G. L. (2017). Implementation of Lung Cancer Screening in the Veterans Health Administration. JAMA Internal Medicine, 177(3), 399–406. https://doi.org/10.1001/jamainternmed.2016.9022
Klotz, L., Vesprini, D., Sethukavalan, P., Jethava, V., Zhang, L., Jain, S., Yamamoto, T., Mamedov, A., & Loblaw, A. (2015). Long-term follow-up of a large active surveillance cohort of patients with prostate cancer.Journal of Clinical Oncology: Official Journal of the American Society of Clinical Oncology, 33(3), 272–277.https://doi.org/10.1200/JCO.2014.55.1192
Kohestani, K., Wallström, J., Dehlfors, N., Sponga, O. M., Månsson, M., Josefsson, A., Carlsson, S., Hellström, M., & Hugosson, J. (2019). Performance and inter-observer variability of prostate MRI (PI-RADS version 2) outside high-volume centres. Scandinavian Journal of Urology, 53(5), 304–311. https://doi.org/10.1080/21681805.2019.1675757
Kondo, S., Takagi, K., Nishida, M., Iwai, T., Kudo, Y., Ogawa, K., Kamiyama, T., Shibuya, H., Kahata, K., & Shimizu, C. (2017). Computer-Aided Diagnosis of Focal Liver Lesions Using Contrast-Enhanced Ultrasonography With Perflubutane Microbubbles. IEEE Transactions on Medical Imaging, 36(7),1427–1437. https://doi.org/10.1109/TMI.2017.2659734
Kramer, B. S. (2004). The science of early detection. Urologic Oncology, 22(4), 344–347. https://doi.org/10.1016/j.urolonc.2003.04.001
Kudo, M. (2009). Multistep human hepatocarcinogenesis: correlation of imaging with pathology. Journal of Gastroenterology, 44 Suppl 19, 112–118. https://doi.org/10.1007/s00535-008-2274-6
Kuipers, E. J., Grady, W. M., Lieberman, D., Seufferlein, T., Sung, J. J., Boelens, P. G., van de Velde, C. J. H., & Watanabe, T. (2015) Colorectal cancer.Nature Reviews. Disease Primers, 1, 15065. https://doi.org/10.1038/nrdp.2015.65
Lam, D. L., Pandharipande, P. V., Lee, J. M., Lehman, C. D., & Lee, C. I. (2014). Imaging-based screening: understanding the controversies.AJR. American Journal of Roentgenology, 203(5), 952–956. https://doi.org/10.2214/AJR.14.13049
Lauritzen, A. D., Rodríguez-Ruiz, A., von Euler-Chelpin, M. C., Lynge, E., Vejborg, I., Nielsen, M., Karssemeijer, N., & Lillholm, M. (2022). An Artificial Intelligence-based Mammography Screening Protocol for Breast Cancer: Outcome and Radiologist Workload. Radiology, 304(1), 41–49. https://doi.org/10.1148/radiol.210948
Leader, J. K., Warfel, T. E., Fuhrman, C. R., Golla, S. K., Weissfeld, J. L., Avila, R. S., Turner, W. D., & Zheng, B. (2005). Pulmonary nodule detection with low-dose CT of the lung: agreement among radiologists. AJR. American Journal of Roentgenology, 185(4), 973–978. https://doi.org/10.2214/ AJR.04.1225
Lehman, C. D., Wellman, R. D., Buist, D. S. M., Kerlikowske, K., Tosteson, A. N. A., Miglioretti, D. L., & Breast Cancer Surveillance Consortium. (2015). Diagnostic Accuracy of Digital Screening Mammography With and Without Computer-Aided Detection. JAMA Internal Medicine, 175(11), 1828–1837. https://doi.org/10.1001/jamainternmed.2015.5231
Leibig, C., Brehmer, M., Bunk, S., Byng, D., Pinker, K., & Umutlu, L. (2022). Combining the strengths of radiologists and AI for breast cancer screening: a retrospective analysis. The Lancet. Digital Health, 4(7), e507–e519. https://doi.org/10.1016/ S2589-7500(22)00070-X
Leitlinienprogramm Onkologie: Prostatakarzinom. (n.d.). Retrieved July 8, 2023, from https://www.leitlinienprogrammonkologie. de/index.php?id=58&type=0
Litjens, G. J. S., Barentsz, J. O., Karssemeijer, N., & Huisman, H. J. (2015). Clinical evaluation of a computer-aided diagnosis system for determining cancer aggressiveness in prostate MRI. European Radiology, 25(11), 3187–3199. https://doi.org/10.1007/ s00330-015-3743-y
Lu, M. T., Raghu, V. K., Mayrhofer, T., Aerts, H. J. W. L., & Hoffmann, U. (2020). Deep Learning Using Chest Radiographs to Identify High-Risk Smokers for Lung Cancer Screening Computed Tomography: Development and Validation of a Prediction Model. Annals of Internal Medicine, 173(9), 704–713. https://doi.org/10.7326/M20-1868
Lung cancer: Screening. (2021, March 9). US Preventive Services Taskforce. https://www.uspreventiveservicestaskforce.org/uspstf/recommendation/lung-cancer-screening
Marmot, M. G., Altman, D. G., Cameron, D. A., Dewar, J. A., Thompson, S. G., & Wilcox, M. (2013). The benefits and harms of breast cancer screening: an independent review. British Journal of Cancer, 108(11), 2205–2240. https://doi.org/10.1038/ bjc.2013.177
Marrero, J. A., Kulik, L. M., Sirlin, C. B., Zhu, A. X., Finn, R. S., Abecassis, M. M., Roberts, L. R., & Heimbach, J. K. (2018). Diagnosis, Staging, and Management of Hepatocellular Carcinoma: 2018 Practice Guidance by the American Association for the Study of Liver Diseases. Hepatology, 68(2), 723–750. https://doi.org/10.1002/hep.29913
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., Etemadi, M., Garcia-Vicente, F., Gilbert, F. J., Halling-Brown, M., Hassabis, D., Jansen, S., Karthikesalingam, A., Kelly, C. J., King, D., … Shetty, S. (2020). International evaluation of an AI system for breast cancer screening. Nature, 577(7788), 89–94. https://doi.org/10.1038/ s41586-019-1799-6
Merriel, S. W. D., Pocock, L., Gilbert, E., Creavin, S., Walter, F. M., Spencer, A., & Hamilton, W. (2022). Systematic review and meta-analysis of the diagnostic accuracy of prostatespecific antigen (PSA) for the detection of prostate cancer in symptomatic patients. BMC Medicine, 20(1), 54. https://doi.org/10.1186/s12916-021-02230-y
Mikhael, P. G., Wohlwend, J., Yala, A., Karstens, L., Xiang, J., Takigami, A. K., Bourgouin, P. P., Chan, P., Mrah, S., Amayri, W., Juan, Y.-H., Yang, C.-T., Wan, Y.-L., Lin, G., Sequist, L. V., Fintelmann, F. J., & Barzilay, R. (2023). Sybil: A Validated Deep Learning Model to Predict Future Lung Cancer Risk From a Single Low-Dose Chest Computed Tomography.Journal of Clinical Oncology: Official Journal of the American Society of Clinical Oncology, 41(12), 2191–2200. https://doi.org/10.1200/ JCO.22.01345
Moawad, F. J., Maydonovitch, C. L., Cullen, P. A., Barlow, D.S., Jenson, D. W., & Cash, B. D. 2010). CT colonography may improve colorectal cancer screening compliance. AJR. American Journal of Roentgenology, 195(5), 1118–1123. https://doi.org/10.2214/AJR.10.4921
Moran, S., & Warren-Forward, H. (2012). The Australian BreastScreen workforce: a snapshot. The Radiographer, 59(1), 26–30. https://doi.org/10.1002/j.2051-3909.2012.tb00169.x
Mottet, N., Bellmunt, J., Bolla, M., Briers, E., Cumberbatch, M. G., De Santis, M., Fossati, N., Gross, T., Henry, A. M., Joniau, S., Lam, T. B., Mason, M. D., Matveev, V. B., Moldovan, P. C., van den Bergh, R. C. N., Van den Broeck, T., van der Poel, H. G., van der Kwast, T. H., Rouvière, O., … Cornford, P. (2017). EAU-ESTRO-SIOG Guidelines on Prostate Cancer. Part 1: Screening, Diagnosis, and Local Treatment with Curative Intent. European Urology 71(4), 618–629. https://doi.org/10.1016/j. eururo.2016.08.003
Muller, B. G., Shih, J. H., Sankineni, S., Marko, J., Rais-Bahrami, S., George, A. K., de la Rosette, J. J. M. C. H., Merino, M. J., Wood, B. J., Pinto, P., Choyke, P. L., & Turkbey, B. (2015). Prostate Cancer: Interobserver Agreement and Accuracy with the Revised Prostate Imaging Reporting and Data System at Multiparametric MR Imaging. Radiology, 277(3), 741–750. https://doi.org/10.1148/radiol.2015142818
National Lung Screening Trial Research Team, Aberle, D. R., Adams, A. M., Berg, C. D., Black, W. C., Clapp, J. D., Fagerstrom, R. M., Gareen, I. F., Gatsonis, C., Marcus, P.M., & Sicks, J. D. (2011). Reduced lung-cancer mortality with low-dose computed tomographic screening. The New England Journal of Medicine, 365(5), 395–409. https://doi.org/10.1056/ NEJMoa1102873
NHS England. (2022). Targeted screening for lung cancer with low radiation dose computed tomography Standard protocol prepared for the Targeted Lung Health Checks Programme. https://www.england.nhs.uk/publication/targeted-screening-for-lung-cancer
OECD. (2012). Screening, survival and mortality for colorectal cancer.In Health at a Glance: Europe 2012 (pp. 110–111). OECD. https://doi.org/10.1787/9789264183896-48-en
Overview | Prostate cancer: diagnosis and management | Guidance | NICE. (n.d.). Retrieved July 8, 2023, from https://www.nice.org.uk/guidance/ng131
Park, H. J., Choi, B. I., Lee, E. S., Park, S. B., & Lee, J. B. (2017). How to Differentiate Borderline Hepatic Nodules in Hepatocarcinogenesis: Emphasis on Imaging Diagnosis. Liver Cancer, 6(3), 189–203. https://doi.org/10.1159/000455949
Penzkofer, T., Tuncali, K., Fedorov, A., Song, S.-E., Tokuda, J., Fennessy, F. M., Vangel, M. G., Kibel, A. S., Mulkern, R. V., Wells, W. M., Hata, N., & Tempany, C. M. C. (2015). Transperineal in-bore 3-T MR imaging-guided prostate biopsy: a prospective clinical observational study.Radiology, 274(1), 170–180. https://doi.org/10.1148/radiol.14140221
Pickhardt, P. J., Choi, J. R., Hwang, I., Butler, J. A., Puckett, M.L., Hildebrandt, H. A., Wong, R. K., Nugent, P. A., Mysliwiec, P. A., & Schindler, W. R. (2003). Computed tomographic virtual colonoscopy to screen for colorectal neoplasia in asymptomatic adults.The New England Journal of Medicine 349(23), 2191–2200. https://doi.org/10.1056/NEJMoa031618
Pickhardt, P. J., Correale, L., Delsanto, S., Regge, D., & Hassan, C. (2018). CT Colonography Performance for the Detection of Polyps and Cancer in Adults ≥ 65 Years Old: Systematic Review and Meta-Analysis.AJR. American Journal of Roentgenology, 211(1), 40–51. https://doi.org/10.2214/ AJR.18.19515
Pickhardt, P. J., Hassan, C., Halligan, S., & Marmo, R. (2011). Colorectal cancer: CT colonography and colonoscopy for detection-systematic review and meta-analysis.Radiology, 259(2), 393–405. https://doi.org/10.1148/radiol.11101887
Raya-Povedano, J. L., Romero-Martín, S., Elías-Cabot, E., Gubern-Mérida, A., Rodríguez-Ruiz, A., & Álvarez-Benito, M. (2021). AI-based Strategies to Reduce Workload in Breast Cancer Screening with Mammography and Tomosynthesis: A Retrospective Evaluation. Radiology, 300(1), 57–65. https://doi. org/10.1148/radiol.2021203555
Reddy, S., Allan, S., Coghlan, S., & Cooper, P. (2020). A governance model for the application of AI in health care. Journal of the American Medical Informatics Association: JAMIA, 27(3), 491–497. https://doi.org/10.1093/jamia/ocz192
Ren, W., Chen, M., Qiao, Y., & Zhao, F. (2022). Global guidelines for breast cancer screening: A systematic review.Breast, 64,85–99. https://doi.org/10.1016/j.breast.2022.04.003
Reyes, M., Meier, R., Pereira, S., Silva, C. A., Dahlweid, F.-M., von Tengg-Kobligk, H., Summers, R. M., & Wiest, R. (2020). On the Interpretability of Artificial Intelligence in Radiology: Challenges and Opportunities. Radiology.Artificial Intelligence, 2(3), e190043. https://doi.org/10.1148/ryai.2020190043
Rimmer, A. (2017). Radiologist shortage leaves patient care at risk, warns royal college.BMJ, 359, j4683. https://doi.org/10.1136/bmj.j4683
Ristvedt, S. L., McFarland, E. G., Weinstock, L. B., & Thyssen, E. P. (2003). Patient preferences for CT colonography, conventional colonoscopy, and bowel preparation. The American Journal of Gastroenterology, 98(3), 578–585.https://doi.org/10.1016/S0002-9270(02)06024-0
Rosenkrantz, A. B., Ayoola, A., Hoffman, D., Khasgiwala, A., Prabhu, V., Smereka, P., Somberg, M., & Taneja, S. S. (2017). The Learning Curve in Prostate MRI Interpretation: Self-Directed Learning Versus Continual Reader Feedback. AJR. American Journal of Roentgenology, 208(3), W92–W100. https://doi.org/10.2214/AJR.16.16876
Saha, A., Hosseinzadeh, M., & Huisman, H. (2021). End-toend prostate cancer detection in bpMRI via 3D CNNs: Effects of attention mechanisms, clinical priori and decoupled false positive reduction. Medical Image Analysis, 73, 102155. https://doi.org/10.1016/j.media.2021.102155
Sanford, T. H., Zhang, L., Harmon, S. A., Sackett, J., Yang, D., Roth, H., Xu, Z., Kesani, D., Mehralivand, S., Baroni, R.H., Barrett, T., Girometti, R., Oto, A., Purysko, A. S., Xu, S., Pinto, P. A., Xu, D., Wood, B. J., Choyke, P. L., & Turkbey, B. (2020). Data Augmentation and Transfer Learning to Improve Generalizability of an Automated Prostate Segmentation Model. AJR. American Journal of Roentgenology 215(6), 1403–1410. https://doi.org/10.2214/AJR.19.22347
Sato, M., Kobayashi, T., Soroida, Y., Tanaka, T., Nakatsuka,T., Nakagawa, H., Nakamura, A., Kurihara, M., Endo, M., Hikita, H., Sato, M., Gotoh, H., Iwai, T., Tateishi, R., Koike,K., & Yatomi, Y. (2022). Development of novel deep multimodal representation learning-based model for the differentiation of liver tumors on B-mode ultrasound images. Journal of Gastroenterology and Hepatology, 37(4), 678–684, https://doi.org/10.1111/jgh.15763
Schmauch, B., Herent, P., Jehanno, P., Dehaene, O., Saillard, C., Aubé, C., Luciani, A., Lassau, N., & Jégou, S. (2019). Diagnosis of focal liver lesions from ultrasound using deep learning.Diagnostic and Interventional Imaging, 100(4), 227–233. https://doi.org/10.1016/j.diii.2019.02.009
Siddiqui, M. M., Rais-Bahrami, S., Turkbey, B., George, A. K., Rothwax, J., Shakir, N., Okoro, C., Raskolnikov, D., Parnes, H.L., Linehan, W. M., Merino, M. J., Simon, R. M., Choyke, P. L., Wood, B. J., & Pinto, P. A. (2015). Comparison of MR/ultrasound fusion-guided biopsy with ultrasound-guided biopsy for the diagnosis of prostate cancer. JAMA: The Journal of the American Medical Association, 313(4), 390–397. https://doi.org/10.1001/jama.2014.17942
Siegel, R. L., Miller, K. D., Fuchs, H. E., & Jemal, A. (2021). Cancer Statistics, 2021. CA: A Cancer Journal for Clinicians, 71(1), 7–33. https://doi.org/10.3322/caac.21654
Singal, A. G., Zhang, E., Narasimman, M., Rich, N. E., Waljee, A. K., Hoshida, Y., Yang, J. D., Reig, M., Cabibbo, G., Nahon, P., Parikh, N. D., & Marrero, J. A. (2022). HCC surveillance improves early detection, curative treatment receipt, and survival in patients with cirrhosis: A meta-analysis Journal of Hepatology, 77(1), 128–139. https://doi.org/10.1016/j.jhep.2022.01.023
Smith, C. P., Harmon, S. A., Barrett, T., Bittencourt, L. K., Law, Y. M., Shebel, H., An, J. Y., Czarniecki, M., Mehralivand, S.,Coskun, M., Wood, B. J., Pinto, P. A., Shih, J. H., Choyke, P. L., & Turkbey, B. (2019). Intra- and interreader reproducibility of PI-RADSv2: A multireader study. Journal of Magnetic Resonance Imaging: JMRI, 49(6), 1694–1703. https://doi.org/10.1002/jmri.26555
Smyth, A. E., Healy, C. F., MacMathuna, P., & Fenlon, H. M. (2013). REVIEW OF CT COLONOGRAPHY: REAL-LIFE EXPERIENCE OF ONE THOUSAND CASES IN A TERTIARY REFERRAL CENTRE. Gut, 62(Suppl 2), A15–A15. https://doi.org/10.1136/gutjnl-2013-305143.36
Song, B., Zhang, G., Lu, H., Wang, H., Zhu, W., J Pickhardt, P., & Liang, Z. (2014). Volumetric texture features from higherorder images for diagnosis of colon lesions via CT colonography. International Journal of Computer Assisted Radiology and Surgery, 9(6), 1021–1031. https://doi.org/10.1007/s11548-014-0991-2
Sonn, G. A., Chang, E., Natarajan, S., Margolis, D. J., Macairan, M., Lieu, P., Huang, J., Dorey, F. J., Reiter, R.E., & Marks, L. S. (2014). Value of targeted prostate biopsy using magnetic resonance-ultrasound fusion in men with prior negative biopsy and elevated prostate-specific antigen. European Urology, 65(4), 809–815. https://doi.org/10.1016/j. eururo.2013.03.025
Stock, C., Ihle, P., Schubert, I., & Brenner, H. (2011). Colonoscopy and fecal occult blood test use in Germany: results from a large insurance-based cohort. Endoscopy, 43(9), 771–781. https://doi.org/10.1055/s-0030-1256504
Sung, H., Ferlay, J., Siegel, R. L., Laversanne, M., Soerjomataram, I., Jemal, A., & Bray, F. (2021). Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA: A Cancer Journal for Clinicians, 71(3), 209–249. https://doi.org/10.3322/ caac.21660
Sunoqrot, M. R. S., Saha, A., Hosseinzadeh, M., Elschot, M., & Huisman, H. (2022). Artificial intelligence for prostate MRI: open datasets, available applications, and grand challenges.European Radiology Experimental, 6(1), 35. https://doi.org/10.1186/s41747-022-00288-8
Tabár, L., Vitak, B., Chen, T. H.-H., Yen, A. M.-F., Cohen, A., Tot, T., Chiu, S. Y.-H., Chen, S. L.-S., Fann, J. C.-Y., Rosell, J., Fohlin, H., Smith, R. A., & Duffy, S. W. (2011). Swedish two-county trial: impact of mammographic screening on breast cancer mortality during 3 decades. Radiology, 260(3), 658–663. https://doi.org/10.1148/radiol.11110469
Ta, C. N., Kono, Y., Eghtedari, M., Oh, Y. T., Robbin, M. L., Barr, R. G., Kummel, A. C., & Mattrey, R. F. (2018). Focal Liver Lesions: Computer-aided Diagnosis by Using Contrast-enhanced US Cine Recordings. Radiology, 286(3), 1062–1071. https://doi.org/10.1148/radiol.2017170365
The burden of cancer. (n.d.). The Cancer Atlas. Retrieved July 8, 2023, from https://canceratlas.cancer.org/the-burden/theburden-of-cancer/
Tiyarattanachai, T., Apiparakoon, T., Marukatat, S., Sukcharoen, S., Yimsawad, S., Chaichuen, O., Bhumiwat, S., Tanpowpong, N., Pinjaroen, N., Rerknimitr, R., & Chaiteerakij, R. (2022). The feasibility to use artificial intelligence to aid detecting focal liver lesions in real-time ultrasound: a preliminary study based on videos. Scientific Reports 12(1), 7749. https://doi.org/10.1038/s41598-022-11506-z
Turco, S., Tiyarattanachai, T., Ebrahimkheil, K., Eisenbrey, J., Kamaya, A., Mischi, M., Lyshchik, A., & Kaffas, A. E. (2022). Interpretable Machine Learning for Characterization of Focal Liver Lesions by Contrast-Enhanced Ultrasound. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 69(5), 1670–1681. https://doi.org/10.1109/TUFFC.2022.3161719
Turkbey, B., & Haider, M. A. (2022). Deep learning-based artificial intelligence applications in prostate MRI: brief summary. The British Journal of Radiology, 95(1131), 20210563. https://doi.org/10.1259/bjr.20210563
Twilt, J. J., van Leeuwen, K. G., Huisman, H. J., Fütterer, J. J., & de Rooij, M. (2021). Artificial Intelligence Based Algorithms for Prostate Cancer Classification and Detection on Magnetic Resonance Imaging: A Narrative Review.Diagnostics (Basel, Switzerland), 11(6). https://doi.org/10.3390/diagnostics11060959
Tzartzeva, K., Obi, J., Rich, N. E., Parikh, N. D., Marrero, J.A., Yopp, A., Waljee, A. K., & Singal, A. G. (2018). Surveillance Imaging and Alpha Fetoprotein for Early Detection of Hepatocellular Carcinoma in Patients With Cirrhosis: A Metaanalysis. Gastroenterology, 154(6), 1706–1718.e1. https://doi.org/10.1053/j.gastro.2018.01.064
Use of colorectal cancer screening tests. (2023, March 31). https://www.cdc.gov/cancer/colorectal/statistics/use-screeningtests-BRFSS.htm
Ushinsky, A., Bardis, M., Glavis-Bloom, J., Uchio, E., Chantaduly, C., Nguyentat, M., Chow, D., Chang, P. D., & Houshyar, R. (2021). A 3D-2D Hybrid U-Net Convolutional Neural Network Approach to Prostate Organ Segmentation of Multiparametric MRI.AJR. American Journal of Roentgenology, 216(1), 111–116. https://doi.org/10.2214/AJR.19.22168
US Preventive Services Task Force, Grossman, D. C., Curry, S. J., Owens, D. K., Bibbins-Domingo, K., Caughey, A. B., Davidson, K. W., Doubeni, C. A., Ebell, M., Epling, J. W., Jr, Kemper, A. R., Krist, A. H., Kubik, M., Landefeld, C. S., Mangione, C. M., Silverstein, M., Simon, M. A., Siu, A. L., & Tseng, C.-W. (2018). Screening for Prostate Cancer: US Preventive Services Task Force Recommendation Statement. JAMA: The Journal of the American Medical Association, 319(18), 1901–1913. https://doi.org/10.1001/jama.2018.3710
van Leeuwen, K. G., Schalekamp, S., Rutten, M. J. C. M., van Ginneken, B., & de Rooij, M. (2021). Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. European Radiology, 31(6), 3797–3804. https://doi.org/10.1007/s00330-021-07892-z
Vayena, E., Blasimme, A., & Cohen, I. G. (2018). Machine learning in medicine: Addressing ethical challenges.PLoS Medicine, 15(11), e1002689. https://doi.org/10.1371/journal. pmed.1002689
Virmani, J., Kumar, V., Kalra, N., & Khandelwal, N. (2014).Neural network ensemble based CAD system for focal liver lesions from B-mode ultrasound. Journal of Digital Imaging, 27(4), 520–537. https://doi.org/10.1007/s10278-014-9685-0
Vogel, A., Meyer, T., Sapisochin, G., Salem, R., & Saborowski, A. (2022). Hepatocellular carcinoma.The Lancet, 400(10360), 1345–1362. https://doi.org/10.1016/S0140-6736(22)01200-4
Walker, S. M., Choyke, P. L., & Turkbey, B. (2020). What You Need to Know Before Reading Multiparametric MRI for Prostate Cancer. AJR. American Journal of Roentgenology, 214(6), 1211–1219. https://doi.org/10.2214/AJR.19.22751
Wang, B., Lei, Y., Tian, S., Wang, T., Liu, Y., Patel, P., Jani, A.B., Mao, H., Curran, W. J., Liu, T., & Yang, X. (2019). Deeply supervised 3D fully convolutional networks with group dilated convolution for automatic MRI prostate segmentation. Medical Physics, 46(4), 1707–1718. https://doi.org/10.1002/mp.13416
Wang, Y., Midthun, D. E., Wampfler, J. A., Deng, B., Stoddard, S. M., Zhang, S., & Yang, P. (2015). Trends in the proportion of patients with lung cancer meeting screening criteria. JAMA: The Journal of the American Medical Association, 313(8), 853–855. https://doi.org/10.1001/jama.2015.413
Welch, H. G., & Albertsen, P. C. (2020). Reconsidering Prostate Cancer Mortality - The Future of PSA Screening. The New England Journal of Medicine, 382(16), 1557–1563. https://doi.org/10.1056/NEJMms1914228
Wesp, P., Grosu, S., Graser, A., Maurus, S., Schulz, C., Knösel, T., Fabritius, M. P., Schachtner, B., Yeh, B. M., Cyran, C. C., Ricke, J., Kazmierczak, P. M., & Ingrisch, M. (2022). Deep learning in CT colonography: differentiating premalignant from benign colorectal polyps. European Radiology, 32(7), 4749–4759. https://doi.org/10.1007/s00330-021-08532-2
Westphalen, A. C., McCulloch, C. E., Anaokar, J. M., Arora, S., Barashi, N. S., Barentsz, J. O., Bathala, T. K., Bittencourt, L. K., Booker, M. T., Braxton, V. G., Carroll, P. R., Casalino, D. D., Chang, S. D., Coakley, F. V., Dhatt, R., Eberhardt, S. C., Foster, B. R., Froemming, A. T., Fütterer, J. J., … Rosenkrantz, A. B. (2020). Variability of the Positive Predictive Value of PI-RADS for Prostate MRI across 26 Centers: Experience of the Society of Abdominal Radiology Prostate Cancer Diseasefocused Panel. Radiology, 296(1), 76–84. https://doi.org/10.1148/radiol. 2020190646
Wing, P., & Langelier, M. H. (2009). Workforce shortages in breast imaging: impact on mammography utilization.AJR. American Journal of Roentgenology, 192(2), 370–378. https://doi.org/10.2214/AJR.08.1665
Winkel, D. J., Tong, A., Lou, B., Kamen, A., Comaniciu, D., Disselhorst, J. A., Rodríguez-Ruiz, A., Huisman, H., Szolar, D., Shabunin, I., Choi, M. H., Xing, P., Penzkofer, T., Grimm, R., von Busch, H., & Boll, D. T. (2021). A Novel Deep Learning Based Computer-Aided Diagnosis System Improves the Accuracy and Efficiency of Radiologists in Reading Biparametric Magnetic Resonance Images of the Prostate: Results of a Multireader, Multicase Study. Investigative Radiology, 56(10), 605–613. https://doi.org/10.1097/RLI.0000000000000780
World Health Organization. Regional Office for Europe. (2022). A short guide to cancer screening: increase effectiveness, maximize benefits and minimize harm. World Health Organization. Regional Office for Europe. https://apps.who.int/iris/handle/10665/351396
Zauber, A. G., Winawer, S. J., O’Brien, M. J., Lansdorp-Vogelaar, I., van Ballegooijen, M., Hankey, B. F., Shi, W., Bond, J. H., Schapiro, M., Panish, J. F., Stewart, E. T., & Waye, J. D. (2012). Colonoscopic polypectomy and long-term prevention of colorectal-cancer deaths. The New England Journal of Medicine, 366(8), 687–696. https://doi.org/10.1056/NEJMoa1100370
Zhang, B.-H., Yang, B.-H., & Tang, Z.-Y. (2004). Randomized controlled trial of screening for hepatocellular carcinoma. Journal of Cancer Research and Clinical Oncology, 130(7), 417–422.https://doi.org/10.1007/s00432-004-0552-0
Guide to Artificial Intelligence in Radiology
Artificial intelligence (AI) is playing a growing role in all our lives and has shown promise in addressing some of the greatest current and upcoming societal challenges we face. The healthcare industry, though notoriously complex and resistant to disruption, potentially has a lot to gain from the use of AI. With an established history of leading digital transformation in healthcare and an urgent need for improved efficiency, radiology has been at the forefront of harnessing AI’s potential.
This book covers how and why AI can address challenges faced by radiology departments, provides an overview of the fundamental concepts related to AI, and describes some of the most promising use cases for AI in radiology. In addition, the major challenges associated with the adoption of AI into routine radiological practice are discussed. The book also covers some crucial points radiology departments should keep in mind when deciding on which AI-based solutions to purchase. Finally, it provides an outlook on what new and evolving aspects of AI in radiology to expect in the near future.
The healthcare industry has experienced a number of trends over the past few decades that demand a change in the way certain things are done. These trends are particularly salient in radiology, where the diagnostic quality of imaging scans has improved dramatically while scan times have decreased. As a result, the amount and complexity of medical imaging data acquired have increased substantially over the past few decades (Smith-Bindman et al., 2019; Winder et al., 2021) and are expected to continue to increase (Tsao, 2020). This issue is complicated by a widespread global shortage of radiologists (AAMC Report Reinforces Mounting Physician Shortage, 2021, Clinical Radiology UK Workforce Census 2019 Report, 2019). Healthcare workers, including radiologists, have an increasing workload (Bruls & Kwee, 2020; Levin et al., 2017) that contributes to burnout and medical errors (Harry et al., 2021). Being an essential service provider to virtually all other hospital departments, staff shortages within radiology have significant effects that spread throughout the hospital and to society as a whole (England & Improvement, 2019; Sutherland et al., n.d.).
With an ageing global population and a rising burden of chronic illnesses, these issues are expected to pose even more of a challenge to the healthcare industry in the future.
AI-based medical imaging solutions have the potential to ameliorate these challenges for several reasons. They are particularly suited to handling large, complex datasets (Alzubaidi et al., 2021). Moreover, they are well suited to automate some of the tasks traditionally performed by radiologists and radiographers, potentially freeing up time and making workflows within radiology departments more efficient (Allen et al., 2021; Baltruschat et al., 2021; Kalra et al., 2020; O’Neill et al., 2021; van Leeuwen et al., 2021; Wong et al., 2019). AI is also capable of detecting complex patterns in data that humans cannot necessarily find or quantify (Dance, 2021; Korteling et al., 2021; Kühl et al., 2020).
The term “artificial intelligence” refers to the use of computer systems to solve specific problems in a way that simulates human reasoning. One fundamental characteristic of AI is that, like humans, these systems can tailor their solutions to changing circumstances. Note that, while these systems are meant to mimic on a fundamental level how humans think, their capacity to do so (e.g. in terms of the amount of data they can handle at one time, the nature and amount of patterns they can find in the data, and the speed at which they do so) often exceeds that of humans.
AI solutions come in the form of computer algorithms, which are pieces of computer code representing instructions to be followed to solve a specific problem. In its most fundamental form, the algorithm takes data as an input, performs some computation on that data, and returns an output.
An AI algorithm can be explicitly programmed to solve a specific task, analogous to a step-by-step recipe for baking a cake. On the other hand, the algorithm can be programmed to look for patterns within the data in order to solve the problem. These types of algorithms are known as machine learning algorithms. Thus, all machine learning algorithms are AI, but not all AI is machine learning. The patterns in the data that the algorithm can be explicitly programmed to look for or that it can “discover” by itself are known as features. An important characteristic of machine learning is that such algorithms learn from the data itself, and their performance improves the more data they are given.
One of the most common uses of machine learning is in classification - assigning a piece of data a particular label. For example, a machine learning algorithm might be used to tell if a photo (the input) shows a dog or a cat (the label). The algorithm can learn to do so in a supervised or unsupervised way.
In supervised learning, the machine learning algorithm is given data that has been labelled with the ground truth, in this example, photos of dogs and cats that have been labelled as such. The process then goes through the following phases:
1.Training phase: The algorithm learns the features associated with dogs and cats using the aforementioned data (training data).
2.Test phase: The algorithm is then given a new set of photos (the test data), it labels them and the performance of the algorithm on that data is assessed.
In some cases, there is a phase in between training and test, known as the validation phase. In this phase, the algorithm is given a new set of photos (not included in either the training or test data), its performance is assessed on this data, and the model is tweaked and retrained on the training data. This is repeated until some predefined performance-based criterion is reached, and the algorithm then enters the test phase.
In unsupervised learning, the algorithm identifies features within the input data that allow it to assign classes to the individual data points without being told explicitly what those classes are or should be. Such algorithms can identify patterns or group data points together without human intervention and include clustering and dimensionality reduction algorithms. Not all machine learning algorithms perform classification. Some are used to predict a continuous metric (e.g. the temperature in four weeks’ time) instead of a discrete label (e.g. cats vs dogs). These are known as regression algorithms.
Neural networks and deep learning
A neural network is made up of an input layer and an output layer, which are themselves composed of nodes. In simple neural networks, features that are manually derived from a dataset are fed into the input layer, which performs some computations, the results of which are relayed to the output layer. In deep learning, multiple “hidden” layers exist between the input and output layers. Each node of the hidden layers performs calculations using certain weights and relays the output to the next hidden layer until the output layer is reached.
In the beginning, random values are assigned to the weights and the accuracy of the algorithm is calculated. The values of the weights are then iteratively adjusted until a set of weight values that maximize accuracy is found. This iterative adjustment of the weight values is usually done by moving backwards from the output layer to the input layer, a technique called backpropagation. This entire process is done on the training data.
Understanding how the performance of AI algorithms is assessed is key to interpreting the AI literature. Several performance metrics exist for assessing how well a model performs certain tasks. No single metric is perfect, so a combination of several metrics provides a fuller picture of model performance.
In regression, the most commonly used metrics include:
- Mean absolute error (MAE): the average difference between the predicted values and the ground truth.
- Root mean square error (RMSE): the differences between the predicted values and the ground truth are squared and then averaged over the sample. Then the square root of the average is taken. Unlike the MAE, the RMSE thus gives higher weight to larger differences.
- R2: the proportion of the total variance in the ground truth explained by the variance in the predicted values. It ranges from 0 to 1.
The following metrics are commonly used in classification tasks:
- Accuracy: this is the proportion of all predictions that were predicted correctly. It ranges from 0 to 1.
- Sensitivity: also known as the true positive rate (TPR) or recall, this is the proportion of true positives that were predicted correctly. It ranges from 0 to 1.
- Specificity: Also known as the true negative rate (TNR), this is the proportion of true negatives that were predicted correctly. It ranges from 0 to 1.
- Precision: also known as positive predictive value (PPV), this is the proportion of positive classifications that were predicted correctly. It ranges from 0 to 1.
An inherent trade-off exists between sensitivity and specificity. The relevant importance of each, as well as their interpretation, highly depends on the specific research question and classification task.
Importantly, although classification models are meant to reach a binary conclusion, they are inherently probability-based. This means that these models will output a probability that a data point belongs to one class or another. In order to reach a conclusion on the most likely class, a threshold is used. Metrics such as accuracy, sensitivity, specificity and precision refer to the performance of the algorithm based on a certain threshold. The area under the receiver operating characteristic curve (AUC) is a threshold-independent performance metric. The AUC can be interpreted as the probability that a random positive example is ranked higher by the algorithm than a random negative example.
In image segmentation tasks, which are a type of classification task, the following metrics are commonly used:
- Dice similarity coefficient: a measure of overlap between two sets (e.g. two images) that is calculated as two times the number of elements common to the sets divided by the sum of the number of elements in each set. It ranges from 0 (no overlap) to 1 (perfect overlap).
- Hausdorff distance: a measure of how far two sets (e.g. two images) within a space are far from each other. It is basically the largest distance from one point in one set to the closest point in the other set.
Internal and external validity
Internally valid models perform well in their task on the data being used to train and validate them. The degree to which they are internally valid is assessed using the performance metrics outlined above and depends on the characteristics of the model itself and the quality of the data that the model was trained and validated on.
Externally valid models perform well in their tasks on new data (Ramspek et al., 2021). The better the model performs on data that differs from the data the models were trained and validated on, the higher the external validity. In practice, this often requires the performance of the models to be tested on data from hospitals or geographical areas that were not part of the model’s training and validation datasets.
Guidelines for evaluating AI research
Several guidelines have been developed to assess the evidence behind AI-based interventions in healthcare (X. Liu et al., 2020; Mongan et al., 2020; Shelmerdine et al., 2021; Weikert et al., 2021). These provide a template for those doing AI research in healthcare and ensure that relevant information is reported transparently and comprehensively, but can also be used by other stakeholders to assess the quality of published research. This helps ensure that AI-based solutions with substantial potential or actual limitations, particularly those caused by poor reporting (Bozkurt et al., 2020; D. W. Kim et al., 2019; X. Liu et al., 2019; Nagendran et al., 2020; Yusuf et al., 2020), are not prematurely adopted (CONSORT-AI and SPIRIT-AI Steering Group, 2019). Guidelines have also been proposed for evaluating the trustworthiness of AI-based solutions in terms of transparency, confidentiality, security, and accountability (Buruk et al., 2020; Lekadir et al., 2021; Zicari et al., 2021).
Over the past few years, AI has shown great potential in addressing a broad range of tasks within a medical imaging department, including many that happen before the patient is scanned. Implementations of AI to improve the efficiency of radiology workflows prior to patient scanning are sometimes referred to as “upstream AI” (Kapoor et al., 2020; M. L. Richardson et al., 2021).
One promising upstream AI application is predicting whichpatients arelikelytomisstheirscanappointments. Missed appointments are associated with significantly increased workload and costs (Dantas et al., 2018). Using a Gradient Boosting approach, Nelson et al. predicted missed hospital magnetic resonance imaging (MRI) appointments in the United Kingdom’s National Health Service (NHS) with high accuracy (Nelson et al., 2019). Their simulations also suggested that acting on the predictions of this model by targeting patients who are likely to miss their appointments would potentially yield a net benefit of several pounds per appointment across a range of model thresholds and missed appointment rates (Nelson et al., 2019). Similar results were recently found in a study of a single hospital in Singapore. For the 6-month period following the deployment of the predictive tool they were able to significantly reduce the no show rate from 19.3 % tp 15.9 % which translated into a potential economic benefit of $180,000 (Chong et. al., 2020).
Scheduling scans in a radiology department is a challenging endeavour because, although it is largely an administrative task, it depends heavily on medical information. The task of assigning patients to specific appointments thus often requires the input of someone with domain knowledge, which stipulates that either the person making the appointments must be a radiologist or radiology technician, or these people will have to provide input regularly. In either scenario, the process is somewhat inefficient and can potentially be streamlined using AI-based algorithms that check scan indications and contraindications and provide the people scheduling the scans with information about scan urgency (Letourneau-Guillon et al., 2020).
Depending on hospital or clinic policy, the decision on what exact scan protocol a patient receives is usually made based on the information on the referring physician’s scan request and the judgement of the radiologist. This is often supplemented by direct communication between the referring physician and radiologist and the radiologist’s review of the patient’s medical information. This process improves patient care (Boland et al., 2014) but can be time-consuming and inefficient, particularly with modalities like MRI, where a large number of protocol permutations exist. In one study, protocolling alone accounted for about 6 % of the radiologist’s working time (Schemmel et al., 2016). Radiologists are also often interrupted by tasks such as protocolling when interpreting images, despite the fact that the latter is considered a radiologist’s primary responsibility (Balint et al., 2014; J.-P. J. Yu et al., 2014).
Interpretation of the narrative text of the referring physician’s scan request has been attempted using natural language classifiers, the same technology used in chatbots and virtual assistants. Natural language classifiers based on deep learning have shown promise in assigning patients to either a contrast-enhanced or non-enhanced MRI protocol for musculoskeletal MRI, with an accuracy of 83 % (Trivedi et al., 2018) and 94 % (Y. H. Lee, 2018). Similar algorithms have shown an accuracy of 95 % for predicting the appropriate brain MRI protocol using a combination of up to 41 different MRI sequences (Brown & Marotta, 2018). Across a wide range of body regions, a deep-learning-based natural language classifier decided based on the narrative text of the scan requests whether to automatically assign a specific computed tomography (CT) or MRI protocol (which it did with 95 % accuracy) or, in more difficult cases, recommend a list of three most appropriate protocols to the radiologist (which it did with 92 % accuracy) (Kalra et al., 2020).
AI has also been used to decide whether already protocolled scans need to be extended, a decision which has to be made in real-time while the patient is inside the scanner. One such example is in prostate MRI, where a decision on whether to administer a contrast agent is often made after the non-contrast sequences. Hötker et al. found that a convolutional neural network (CNN) assigned 78 % of patients to the appropriate prostate MRI protocol (Hötker et al., 2021). The sensitivity of the CNN for the need for contrast was 94.4 % with a specificity of 68.8 % and only 2 % of patients in their study would have had to be called back for a contrast- enhanced scan (Hötker et al., 2021).
Image quality improvement and monitoring
Many AI-based solutions that work in the background of radiology workflows to improve image quality have recently been established. These include solutions for monitoring image quality, reducing image artefacts, improving spatial resolution, and speeding up scans.
Such solutions are entering the radiology mainstream, particularly for computed tomography, which for decades used established but artefact-prone methods for reconstructing interpretable images from the raw sensor data (Deák et al., 2013; Singh et al., 2010).
These are gradually being replaced by deep-learning- based reconstruction methods, which improve image quality while maintaining low radiation doses (Akagi et al., 2019; H. Chen et al., 2017; Choe et al., 2019; Shan et al., 2019). This reconstruction is performed on supercomputers on the CT scanner itself or on the cloud. The balance between radiation dose and image quality can be adjusted on a protocol-specific basis to tailor scans to individual patients and clinical scenarios (McLeavy et al., 2021; Willemink & Noël, 2019). Such approaches have found particular use when scanning children, pregnant women, and obese patients as well as CT scans of the urinary tract and heart (McLeavy et al., 2021).
AI-based solutions have also been used to speed up scans while maintaining diagnostic quality. Scan time reduction not only improves overall efficiency but also contributes to an overall better patient experience and compliance with imaging examination. A multi- centre study of spine MRI showed that a deep-learning- based image reconstruction algorithm that enhanced images using filtering and detail-preserving noise reduction reduced scan times by 40 % (Bash, Johnson, et al., 2021). For T1-weighted MRI scans of the brain, a similar algorithm that improves image sharpness and reduces image noise reduced scan times by 60 % while maintaining the accuracy of brain region volumetry compared to standard scans (Bash, Wang, et al., 2021).
In routine radiological practice, images often contain artefacts that reduce their interpretability. These artefacts are the result of characteristics of the specific imaging modality or protocol used or factors intrinsic to the patient being scanned, such as the presence of foreign bodies or the patient moving during the scan. Particularly with MRI, imaging protocols that demand fast scanning often introduce certain artefacts to the reconstructed image. In one study, a deep-learning- based algorithm reduced banding artefacts associated with balanced steady-state free precession MRI sequences of the brain and knee (K. H. Kim & Park, 2017). For real-time imaging of the heart using MRI, another study found that the aliasing artefacts introduced by the data undersampling were reduced by using a deep-learning-based approach (Hauptmann et al., 2019). The presence of metallic foreign bodies such as dental, orthopaedic or vascular implants is a common patient-related factor causing image artefacts in both CT and MRI (Boas & Fleischmann, 2012; Hargreaves et al., 2011). Although not yet well established, several deep-learning-based approaches for reducing these artefacts have been investigated (Ghani & Clem Karl, 2019; Puvanasunthararajah et al., 2021; Zhang & Yu, 2018). Similar approaches are being tested for reducing motion-related artefacts in MRI (Tamada et al., 2020; B. Zhao et al., 2022).
AI-based solutions for monitoring image quality potentially reduce the need to call patients back to repeat imaging examinations, which is a common problem (Schreiber-Zinaman & Rosenkrantz, 2017). A deep-learning-based algorithm that identifies the radiographic view acquired and extracts quality-related metrics from ankle radiographs was able to predict image quality with about 94 % accuracy (Mairhöfer et al., 2021). Another deep-learning-based approach was capable of predicting nondiagnostic liver MRI scans with a negative predictive value of between 86 % and 94 % (Esses et al., 2018). This real-time automated quality control potentially allows radiology technicians to rerun scans or run additional scans with greater diagnostic value.
Scan reading prioritization
With staff shortages and increasing scan numbers, radiologists face long reading lists. To optimize efficiency and patient care, AI-based solutions have been suggested as a way to prioritize which scans radiologists read and report first, usually by screening acquired images for findings that require urgent intervention (O’Connor & Bhalla, 2021). This has been most extensively studied in neuroradiology, where moving CT scans that were found to have intracranial haemorrhage by an AI-based tool to the top of the reading list reduced the time it took radiologists to view the scans by several minutes (O’Neill et al., 2021). Another study found that the time-to diagnosis (which includes the time from image acquisition to viewing by the radiologist and the time to read and report the scans) was reduced from 512 to 19 minutes in an outpatient setting when such a worklist prioritization was used (Arbabshirani et al., 2018). A simulation study using AI-based worklist prioritization based on identifying urgent findings on chest radiographs (such as pneumothorax, pleural effusions, and foreign bodies) also found a substantial reduction in the time it took to view and report the scans compared to standard workflow prioritization (Baltruschat et al., 2021).
Currently, the majority of commercially available AI- based solutions in medical imaging focus on some aspect of analyzing and interpreting images (Rezazade Mehrizi et al., 2021; van Leeuwen et al., 2021). This includes segmenting parts of the image (for surgical or radiation therapy targeting, for example), bringing suspicious areas to radiologists’ attention, extracting imaging biomarkers (radiomics), comparing images across time, and reaching specific imaging diagnoses.
¡ 29–38 % of commercially available AI-based applications in radiology (Rezazade Mehrizi et al., 2021; van Leeuwen et al., 2021).
Most commercially available AI-based solutions targeted at neuroimaging data aim to detect and characterize ischemic stroke, intracranial haemorrhage, dementia, and multiple sclerosis (Olthof et al., 2020). Several studies have shown excellent accuracy of AI- based methods for the detection and classification of intraparenchymal, subarachnoid, and subdural haemorrhage on head CT (Flanders et al., 2020; Ker et al., 2019; Kuo et al., 2019). Subsequent studies showed that, compared to radiologists, some AI-based solutions have substantially lower false positive and negative rates (Ginat, 2020; Rao et al., 2021). In ischemic stroke, AI-based solutions have largely focused on the quantification of the infarct core (Goebel et al., 2018; Maegerlein et al., 2019), the detection of large vessel occlusion (Matsoukas et al., 2022; Morey et al., 2021; Murray et al., 2020; Shlobin et al., 2022), and the prediction of stroke outcomes (Bacchi et al., 2020; Nielsen et al., 2018; Y. Yu et al., 2020, 2021).
In multiple sclerosis, AI has been used to identify and segment lesions (Nair et al., 2020; S.-H. Wang et al., 2018), which can be particularly helpful for the longitudinal follow-up of patients. It has also been used to extract imaging features associated with progressive disease and conversion from clinically isolated syndrome to definite multiple sclerosis (Narayana et al., 2020; Yoo et al., 2019). Other applications of AI in neuroradiology include the detection of intracranial aneurysms (Faron et al., 2020; Nakao et al., 2018; Ueda et al., 2019) and the segmentation of brain tumours (Kao et al., 2019; Mlynarski et al., 2019; Zhou et al., 2020) as well as the prediction of brain tumour genetic markers from imaging data (Choi et al., 2019; J. Zhao et al., 2020)
¡ 24 %–31 % of commercially available AI-based applications in radiology (Rezazade Mehrizi et al., 2021; van Leeuwen et al., 2021).
When interpreting chest radiographs, radiologists detected substantially more critical and urgent findings when aided by a deep-learning-based algorithm, and did so much faster than without the algorithm (Nam et al., 2021). Deep-learning-based image interpretation algorithms have also been found to improve radiology residents’ sensitivity for detecting urgent findings on chest radiographs from 66 % to 73 % (E. J. Hwang, Nam, et al., 2019). Another study which focused on a broader range of findings on chest radiographs also found that radiologists aided by a deep-learning-based algorithm had higher diagnostic accuracy than radiologists who read the radiographs without assistance (Seah et al., 2021). The uses of AI in chest radiology also extend to cross-sectional imaging like CT. A deep learning algorithm was found to detect pulmonary embolism on CT scans with high accuracy (AUC = 0.85) (Huang, Kothari, et al., 2020). Moreover, a deep learning algorithm was 90 % accurate in detecting aortic dissection on non-contrast-enhanced CT scans, similar to the performance of radiologists (Hata et al., 2021).
Outside the emergency setting, AI-based solutions have been widely tested and implemented for tuberculosis screening on chest radiographs (E. J. Hwang, Park, et al., 2019; S. Hwang et al., 2016; Khan et al., 2020; Qin et al., 2019; WHO Operational Handbook on Tuberculosis Module 2: Screening – Systematic Screening for Tuberculosis Disease, n.d.). In addition, they have been useful for lung cancer screening both in terms of detecting lung nodules on CT (Setio et al., 2017) and chest radiographs (Li et al., 2020) and by classifying whether nodules are likely to be malignant or benign (Ardila et al., 2019; Bonavita et al., 2020; Ciompi et al., 2017; B. Wu et al., 2018). AI-based solutions also show great promise for the diagnosis of pneumonia, chronic obstructive pulmonary disease, and interstitial lung disease (F. Liu et al., 2021).
¡ 11 % of commercially available AI-based applications in radiology (Rezazade Mehrizi et al., 2021; van Leeuwen et al., 2021).
So far, many of the AI-based algorithms targeting breast imaging aim to reduce the workload of radiologists reading mammograms. Ways to do this have included using AI-based algorithms to triage out negative mammograms, which in one study was associated with a reduction in radiologists’ workload by almost one-fifth (Yala et al., 2019). Other studies that have replaced second readers of mammograms with AI- based algorithms have shown that this leads to fewer false positives and false negatives as well as reduces the workload of the second reader by 88 % (McKinney et al., 2020).
AI-based solutions for mammography have also been found to increase the diagnostic accuracy of radiologists (McKinney et al., 2020; Rodríguez-Ruiz et al., 2019; Watanabe et al., 2019) and some have been found to be highly accurate in independently detecting and classifying breast lesions (Agnes et al., 2019; Al- Antari et al., 2020; Rodriguez-Ruiz et al., 2019).
Despite this, a recent systematic review of 36 AI- based algorithms found that these studies were of poor methodological quality and that all algorithms were less accurate than the consensus of two or more radiologists (Freeman et al., 2021). AI-based algorithms have nonetheless shown potential for extracting cancer-predictive features from mammograms beyond mammographic breast density (Arefan et al., 2020; Dembrower et al., 2020; Hinton et al., 2019). Beyond mammography, AI-based solutions have been developed for detecting and classifying breast lesions on ultrasound (Akkus et al., 2019; Park et al., 2019; G.- G. Wu et al., 2019) and MRI (Herent et al., 2019).
¡ 11 % of commercially available AI-based applications in radiology (Rezazade Mehrizi et al., 2021; van Leeuwen et al., 2021).
Cardiac radiology has always been particularly challenging because of the difficulties inherent in acquiring images of a constantly moving organ. Because of this, it has benefited immensely from advances in imaging technology and seems set to benefit greatly from AI as well (Sermesant et al., 2021). Most of the AI-based applications of the cardiovascular system use MRI, CT or ultrasound data (Weikert et al., 2021). Prominent examples include the automated calculation of ejection fraction on echocardiography, quantification of coronary artery calcification on cardiac CT, determination of right ventricular volume on CT pulmonary angiography, and determination of heart chamber size and thickness on cardiac MRI (Medical AI Evaluation, n.d., The Medical Futurist, n.d.). AI-based solutions for the prediction of patients likely to respond favourably to cardiac interventions, such as cardiac resynchronization therapy, based on imaging and clinical parameters have also shown great promise (Cikes et al., 2019; Hu et al., 2019). Changes in cardiac MRI not readily visible to human readers but potentially useful for differentiating different types of cardiomyopathies can also be detected using AI through texture analysis (Neisius et al., 2019; J. Wang et al., 2020) and other radiomic approaches (Mancio et al., 2022).
¡ 7–11 % of commercially available AI-based applications in radiology (Rezazade Mehrizi et al., 2021; van Leeuwen et al., 2021).
Promising applications of AI in the assessment of muscles, bones and joints include applications where human readers generally show poor between- and within-rater reliability, such as the determination of skeletal age based on bone radiographs (Halabi et al., 2019; Thodberg et al., 2009) and screening for osteoporosis on radiographs (Kathirvelu et al., 2019; J.-S. Lee et al., 2019) and CT (Pan et al., 2020). AI- based solutions have also shown promise for detecting fractures on radiographs and CT (Lindsey et al., 2018; Olczak et al., 2017; Urakawa et al., 2019). One systematic review of AI-based solutions for fracture detection in several different body parts showed AUCs ranging from 0.94 to 1.00 and accuracies of 77 % to 98 % (Langerhuizen et al., 2019). AI-based solutions have also achieved accuracies similar to radiologists for classification of the severity of degenerative changes of the spine (Jamaludin et al., 2017) and extremity joints (F. Liu et al., 2018; Thomas et al., 2020). AI-based solutions have also been developed to determine the origin of skeletal metastases (Lang et al., 2019) and the classification of primary bone tumours (Do et al., 2017).
Abdomen and pelvis
¡ 4 % of commercially available AI-based applications in radiology (Rezazade Mehrizi et al., 2021; van Leeuwen et al., 2021).
Much of the efforts in using AI in abdominal imaging have thus far concentrated on the automated segmentation of organs such as the liver (Dou et al., 2017), spleen (Moon et al., 2019), pancreas (Oktay et al., 2018), and kidneys (Sharma et al., 2017). In addition, a systematic review of 11 studies using deep learning for the detection of malignant liver masses showed accuracies of up to 97 % and AUCs of up to 0.92 (Azer, 2019).
Other applications of AI in abdominal radiology include the detection of liver fibrosis (He et al., 2019; Yasaka et al., 2018), fatty liver disease, hepatic iron content, the detection of free abdominal gas on CT, and automated volumetry and segmentation of the prostate (AI for Radiology, n.d.).
Despite the great potential of AI in medical imaging, it has yet to find widespread implementation and impact in routine clinical practice. This research-to- clinic translation is being hindered by several complex and interrelated issues that directly or indirectly lower the likelihood of AI-based solutions being adopted. One major way they do so is by creating a lack of trust in AI- based solutions by key stakeholders such as regulators, healthcare professionals and patients (Cadario et al., 2021; Esmaeilzadeh, 2020; J. P. Richardson et al., 2021; Tucci et al., 2022).
One major challenge is to develop AI-based solutions that continue to perform well in new, real-world scenarios. In a large systematic review, almost half of the studied AI-based medical imaging algorithms reported a greater than 0.05 decrease in the AUC when tested on new data (A. C. Yu et al., 2022). This lack of generalizability can lead to adverse effects on how well the model performs in a real-world scenario.
If a solution performs poorly when tested on a dataset with a similar or identical distribution to the training dataset, it is said to lack narrow generalizability and is often a consequence of overfitting (Eche et al., 2021). Potential solutions for overfitting are using larger training datasets and reducing the model’s complexity. If a solution performs poorly when tested on a dataset with a different distribution to the training dataset (e.g. a different distribution of patient ethnicities), it is said to lack broad generalizability (Eche et al., 2021). Solutions to poor broad generalizability include stress-testing the model on datasets with different distributions from the training dataset (Eche et al., 2021).
AI solutions are often developed in a high-resource environment such as large technology companies and academic medical centres in wealthy countries. It is likely that findings and performance in these high-resource contexts will fail to generalize to lower- resource contexts such as smaller hospitals, rural areas or poorer countries (Price & Nicholson, 2019), which complicates the issue further.
Risk of bias
Biases can arise in AI-based solutions due to data or human factors. The former occurs when the data used to train the AI solution does not adequately represent the target population. Datasets can be unrepresentative when they are too small or have been collected in a way that misrepresents a certain population category. AI solutions trained on unrepresentative data perpetuate biases and perform poorly in the population categories underrepresented or misrepresented in the training data. The presence of such biases has been empirically shown in many AI-based medical imaging studies (Larrazabal et al., 2020; Seyyed-Kalantari et al., 2021).
AI-based solutions are prone to several subjective and sometimes implicitly or explicitly prejudiced decisions during their development by humans. These human factors include how the training data is selected, how it is labelled, and how the decision is made to focus on the specific problem the AI-based solution intends to solve (Norori et al., 2021). Some recommendations and tools are available to help minimize the risk of bias in AI research (AIF360: A Comprehensive Set of Fairness Metrics for Datasets and Machine Learning Models, Explanations for These Metrics, and Algorithms to Mitigate Bias in Datasets and Models, n.d., IBM Watson Studio - Model Risk Management, n.d.; Silberg & Manyika, 2019).
Data quantity, quality and variety
Problems such as bias and lack of generalizability can be mitigated by ensuring that training data is of sufficient quantity, quality and variety. However, this is difficult to do because patients are often reluctant to share their data for commercial purposes (Aggarwal, Farag, et al., 2021; Ghafur et al., 2020; Trinidad et al., 2020), hospitals and clinics are usually not equipped to make this data available in a useable and secure manner, and organizing and labelling the data is time- consuming and expensive.
Many datasets can be used for a number of different purposes, and sharing data between companies can help make the process of data collection and organization more efficient, as well as increase the amount of data available for each application. However, developers are often reluctant to share data with each other, or even reveal the exact source of their data, to stay competitive.
Data protection and privacy
The development and implementation of AI-based solutions require that patients are explicitly informed about, and give their consent to, the use of their data for a particular purpose and by certain people. This data also has to be adequately protected from data breaches and misuse. Failure to ensure this greatly undermines the public’s trust in AI-based solutions and hinders their adoption. While regulations governing health data privacy state that the collection of fully anonymized data does not require explicit patient consent (General Data Protection Regulation (GDPR) – Official Legal Text, 2016; Office for Civil Rights (OCR), 2012) and in theory protects from the data being misused, whether or not imaging data can be fully anonymized is controversial (Lotan et al., 2020; Murdoch, 2021). Whether consent can be truly informed considering the complexity of the data being acquired, and the resulting myriad of potential future uses of the data, is also disputed (Vayena & Blasimme, 2017).
Among hospital departments, radiology has always been at the forefront ofdigitalization. AI-based solutions that focus on image processing and interpretation are likely to find the prerequisite infrastructure in most radiology departments, for example for linking imaging equipment to computers for analysis and for archiving images and other outputs. However, most radiology departments are likely to require significant infrastructure upgrades for other applications of AI, particularly those requiring the integration of information from multiple sources and having complex outputs. Moreover, it is important to keep in mind that the distribution of necessary infrastructure is highly unequal across and within countries (Health Ethics & Governance, 2021).
In terms of computing power, radiology departments will either have to invest resources into the hardware and personnel necessary to run these AI-based solutions or opt for cloud-based solutions. The former comes with an extra cost but allows data processing within the confines of the hospital or clinic’s local network. Cloud-based solutions for computing (known as “infrastructure as a service” or “IaaS”) are often considered the less secure and less trustworthy option, but this depends on a number of factors and is thus not always true (Baccianella & Gough, n.d.). Guidelines on what to consider when procuring cloud-based solutions in healthcare are available (Cloud Security for Healthcare Services, 2021).
Lack of standardization, interoperability, and integrability
The problem of infrastructure becomes even more complicated when considering how fragmented the AI medical imaging market currently is (Alexander et al., 2020). It is therefore likely that in the near future a single department will have several dozen AI-based solutions from different vendors running simultaneously. Having a separate self-contained infrastructure (e.g. a workstation or server) for each of these would be incredibly complicated and difficult to manage. Suggested solutions for this have included AI solution “marketplaces”, similar to app stores (Advanced AI Solutions for Radiology, n.d., Curated Marketplace, 2018, Imaging AI Marketplace - Overview, n.d., Sectra Amplifier Marketplace, 2021, The Nuance AI Marketplace for Diagnostic Imaging, n.d.), and development of an overarching vendor-neutral infrastructure (Leiner et al., 2021). The successful implementation of such solutions requires close partnerships between AI solution developers, imaging vendors and information technology companies.
It is often impossible to understand exactly how AI- based solutions come to their conclusions, particularly with complex approaches like deep learning. This reduces how transparent the decision-making process for procuring and approving these solutions can be, makes the identification of biases difficult, and makes it harder for clinicians to explain the outputs of these solutions to their patients and to determine whether a solution is working properly or has malfunctioned (Char et al., 2018; Reddy et al., 2020; Vayena et al., 2018; Whittlestone et al., 2019). Some have suggested that techniques that help humans understand how AI- based algorithms made certain decisions or predictions (“interpretable” or “explainable” AI) might help mitigate these challenges. However, others have argued that currently available techniques are unsuitable for understanding individual decisions of an algorithm and have warned against relying on them for ensuring that algorithms work in a safe and reliable way (Ghassemi et al., 2021).
In healthcare systems, a framework of accountability ensures that healthcare workers and medical institutions can be held responsible for adverse effects resulting from their actions. The question of who should be held accountable for the failures of an AI- based solution is complicated. For pharmaceuticals, for example, the accountability for inherent failures in the product or its use often lies with either the manufacturer or the prescriber. One key difference is that AI-based systems are continuously evolving and learning, and so inherently work in a way that is independent of what their developers could have foreseen (Yeung, 2018). To the end-user such as the healthcare worker, the AI- based solution may be opaque and so they may not be able to tell when the solution is malfunctioning or inaccurate (Habli et al., 2020; Yeung, 2018).
Despite substantial progress in their development over the past few years, deep learning algorithms are still surprising brittle. This means that, when the algorithm faces a scenario that differs substantially from what it faced during training, it cannot contextualize and often produces nonsensical or inaccurate results. This happens because, unlike humans, most algorithms learn to perceive things within the confines of certain assumptions, but fail to generalize outside these assumptions. As an example of how this can be abused with malicious intent, subtle changes to medical images, imperceptible by humans, can render the results of disease-classifying algorithms inaccurate (Finlayson et al., 2018). The lack of interpretability of many AI-based solutions compounds this problem because it makes it difficult to troubleshoot how they reached the wrong conclusion.
So far, more than 100 AI-based products have gained conformité européenne (CE) marking or Food and Drug Adminstration (FDA) clearance. These products can be found in continuously updated and searchable online databases curated by the FDA (Center for Devices & Radiological Health, n.d.), the American College of Radiology (Assess-AI, n.d.), and others (AI for Radiology, n.d., The Medical Futurist, n.d.; E. Wu et al., 2021). The increasing number of available products, the inherent complexity of many of these solutions, and the fact that many people who usually make purchasing decisions in hospitals are not familiar with evaluating such products make it important to think carefully when deciding on which product to purchase. Such decisions will need to be made after incorporating input from healthcare workers, information technology (IT) professionals, as well as management, finance, legal, and human resources professionals within hospitals.
Deciding on whether to purchase an AI-based solution in radiology, as well as which of the increasing number of commercially available solutions to purchase, includes considerations of quality, safety, and finances. Over the past few years, several guidelines have emerged to help potential buyers make these decisions (A Buyer’s Guide to AI in Health and Care, 2020; Omoumi et al., 2021; Reddy et al., 2021), and these guidelines are likely to evolve in the future with changing expectations from customers, regulatory bodies, and stakeholders involved in reimbursement decisions.
First of all, it has to be clear to the potential buyer what the problem is and whether AI is the appropriate approach to this solution, or whether alternatives exist that are more advantageous on balance. If AI is the appropriate approach, buyers should know exactly what a potential AI-based product’s scope of the solution is - i.e. what specific problem the AI-based solution is designed to solve and in what specific circumstances. This includes whether the solution is intended for screening, diagnosis, monitoring, treatment recommendation or another application. It also includes the intended users of the solution and what kind of specific qualifications or training they are expected to have in order to be able to operate the solution and interpret its outputs. It needs to be clear to buyers whether the solution is intended to replace certain tasks that would normally be performed by the end-user, act as a double-reader, as a triaging mechanism, or for other tasks like quality control. Buyers should also understand whether the solution is intended to provide “new” information (i.e. information that would otherwise be unavailable to the user without the solution), improve the performance of an existing task beyond a human’s or other non-AI-based solution’s performance or if it is intended to save time or other resources.
Buyers should also have access to information that allows them to assess the potential benefits of the AI solution, and this should be backed up by published scientific evidence for the efficacy and cost-efficiency of the solution. How this is done will depend highly on the solution itself and the context in which it is expected to be deployed, but guidelines for this are available (National Institute for Health and Care Excellence (NICE), n.d.). Some questions to ask here would be: How much of an influence will the solution have on patient management? Will it improve diagnostic performance? Will it save time and money? Will it affect patients’ quality of life? It should also be clear to the buyer who exactly is expected to benefit from the use of this solution (Radiologists? Clinicians? Patients? The healthcare system or society as a whole?).
As with any healthcare intervention, all AI-based solutions come with potential risks, and these should be made clear to the buyer. Some of these risks might have legal consequences, such as the potential for misdiagnosis. These risks should be quantified, and potential buyers should have a framework for dealing with them, including identifying a framework for accountability within the organizations implementing these solutions. Buyers should also ensure they clearly understand the potential negative effects on radiologists’ training and the potential disruption to radiologists’ workflows associated with the use of these solutions.
Specifics of the AI solution’s design are also relevant to the decision on whether or not to purchase it. These include how robust the solution is to differences between vendors and scanning parameters, the circumstances under which the algorithm was trained (including potential confounding factors), and the way that performance was assessed. It should also be clear to buyers if and how potential sources of bias were accounted for during development. Because a core characteristic of AI-based solutions is their ability to continuously learn from new data, whether and how exactly this retraining is incorporated into the solution with time should also be clear to the buyer, including whether or not new regulatory approval is needed with each iteration. This also includes whether or not retraining is required, for example, due to changes in imaging equipment at the buyer’s institution.
The main selling points of many AI-based solutions are ease-of-use and improved workflows. Therefore, potential buyers should carefully scrutinize how these solutions are to be integrated into existing workflows, including inter-operability with PACS and electronic medical record systems. Whether or not the solution requires extra hardware (e.g. graphical processing units) or software (e.g. for visualization of the solution’s outputs), or if it can readily be integrated into the existing information technology infrastructure of the buyer’s organization influences the overall cost of the solution for the buyer and is therefore also a critical consideration. In addition, the degree of manual interaction required, both under normal circumstances and for troubleshooting, should be known to the buyer. All potential users of the AI solution should be involved in the purchasing process to ensure that they are familiar with it and that it meets their professional ethical standards and suits their needs.
From a regulatory perspective, it should be clear to the buyer whether the solution complies with medical device and data protection regulations. Has the solution been approved in the buyer’s country? If so, under which risk classification? Buyers should also consider creating data flow maps that display how the data flows in the operation of the AI-based solution, including who has access to the data.
Finally, there are other factors to consider which are not necessarily unique to AI-based solutions and which buyers might be familiar with from purchasing other types of solutions. This includes the licensing model of the solution, how users are to be trained on using the solution, how the solution is maintained, how failures in the solution are dealt with, and whether additional costs are to be expected when scaling up the solution’s implementation (e.g. using the solution for more imaging equipment or more users). This allows the potential buyer to anticipate the current and future costs of purchasing the solution.
The past decade of increasing interest and progress in AI-based solutions for medical imaging has set the stage for a number of trends that are likely to appear or intensify in the near future.
Firstly, there is an increasing sentiment that, although AI holds a great deal of promise for interpretive applications (such as the detection of pathology), non-interpretive AI-based solutions might hold the most potential in terms of instilling efficiency into radiology workflows and improving patient experiences. This trend towards involving AI earlier in the patient management process is likely to extend to AI increasingly acting as a clinical decision support system to guide when and which imaging scans are performed.
For this to happen, AI needs to be integrated into existing clinical information systems, and the specific algorithms used need to be able to handle more varied data. This will likely pave the way for the development of algorithms that are capable of integrating demographic, clinical, and laboratory patient data to make recommendations about patient management (Huang, Pareek, et al., 2020; Rockenbach, 2021). The previously mentioned natural language processing algorithms that have been used to interpret scan requests may be useful candidates for this.
In addition, we are likely to see AI algorithms that can interpret multiple different types of imaging data from the same patient. Currently, less than 5 % of commercially available AI-based solutions in medical imaging work with more than one imaging modality (Rezazade Mehrizi et al., 2021; van Leeuwen et al., 2021) despite the fact that the typical patient in a hospital receives multiple imaging scans during their stay (Shinagare et al., 2014). With this, it is also likely that more AI-based solutions will be developed that target hitherto neglected modalities such as nuclear imaging techniques and ultrasound.
The current market for AI-based solutions in radiology is spread across a relatively large number of companies (Alexander et al., 2020). Potential users are likely to expect a streamlined integration of these products in their workflows, which can be challenging in such a fragmented market. Improved integration can be achieved in several different ways, including with vendor-neutral marketplaces or by the gradual consolidation of providers of AI-based solutions.
With the expanding use of AI, the issue of trust between AI developers, healthcare professionals, regulators, and patients will become more relevant. It is therefore likely that efforts will intensify to take steps towards strengthening that trust. This will potentially include raising the expected standards of evidence for AI- based solutions (Aggarwal, Sounderajah, et al., 2021; X. Liu et al., 2019; van Leeuwen et al., 2021; Yusuf et al., 2020), making them more transparent through the use and improvement of interpretable AI techniques (Holzinger et al., 2017; Reyes et al., 2020; “Towards Trustable Machine Learning,” 2018), and enhancing techniques for maintaining patient data privacy (G. Kaissis et al., 2021; G. A. Kaissis et al., 2020).
Furthermore, while most existing regulations stipulate that AI-based algorithms cannot be modified after regulatory approval, this is likely to change in the future. The potential for these algorithms to learn from data acquired after approval and adapt to changing circumstances is a major advantage of AI. Still, frameworks for doing so have thus far been lacking in the healthcare sector. However, promising ideas have recently emerged, including adapting existing hospital quality assurance and improvement frameworks to monitor AI-based algorithms’ performance and the data they are trained on and update the algorithms accordingly (Feng et al., 2022). This will likely require the development of multidisciplinary teams within hospitals consisting of clinicians, IT professionals, and biostatisticians who closely collaborate with model developers and regulators (Feng et al., 2022).
While the obstacles discussed in previous sections might slow down the adoption of AI in radiology somewhat, the fear of AI potentially replacing radiologists is unlikely to be one of them. A recent survey from Europe showed that most radiologists did not perceive a reduction in their clinical workload after adopting AI-based solutions (European Society of Radiology (ESR), 2022), likely because, at the same time, demand for radiologists’ services has been continuously rising. Studies from around the world have shown that radiology professionals, particularly those with AI exposure and experience, are generally optimistic about the role of AI in their practice (Y. Chen et al., 2021; Huisman et al., 2021; Ooi et al., 2021; Santomartino & Yi, 2022; Scott et al., 2021).
AI has shown promise in positively impacting virtually every facet of a radiology department’s work - from scheduling and protocolling patient scans to interpreting images and reaching diagnoses. Promising research on AI-based tools in radiology has not yet been widely translated to adoption in routine practice, however, because of a number of complex, partially intertwined issues. Potential solutions exist for many of these challenges, but many of these solutions require further refinement and testing. In the meantime, guidelines are emerging to help potential users of AI-based solutions in radiology navigate the increasing number of commercial products. This encourages their adoption in real-world scenarios, thus allowing their true potential to be uncovered, as well as their weaknesses to be identified and addressed in a safe and effective way. As these incremental improvements are made, these tools will likely evolve to handle more varied data, become integrated into consolidated workflows, become more transparent, and ultimately more useful for increasing efficiency and improving patient care.
AAMC Report Reinforces Mounting Physician Shortage. (2021). AAMC. https://www.aamc.org/news-insights/press- releases/aamc-report-reinforces-mounting-physician-shortage
A buyer’s guide to AI in health and care. (2020). NHS Transformation Directorate. https://www.nhsx.nhs.uk/ai-lab/ explore-all-resources/adopt-ai/a-buyers-guide-to-ai-in-health- and-care/
Advanced AI solutions for radiology. (n.d.). Calantic Website. Retrieved July 3, 2022, from https://aivisions.calantic.com/
Aggarwal, R., Farag, S., Martin, G., Ashrafian, H., & Darzi, A. (2021). Patient Perceptions on Data Sharing and Applying Artificial Intelligence to Health Care Data: Cross-sectional Survey. Journal of Medical Internet Research, 23(8), e26162. https://doi.org/10.2196/26162
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021). Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis. NPJ Digital Medicine, 4(1), 65. https://doi.org/10.1038/s41746-021-00438-z
Agnes, S. A., Anitha, J., Pandian, S. I. A., & Peter, J. D. (2019). Classification of Mammogram Images Using Multiscale all Convolutional Neural Network (MA-CNN). Journal of Medical Systems, 44(1), 30. https://doi.org/10.1007/s10916-019-1494-z
AIF360: A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models. (n.d.). Github. Retrieved June 11, 2022, from https:// github.com/Trusted-AI/AIF360
AI for radiology. (n.d.). Retrieved June 26, 2022, from https://grand-challenge.org/
aiforradiology/?subspeciality=Abdomen&modality=All&ce_ under=All&ce_class=All&fda_class=All&sort_by=last %20 modified&search=
Akagi, M., Nakamura, Y., Higaki, T., Narita, K., Honda, Y., Zhou, J., Yu, Z., Akino, N., & Awai, K. (2019). Deep learning reconstruction improves image quality of abdominal ultra- high-resolution CT. European Radiology, 29(11), 6163–6171. https://doi.org/10.1007/s00330-019-06170-3
Akkus, Z., Cai, J., Boonrod, A., Zeinoddini, A., Weston, A. D., Philbrick, K. A., & Erickson, B. J. (2019). A Survey of Deep- Learning Applications in Ultrasound: Artificial Intelligence- Powered Ultrasound for Improving Clinical Workflow. Journal of the American College of Radiology: JACR, 16(9 Pt B), 1318–1328. https://doi.org/10.1016/j.jacr.2019.06.004
Al-Antari, M. A., Al-Masni, M. A., & Kim, T.-S. (2020). Deep Learning Computer-Aided Diagnosis for Breast Lesion in Digital Mammogram. Advances in Experimental Medicine and Biology, 1213, 59–72. https://doi.org/10.1007/978-3-030-33128-3_4
Alexander, A., Jiang, A., Ferreira, C., & Zurkiya, D. (2020). An Intelligent Future for Medical Imaging: A Market Outlook on Artificial Intelligence for Medical Imaging. Journal of the American College of Radiology: JACR, 17(1 Pt B), 165–170. https:// doi.org/10.1016/j.jacr.2019.07.019
Allen, B., Agarwal, S., Coombs, L., Wald, C., & Dreyer, K. (2021). 2020 ACR Data Science Institute Artificial Intelligence Journal of the American College of Radiology: JACR
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021). Review of deep learning: concepts, CNN architectures, challenges, applications, future directions.
Arbabshirani, M. R., Fornwalt, B. K., Mongelluzzo, G. J., Suever, J. D., Geise, B. D., Patel, A. A., & Moore, G. J. (2018). Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration. NPJ Digital Medicine, 1, 9. https://doi.org/10.1038/s41746-017-0015-z
Ardila, D., Kiraly, A. P., Bharadwaj, S., Choi, B., Reicher, J. J., Peng, L., Tse, D., Etemadi, M., Ye, W., Corrado, G., Naidich, D. P., & Shetty, S. (2019). End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nature Medicine, 25(6), 954–961. https://doi. org/10.1038/s41591-019-0447-x
Arefan, D., Mohamed, A. A., Berg, W. A., Zuley, M. L., Sumkin, J. H., & Wu, S. (2020). Deep learning modeling using normal mammograms for predicting breast cancer risk. Medical Physics, 47(1), 110–118. https://doi.org/10.1002/mp.13886
Assess-AI. (n.d.). Retrieved July 2, 2022, from https://www. acrdsi.org/DSI-Services/Assess-AI
Azer, S. A. (2019). Deep learning with convolutional neural networks for identification of liver masses and hepatocellular carcinoma: A systematic review. World Journal of Gastrointestinal Oncology, 11(12), 1218–1230. https://doi.org/10.4251/wjgo.v11. i12.1218
Bacchi, S., Zerner, T., Oakden-Rayner, L., Kleinig, T., Patel, S., & Jannes, J. (2020). Deep Learning in the Prediction of Ischaemic Stroke Thrombolysis Functional Outcomes: A Pilot Study. Academic Radiology, 27(2), e19–e23. https://doi. org/10.1016/j.acra.2019.03.015
Baccianella, S., & Gough, T. (n.d.). Why cloud computing is the best option for hospitals adopting AI. Retrieved June 11, 2022, from https://www.aidence.com/articles/cloud-best-option- imaging-ai/
Balint, B. J., Steenburg, S. D., Lin, H., Shen, C., Steele, J. L., & Gunderman, R. B. (2014). Do telephone call interruptions have an impact on radiology resident diagnostic accuracy? Academic Radiology, 21(12), 1623–1628. https://doi.org/10.1016/j. acra.2014.08.001
Baltruschat, I., Steinmeister, L., Nickisch, H., Saalbach, A., Grass, M., Adam, G., Knopp, T., & Ittrich, H. (2021). Smart chest X-ray worklist prioritization using artificial intelligence: a clinical workflow simulation. European Radiology, 31(6), 3837– 3845. https://doi.org/10.1007/s00330-020-07480-7
Bash, S., Johnson, B., Gibbs, W., Zhang, T., Shankaranarayanan, A., & Tanenbaum, L. N. (2021). Deep Learning Image Processing Enables 40 % Faster Spinal MR Scans Which Match or Exceed Quality of Standard of Care : A Prospective Multicenter Multireader Study. Clinical Neuroradiology. https://doi.org/10.1007/s00062-021-01121-2
Bash, S., Wang, L., Airriess, C., Zaharchuk, G., Gong, E., Shankaranarayanan, A., & Tanenbaum, L. N. (2021). Deep Learning Enables 60 % Accelerated Volumetric Brain MRI While Preserving Quantitative Performance: A Prospective, Multicenter, Multireader Trial. AJNR. American Journal of Neuroradiology, 42(12), 2130–2137. https://doi.org/10.3174/ajnr.A7358
Boas, F. E., & Fleischmann, D. (2012). CT artifacts: causes and reduction techniques. Imaging in Medicine, 4(2), 229–240. https://doi.org/10.2217/iim.12.13
Boland, G. W., Duszak, R., Jr, & Kalra, M. (2014). Protocol design and optimization. Journal of the American College of Radiology: JACR, 11(5), 440–441. https://doi.org/10.1016/j. jacr.2014.01.021
Bonavita, I., Rafael-Palou, X., Ceresa, M., Piella, G., Ribas, V., & González Ballester, M. A. (2020). Integration of convolutional neural networks for pulmonary nodule malignancy assessment in a lung cancer classification pipeline. Computer Methods and Programs in Biomedicine, 185, 105172. https://doi.org/10.1016/j.cmpb.2019.105172
Bozkurt, S., Cahan, E. M., Seneviratne, M. G., Sun, R., Lossio- Ventura, J. A., Ioannidis, J. P. A., & Hernandez-Boussard, T. (2020). Reporting of demographic data and representativeness in machine learning models using electronic health records.
Journal of the American Medical Informatics Association: JAMIA, 27(12), 1878–1884. https://doi.org/10.1093/jamia/ocaa164
Brown, A. D., & Marotta, T. R. (2018). Using machine learning for sequence-level automated MRI protocol selection in neuroradiology. Journal of the American Medical Informatics Association: JAMIA, 25(5), 568–571. https://doi.org/10.1093/ jamia/ocx125
Bruls, R. J. M., & Kwee, R. M. (2020). Workload for radiologists during on-call hours: dramatic increase in the past 15 years.
Insights into Imaging, 11(1), 121. https://doi.org/10.1186/ s13244-020-00925-z
Buruk, B., Ekmekci, P. E., & Arda, B. (2020). A critical perspective on guidelines for responsible and trustworthy artificial intelligence. Medicine, Health Care, and Philosophy, 23(3), 387–399. https://doi.org/10.1007/s11019-020-09948-1
Cadario, R., Longoni, C., & Morewedge, C. K. (2021). Understanding, explaining, and utilizing medical artificial intelligence. Nature Human Behaviour, 5(12), 1636–1642. https://doi.org/10.1038/s41562-021-01146-0
Center for Devices, & Radiological Health. (n.d.). Artificial Intelligence and Machine Learning (AI/ML)-Enabled Medical Devices. U.S. Food and Drug Administration; FDA. Retrieved July 2, 2022, from https://www.fda.gov/medical-devices/software- medical-device-samd/artificial-intelligence-and-machine- learning-aiml-enabled-medical-devices
Char, D. S., Shah, N. H., & Magnus, D. (2018). Implementing Machine Learning in Health Care - Addressing Ethical Challenges. The New England Journal of Medicine, 378(11), 981–983. https://doi.org/10.1056/NEJMp1714229
Chen, H., Zhang, Y., Kalra, M. K., Lin, F., Chen, Y., Liao, P., Zhou, J., & Wang, G. (2017). Low-Dose CT With a Residual Encoder-Decoder Convolutional Neural Network. IEEE Transactions on Medical Imaging, 36(12), 2524–2535. https://doi.org/10.1109/TMI.2017.2715284
Chen, Y., Stavropoulou, C., Narasinkan, R., Baker, A., & Scarbrough, H. (2021). Professionals’ responses to the introduction of AI innovations in radiology and their implications for future adoption: a qualitative study. BMC Health Services Research, 21(1), 813. https://doi.org/10.1186/ s12913-021-06861-y
Choe, J., Lee, S. M., Do, K.-H., Lee, G., Lee, J.-G., Lee, S. M., & Seo, J. B. (2019). Deep Learning-based Image Conversion of CT Reconstruction Kernels Improves Radiomics Reproducibility for Pulmonary Nodules or Masses. Radiology, 292(2), 365–373. https://doi.org/10.1148/radiol.2019181960
Choi, K. S., Choi, S. H., & Jeong, B. (2019). Prediction of IDH genotype in gliomas with dynamic susceptibility contrast perfusion MR imaging using an explainable recurrent neural network. Neuro-Oncology, 21(9), 1197–1209. https://doi.org/10.1093/neuonc/noz095
Chong, L. R., Tsai, K. T., Lee, L. L., Foo, S. G., & Chang, P. C. (2020). Artificial Intelligence Predictive Analytics in the Management of Outpatient MRI Appointment No-Shows. AJR. American Journal of Roentgenology, 215(5), 1155–1162. https://doi.org/10.2214/AJR.19.22594
Cikes, M., Sanchez-Martinez, S., Claggett, B., Duchateau, N., Piella, G., Butakoff, C., Pouleur, A. C., Knappe, D., Biering- Sørensen, T., Kutyifa, V., Moss, A., Stein, K., Solomon, S. D., & Bijnens, B. (2019). Machine learning-based phenogrouping in heart failure to identify responders to cardiac resynchronization therapy. European Journal of Heart Failure, 21(1), 74–85. https://doi.org/10.1002/ejhf.1333
Ciompi, F., Chung, K., van Riel, S. J., Setio, A. A. A., Gerke, P. K., Jacobs, C., Scholten, E. T., Schaefer-Prokop, C., Wille,
M. M. W., Marchianò, A., Pastorino, U., Prokop, M., & van Ginneken, B. (2017). Towards automatic pulmonary nodule management in lung cancer screening with deep learning. Scientific Reports, 7, 46479. https://doi.org/10.1038/srep46479
Clinical radiology UK workforce census 2019 report. (2019). https://www.rcr.ac.uk/publication/clinical-radiology-uk- workforce-census-2019-report
Cloud security for healthcare services. (2021, January 14). ENISA. https://www.enisa.europa.eu/publications/cloud- security-for-healthcare-services/
CONSORT-AI and SPIRIT-AI Steering Group. (2019). Reporting guidelines for clinical trials evaluating artificial intelligence interventions are needed. Nature Medicine, 25(10), 1467–1468. https://doi.org/10.1038/s41591-019-0603-3
Curated marketplace. (2018, May 22). Blackford. https://www.blackfordanalysis.com/applications/
Dance, A. (2021). AI spots cell structures that humans can’t. Nature. 592 (7852), 154–155.
Dantas, L. F., Fleck, J. L., Cyrino Oliveira, F. L., & Hamacher, S. (2018). No-shows in appointment scheduling - a systematic literature review. Health Policy, 122(4), 412–421. https://doi.org/10.1016/j.healthpol.2018.02.002
Deák, Z., Grimm, J. M., Treitl, M., Geyer, L. L., Linsenmaier, U., Körner, M., Reiser, M. F., & Wirth, S. (2013). Filtered back projection, adaptive statistical iterative reconstruction, and a model-based iterative reconstruction in abdominal CT: an experimental clinical study. Radiology, 266(1), 197–206. https://doi.org/10.1148/radiol.12112707
Dembrower, K., Liu, Y., Azizpour, H., Eklund, M., Smith, K., Lindholm, P., & Strand, F. (2020). Comparison of a Deep
Learning Risk Score and Standard Mammographic Density Score for Breast Cancer Risk Prediction. Radiology, 294(2), 265–272. https://doi.org/10.1148/radiol.2019190872
Do, B. H., Langlotz, C., & Beaulieu, C. F. (2017). Bone Tumor Diagnosis Using a Naïve Bayesian Model of Demographic and Radiographic Features. Journal of Digital Imaging, 30(5), 640–647. https://doi.org/10.1007/s10278-017-0001-7
Dou, Q., Yu, L., Chen, H., Jin, Y., Yang, X., Qin, J., & Heng, P.-A. (2017). 3D deeply supervised network for automated segmentation of volumetric medical images. Medical Image Analysis, 41, 40–54. https://doi.org/10.1016/j. media.2017.05.001
Eche, T., Schwartz, L. H., Mokrane, F.-Z., & Dercle, L. (2021). Toward Generalizability in the Deployment of Artificial Intelligence in Radiology: Role of Computation Stress Testing to Overcome Underspecification. Radiology. Artificial Intelligence, 3(6), e210097. https://doi.org/10.1148/ryai.2021210097
England, N. H. S., & Improvement, N. H. S. (2019). NHS diagnostic waiting times and activity data. NHS. https://www. england.nhs.uk/statistics/wp-content/uploads/sites/2/2021/12/ DWTA-Report-October-2021_M43D4.pdf
Esmaeilzadeh, P. (2020). Use of AI-based tools for healthcare purposes: a survey study from consumers’ perspectives. BMC Medical Informatics and Decision Making, 20(1), 170. https://doi. org/10.1186/s12911-020-01191-1
Esses, S. J., Lu, X., Zhao, T., Shanbhogue, K., Dane, B., Bruno, M., & Chandarana, H. (2018). Automated image quality evaluation of T2 -weighted liver MRI utilizing deep learning architecture. Journal of Magnetic Resonance Imaging: JMRI, 47(3), 723–728. https://doi.org/10.1002/jmri.25779
European Society of Radiology (ESR). (2022). Current practical experience with artificial intelligence in clinical radiology:
a survey of the European Society of Radiology. Insights into Imaging, 13(1), 107. https://doi.org/10.1186/s13244-022- 01247-y
Ferlay, J., Colombet, M., Soerjomataram, I., Dyba, T., Randi,G., Bettio, M., Gavin, A., Visser, O., & Bray, F.(2018). Cancer incidence and mortality patterns in Europe: Estimates for 40 countries and 25 major cancers in 2018. European Journal of Cancer, 103, 356–387. https://doi.org/10.1016/j.ejca.2018.07.005