Artificial intelligence unlocks the healthcare data lake
0 Abstract
Artificial intelligence (AI) is poised to revolutionize surgical care by leveraging the vast and complex “data lake” of healthcare information. This perspective piece outlines how AI may harness structured and unstructured data to improve patient outcomes. Advances in deep learning and foundational models have enabled the development of predictive analytics, automated clinical documentation, personalized patient chatbots, remote monitoring, and enhanced medical imaging. Examples include the ACS NSQIP risk calculator, Sepsis ImmunoScore, startups in ambient transcription, and cutting-edge AI applications in intraoperative imaging and real-time diagnostics. However, the adoption of AI in healthcare requires overcoming challenges, including data privacy, bias, integration into clinical workflows, interoperability, cost, ethical concerns, and regulatory hurdles. As AI technologies evolve, collaboration between surgeons and scientists will be critical to ensure ethical, patient-centered designs. This manuscript calls for surgeons to lead AI applications role in surgery, bridging technology with meaningful use cases to positively align with clinical practice.
Keywords
INTRODUCTION
High-value healthcare, including surgical care, is increasingly informed by evidence, making patient-centered approaches more effective and improving outcomes. Over the past two decades, big data has revolutionized decision making across industries. Yet, as healthcare data grow exponentially, we remain at the threshold of fully realizing its potential. Modern computing now enables us to harness the data, guiding the next generation of surgical care. This perspective aims to help surgeons understand how artificial intelligence (AI) can capture the promise of big data in healthcare.
THE HEALTHCARE DATA LAKE
Healthcare data availability and complexity have grown immensely. A single patient may generate up to 80 megabytes of electronic health record (EHR) data annually[1]. The breadth of EHR information can be conceptualized as a “data lake” [Figure 1], an amalgamation of heterogeneous data[2]. The surface layer consists of traditional, coded metrics [e.g., demographics, body mass index (BMI), International Classification of Diseases (ICD) codes, Current Procedural Terminology (CPT) codes] that are “structured” and easily interpreted. Beyond the surface, the “semi-structured” or “unstructured” data - like imaging, digital pathology, clinical notes, and sensor data - cannot be fully utilized with conventional analytics. Each patient’s EHR is its own data lake. Modern computing can selectively extract data for analysis, creating human-designed, machine-powered decision-making tools[3].
Figure 1. An original representation of the healthcare data lake. The surface level is comprised of traditional, coded metrics such as patient demographics (e.g., age, sex, BMI) as well as diagnostic and procedural codes (e.g., ICD and CPT codes). These data are highly “structured”, stored in ways that are easily interpretable by computers and humans alike. Deeper regions of the data lake contain a vast amount of “semi-structured” or “unstructured” patient data. This includes radiology and imaging, digital pathology, clinical documentation, patient-reported outcomes, nutritional data, wearable biotechnology and sensors, genetics, environmental data, and population/epidemiological data. Modern computing and methods in AI can harness the wealth of information in the EHR data lake to enable human-designed, machine-powered decision making in surgery. BMI: Body mass index; ICD: International Classification of Diseases; CPT: Current Procedural Terminology; EHR: electronic health record.
AI IN DATA SCIENCE
The early 21st century saw a proliferation of machine learning (ML) using curated datasets to build statistical models. For example, the ACS NSQIP surgical risk calculator uses 20 patient factors and procedure codes to predict outcomes[4]. This era of data science gave us a thorough understanding of the structured, surface-level factors from the healthcare data lake. However, more advanced AI techniques are needed to unlock insights from the deeper layers of the data lake. Neural networks are advanced ML algorithms that are designed like the human brain, with layers of interconnected “neurons”. Introducing more layers of “neurons” to the network creates a deep learning (DL) network[5]. DL excels at processing complex data types such as imaging and unstructured notes, delivering predictions with remarkable accuracy[6].
The barriers to implementing DL have previously been the immense computational and data requirements. The emergence of foundation models and advancements in computing power [i.e., graphic processing units (GPUs)] have improved computational efficiency and cost. Further, the paradigm shift to generative AI models has allowed scientists to create new user-friendly multimedia. These combined advancements have propelled the recent revolution in AI.
THE AI REVOLUTION: FOUNDATION MODELS
Foundation models have democratized access to powerful DL. Foundation models are deep neural networks pre-trained on extremely large and diverse datasets[7]. Popular examples include GPT-4o, Llama 3, Mistral, segment anything model (SAM), and Gemini.
Foundation models can be thought of as general-purpose technology, which scientists can further adapt. Unlike the past generation of ML, in which models were built for each application de novo, foundation models can be repurposed for many applications. Further training of a foundation model is considered “transfer learning”, as scientists can transfer the network of a base model and build layers on top of them to create downstream models[7]. Saha et al. recently used transfer learning on SAM to help create a model, which, in preliminary studies, identified lesions on mammograms as benign or malignant with > 99.9% accuracy[8]. “Few-shot learning” is a case of transfer learning, whereby the model can be adjusted for a specific task by training on minimal additional data [Figure 2][9]. This is compared to traditional ML models, which require training on many cases to perform a specific task.
Figure 2. Differences between traditional supervised ML, and “few-shot” learning using foundation models. The latter allows the development of AI applications in smaller teams, with fewer resources and less training data. In this way, foundation models will be essential in democratizing AI for healthcare applications. ML: Machine learning; AI: artificial intelligence.
Because of the sensitive nature of health information, the development of clinical AI applications has been piecemeal. Still, there is great potential. In this manuscript, we outline the broad areas of AI applications in surgery and highlight specific examples. AI has the potential to dip into each patient’s healthcare data lake, process information, and vastly improve the quality of surgical care.
APPLICATIONS OF AI IN SURGERY
Surgical data science
To date, the most common application of AI in surgery is for data science. This is best encapsulated by the ACS NSQIP calculator for surgical outcomes. The original set of models developed in 2013 were based on regressions[4]. Most recently, in 2023, Liu et al. published a new set of models using a more complex ML algorithm on the same dataset and found it was more accurate[10]. Future work should improve upon this performance with more advanced techniques. Another healthcare startup, Prenosis©, developed the Sepsis ImmunoScore, an AI software to predict patients at risk of sepsis. Their team recently published their model development and validation in NEJM AI, demonstrating the high accuracy of their tool (C-statistic of 0.85)[11]. Surgeons should continue to harness the growing healthcare data lake for predictive analytics that can forecast surgical outcomes, anticipate complications, and guide surgical management.
Clinical documentation and coding
Several companies have made strides in automating clinical documentation. Ambience© is attempting to generate clinical documentation in real time during clinical encounters. Industry leader Nuance© also announced a new service that will incorporate GPT-4. These systems may also be trained to learn the ICD and CPT coding systems to automatically assign codes for specific patient-provider interactions[12]. Automating documentation will reduce administrative burden, allowing all healthcare providers to focus more on patient care.
Moreover, AI should also assist in translating clinical materials between languages. This could streamline care for non-native English speakers, or who have other non-native English providers. A recent international study found GPT-4, Llama 3, and Mistral models were all able to translate free text radiology reports with high overall accuracy and quality, but with room for improvement in the accuracy of medical terminology[13].
Personalized chatbots
Patient-facing chatbots may assist with answering basic questions, scheduling appointments, and medication reminders, providing personalized information for patients in real time. By delivering information in an accessible and user-friendly format, they can empower patients to take an active role in their care, improving patient engagement and adherence. The ideal chatbot will seamlessly navigate surgical patients through preoperative and postoperative care.
Indeed, several specialties have studied how untrained AI chatbots perform at answering patient questions, usually with ChatGPT. The consensus seems that answers to the untrained chatbot are generally accurate, unbiased, and deferential to formal surgical consultations[14-16]. Many other surgical specialties see the value as well[17-19]. Further development will require training models on specific surgical details and continuously updating them with the best evidence.
Remote patient monitoring
AI extends its impact to postoperative recovery by enabling remote patient monitoring (RPM) systems. RPM systems can use wearable devices to track important physiologic markers around the clock, including vital signs, activity levels, glucose levels, and wound healing progress. Some examples have been developed and tested in cardiac surgery and oncologic surgery[20,21]. Basic outpatient monitors have been shown to improve patient outcomes. Indeed, Nagappa et al. conducted a randomized control trial and found thoracic surgery patients using their digital home monitor had fewer ED visits, unplanned admissions, and postoperative complications[22]. AI models may unlock RPM by analyzing more complex physiologic markers alongside other patient inputs to predict complications, alert healthcare providers, recommend interventions, and personalize rehabilitation or nutrition regimens.
Medical image analysis
Medical imaging is a domain where AI’s impact is particularly pronounced. DL models excel at analyzing complex visual data, enabling them to detect subtle patterns that may be missed by human observers. Companies like Aidoc© and Viz© have pioneered stroke and PE diagnosis by using AI to identify acute hemorrhagic strokes on CT. Recent studies have shown the benefits of employing these systems in the clinical setting[23,24]. More generally, scientists are creating comprehensive vision models to read all types of radiology[25]. These advancements hold significant potential for early detection and intervention, ultimately saving lives and reducing healthcare costs.
AI can also be applied to imaging from robotic surgery, endoscopy, pathology, and other use cases to improve diagnostic and predictive ability. For example, Activ Surgical© has developed a system to improve the accuracy of intra-operative ICG for assessment of tissue perfusion[26]. Authors from Stanford additionally published a vision model capable of analyzing intraoperative video from laparoscopic cholecystectomy to predict blood loss[27]. Their model’s binary predictive ability of low vs. moderate blood loss had a C-statistic of 0.81. Future AI systems may be able to process live video feeds from surgeries to identify key anatomical landmarks and detect potential errors in real time.
CHALLENGES AND LIMITATIONS
While the potential of AI in healthcare is immense, several challenges must be addressed. A significant challenge in clinical AI development has been ensuring patient data are protected and used ethically. Robust encryption and access controls are essential and represent major concerns for health systems. AI model development requires rigorous data quality checks to ensure dataset diversity to minimize bias. The model interface is also important and must be integrated into clinical workflows. Innovators will have to consider how to adapt their technology across several different EHRs. Creating patient- and provider-centered designs will facilitate adoption to achieve widespread surgeon support. Working with datasets across the data lake is currently challenging because data are often siloed into distinct programs or operations within or outside the EHR. Our discipline will need to invest significantly in interoperability and standardization to achieve the potential of AI.
Integrating AI into surgical care demands a commitment to scientific and ethical principles. While AI can process extensive datasets to assist decision making, the surgeon’s experience, judgment, and ethical discernment remain paramount. AI should function as a supportive adjunct rather than a replacement, ensuring that nuanced, patient-specific decisions incorporate human insight. For example, in developing the aforementioned vision model predicting blood loss in laparoscopic cholecystectomy, surgeons were fundamental in designing the research objectives and identifying 114 critical surgical actions to train the AI model[27].
A potential guide for the development of AI-based surgical decision support tools may be the algorithm-based clinical decision support (ABCDS) model of oversight[28]. This framework consists of four stages for AI development: model development, silent evaluation/feasibility, prospective effectiveness evaluation, and general deployment. The framework emphasizes several checkpoints for an oversight committee to monitor the tool in production. It also calls for more scientific validation so that AI tools can be assessed on performance metrics. This type of AI governance ecosystem promotes ethical model development, impact, quality control, accountability, and collaboration. Other ethical guidelines share these concepts and should also be considered[29,30].
Innovators must also consider the shifting regulatory landscape. The European Union has several legislations to consider: the Medical Device Regulation (MDR), the European Artificial Intelligence Act (AIA), the Data Governance Act (DGA), and the Open Data Directive (ODD)[31]. Together, these regulations deem AI in healthcare to be high-risk due to its critical impact on human lives. They collectively outline guidelines for healthcare AI development that is human-centered, privacy-conscious, and innovation-friendly. In the U.S., the regulatory standards have been murkier as there is still no federal legislation. However, in April 2024, Prenosis’s Sepsis ImmunoScore became the first AI/ML model approved by the FDA through the De Novo regulatory pathway, paving the way for future AI/ML technologies[11].
CONCLUSION: A CALL TO SURGEONS
AI’s potential to transform surgery lies in its ability to harness the full spectrum of data within the healthcare data lake. By leveraging advanced models and aligning their development with patient-centered goals, we can usher in a new era of evidence-informed, technology-driven surgical care. However, this goal requires collaboration between clinicians and scientists. Now more than ever, it is crucial for surgeons to have a basic proficiency in AI methods and current developments. By collaborating with AI researchers and developers, surgeons can identify meaningful use cases, address challenges, and guide the ethical and practical implementation of AI systems.
DECLARATIONS
Authors’ contributions
Made substantial contributions to the conception and design of the review: Talwar A, Talwar AA, Broach RB, Ungar LH, Hashimoto DA, Fischer JP
Availability of data and materials
Not applicable.
Financial support and sponsorship
None.
Conflicts of interest
All authors declared that there are no conflicts of interest.
Ethical approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Copyright
© The Author(s) 2025.
REFERENCES
1. Huesch MD, Mosher TJ. Using it or losing it? The case for data scientists inside health care. NEJM Catal. 2017. Available from: https://catalyst.nejm.org/doi/full/10.1056/CAT.17.0493. [Last accessed on 28 Apr 2025]
2. Stein B, Morrison A. The enterprise data lake: better integration and deeper analytics. 2014. Available from: https://www.smallake.kr/wp-content/uploads/2017/03/20170313_074222.pdf. [Last accessed on 28 Apr 2025].
3. Moor M, Banerjee O, Abad ZSH, et al. Foundation models for generalist medical artificial intelligence. Nature. 2023;616:259-65.
4. Bilimoria KY, Liu Y, Paruch JL, et al. Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg. 2013;217:833-42.e1.
5. Hashimoto DA, Rosman G, Rus D, Meireles OR. Artificial intelligence in surgery: promises and perils. Ann Surg. 2018;268:70-6.
6. Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. arXiv 2017; arXiv:1706.03762. Available from: https://doi.org/10.48550/arXiv.1706.03762. [accessed 28 Apr 2025]
7. Bommasani R, Hudson DA, Adeli E, et al. On the opportunities and risks of foundation models. arXiv 2021; arXiv:2108.07258. Available from: https://doi.org/10.48550/arXiv.2108.07258. [accessed 28 Apr 2025]
8. Saha DK, Hossain T, Safran M, Alfarhood S, Mridha MF, Che D. Segmentation for mammography classification utilizing deep convolutional neural network. BMC Med Imaging. 2024;24:334.
9. Brown TB, Mann B, Ryder N, et al. Language models are few-shot learners. arXiv 2020; arXiv:2005.14165. Available from: https://doi.org/10.48550/arXiv.2005.14165. [accessed 28 Apr 2025]
10. Liu Y, Ko CY, Hall BL, Cohen ME. American college of surgeons NSQIP risk calculator accuracy using a machine learning algorithm compared with regression. J Am Coll Surg. 2023;236:1024-30.
11. Bhargava A, López-Espina C, Schmalz L, et al. FDA-authorized AI/ML tool for sepsis prediction: development and validation. NEJM AI. 2024;1:867.
12. Dong H, Falis M, Whiteley W, et al. Automated clinical coding: what, why, and where we are? NPJ Digit Med. 2022;5:159.
13. Meddeb A, Lüken S, Busch F, et al. Large language model ability to translate CT and MRI free-text radiology reports into multiple languages. Radiology. 2024;313:e241736.
14. Pan A, Musheyev D, Bockelman D, Loeb S, Kabarriti AE. Assessment of artificial intelligence chatbot responses to top searched queries about cancer. JAMA Oncol. 2023;9:1437-40.
15. Mika AP, Martin JR, Engstrom SM, Polkowski GG, Wilson JM. Assessing ChatGPT responses to common patient questions regarding total hip arthroplasty. J Bone Joint Surg Am. 2023;105:1519-26.
16. Seth I, Cox A, Xie Y, et al. Evaluating chatbot efficacy for answering frequently asked questions in plastic surgery: a ChatGPT case study focused on breast augmentation. Aesthet Surg J. 2023;43:1126-35.
17. Li W, Zhang Y, Chen F. ChatGPT in colorectal surgery: a promising tool or a passing fad? Ann Biomed Eng. 2023;51:1892-7.
18. Garcia Valencia OA, Thongprayoon C, Jadlowiec CC, Mao SA, Miao J, Cheungpasitporn W. Enhancing kidney transplant care through the integration of chatbot. Healthcare. 2023;11:2518.
19. Lechien JR. Generative AI and otolaryngology-head & neck surgery. Otolaryngol Clin North Am. 2024;57:753-65.
20. van der Stam JA, Mestrom EHJ, Nienhuijs SW, et al. A wearable patch based remote early warning score (REWS) in major abdominal cancer surgery patients. Eur J Surg Oncol. 2023;49:278-84.
21. Londral A, Azevedo S, Dias P, et al. Developing and validating high-value patient digital follow-up services: a pilot study in cardiac surgery. BMC Health Serv Res. 2022;22:680.
22. Nagappa M, Subramani Y, Yang H, et al. Enhancing quadruple health outcomes after thoracic surgery: feasibility pilot randomized controlled trial using digital home monitoring. JMIR Perioper Med. 2025;8:e58998.
23. Odland I, Liu KJ, Wu D, et al. Real-world evaluation of the accuracy of the Viz.AI automated intracranial hemorrhage volume calculation tool. J Neurointerv Surg. 2025:jnis-2024-022564.
24. Graeve VIJ, Laures S, Spirig A, et al. Implementation of an AI algorithm in clinical practice to reduce missed incidental pulmonary embolisms on chest CT and its impact on short-term survival. Invest Radiol. 2025;60:260-6.
25. Ma J, He Y, Li F, Han L, You C, Wang B. Segment anything in medical images. Nat Commun. 2024;15:654.
26. Nwaiwu CA, Buharin VE, Mach A, et al. Feasibility and comparison of laparoscopic laser speckle contrast imaging to near-infrared display of indocyanine green in intraoperative tissue blood flow/tissue perfusion in preclinical porcine models. Surg Endosc. 2023;37:1086-95.
27. Aklilu JG, Sun MW, Goel S, et al. Artificial intelligence identifies factors associated with blood loss and surgical experience in cholecystectomy. NEJM AI. 2024;1:88.
28. Duke AI Health. Algorithm-Based Clinical Decision Support (ABCDS) Oversight. Available from: https://aihealth.duke.edu/algorithm-based-clinical-decision-support-abcds/. [Last accessed on 28 Apr 2025].
29. World Health Organization. Ethics and governance of artificial intelligence for health. 2021. Available from: https://www.who.int/publications-detail-redirect/9789240029200. [Last accessed on 28 Apr 2025].
30. Solanki P, Grundy J, Hussain W. Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers. AI Ethics. 2023;3:223-40.
Cite This Article
Open AccessHow to Cite
Download Citation
Export Citation File:
Type of Import
Tips on Downloading Citation
Citation Manager File Format
Type of Import
Direct Import: When the Direct Import option is selected (the default state), a dialogue box will give you the option to Save or Open the downloaded citation data. Choosing Open will either launch your citation manager or give you a choice of applications with which to use the metadata. The Save option saves the file locally for later use.
Indirect Import: When the Indirect Import option is selected, the metadata is displayed and may be copied and pasted as needed.
About This Article
Copyright
Data & Comments
Data
0
Comments
Comments must be written in English. Spam, offensive content, impersonation, and private information will not be permitted. If any comment is reported and identified as inappropriate content by OAE staff, the comment will be removed without notice. If you have any queries or need any help, please contact us at support@oaepublish.com.