Medical Working Group

Mission

Design benchmarks and propose best practices to accelerate the development of AI and machine learning for healthcare, catalyze new markets, and ultimately improve Patient outcome and improve Providers’ experience.

Purpose

Medical AI has tremendous potential to advance healthcare by supporting the evidence-based practice of medicine, personalizing patient treatment, reducing costs, and improving provider and patient experience. We believe that unlocking this potential requires a systematic way to measure the performance of medical AI models on large-scale heterogeneous data. To meet this need, we are building MedPerf, an open framework for benchmarking machine learning in the medical domain. MedPerf will enable federated evaluation in which models are securely distributed to different facilities for evaluation, thereby empowering healthcare organizations to assess and verify the performance of AI models in an efficient and human-supervised process, while prioritizing privacy. In our paper titled "MedPerf: Open Benchmarking Platform for Medical Artificial Intelligence using Federated Evaluation" we describe the current challenges healthcare and AI communities face, the need for an open platform, the design philosophy of MedPerf, its current implementation status, and our roadmap.

Within our working group we strongly believe that these efforts will give key stakeholders the confidence to trust models and the data/processes they relied on, therefore accelerating ML adoption in the clinical settings, possibly improving patient outcomes, optimizing healthcare costs, and improving provider experiences.

Deliverables

1. Initial efforts will focus on developing a Proof-of-Concept (PoC):
       - Build confidence for discussions of participation with real clinical data
       - Validate our technological approach
       - Enhance our understanding of system requirements
       - Serve as a public example of federated learning in the medical space
       - Publish white papers with findings

2. Long Term efforts will focus on:
       - Extending initial PoC features with benchmarks for methods in data utility, model evaluation, model security and data privacy
       - Improving technical integration with multiple Federated Learning frameworks
       - Building partnerships with medical organizations
       - Facilitate clinical data access
       - Research on various open challenges: noisy labels, data utility, privacy

Meeting Schedule

Bi-Weekly on Monday from 10:00-11:00AM Pacific.

How to Join

Use this link to request to join the group/mailing list, and receive the meeting invite:
Medicine Google Group.
Requests are manually reviewed, so please be patient.

Working Group Resources

Shared documents and meeting minutes:

1. Associate a Google account with your e-mail address.
2. Ask to join our Public Google Group.
3. Once approved, go to the Medical folder in our Public Google Drive.

Working Group Chair Emails

Alexandros Karargyris (alexandros.karargyris@ihu-strasbourg.eu)

Renato Umeton (renato_umeton@mail.dfci.harvard.edu)

Micah Sheller (micah.j.sheller@intel.com), Vice Chair

Spyridon Bakas (spyridon.Bakas@pennmedicine.upenn.edu), Vice Chair for Benchmarking & Clinical Translation

Working Group Chair Bios

Alexandros Karargyris is a senior researcher at the Institute of Image-Guided Surgery of Strasbourg, a unique place for translative clinical research. He is leading projects related to  applications in the intersection of surgery and artificial intelligence (AI).  Previously, he worked as a researcher at IBM and NIH for more than 10 years. His research interests lie in the space of medical imaging, machine learning and mobile health. He has contributed to healthcare commercial products and imaging solutions deployed in under-resourced areas. His work has been published in peer-reviewed journals and conferences.

LinkedIn

Renato Umeton heads artificial intelligence (AI) and data science in the Informatics & Analytics department of Dana-Farber Cancer Institute, a teaching-affiliate of Harvard Medical School. His work focuses on Operations, translating AI from the research realm into the Clinic and into Enterprise software offerings that benefit patients and cancer researchers, at scale. Renato started working on artificial intelligence, data science and big data in 2007, when these areas were yet to be well defined; since then he has published in several scientific fields, he has worked in academia, in consulting, in hospital settings, in biotech, and is currently affiliated also with Harvard School of Public Health, Massachusetts Institute of Technology, and Weill Cornell Medicine.

LinkedIn

Micah Sheller currently works as a senior research scientist in Intel's Security and Privacy Research Labs, where he leads secure federated learning research. He developed the first version of the OpenFL open-source federated learning platform and is the technical federated learning lead for the FeTS initiative, which recently trained a 3DResUNet across 53 hospitals. Micah has had the pleasure of working on a wide range of projects since his first Intel internship in 1999, when he worked on the Intel Web Tablet. Work in USB 3.0, the prototype Intel SGX software runtime, passive and continuous biometrics and more has kept Micah happily learning and making friends throughout his career.

Intel Bio

Spyridon Bakas is an Assistant Professor at the Perelman School of Medicine at the University of Pennsylvania, focusing on the development, application, and benchmarking of computational algorithms in medical imaging, with the intention of improving disease assessment and diagnosis in the current clinical practice. He has been leading projects on image quantification, radiogenomics, and federated learning, towards enabling treatment selection models customized on an individual patient basis, while addressing health disparities and inequities. He has published in numerous peer-reviewed journals and conferences, he is a board member of the MICCAI Society's Special Interest Group on Biomedical Image Analysis Challenges (SIG-BIAS) and has served as the organizer and chair of numerous computational challenges, workshops, and tutorials at both technical and clinical scientific meetings.

Academic Profile

Technical Leads

Johnu George is a staff engineer at Nutanix with a wealth of experience in building production grade cloud native platforms. He has a strong distributed systems background and has led efforts in building large scale hybrid data pipelines. He holds multiple patents in this area and has been an invited speaker at various conferences like Kubecon, Apache Big Data etc. He is an active open source contributor and has steered several industry collaborations on projects like Kubeflow, Apache Mnemonic and Knative. His current research interests include machine learning system design, distributed learning infrastructure improvements and ML workloads characterization. He is an Apache PMC member and currently chairing Kubeflow Training and AutoML Working groups.

Alejandro Aristizabal is a machine learing engineer with Factored. He has more than 4 years of experience in Software Development. He holds a bachelors degree in sound engineering. He is very interested in AI Safety, Security and Privacy, as well as human-machine interaction.

Alexander Getka is a software engineer with a focus on scientific computing and artificial intelligence. He has been working in imaging analysis and machine learning for medicine at the University of Pennsylvania, and holds a Bachelor’s degree in Bioengineering. In his free time, he enjoys experimenting with computing techniques in game development.

Hasan Kassem is a machine learning engineer with a background in MLOps and Federated Learning. He has been working for more than three years in the fields of software engineering and AI. He holds a master's degree in robotics and intelligent systems, where his focus was on federated machine learning for surgical applications. He enjoys his free time playing chess and understanding the psychology of human personalities.

Theme Leads

Prakash Narayana Moorthy is a research software engineer at the Security and Privacy research labs, Intel Corporation, where he has been working on the design and implementation of trustworthy distrusted system for the past 3 years. Prakash currently works on architectures for privacy-preserving smart contract platforms, exploring the role of blockchains in establishing trustworthy federated learning pipelines, and also acts as a security architect for the OpenFL federated learning platform. Prior to joining Intel, Prakash was a postdoc at the EECS department, Massachusetts Institute of Technology, where he invented multiple algorithms for consistent distributed data storage. Prakash holds a Ph.D. in electrical engineering from Indian Institute of Science, Bangalore. Past industry experiences include multiple internships with NetApp as well as working for more than 2 years as a wireless communications engineer for Beceem, a 4G fabless semiconductor company that got acquired by Broadcom.