Sebastian Caldas

I’m a Ph.D. student in Machine Learning at Carnegie Mellon University. I’m broadly interested in building collaborative intelligent systems that are useful in the real world. Currently, I’m focused on federated scenarios, particularly healthcare domains. I’m a member of the AutonLab, advised by Prof. Artur Dubrawski.

selected publications

MLHC

Understanding Clinical Collaborations Through Federated Classifier Selection

Sebastian Caldas, Joo Heung Yoon, Michael R Pinsky, Gilles Clermont, and Artur Dubrawski

In Machine Learning for Healthcare Conference 2021

Abs HTML PDF Poster Slides Talk

Deriving true clinical utility from models trained on multiple hospitals’ data is a key challenge in the adoption of Federated Learning (FL) systems in support of clinical collaborations. When utility is equated to predictive power, population heterogeneity between centers becomes a key bottleneck in training performant models. Nevertheless, there are other aspects to clinical utility that have frequently been overlooked in this context. Among them, we argue for the importance of understanding how a collaboration may be affecting the quality of a center’s predictions. Insights into how and when external knowledge is being useful can lead to strategic decisions by stakeholders, such as better allocation of local resources or even identifying best practices outside of the current organization. We take a step towards deriving such utility through FedeRated CLassifier Selection (FRCLS, pronounced “freckles”): an algorithm that reuses classifiers trained in outside institutions. It identifies regions of the feature space where the collaborators’ models will outperform the local center’s classifier, and can provide interpretable rules to describe these regions of beneficial expertise. We apply FRCLS to a sepsis prediction task in two different hospital systems, demonstrating its benefits in terms of understanding the types of patients for which the collaboration is useful and reasoning about the strategic decisions that may stem out of these analyses.
FL-NeurIPS

LEAF: A benchmark for federated settings

Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečnỳ, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar

Workshop on Federated Learning for Data Privacy and Confidentiality, FL-NeurIPS 2019

Abs arXiv PDF Code Poster Website

Modern federated networks, such as those comprised of wearable devices, mobile phones, or autonomous vehicles, generate massive amounts of data each day. This wealth of data can help to learn models that can improve the user experience on each device. However, the scale and heterogeneity of federated data presents new challenges in research areas such as federated learning, meta-learning, and multi-task learning. As the machine learning community begins to tackle these challenges, we are at a critical time to ensure that developments made in these areas are grounded with realistic benchmarks. To this end, we propose LEAF, a modular benchmarking framework for learning in federated settings. LEAF includes a suite of open-source federated datasets, a rigorous evaluation framework, and a set of reference implementations, all geared towards capturing the obstacles and intricacies of practical federated environments.