Research
My interests broadly lie in the field of trustworthy machine learning (ML). During my PhD, I focused on algorithmic transparency and its effects on AI-assisted decision-making. I am currently build tools for routing decision-makers to appropriate forms of decision support and for cataloging how AI systems are used in decision-making contexts all over the world.
You can find the most up-to-date list of my work on Google Scholar. The list below is updated periodically.
Journal and Conference Publications (Refereed and Archived)
-
Katherine Collins*, Ilia Sucholutsky*, Umang Bhatt*, Kartik Chandra*, Lionel Wong*, Mina Lee, Cedegao Zhang, Tan Zhi-Xuan, Mark Ho, Vikash Mansinghka, Adrian Weller, Joshua Tenenbaum, Thomas Griffiths
Building Machines that Learn and Think with People
Nature Human Behavior, 2024.
-
Umang Bhatt*, Holli Sargeant*
When Should Algorithms Resign? A Proposal for AI Governance
IEEE Computer, 2024.
-
Katherine Collins, Albert Jiang, Simon Frieder, Lionel Wong, Miri Zilka, Umang Bhatt, Thomas Lukasiewicz, Yuhuai Wu, Joshua Tenenbaum, William Hart, Timothy Gowers, Wenda Li, Adrian Weller, Mateja Jamnik
Evaluating Language Models for Mathematics through Interactionse
Proceedings of the National Academy of Sciences, 2024.
-
Sreejan Kumar, Raja Marjieh, Byron Zhang, Declan Campbell, Michael Hu, Umang Bhatt, Brenden Lake, Thomas Griffiths
Comparing Abstraction in Humans and Large Language Models Using Multimodal Serial Reproduction
46th Annual Conference of the Cognitive Science Society (CogSci), 2024.
-
Valerie Chen*, Umang Bhatt*, Hoda Heidari, Adrian Weller, Ameet Talwalkar
Perspectives on Incorporating Expert Feedback into Model Updates
Patterns, 2023.
-
Isa Inuwa-Dutse, Alice Toniolo, Adrian Weller, Umang Bhatt
Algorithmic Loafing and Mitigation Strategies in Human-AI Teams
Computers in Human Behavior: Artificial Humans, 2023.
-
Matthew Barker, Katherine Collins, Krishnamurthy Dvijotham, Adrian Weller, Umang Bhatt
Selective Concept Models: Permitting Stakeholder Customisation at Test-Time
AAAI Conference on Human Computation and Crowdsourcing (HCOMP), 2023.
-
Matthew Barker, Emma Kallina, Dhananjay Ashok, Katherine Collins, Ashley Casovan, Adrian Weller, Ameet Talwalkar, Valerie Chen, Umang Bhatt
FeedbackLogs: Recording and Incorporating Stakeholder Feedback into Machine Learning Pipeline
ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO), 2023.
-
Alan Chan, Rebecca Salganik, Alva Markelius, Chris Pang, Nitarshan Rajkumar, Dmitrii Krasheninnikov, Lauro Langosco, Zhonghao He, Yawen Duan, Micah Carroll, Michelle Lin, Alex Mayhew, Katherine Collins, Maryam Molamohammadi, John Burden, Wanru Zhao, Shalaleh Rismani, Konstantinos Voudouris, Umang Bhatt, Adrian Weller, David Krueger, Tegan Maharaj
Harms from Increasingly Agentic Algorithmic Systems
ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2023.
-
Katherine Collins, Matthew Barker, Mateo Espinosa Zarlenga, Naveen Raman, Umang Bhatt, Mateja Jamnik, Ilia Sucholutsky, Adrian Weller, Krishnamurthy Dvijotham
Human Uncertainty in Concept-Based AI Systems
AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES), 2023.
-
Zeju Qiu, Weiyang Liu, Tim Xiao, Zhen Liu, Yucen Luo, Umang Bhatt, Adrian Weller, Bernhard Scholkopf
Iterative Teaching by Data Hallucination
International Conference on Artificial Intelligence and Statistics (AISTATS), 2023.
-
Javier Abad Martinez, Umang Bhatt, Adrian Weller, Giovanni Cherubin
Approximating Full Conformal Prediction at Scale via Influence Functions
AAAI Conference on Artificial Intelligence (AAAI), 2023.
-
Mateo Zarlenga, Pietro Barbiero, Zohreh Shams, Dmitry Kazhdan, Umang Bhatt, Adrian Weller, Mateja Jamnik
Towards Robust Metrics for Concept Representation Evaluation
AAAI Conference on Artificial Intelligence (AAAI), 2023.
-
Katherine Collins, Umang Bhatt, Weiyang Liu, Vihari Piratla, Ilia Sucholutsky, Bradley Love, Adrian Weller
Human-in-the-Loop Mixup
Conference on Uncertainty in Artificial Intelligence (UAI), 2023. (Oral)
-
Ilia Sucholutsky, Ruairidh Battleday, Katherine Collins, Raja Marjieh, Joshua Peterson, Pulkit Singh, Umang Bhatt, Nori Jacoby, Adrian Weller, Thomas Griffiths
On the Informativeness of Supervision Signals
Conference on Uncertainty in Artificial Intelligence (UAI), 2023.
-
John Zerilli, Umang Bhatt, Adrian Weller
How Transparency Modulates Trust in Artificial Intelligence
Patterns, 2022.
-
Yuxin Xiao, Paul Pu Liang, Umang Bhatt, Willie Neiswanger, Ruslan Salakhutdinov, Louis-Philippe Morency
Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis
Findings of the Association for Computational Linguistics (EMNLP), 2022.
-
Katherine Collins*, Umang Bhatt*, Adrian Weller
Eliciting and Learning with Soft Labels from Every Annotator
AAAI Conference on Human Computation and Crowdsourcing (HCOMP), 2022.
-
Varun Babbar, Umang Bhatt, Adrian Weller
On the Utility of Prediction Sets in Human-AI Teams
International Joint Conference on Artificial Intelligence (IJCAI), 2022. (Oral)
-
Dan Ley, Umang Bhatt, Adrian Weller
Diverse and Amortised Counterfactual Explanations for Uncertainty Estimates
AAAI Conference on Artificial Intelligence (AAAI), 2022.
-
Julius von Kugelgen, Amir-Hossein Karimi, Umang Bhatt, Isabel Valera, Adrian Weller, Bernhard Scholkopf
On the Fairness of Causal Algorithmic Recourse
AAAI Conference on Artificial Intelligence (AAAI), 2022. (Oral)
-
Umang Bhatt, Javier Antoran, Yunfeng Zhang, Q. Vera Liao, Prasanna Sattigeri, Riccardo Fogliato, Gabrielle Melancon, Ranganath Krishnan, Jason Stanley, Omesh Tickoo, Lama Nachman, Rumi Chunara, Madhulika Srikumar, Adrian Weller, Alice Xiang
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES) 2021.
-
Javier Antoran, Umang Bhatt, Tameem Adel, Adrian Weller, Jose Miguel Hernandez-Lobato
Getting a CLUE: A Method for Explaining Uncertainty Estimates
International Conference on Learning Representations (ICLR), 2021. (Oral)
-
Matt Chapman-Rounds, Umang Bhatt, Erik Pazos, Marc-Andre Schulz, Konstantinos Georgatzis
FIMAP: Feature Importance by Minimal Adversarial Perturbation
AAAI Conference on Artificial Intelligence (AAAI), 2021.
-
Umang Bhatt, Adrian Weller, José M. F. Moura
Evaluating and Aggregating Feature-based Model Explanations
International Joint Conference on Artificial Intelligence (IJCAI), 2020.
-
Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jia, Joydeep Ghosh, Ruchir Puri, José M. F. Moura, Peter Eckersley
Explainable Machine Learning in Deployment
ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2020.
-
Botty Dimanov, Umang Bhatt, Mateja Jamnik, Adrian Weller
You shouldn’t trust me: Learning models which conceal unfairness from multiple explanation methods
European Conference on Artificial Intelligence (ECAI), 2020.
-
Brian Davis*, Umang Bhatt*, Kartikeya Bhardwaj*, Radu Marculescu, José M. F. Moura
On Network Science and Mutual Information for Explaining Deep Neural Networks
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020.
-
Aaron Roth, Samantha Reig, Umang Bhatt, Jonathan Shulgach, Tamara Amin, Afsaneh Doryab, Fei Fang, Manuela Veloso
A Robot’s Expressive Language Affects Human Strategy and Perceptions in a Competitive Game
IEEE International Conference on Robot and Human Interactive Communication (ROMAN), 2019.
-
Umang Bhatt*, Edgar Xi*, Shouvik Mani*, Zico Kolter
Intelligent Pothole Detection and Road Condition Assessment
Bloomberg Data for Good Exchange (D4GX), 2017.
Talk at UChicago Data Science for Social Good (DSSG), 2017.
Theses
- Umang Bhatt
Trustworthy Machine Learning: From Algorithmic Transparency to Decision Support
Universty of Cambridge, 2024.
Book Chapters (Refereed and Archived)
- Umang Bhatt, Zohreh Shams
Trust in Artificial Intelligence: Clinicians Are Essential
Healthcare Information Technology for Cardiovascular Medicine, 2021.
Workshop Publications (Refereed) – Not Updated Frequently
-
Ana Lucic, Sheeraz Ahmad, Amanda Furtado Brinhosa, Vera Liao, Himani Agrawal, Umang Bhatt, Krishnaram Kenthapadi, Alice Xiang, Maarten de Rijke, Nicholas Drabowski
Towards the Use of Saliency Maps for Explaining Low-Quality Electrocardiograms to End Users
ICML Workshop on Interpretable ML in Healthcare, 2022.
-
Valerie Chen*, Umang Bhatt*, Hoda Heidari, Adrian Weller, Ameet Talwalkar
Perspectives on Incorporating Expert Feedback into Model Updates
ICML Workshop on Updatable Machine Learning, 2022.
-
Umang Bhatt, Adrian Weller, Giovanni Cherubin
Fast Conformal Classification using Influence Functions
10th Symposium on Conformal and Probabilistic Prediction with Applications (COPA), 2021.
-
Dan Ley, Umang Bhatt, Adrian Weller
Diverse and Amortised Counterfactual Explanations for Uncertainty Estimates
ICML Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI, 2021.
-
Andrei Margeloiu*, Matthew Ashman*, Umang Bhatt*, Yanzhi Chen, Mateja Jamnik, Adrian Weller
Do Concept Bottleneck Models Learn As Intended?
ICLR Workshop on Responsible AI, 2021.
-
Dan Ley, Umang Bhatt, Adrian Weller
δ-CLUE: Diverse Sets of Explanations for Uncertainty Estimates
ICLR Workshop on Security and Safety in Machine Learning Systems, 2021.
-
Ana Lucic, Madhulika Srikumar, Umang Bhatt, Alice Xiang, Ankur Taly, Q. Vera Liao, Maarten de Rijke
A Multistakeholder Approach Towards Evaluating AI Transparency Mechanisms
CHI Workshop on Operationalizing Human-centered Perspectives in Explainable AI, 2021.
-
Torgyn Shaikhina*, Umang Bhatt*, Roxanne Zhang, Konstantinos Georgatzis, Alice Xiang, Adrian Weller
Effects of Uncertainty on the Quality of Feature Importance Estimates
AAAI Workshop on Explainable Agency in Artificial Intelligence, 2021.
-
Julius von Kugelgen, Umang Bhatt, Amir-Hossein Karimi, Isabel Valera, Adrian Weller, Bernhard Scholkopf
On the Fairness of Causal Algorithmic Recourse
NeurIPS Workshop on Algorithmic Fairness through the Lens of Causality and Interpretability, 2020.
-
Umang Bhatt, McKane Andrus, Adrian Weller, Alice Xiang
Machine Learning Explainability for External Stakeholders
ICML Workshop on Extending Explainable AI: Beyond Deep Models and Classifiers, 2020.
ICML Workshop on Human Interpretability, 2020. (Spotlight)
IJCAI Workshop on Explainable AI, 2020.
-
Javier Antoran, Umang Bhatt, Jose Miguel Hernandez-Lobato, Adrian Weller, Tameem Adel
Getting a CLUE: A Method for Explaining Uncertainty Estimates
ICLR Workshop on Machine Learning in Real Life (ML-IRL), 2020. (Oral)
-
Umang Bhatt, Adrian Weller, Muhammad Bilal Zafar, Krishna Gummadi
Counterfactual Accuracies of Alternative Models
ICLR Workshop on Machine Learning in Real Life (ML-IRL), 2020.
-
Botty Dimanov, Umang Bhatt, Mateja Jamnik, Adrian Weller
Models can be learned to conceal unfairness from explanation methods
AAAI Workshop on Safe AI, 2020. (Oral)
-
Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jia, Joydeep Ghosh, Ruchir Puri, José M. F. Moura, Peter Eckersley
Explainable Machine Learning in Deployment
NeurIPS Workshop on Human-Centric Machine Learning (HCML), 2019.
-
Umang Bhatt, Brian Davis, José M. F. Moura
Diagnostic Model Explanations: A Medical Narrative
AAAI Spring Symposium on Interpretable AI for Well-being, 2019.
Best Paper Award
-
Brian Davis*, Umang Bhatt*, Kartikeya Bhardwaj*, Radu Marculescu, José M. F. Moura
NIF: A Framework for Quantifying Neural Information Flow in Deep Networks
AAAI Workshop on Network Interpretability for Deep Learning, 2019.
-
Umang Bhatt, Pradeep Ravikumar, José M. F. Moura
Towards Aggregating Weighted Feature Attributions
AAAI Workshop on Network Interpretability for Deep Learning, 2019.
-
Umang Bhatt, Pradeep Ravikumar, José M. F. Moura
Building Human-Machine Trust via Interpretability
AAAI Conference on Artificial Intelligence (AAAI), 2019. (Student Abstract)
-
Aaron Roth*, Umang Bhatt*, Tamara Amin*, Afsaneh Doryab, Fei Fang, Manuela Veloso
The Impact of Humanoid Affect Expression on Human Behavior in Game-Theoretic Setting
IJCAI Workshop on Humanizing Artificial Intelligence, 2018. (Oral)
-
Umang Bhatt
Maintaining the Humanity of Our Models
AAAI Spring Symposium on AI and Society: Ethics, Safety and Trustworthiness in Intelligent Agents, 2018.