Trustworthy & Responsible AI in Drug Discovery

Dr. Oliver Crook · University of Oxford

Right Answer, Wrong Reason

AI is now better than humans at many challenging scientific scenarios, predicting outcomes and modelling data, yet this alone is often insufficient for scientific discovery. A model may produce the correct answer while relying on spurious correlations, hidden biases, or shortcuts that fail outside the conditions in which it was trained.

This session explores the question of what information is AI really using and how to quantify. We will discuss how we can get a better understanding of what information an AI model is using with different approaches and discuss the implication for pharmaceutical science and trade-off against simpler, mechanistic models. We will draw examples from molecular and structural biology and chemistry.

Prof. Dr. Kerstin Lenhof · University Medical Center Göttingen & CAIMed

Trustworthy machine learning in medicine and medical research: concepts, tensions, and practice

Given the current machine learning (ML) adoption rate across the life sciences, ensuring trustworthiness, particularly in high-stakes domains that heavily impact human lives, e.g., medical decision-making, is paramount. However, defining what trustworthiness means and how to implement it in practice are intricate quests that require attention from various disciplines beyond the specific application domain and mathematics, e.g., philosophy, ethics, law, and the social sciences. In this talk, I will reconcile an ethical and technical perspective on trustworthy ML. To this end, I will briefly introduce trust(worthiness) from an ethical perspective, and then give an overview of technical design features of ML models that can contribute to the trustworthiness of, and trust in, ML models, including, amongst others, generalizability, reliability, robustness, privacy, security, interpretability, explainability, transparency, and fairness. By doing so, I will uncover ambiguities in definitions as well as interrelations and tensions between these concepts. In addition, I will provide illustrative examples that demonstrate how these concepts can enhance the trustworthiness of ML models in the medical domain.

The talk is loosely based on the preprint: Lenhof, K., Rolli, L. M., Buhr, L., Roth, S., Binkyte, R., Schicktanz, S., … & Beerenwinkel, N. (2025). The trustworthiness landscape in machine learning: a conceptual guide with applications in medicine.

Dr. Philipp Flotho · Saarland University

Sim-to-Real: Generalizing from Synthetic to Real-World Data

Real-world datasets are often scarce, noisy, biased, and lack reliable ground truth. As a result, training on synthetic data has emerged as a competitive strategy that can match or outperform training on labelled real data despite substantial realism gaps. However, successful sim-to-real transfer depends not only on realism, but on where synthetic data abstracts from reality. In this talk, we trace this idea from synthetic benchmarks and training data in computer vision to appearance-driven synthesis and biophysical simulation and connect these examples to autonomous driving, robotics, and learned world models. Across these domains, the central question is which exact factor must be modeled faithfully, which can be randomized, and which gaps can be closed using augmentation and real data. In single-cell RNA sequencing, existing simulators support controlled evaluation but not yet broad synthetic pretraining, and we discuss preliminary results on transfer to real biological data with hierarchical high-throughput procedural simulators.

Dr. Sofia Imperatore · Eindhoven University of Technology

A (deep) learning journey from point clouds to proteins

Establishing physically and chemically valid protein representations is a key problem in computational biology, and a wide range of deep learning methods have been developed in recent years to address it. Geometric deep learning, a deep learning paradigm that tackles non-Euclidean domains, results fundamental to learning from structural protein information. In this talk, we focus on point cloud representations, their geometric interpretation, and suitable deep learning architectures. We examine the problem of manifold (re-)construction, first addressing its geometric conceptualization for general point clouds and subsequently moving to its chemical formulation for protein point clouds. We analyze the similarities and differences between geometric and chemical point clouds, and highlight the fundamental information that protein point clouds and their models should embed. Finally, we also turn to the emerging topic of AI-assisted research, investigating how agents may support a variety of research tasks, among them literature review and coding, with particular emphasis on protein representation learning.

Dr. Giovanna Jaramillo-Gutierrez · EU AI Office, European Commission

The dual-use nature of General Purpose AI increasingly raises biosecurity concerns

The convergence of general-purpose AI models, and AI-enabled biodesign tools is reshaping the biosecurity landscape. While these technologies can be used to accelerate health research and preparedness, they can also lower barriers to misuse by supporting threat ideation, design, planning and access to dispersed technical know-how. The AI Act is Europe’s key tool (Code of Practice) to ensure providers of the most advanced AI models adequately assess and mitigate such risks at model level. The AI Office, through its role in supervising and enforcing the AI Acts’ obligations for providers of the most advanced models, has unique insights into model capabilities and guardrails. The AI Office’s activities include monitoring the risk landscape, detailed technical discussion with providers (Frontier AI labs), and recommending or if appropriate, enforcing, specific risk mitigations on the General Purpose AI model level. However, not all risks can presently be mitigated at model level, considering in particular the state of the art of model guardrails and the challenges posed by open-weight models. We argue that a defence-in-depth approach is thus needed to ensure that advanced AI can be widely used for beneficial purposes without materially increasing risks.

Dr. Jonas Fischer · Max Planck Institute for Informatics

Robust and Faithful Explanations of AI Decision-Making

Neural Networks provide outstanding decision-making in complex domains, including high-stakes biomedical applications. However, their reasoning remains largely opaque, making them hard to understand, which erodes trust and prevents locating and fixing mistakes. In this talk, I will discuss our work on building high performance AI systems that provide human-interpretable decision-making, and how to equip such systems with formal guarantees on the robustness of the given explanations.

Daniel Probst · Wageningen University & Research

Can You Trust This Prediction? A Workshop on AI in Drug Discovery

Machine learning is now used across drug discovery, from proposing candidate molecules to flagging likely ADME-Tox problems. The accuracy figures usually look good. What they do not tell you is whether to act on any single prediction, and most of the models producing them are black boxes. This workshop is about closing that gap in practice. We will work with real models and the messy datasets they are trained and evaluated on. Starting from the data itself, participants will see how its gaps and imbalances quietly limit what a model can be trusted to do. We will then open up an explainable model for enzyme-catalysed reactions, take its predictions, and follow each one back to the molecular fragments that drove it, so a chemist can check the reasoning. By the end, we will have a short checklist to run before trusting a model’s output.

Dr. Johannes Schimunek · Johannes Kepler University Linz

Earning Confidence in AI for Drug Discovery through Rigorous Research

AI is now woven through the pharmaceutical pipeline, from predicting molecular properties to designing new compounds. But as these models move closer to real decisions, a harder question follows: when can we actually trust them? Interpretability is one canonical answer. This talk emphasizes another: that the everyday work of rigorous ML research in drug discovery itself paves a path toward trustworthy models. Using scenarios typical of how AI is applied in pharma — from QSAR modeling to large language models as molecular reasoning systems — I show how asking the right research questions builds confidence in the models we rely on.

Dr. Miriam Mathea · BASF

Trustworthy and Reliable Molecular Property Prediction: Requirements, Challenges and Best Practices

Trustworthy molecular property prediction requires a lot of more things than strong benchmark performance. The reliability of predictions is crucial for decision-making, guiding research and establishing models in industry. Building on recent advances in applicability domains, uncertainty quantification, calibration, transformer-based architectures, and out-of-distribution (OOD) evaluation, this talk discusses how methodological progress in molecular machine learning can be translated into reliable decision support for chemical research and development.

The talk will give an overview of the concepts for building trustworthy machine learning models and report on latest research findings. We highlight that calibration and uncertainty estimates can directly influence model selection, particularly for imbalanced endpoints, and that in-distribution performance is not necessarily predictive of behavior of data that are OOD. Best practices therefore include validation strategies aligned with the intended use-case, explicit applicability-domain and OOD analysis, calibration-aware evaluation, and reproducible benchmarking. From an industrial perspective, these requirements must be met under additional constraints, including limited and heterogeneous data, evolving chemical spaces, class imbalance, data drifts, and the need to integrate everything into (semi)-automated retrainable workflows.

Dr. Gaël Varoquaux · Inria

Uncertainty in LLMs

Large language models, used as AIs, generate answers that are at odds with the information on which the model was trained or pre-trained. A good representation of uncertainty should enable to design AI systems less burdened by these shortcomings.

I will discuss how to capture and characterize the uncertainty that an LLMs has on embedded facts as well as given queries. I will detail the different underlying notions, and how they can be used to reject hallucinations and make better decisions from LLMs.

Meelis Lootus · CEO & Founder, Tehistark

The physics of trust in drug go-to-market: making your discoveries legible to regulators

The distance between a molecule that works in silico, and its prescription to a patient, is a decade, and a fortune. On its way to marketing authorisation, a molecule has to survive a gauntlet to earn regulators’ trust: pilot then industrial batches, stability and analytical assays, cells, organoids and animals, three phases of human trials, manufacturing scale-up and quality control. The award for the work is an approved “label” for the “drug product”, written against a “target product profile”. The rules governing the journey to market and the submission documents required on the way are easy to classify as administrative bureaucracy, partly correctly, partly naively. In this talk, I ask, (1) why is it that way, (2) does it have to be that way, (3) how this might impact your work in drug discovery, (4) how the system is changing today. I will answer these questions, drawing on my experience as startup operator and developer of AI systems to remove friction in drugs’ post-discovery journey.