Reproducibility and AI: Potential, Challenges, and Other Preliminary Thoughts

Presentation: https://larsvilhuber.github.io/reproducibility-for-llm/presentation/

Abstract

This talk will explore how AI, while offering great potential for enhancing research, introduces challenges that parallel the historical difficulties researchers have faced when using black-box systems, commercial software, or external APIs. By examining AI’s role in reshaping traditional research workflows, we will discuss issues such as algorithmic transparency, data dependencies, and the difficulty of archiving machine learning models and outputs. The talk will highlight key strategies for ensuring reproducibility in AI-driven research and propose pathways for researchers to navigate this evolving landscape.