#158 Bayesian Workflows & Foundation Models, with Stefan Radev

Support & Resources
→ Support the show on Patreon
→ Bayesian Modeling Course (first 2 lessons free)

Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work

Takeaways:

Q: Why are prior predictive checks so underused in practice, and how do simulations help?
A: They're underused because researchers don't always think to run them before seeing data -- but also because doing them rigorously (in the style Michael Betancourt advocates, with prior push-forward checks on interpretable summaries) takes effort. Simulations make it cheap to generate thousands of “what-if world” datasets from your model and check whether they look plausible, catching bad priors before you ever touch real data.

Q: How can generative AI help with prior elicitation?
A: Rather than forcing a domain expert to choose a distributional family and parameterize it, you can use a generative model to translate their qualitative knowledge directly into a prior. The expert describes what realistic data should look like; the generative model produces synthetic datasets matching that description; those datasets are used to fit a prior distribution. It removes the assumption that experts can think in terms of parameters and replaces it with the more natural question: does this look like your data?

Q: What would a foundation model for Bayesian inference actually look like?
A: Stefan's bet is that it won't be a fine-tuned general LLM. The right analogy is chess: you don't fine-tune GPT to play chess, you teach it when to call Stockfish. For Bayesian inference, you'd want a semantic layer – an LLM that understands the analysis goal – calling specialized numerical engines (MCMC samplers, amortized inference networks) that do the actual computation. Agent skills are already a step in this direction; the longer-term vision is engines that have been trained from scratch to generalize across large families of models and priors.

Full takeaways here.

Chapters:
00:00 How does amortized inference fit into modern Bayesian workflows?
06:01 What role do simulations play across the full Bayesian workflow?
12:12 How do you elicit priors from a domain expert who doesn't think in distributions?
19:01 What would a foundation model for Bayesian inference actually look like?
35:32 What is self-consistency in amortized inference and why does it matter?
39:22 How does semi-supervised learning improve simulation-based inference?
43:16 Why is sensitivity analysis so important yet so underused in Bayesian practice?
47:40 What is multiverse analysis and how does it change how we report Bayesian results?
51:32 How does amortized inference make sensitivity and multiverse analysis affordable?
01:02:47 How do amortized inference and classical MCMC complement each other?
01:10:08 What are the next major directions for BayesFlow and amortized inference research?

Thank you to my Patrons for making this episode possible!

Links from the show here.