Google’s Med-PaLM and Med-PaLM2 with Vivek Natarajan

Watch Episode Here

Video Description

Nathan sits down with Vivek Natarajan, research scientist at Google Health. Vivek leads the Google Brain moonshot behind Med-PaLM, Google’s flagship medical large language model, featured in The Economist, The Scientific American, CNBC, and Forbes. In this episode, they discuss the foundational models that Vivek and team built before Med-PaLM, the techniques used to develop Med-PaLM which will be of interest to anyone developing AI systems for high-stakes use cases, and the capabilities for Med-PaLM to equalize access to medical knowledge and care.

This episode is part of a series centered on talking to the people at the cutting edge of building AI-driven solutions in medicine.

PODCAST RECOMMENDATION:
Youtube: @UpstreamwithErikTorenberg
Audio: https://link.chtbl.com/Upstream

LINKS:
https://sites.research.google/med-palm/

FEEDBACK / COLLABORATE WITH NATHAN:
TCR@turpentine.co

TIMESTAMPS:
(00:00) Episode preview
(03:43) The story of how Med-PaLM came to be
(09:41) Building Med-PaLM’s infrastructure
(13:10) The US medical licensing exam as a measure of AI progress
(15:23) Sponsor: Omneky
(18:17) Practicality of benchmarking in real-world usage
(21:39) Overcoming the shortfalls of Flan-PaLM with Med-PaLM
(25:08) Choosing to use soft prompting over few shot prompting
(30:36) The process of training Flan-PaLM
(37:31) A curriculum approach to soft-prompting
(38:43) Layperson vs expert interactions with LLMs
(43:54) How did the Google team facilitate user exploration of the model’s capabilities?
(46:58) Shift in techniques from Med-PaLM to Med-PaLM2
(50:21) Using different prompting strategies with Med-PaLM2
(57:33) Is Med-PaLM 2 preferred over clinicians?
(01:02:28) Will there be a multimodal version of Med-PaLM?
(01:04:52) Breakthroughs required for AI to further advance human potential
(01:10:23) The Med-PaLM business plan
(01:12:08) Is there a vision for a consumer product?
(01:15:46) The pros and cons of pre-training a model
(01:19:45) Vivek’s favorite AI products
(01:21:01) Would Vivek get a Neuralink implant?
(01:23:08) AI hopes and fears

TWITTER:
@CogRev_Podcast
@vivnat (Vivek)
@labenz (Nathan)
@eriktorenberg (Erik)

Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

Music Credit: MusicLM

More show notes and reading material released in our Substack: https://cognitiverevolution.substack.com/