AI Scouting Report – Part 1 of 3: Fundamentals

Nathan Labenz demystifies AI fundamentals, aiming to provide an AP course level understanding in 90 minutes, with insights on data, compute, and algorithms.

1970-01-01T02:05:48.000Z

Watch Episode Here

Video Description

Nathan Labenz synthesizes the core fundamentals of AI as simply, clearly, and literally as possible. Nathan's aim is to impart the equivalent of a high school AP course understanding to listeners in 90 minutes. This special episode is in part brought to you by Athena. You can use Nathan’s referral link (http://athenago.me/nathan-labenz) for a free first month of Athena services for qualified customers.

Questions or topics you want us to review for Part 2? Email TCR@turpentine.co
The Cognitive Revolution is a part of the Turpentine podcast network. Learn more: Turpentine.co

TIMESTAMPS:
(00:00) Context for the AI Scouting report and Athena’s offer for The Cognitive Revolution listeners.
(06:16) Headlines
(09:03) We now have human-level AI (That does not mean human)
(23:18) Is memory the next big shift in AI?
(16:42) Sponsor: Omneky
(28:02) How did we get here? Data, Compute, Algorithms.
(46:40) Foundation model foundations
(01:02:00) Scaling, Grokking, Emergence
(01:29:46) Certain capabilities remain hard to predict

PODCAST INTERVIEWS MENTIONED IN REPORT
- Red Teaming GPT-4 https://youtu.be/oLiheMQayNE
- Vivek Natarajan from Google's Med PaLM https://youtu.be/nPBd7i5tnEE
- The Medical AI Revolution with Dr. Zak Kohane https://youtu.be/pS5Vye671Xg
- Eugenia Kuyda of Replika https://youtu.be/SFKA7T-v6WE
- Seeing is Believing with Ziming Liu of MIT https://youtu.be/VKxwlxb2XWw
- The Tiny Stories Revolution https://youtu.be/mv3SIgDP_y4

TWITTER:
@CogRev_Podcast
@labenz (Nathan)
@eriktorenberg (Erik)

SPONSOR:
Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.


Full Transcript

Transcript

Nathan Labenz: 0:00 Hello, and welcome back to the Cognitive Revolution. Today, we're changing up the format, and I'm presenting part 1 of my AI scouting report for June 2023. The AI scouting report is my attempt to zoom out from all the day to day AI news and hype and help you focus on what really matters. I'm synthesizing the most important concepts and trends that I've obsessively studied for the last 2 years so that you can begin to engage with AI research, products, and discourse with a new level of sophistication. This is the first edition of the AI scouting report, and it comes in 2 parts. Part 1, which I'm presenting today, aims to cover the core fundamentals of AI as simply, clearly, and literally as possible, starting with a headline summary of current general purpose AI capabilities and then surveying how modern AIs are designed and trained, what the big capability unlocks have been, and what we know about how they work and what they truly understand. I'll consider it a huge success if you come away with the equivalent of a high school AP course level understanding of machine learning fundamentals. I've curated a bunch of the very best graphs, illustrations, and figures that I've seen, and I hope to explain them so that they make as much of an impression on you as they have on me. To do that in just 90 minutes is ambitious, and I do need the help of visual aids. So for this reason, the AI scouting report will be a YouTube series. We'll continue to post short audio only episodes to the podcast feed to announce each new episode as it's released, but you will need to watch the video to get the full experience. Part 2 of the AI scouting report coming soon will cover AI as it interfaces with the world, the latest trends in fine tuning, agent design, and memory systems, as well as market dynamics, economic impact and societal fallout, AGI timelines, live players, and the outlook for AI safety. Before getting started, I also want to share that the AI scouting report is brought to you in part by Athena. Athena is an executive assistant services company that pairs EAs in The Philippines with startup executives and other opportunity rich, time poor clients around the world. If you're a regular listener to the Cognitive Revolution, you know that I've mentioned my AI advisory work with Athena a couple times over the last few months. But what I haven't told is the backstory of how I originally started with Athena as a client 2 and a half years ago while I was still CEO of Waymark. I matched with a woman named Charisse. We started with typical EA tasks, email and calendar management, social media, ad hoc research projects. She did well, and I got a lot of projects done that I simply wouldn't otherwise have been able to get to. But things got really interesting when we made the move to generative AI. I found that the same core qualities that made Charisse a strong EA, natural organization, high attention to detail, a sense for the importance of outlier data points, facility with software, tolerance for some repetitive tasks, ability to balance speed and accuracy, the curiosity to ask why and the inclination to be proactive. All of these made her a natural AI operations pro as well. So over the last 2 years, working on our time zone, Therese has built a variety of critical datasets, has developed tons of prompts, fine tuned and validated more than a 100 custom models, maintained dozens of elaborate spreadsheets, and pitched in in many other ways. At this point, it's no exaggeration to say that she's become a core part of the Waymark team. And it was this experience with Trees that inspired me to think about working with Athena as an AI adviser to see if I could help the company replicate this success story across its unique client base. I've been advising the company for about 6 months now with a focus on foundational AI education for EAs and the creation of a task automation practice within Athena. We've now developed an AI 1 0 1 core skills class that covers prompting and savvy shopping. We've extended many of the Athena delegation playbooks with AI prompts, and we've developed some early task automation best practices and templates as well. Fun fact, the first audience after my wife, Amy, to watch this AI scouting report was in fact a group of 200 Athena EAs. 1 of Athena's core values as a company is always long term. That, of course, is a bit tricky in this crazy fast moving AI environment, but the company is investing in the human capital needed to ride the AI wave. And we're planning to share our experiments, findings, and automation templates with clients as much as possible along the way. Like everything in AI right now, it's still a work in progress and subject to all sorts of change. But we do feel at this point that we are as well positioned to help clients ride the AI wave as any EA company in the world. Now to be totally upfront, this service definitely isn't for everyone. It runs $3,000 a month, and that's enough to fund a strong salary in The Philippines, enough, in fact, to recruit management level talent that will work on your time zone and really integrate and fully support your life and team. Athena is willing to invest in listeners of the Cognitive Revolution by offering a first month of service entirely free for qualified customers. To claim this offer, you can visit my personal referral link. That is athenago.me/nathan-labenz.

Nathan Labenz: 5:50 That's athenago.me/nathan-labenz.

Nathan Labenz: 5:54 Of course, we'll place a link in the show notes as well. Now without further ado, follow the link to our YouTube channel for my AI scouting report from June 2023.

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to The Cognitive Revolution.

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.