{"version":1,"type":"rich","provider_name":"Libsyn","provider_url":"https:\/\/www.libsyn.com","height":90,"width":600,"title":"38.2 - Jesse Hoogland on Singular Learning Theory","description":"You may have heard of singular learning theory, and its &quot;local learning coefficient&quot;, or LLC - but have you heard of the refined LLC? In this episode, I chat with Jesse Hoogland about his work on SLT, and using the refined LLC to find a new circuit in language models. &amp;nbsp; Patreon: https:\/\/www.patreon.com\/axrpodcast Ko-fi: https:\/\/ko-fi.com\/axrpodcast The transcript:  https:\/\/axrp.net\/episode\/2024\/11\/27\/38_2-jesse-hoogland-singular-learning-theory.html FAR.AI: https:\/\/far.ai\/ FAR.AI on X (aka Twitter): https:\/\/x.com\/farairesearch &amp;nbsp;FAR.AI on YouTube: https:\/\/www.youtube.com\/@FARAIResearch The Alignment Workshop: https:\/\/www.alignment-workshop.com\/ &amp;nbsp; Topics we discuss, and timestamps: 00:34 - About Jesse 01:49 - The Alignment Workshop 02:31 - About Timaeus 05:25 - SLT that isn't developmental interpretability 10:41 - The refined local learning coefficient 14:06 - Finding the multigram circuit &amp;nbsp; Links: Differentiation and Specialization of Attention Heads via the Refined Local Learning Coefficient: https:\/\/arxiv.org\/abs\/2410.02984 Investigating the learning coefficient of modular addition: hackathon project:  https:\/\/www.lesswrong.com\/posts\/4v3hMuKfsGatLXPgt\/investigating-the-learning-coefficient-of-modular-addition &amp;nbsp; Episode art by Hamish Doodles:&amp;nbsp;hamishdoodles.com ","author_name":"AXRP - the AI X-risk Research Podcast","author_url":"https:\/\/axrp.net","html":"<iframe title=\"Libsyn Player\" style=\"border: none\" src=\"\/\/html5-player.libsyn.com\/embed\/episode\/id\/34173675\/height\/90\/theme\/custom\/thumbnail\/yes\/direction\/forward\/render-playlist\/no\/custom-color\/88AA3C\/\" height=\"90\" width=\"600\" scrolling=\"no\"  allowfullscreen webkitallowfullscreen mozallowfullscreen oallowfullscreen msallowfullscreen><\/iframe>","thumbnail_url":"https:\/\/assets.libsyn.com\/secure\/content\/181655600"}