Bit of a Tangent
025 | Self-Supervised Machine Learning: Introduction, Intuitions, and Use-Cases

025 | Self-Supervised Machine Learning: Introduction, Intuitions, and Use-Cases

March 5, 2020

On this episode of Bit of A Tangent, we discuss the emerging field of self-supervised machine learning. This is an immensely exciting area of active research in machine learning and AI - one which most people haven’t even heard about yet! We build up to the intuition for the topic by covering supervised and unsupervised learning; autoencoders and dimensionality reduction, and exploring how these techniques could be applied to Gianluca’s Quantified Self n=1 sleep quality dataset. We culminate in a detailed discussion of the state-of-the-art Contrastive Predictive Coding model, and how it allows us to learn about the structure of the world, without tonnes of labelled training data!

--------

Shownotes:

--------

Jared on Twitter: www.twitter.com/jnearestn

Gianluca on Twitter: www.twitter.com/QVagabond

Bit of a Tangent on Twitter (www.twitter.com/podtangent) and Instagram (instagram.com/podtangent/)

Summer school on Computational Neuroscience: http://imbizo.africa/

Control problem in AI: https://intelligence.org/stanford-talk/

Coordination problem: https://conceptually.org/concepts/coordination-problems

Deep learning overview: https://lilianweng.github.io/lil-log/2017/06/21/an-overview-of-deep-learning.html

t-SNE explained: https://mlexplained.com/2018/09/14/paper-dissected-visualizing-data-using-t-sne-explained/

Variational autoencoders explained: https://anotherdatum.com/vae.html

Self-supervised learning by fast.ai: https://www.fast.ai/2020/01/13/self_supervised/

CPC model papers on Arxiv: https://arxiv.org/pdf/1807.03748.pdf https://arxiv.org/pdf/1905.09272.pdf 

Blog posts explaining CPC: https://lilianweng.github.io/lil-log/2019/11/10/self-supervised-learning.html

https://yann-leguilly.gitlab.io/post/2019-09-29-representation-learning-with-contrastive-predictive-coding/

https://mf1024.github.io/2019/05/27/contrastive-predictive-coding/

024 | Replacing Guilt

024 | Replacing Guilt

February 4, 2020

Gianluca announces his latest project—the audio version of Nate Soares’ Replacing Guilt series—and Jared explains why it’s a notable contribution to the rationalist community. Together, they discuss how to gain the most from listening to the series and why it’s something you should strongly consider doing if you often find yourself binging Netflix until 3 AM.  

--------

Shownotes:

--------

Replacing Guilt podcast: https://anchor.fm/guilt

Rationality: From AI to Zombies podcast: www.from-ai-to-zombies.eu/

SlateStarCodex podcast: https://podcasts.apple.com/us/podcast/slate-star-codex-podcast/id1295289140

Harry Potter and the Methods of Rationality (audio): http://www.hpmorpodcast.com/

Jared on Twitter: www.twitter.com/jnearestn

Gianluca on Twitter: www.twitter.com/QVagabond

Bit of a Tangent on Twitter (twitter.com/podtangent) and Instagram (instagram.com/podtangent/)

023 | Are Brains Just Neural Nets?

023 | Are Brains Just Neural Nets?

January 29, 2020

Jared and Gianluca have an uncompromising exchange about computational neuroscience and its implications for life and AI research.

---------------

Shownotes:

---------------

Jared on Twitter: www.twitter.com/jnearestn

Gianluca on Twitter: www.twitter.com/QVagabond

Bit of a Tangent on Twitter (twitter.com/podtangent) and Instagram (instagram.com/podtangent/)

Last episode: www.podtangent.com/e/022-6-books-you-need-to-read-in-2020/

Ideas episode: www.podtangent.com/e/021-the-top-19-ideas-from-2019/

The Portal podcast episode with Garrett Lisi: podcasts.apple.com/us/podcast/15-garrett-lisi-my-arch-nemesis-myself/id1469999563?i=1000458879827

Don Hoffman on Sam Harris’ podcast: https://samharris.org/subscriber-extras/178-reality-illusion/

022 | 6 Books You Need to Read in 2020… and 1 You Probably Shouldn’t

022 | 6 Books You Need to Read in 2020… and 1 You Probably Shouldn’t

January 21, 2020

Gianluca presents his much-requested three non-fiction and three fiction recommendations for 2020. Jared recommends a book so long and gripping that you should probably avoid it if you have anything productive to do in the next six months. 

---------------

Shownotes:

---------------

Jared (twitter.com/jnearestn) and Gianluca (twitter.com/QVagabond) on Twitter

Bit of a Tangent on Twitter (twitter.com/podtangent) and Instagram (www.instagram.com/podtangent/)

Last week’s episode: www.podtangent.com/e/021-the-top-19-ideas-from-2019/

Gianluca on Goodreads: www.goodreads.com/user/show/56535709-gianluca

Books Gianluca read in 2019: www.goodreads.com/user_challenges/14773941

Algorithms to Live By, by Brian Christian and Tom Griffiths: www.goodreads.com/book/show/25666050-algorithms-to-live-by

80000 Hours podcast with one of the authors of Algorithms to Live By: https://80000hours.org/podcast/episodes/brian-christian-algorithms-to-live-by/

Inadequate Equilibria by Eliezer Yudkowsky: https://equilibriabook.com/

Eliezer Yudkowsky’s Sequences: https://wiki.lesswrong.com/wiki/Sequences

007 | Game Theory, Alignment, and Civilisational Inadequacy: https://www.podtangent.com/e/007-game-theory-alignment-and-civilisational-inadequacy/

Meditations on Moloch: https://slatestarcodex.com/2014/07/30/meditations-on-moloch/

The Replacing Guilt Series by Nate Soares: http://mindingourway.com/guilt/

Half-assing it with everything you’ve got: http://mindingourway.com/half-assing-it-with-everything-youve-got/

Discussions of Replacing Guilt on The Bayesian Conspiracy podcast: www.thebayesianconspiracy.com/2019/08/92-replacing-guilt/

Surfing Uncertainty by Andy Clark: https://www.goodreads.com/book/show/25823558-surfing-uncertainty

Good Omens by Terry Pratchett and Neil Gaiman: www.goodreads.com/book/show/12067.Good_Omens

Neil Gaiman on the Tim Ferriss show: https://tim.blog/2019/03/28/neil-gaiman/

The Metropolitan Man by Alexander Wales: www.goodreads.com/book/show/22872436-the-metropolitan-man

Alexander Wales on the Bayesian Conspiracy podcast: www.thebayesianconspiracy.com/2018/07/63-rational-fiction/

The Metropolitan Man audiobook (free podcast): www.hpmorpodcast.com/?page_id=1705

The Three-Body Problem by Liu Cixin: www.goodreads.com/book/show/20518872-the-three-body-problem

The Paper Menagerie by Ken Liu: www.goodreads.com/book/show/39803561-the-paper-menagerie

Worm (the webseries you should probably never read): https://parahumans.wordpress.com/

021 | The Top 19 Ideas from 2019

021 | The Top 19 Ideas from 2019

January 14, 2020

Bit of a Tangent is back! In this episode, Jared shares the 19 ideas he couldn’t stop thinking about in 2019, whilst Gianluca does two B-grade Joe Rogan impressions. Together, they discuss everything from mechanism design to personal fitness hacks, and throw in some recommendations for supercharging your learning in the new year. Strap yourselves in for a taste of the fascinating conversations that lie ahead in 2020!

---------------

Shownotes:

---------------

Jared (twitter.com/jnearestn) and Gianluca (twitter.com/QVagabond) on Twitter

Jared’s Twitter thread (19 ideas I couldn't stop thinking about in 2019): https://twitter.com/jnearestn/status/1211681767742156803

Eliezer Yudkowsky’s Sequences: https://wiki.lesswrong.com/wiki/Sequences

Meditations on Moloch: https://slatestarcodex.com/2014/07/30/meditations-on-moloch/

Radical Markets: https://www.goodreads.com/book/show/36515770-radical-markets

007 | Game Theory, Alignment, and Civilisational Inadequacy: https://www.podtangent.com/e/007-game-theory-alignment-and-civilisational-inadequacy/

Glen Weyl on Twitter: https://twitter.com/glenweyl

Mechanism design: https://en.wikipedia.org/wiki/Mechanism_design

Tyler Cowen on Twitter: https://twitter.com/tylercowen

Surfing Uncertainty by Andy Clark: https://www.goodreads.com/book/show/25823558-surfing-uncertainty

Fast.AI courses: https://www.fast.ai/

When someone tries to “Euler” you: https://slatestarcodex.com/2014/08/12/does-race-exist-does-culture/

Body by Science, by Little and McGuff: https://www.goodreads.com/book/show/4642174-body-by-science

FitNotes app for Android: https://www.fitnotesapp.com/

All Debates are Bravery Debates on SlateStarCodex: https://slatestarcodex.com/2013/06/09/all-debates-are-bravery-debates/

Tim Ferriss on Twitter: https://twitter.com/tferriss

Naval Ravikant on Twitter: https://twitter.com/naval

011 | Slack, attention and focus: https://www.podtangent.com/e/slack-attention-and-focus-or-why-more-is-less/

010 | Flow states, optimal performance, and PhD hunter-gathering: https://www.podtangent.com/e/010-flow-states-optimal-performance-and-phd-hunter-gathering

Michael Nielsen on Twitter: https://twitter.com/michael_nielsen

Anki for spaced repetition: https://apps.ankiweb.net/

013 | How To Learn Anything: https://www.podtangent.com/e/013-how-to-learn-anything/

The 7 +-2 principle for memory: https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus_or_Minus_Two

Chunking: https://en.wikipedia.org/wiki/Chunking_(psychology)

The Elephant in the Brain by Kevin Simler, Robin Hanson: https://www.goodreads.com/book/show/28820444-the-elephant-in-the-brain

Inadequate Equilibria by Eliezer Yudkowsky: https://www.goodreads.com/book/show/36606376-inadequate-equilibria

Robin Hanson on Twitter: https://twitter.com/robinhanson

Awareness by Anthony de Mello: https://www.goodreads.com/book/show/94318.Awareness

The Cook and the Chef on WaitButWhy: https://waitbutwhy.com/2015/11/the-cook-and-the-chef-musks-secret-sauce.html

Sam Harris on Twitter: https://twitter.com/SamHarrisOrg

Sam’s Waking Up app: https://wakingup.com/

Podbean App

Play this podcast on Podbean App