Machine Learning Street Talk (MLST)

chevron_right

#107 - Dr. RAPHAËL MILLIÈRE - Linguistics, Theory of Mind, Grounding

whatshot 10 snips

Mar 13, 2023

Guest

Raphaël Millière

In this engaging discussion, Dr. Raphaël Millière, a Columbia University lecturer in philosophy, delves into the intersection of AI, linguistics, and cognition. He explores how deep learning challenges traditional notions of self-representation and consciousness. Millière tackles the complexities of mimicry in AI, uncovering biases it may perpetuate. He also analyzes the limitations of large language models, emphasizing the grounding problem and the intricacies of human-like understanding, raising thought-provoking questions about the future of AI and its ethical implications.

01:43:54

forum

Ask episode

web_stories

AI Snips

view_agenda

Chapters

menu_book

Books

auto_awesome

Transcript

info_circle

Episode notes

insights

INSIGHT

Semantic Competence vs. Understanding

'Understanding' in language models should be viewed as semantic competence, encompassing lexical and structural competence.
Lexical competence can be referential (mapping words to real-world objects) or inferential (relating words within language).

question_answer

ANECDOTE

SHRDLU vs. Language Models

SHRDLU, a classical symbolic AI, demonstrates referential competence by manipulating virtual objects.
Language models lean towards inferential competence, deriving meaning from co-occurrence patterns.

insights

INSIGHT

Compression and Generalization

Language models compress data by learning underlying patterns, enabling generalization.
Lossy compression analogy is misleading, as inference isn't simply regurgitating degraded data.

Get the Snipd Podcast app to discover more snips from this episode

Exploring AI's Understanding: Beyond Language and Perception

02:32 • 3min

chevron_right

The Complexities of Mimicry in Machine Learning

05:15 • 2min

chevron_right

Decoding Neural Networks and Human Cognition

07:12 • 23min

chevron_right

Unraveling Language Models: Beyond Mimicry

30:18 • 16min

chevron_right

Exploring Intelligence in Language Models

46:10 • 20min

chevron_right

Exploring Analogies and Analogical Reasoning in Language Models

01:06:15 • 4min

chevron_right

Grounding Language Models: Concepts and Challenges

01:09:47 • 15min

chevron_right

Language Models and the Theory of Mind

01:24:32 • 19min

chevron_right

#17311

• Mentioned in 3 episodes

The Little Big Things

Henry Fraser

The Little Big Things is Henry Fraser's memoir, detailing his journey after a tragic accident left him paralyzed. The book shares his insights on resilience, gratitude, and the importance of focusing on what can be done rather than what cannot. It offers a powerful message of hope and inspiration for anyone facing obstacles.

#97

• Mentioned in 177 episodes

Tao te ching

Laozi

The Tao Te Ching is a central text in Taoist philosophy and religion. It consists of 81 brief chapters or sections that discuss the nature of the Tao, which is described as the source and ideal of all existence. The text emphasizes living in harmony with nature, the importance of simplicity, humility, and the interconnectedness of all things. It critiques unnatural actions and social activism based on abstract moralism, advocating for a life of 'nonaction' (wu wei) and spontaneity. The text has been highly influential in Chinese philosophy and has been translated numerous times, making it one of the most translated texts in world literature.

#89896

The Symbol Grounding Problem

Stevan Harnad

The symbol grounding problem explores how symbols, such as words or abstract representations, acquire meaning by being tied to the physical world. It is a central issue in understanding how mental states become meaningful and is closely related to the problem of consciousness. While there isn't a specific book titled 'The Symbol Grounding Problem', the concept is extensively discussed in academic literature.

#18152

• Mentioned in 3 episodes

Connectionism and Cognitive Architecture: A Critical Analysis

Jerry A. Fodor

Zenon W. Pylyshyn

In this influential paper, Fodor and Pylyshyn challenge connectionist theorists by arguing that connectionist explanations, at best, can only inform us about details of the neural substrate but fail to explain the systematicity and complexity of adult human cognition. They contend that classical cognitive architectures, which operate on symbols, are necessary to account for the systematic nature of human thought and behavior. The paper contrasts connectionist models with classical models derived from the structure of Turing and Von Neumann machines, emphasizing the importance of symbolic representations in understanding cognition.

#86010

Big Bench

Rett Warren

No information available about this book.

#41221

The New Zealand experiment

Jane Kelsey

Jane Kelsey's book provides a detailed critique of the economic reforms implemented in New Zealand, examining their social and economic consequences. It offers a sobering perspective on the effects of neoliberal policies.

#43612

Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data

A Position Paper on Natural Language Understanding

Emily M. Bender

Alexander Koller

This paper critiques the notion that large neural language models truly 'understand' language, arguing that they primarily learn form rather than meaning. It emphasizes the need for clarity in distinguishing between form and meaning in NLP research to progress towards genuine NLU.

#102772

Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data

Emily M. Bender

Alexander Koller

This paper argues that large neural language models, despite their successes, do not truly understand language. It emphasizes the distinction between linguistic form and meaning, suggesting that meaning cannot be learned solely from form. The authors advocate for a clearer understanding of these concepts to guide future research in natural language understanding.

#50905

Do Language Models Know the Way to Rome?

Mostafa Abdou

Bastien Liétard

Anders Søgaard

This paper examines the extent to which language models can represent geographic knowledge, such as understanding the relationship between city and country names. It finds that larger models perform better in encoding geographic information, which can be inferred from higher-order co-occurrence statistics in their training data.

#87217

Othello-GPT Papers

Research on Transformer Models for Othello

Neel Nanda

Kenneth Li et al.

The Othello-GPT papers involve training transformer models to predict next moves in the game of Othello. These models learn to internally represent the board state, even when not explicitly given. The research demonstrates that such models can achieve high accuracy in predicting legal moves and exhibit emergent world representations, which can be probed and intervened upon.

#67406

Computational Language Acquisition with Theory of Mind

This paper investigates how incorporating Theory of Mind (ToM) into language acquisition models can enhance their performance. It uses an image referential game setting to test the effects of ToM on language learning, finding that models with a strong ToM component perform better and produce more fluent utterances.

Support us! https://www.patreon.com/mlst

MLST Discord: https://discord.gg/aNPkGUQtc5

Dr. Raphaël Millière is the 2020 Robert A. Burt Presidential Scholar in Society and Neuroscience in the Center for Science and Society, and a Lecturer in the Philosophy Department at Columbia University. His research draws from his expertise in philosophy and cognitive science to explore the implications of recent progress in deep learning for models of human cognition, as well as various issues in ethics and aesthetics. He is also investigating what underlies the capacity to represent oneself as oneself at a fundamental level, in humans and non-human animals; as well as the role that self-representation plays in perception, action, and memory. In a world where technology is rapidly advancing, Dr. Millière is striving to gain a better understanding of how artificial neural networks work, and to establish fair and meaningful comparisons between humans and machines in various domains in order to shed light on the implications of artificial intelligence for our lives.

https://www.raphaelmilliere.com/

https://twitter.com/raphaelmilliere

Here is a version with hesitation sounds like "um" removed if you prefer (I didn't notice them personally): https://share.descript.com/view/aGelyTl2xpN

YT: https://www.youtube.com/watch?v=fhn6ZtD6XeE

TOC:

Intro to Raphael [00:00:00]

Intro: Moving Beyond Mimicry in Artificial Intelligence (Raphael Millière) [00:01:18]

Show Kick off [00:07:10]

LLMs [00:08:37]

Semantic Competence/Understanding [00:18:28]

Forming Analogies/JPG Compression Article [00:30:17]

Compositional Generalisation [00:37:28]

Systematicity [00:47:08]

Language of Thought [00:51:28]

Bigbench (Conceptual Combinations) [00:57:37]

Symbol Grounding [01:11:13]

World Models [01:26:43]

Theory of Mind [01:30:57]

Refs (this is truncated, full list on YT video description):

Moving Beyond Mimicry in Artificial Intelligence (Raphael Millière)

https://nautil.us/moving-beyond-mimicry-in-artificial-intelligence-238504/

On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 (Bender et al)

https://dl.acm.org/doi/10.1145/3442188.3445922

ChatGPT Is a Blurry JPEG of the Web (Ted Chiang)

https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web

The Debate Over Understanding in AI's Large Language Models (Melanie Mitchell)

https://arxiv.org/abs/2210.13966

Talking About Large Language Models (Murray Shanahan)

https://arxiv.org/abs/2212.03551

Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data (Bender)

https://aclanthology.org/2020.acl-main.463/

The symbol grounding problem (Stevan Harnad)

https://arxiv.org/html/cs/9906002

Why the Abstraction and Reasoning Corpus is interesting and important for AI (Mitchell)

https://aiguide.substack.com/p/why-the-abstraction-and-reasoning

Linguistic relativity (Sapir–Whorf hypothesis)

https://en.wikipedia.org/wiki/Linguistic_relativity

Cooperative principle (Grice's four maxims of conversation - quantity, quality, relation, and manner)

https://en.wikipedia.org/wiki/Cooperative_principle

Home Top podcasts Popular guests Top books