Learning the Past Tense in AI

This episode revisits the famous debate over whether language is learned through symbolic rules or distributed neural patterns, using the classic irregular verb U-shaped curve as the battleground. It then compares child language development with training snapshots from modern AI models, showing that transformers learn past tense in a strikingly different way: starting with the broad rule and gradually memorizing the exceptions.