Arthur Samuel's 1959 Checkers Program & Machine Learning

A description of learning, not a slogan

Samuel's paper describes its subject as programming a computer "to behave in a way which, if done by human beings or animals, would be described as involving the process of learning." Its abstract is more concrete: testing verified a computer could be programmed to learn "a better game of checkers than can be played by the person who wrote the program," in 8 or 10 hours of machine-playing time. [2]

The tidy one-liner now attached to his name, that machine learning gives computers "the ability to learn without being explicitly programmed," appears nowhere in this paper: not the abstract, the introduction, or the results. [2]

Atlas interpretation: Three years after the Dartmouth workshop had already named artificial intelligence, Samuel needed no borrowed slogan. His own line, a program that outplays its author, is blunter and easier to check than the phrase later writers put in his mouth. [2]

Two ways the program got better

The program used two learning procedures. Rote learning saved every board position reached in play with the backed-up score a minimax search had assigned it; a recurring position deepened its next search by the levels already analyzed. A ply-based discount, shrinking a score slightly each level backed up, gave the program a sense of direction toward quicker wins, slower losses. [2]

Generalization learning instead adjusted the coefficients of the scoring polynomial, a weighted sum of board features (piece advantage, denial of occupancy, mobility, center control, in the four-term version tested). After each move, the program compared its prior evaluation with the backed-up score once the move was made, and shifted each coefficient by whether its sign matched that difference. [2]

Atlas interpretation: Sutton and Barto later read this as an early, incomplete relative of temporal-difference learning: nudging a value estimate toward a later, better-informed estimate of the same position, without the explicit rewards a modern system would supply. [3]

Lookahead, bounded by a clock

Move selection used a minimax procedure, working backward through a move tree to pick the move whose worst case scored best against a like-minded opponent. A ply was one proposed move plus one anticipated reply. The program searched a minimum of three plies, extended when a jump was available or had just occurred, and cut off at twenty plies once memory ran out; a move was budgeted about 30 seconds. [2]

Better than its author, not the masters

By Samuel's own account, the rote-learning version "now qualifies as a rather better-than-average novice, but definitely not an expert." [2]

Samuel's 1967 follow-up, after adding alpha-beta pruning and a longer parameter list, reported playing ability "greatly improved," the program still "unable to outplay checker masters." [4]

Atlas interpretation: That gap collapsed in retelling. A single 1962 win over a Connecticut player IBM billed as a top player, though his record never supported it, hardened into a claim the program had solved checkers, still repeated decades later. [5]