Blackjack AI

Blackjack's per-round reward has a standard deviation near 1.15, so a short test of the same agent can read anywhere from brilliant to broken — which is why most RL demos quietly report win rate and declare victory. The learning problem is equally adversarial: rewards arrive only when a hand settles, doubling is rare and pays double so its Q-value stays noisy, and a split turns one chain of decisions into two that both need credit. A policy can also look right and be wrong — an early agent that mis-split 27 of 100 pair cells still audited well.

State is bit-packed into a 16-bit hash indexing a flat 65,536-slot Q-table, with canSplit/canDouble in the key so the opening, already-hit and pair policies learn independently, plus per-action visit counts that separate 'evaluated and rejected' from 'never tried'. A visit-scaled step size (α = 1/(1+n)) fixed doubling, and SPLIT bootstraps from both resulting hands so a split is valued as two. Every claim runs through one seeded evaluator against in-engine basic-strategy and random baselines, and experiments are pre-registered in git: candidates picked on exploration seeds, then confirmed on fresh ones.

The agent recovers 99.0% of the EV gap between random play and the book, with a residual shortfall of 0.0017 per round confirmed on ten fresh seeds at 1M rounds each, and matches basic strategy on 94.9% of audited cells. Learned bet sizing off the Hi-Lo true count turns a −0.58% game into +0.27% per unit wagered — beating the textbook ramp outright and tying Hi-Lo + Illustrious 18 while winning twice as much per round; count-aware play failed to learn and is published as a negative result. Ships with 166 tests across GCC, Clang and sanitizers in CI, and the same C++ compiled to WebAssembly for a browser demo deployed to Pages.