The problem that hides in a single word
Imagine you're building an app that watches over an aging parent and quietly emails their family if something seems off. At some point the app has to write a sentence like "Skips her normal quiet times like an afternoon nap." Innocent enough — until you realize the software has no idea whether the person being watched is a "her," a "him," or a "them."
A human never thinks about this. You meet someone, you learn their name and a few facts, and the right pronoun falls out automatically. For a computer, that reflex has to be manufactured from scratch — and there are really only a few ways to do it, each with its own failure mode. The history of those methods is, in miniature, the history of how machines learned to read.
Method one: the giant list of names
The oldest trick is also the dumbest, and it works more often than you'd expect: keep a giant list. Somewhere in the program sits a table that says "Michael → male, Mary → female, Jordan → either," and the software simply looks your name up like a word in a dictionary. The U.S. Social Security Administration publishes decades of baby-name data, so building such a list is easy. Type a name in, get a gender out.
This is fast, cheap, and completely transparent — you can see exactly why it decided what it decided. It's also brittle in ways that get people misgendered constantly. "Jordan," "Taylor," "Alex," and "Casey" shrug and refuse to commit. "Andrea" is overwhelmingly female in the United States and male in Italy. Nicknames like "Sam" could be Samuel or Samantha. And a list of names knows nothing about the actual human in front of it — it's playing the odds on a spreadsheet, nothing more.
Method two: grammar rules and the "agreement" trick
Most of the time, software doesn't actually need to guess gender from a name — it needs to keep pronouns straight inside a chunk of text it's already been handed. "John went to the store. He bought milk." Which person does "he" point back to? Linguists call this coreference resolution: figuring out which words refer to the same thing. The pronoun "he" is an arrow, and the computer's job is to find what it's pointing at.
For decades, the workhorse approach was a stack of hand-written rules. Match on gender ("he" goes with a male name, not a female one). Match on number ("they" goes with a plural, "it" with a singular thing). Prefer the nearest sensible noun. Walk through the sentence applying filter after filter until one candidate survives. These rule-based "sieves" were the backbone of serious language tools like Stanford's CoreNLP, and when the cues line up neatly, they're remarkably effective.
The catch is that grammar rules only carry you as far as grammar goes. The moment meaning enters the picture, a rulebook is hopelessly outgunned.
The sentence that broke the rulebook
Computer scientists have a favorite torture test for this, and it's worth seeing because it makes the whole problem click. It's called a Winograd schema, named after researcher Terry Winograd, and the classic example is a pair of nearly identical sentences:
- "The city councilmen refused the demonstrators a permit because they feared violence."
- "The city councilmen refused the demonstrators a permit because they advocated violence."
One word changes — feared becomes advocated — and the meaning of "they" flips entirely. In the first sentence "they" is the councilmen; in the second it's the demonstrators. No rule about gender, number, or word order can tell these apart, because the grammar is identical. You can only solve it if you actually understand how the world works: who tends to fear violence, who tends to advocate it. For a long time, this was considered a clean line between genuine understanding and mere statistical mimicry — a test a machine couldn't fake its way through.
Method three: the AI that reads the whole room
Then the line moved. Modern AI assistants don't follow a rulebook at all. They were trained on staggering quantities of human writing, and in the process absorbed the statistical shape of how language actually behaves — including which way a "they" leans when the only clue is meaning. By 2019, transformer-based language models were scoring above 90% on the Winograd Schema Challenge, and the test that was supposed to separate real comprehension from tricks was essentially declared beaten.
So an assistant like Claude, handed "the councilmen refused the demonstrators because they feared violence," will quietly land on the councilmen — not because a rule told it to, but because in everything it has read, that's how the sentence resolves. It's reading the whole room instead of checking boxes. The difference in approach is stark:
| Traditional code | Modern AI | |
|---|---|---|
| How it decides | Looks up lists, applies fixed grammar rules | Pattern learned from billions of sentences |
| Handles meaning? | No — grammar only | Often yes — that's its whole trick |
| Can you see why? | Yes, every step is inspectable | Rarely — the reasoning is buried in the math |
| Typical failure | Trips on ambiguity and uncommon names | Confidently inherits society's stereotypes |
Smart doesn't mean unbiased
Here's the uncomfortable part. Being able to read the room means an AI also picks up the room's prejudices. Researchers built a clever test set called Winogender that takes a sentence about, say, a surgeon or a nurse and swaps only the pronoun. Flip "he" to "she" around the word "surgeon," and coreference systems start making different — and worse — choices, because their training data is steeped in the assumption that surgeons are men. The bias isn't typed in by a programmer; it's quietly inherited from a world that wrote those patterns down for centuries.
Gender-neutral language exposes the seams even more. Datasets built specifically to test singular "they" and non-binary references reveal that many systems, traditional and modern alike, stumble badly the moment a person doesn't fit a tidy he-or-she box. The machine that aces the famous brain-teaser can still fumble the most basic act of respect: referring to someone the way they actually are.
What comes next
The trajectory is fairly clear, even if the destination isn't. Pure name-list lookups are fading into the background, useful only as a rough fallback. Hand-written grammar sieves are becoming a legacy layer — reliable for the easy 80%, quietly handing off the hard cases. The frontier now is teaching large models to handle the cases they're worst at: gender-neutral pronouns, non-binary identities, names and languages underrepresented in their training, and the basic humility to say "I'm not sure" instead of guessing wrong with total confidence.
And increasingly the smartest move isn't a better guess at all — it's a better question. Rather than infer your gender from your name and risk an insult, well-designed software is learning to simply ask once and remember the answer. A century of computer science went into making machines clever enough to deduce a pronoun from context. It turns out the most respectful thing they can do with that cleverness is know when not to use it.