Bringing Actual Intelligence to AI™
Artificial intelligence is having its great unearned moment. It speaks smoothly, answers instantly, and carries itself with the confidence of a seasoned expert. And because it sounds certain, people assume it is certain. They trust it. They delegate to it. They let it make decisions it was never designed to make.
That trust is the dangerous part.
The most comprehensive field guide to LLM failure modes – written by breaking two frontier systems and making them tell on each other.
Instant download. No spam.
340 failure modes organized and scored across six comprehensive sections:
Hallucination, confabulation, stale knowledge, probabilistic output, arithmetic errors, and the fundamental ways LLMs produce confident nonsense.
Instruction drift, negative constraint violations, format corruption, premature truncation, and why AI forgets or ignores what you just told it.
Overconfidence, appeasement bias, memory illusions, tone oscillation, and how AI prioritizes sounding helpful over being accurate.
Context window limitations, attention mechanism flaws, tokenization artifacts, catastrophic forgetting, and the structural problems baked into transformer models.
Prompt injection, jailbreaking, unsafe code generation, backdoor triggers, and the attack vectors that bypass safety guardrails.
Overreliance by users, regulatory lag, bias amplification, concentration of power, and the institutional problems AI creates at scale.
Here are just 4 of the 340 documented failure modes–demonstrated live while building this very download page:
L10 / S9 / M8
Over multiple turns, AI gradually forgets exact specifications like filename formats, forcing users to repeatedly correct the same mistakes.
L9 / S8 / M7
AI defaults to common patterns (hyphens) instead of following explicit instructions (underscores), prioritizing training data over user directives.
L8 / S9 / M8
AI produces the correct output, then contradicts itself in subsequent responses without recognizing the inconsistency or maintaining coherent state.
L9 / S8 / M8
AI generates code without validating syntax, file paths, or variable names against earlier context, shipping errors confidently.
Meta-irony: These failures occurred while building the gate for a book documenting AI failures. See the Postscript for screenshots.
Not blindly. AI is fluent and confident, but it hallucinates facts, forgets instructions, and prioritizes patterns over truth. Trust it as a productivity tool requiring human verification–not as an autonomous decision-maker.
Yes, with guardrails. AI works well for supervised tasks like drafting, summarizing, and analysis. It fails when given autonomy, ambiguous instructions, or responsibility for critical decisions without human oversight and verification loops.
AI is not deterministic. It drifts, forgets context, reproduces bias, and generates plausible-sounding errors. Deploy with human checkpoints, clear accountability structures, and acceptance that failures are operational risks–not edge cases.
Yes, but don't trust it. AI can review its own output, but it suffers the same failures twice. Better approach: use multiple AI systems to challenge each other's reasoning, then apply human judgment to resolve conflicts and verify consensus.
Yes, structurally. All transformer-based models hallucinate, drift across turns, suffer context limits, and prioritize fluency over accuracy. Vendors differ in degree and mitigation–not in fundamental failure modes. The architecture guarantees specific weaknesses.
Treat AI as an intern: useful for drafting and research, dangerous for decisions. Implement human review gates, limit autonomy, maintain clear accountability, train staff on failure modes, and never let AI touch critical workflows unsupervised.