The problem
Your next customer might not Google you. They might ask ChatGPT or Perplexity "who's the best estate attorney near me" and go with whatever name comes back. Answer-engine optimization (AEO) is the work of measuring and improving whether AI answers actually mention your brand — and most businesses have no idea what those answers currently say. Asking once doesn't count either: model output is stochastic, so a real measurement needs sampling and statistics, not a single screenshot.
What we built
A serverless pipeline on AWS that measures it properly.
- Multi-engine sampling. Buyer-intent prompts are generated per catalog, then run across several models — Claude, Llama, and Mistral on Amazon Bedrock, plus Perplexity — each sampled multiple times. Raw answers are archived to S3 before anything touches them.
- LLM-as-judge scoring. A separate model scores each answer for presence, rank, sentiment, and named competitors through a forced tool-call schema, cross-checked against AWS Comprehend. Malformed output is marked unparseable, never guessed.
- Honest statistics. Results aggregate with Wilson confidence intervals, so one noisy run is never dressed up as a finding — "improved" only shows when the intervals don't overlap.
- Guardrailed fixes. For each loss, the engine drafts copy and schema fixes against the winning competitor — and refuses to invent product details it doesn't have. Suggestions only, never auto-applied.
- AWS Step Functions orchestrates the whole run, and an eval gate scores the judge and fixer against labeled fixtures before any prompt change ships.
Status
Open source and running weekly against a demo store. Same pattern we use for client automations — serverless AWS around an LLM, pointed at one real question and left to run — but held to a measurement standard, with the statistics and the eval gate doing the work a single screenshot can't.