Prompt evals for coding agents
Prove your prompts work. Ship changes without redeploying.
The router decides before you pay
Three tiers. Simple requests resolve on deterministic rules with no LLM call; only genuine reasoning reaches an LLM.
"Format this JSON response…"
Rules-based layer
Pattern match, resolved by rules
No LLM call
"Analyze our competitive position in Q1…"
LLM tier
No match — escalated for reasoning
LLM call · cost varies
"Always prioritize safety above clarity…"
Value hierarchy
Non-negotiable rule — routing floor applied
Hybrid tier or higher
Cost impact depends on your prompt mix, not a fixed percentage. Duplicate requests are cached at zero cost.
Prompts stop shipping with deploys
Fetch the compiled prompt at runtime. Update once; live everywhere, no redeploy.
Runtime fetch
Always serve the latest optimized version — no redeploy needed.
{{variable}} interpolation
Inject user-specific values at serve time. One template, infinite variations.
Webhook on change
HMAC-signed webhooks notify your system the moment a prompt is updated.
Proof, not vibes
Deterministic assertions, constraint checks and opt-in semantic drift scoring — so you know what changed, not just that it did.
11
Assertion types
Length limits, required phrases, JSON validity, regex, latency — plus LLM-rubric scores. A measurable pass rate, not a gut feeling.
5
Constraint rules
Deterministic checks for no bullets, no markdown, banned words, max length and required prefix. Partial scores show which rule broke.
0–1
Semantic drift
Cosine similarity between original and optimized embeddings flags when "improvement" changed what you were asking for.
Inspectable quality scores
Evaluations return a 0–1.0 score alongside the assertion results, so you see what passed and what failed rather than a black box.
Scoped repairs: published templates are never touched (Pro and Enterprise)
A low-scoring evaluation (below 0.65) can trigger up to 3 repair attempts, each re-scored. A repair is kept only if it clears the quality gate and stays close to your original wording; otherwise it escalates for review. Kept repairs are saved to your optimization history, and a linked draft or unpublished template is updated with the repair (the previous version stays in version history). Published templates are never overwritten automatically.
Live repair events
In the real-time optimizer, repair events arrive over WebSocket as they happen.
Four ways to run it
Know what changed. Prove it worked. Undo it if it didn't. Cloud, MCP, guardrails and governance for versioned, environment-scoped templates.
Cloud
Cloud Edition
Zero-setup, always up to date. Use it from the web app, the REST API, or any MCP client via the npm package.
MCP
MCP Server
Connect Claude Desktop, Cursor, Windsurf or VS Code to the cloud service with the mcp-prompt-optimizer npm package. The standalone local server is deprecated.
Guardrails
Quality Guardrails
Define ranked, non-negotiable rules for your prompts and check outputs against deterministic assertions such as format, length, banned words and required phrases.
Ranked quality rules
Deterministic output assertions
Cost-aware auto-routing
Governance
Enterprise Governance
Semantic versioning, environment scoping (dev/staging/prod), and API rollback to any earlier version — built into every template.
Auto-versioned on edit
Environment labels (Enterprise)
Rollback to any version
Draft and published states
Everywhere you already write prompts
The Chrome extension optimizes in place — ChatGPT, Claude, Gemini, Perplexity and other AI tools. Currently in private access.
Ship prompts you can prove.
$0 to start · $19/mo Pro · Enterprise on request