Lower LLM costs without lowering the bar.

Anyone who needs an LLM prompt to be cheaper, more accurate, or both. If you're an engineer, agent developer, or product manager shipping AI features — and manual prompt tuning is eating your time — this is your next step.

The sweet spot is a well-defined LLM task inside a workflow or agent: entity extraction, classification, summarization, tool-calling, and similar. You provide a way to measure quality (ground-truth data or a clear rubric), and EigenPrompt systematically finds prompts that perform better — often enabling a switch to a cheaper model at the same quality level.

Because it's the only unit of LLM behavior you can rigorously measure and improve.

Every model call carries a non-trivial error rate — hallucinations, format violations, subtle reasoning drift. For a single task with ground-truth data, that rate is quantifiable and reducible. But chain multiple calls without validation and correction at each step, and the errors compound: a five-step pipeline at 95% per-step accuracy lands below 80% end-to-end. At 90%, you're under 60%. The enthusiasm for autonomous agents chaining dozens of free-form LLM calls glosses over this arithmetic — without a deterministic oracle checking every intermediate output, long chains are unreliable at production scale.

EigenPrompt focuses on the lever that actually works today: making each prompt in your system as accurate and cost-effective as possible. That's the foundation that compounds in your favor — whether you're running a two-step pipeline or a twenty-step agent. Full agent-system optimization, where you bring your own evaluation harness and we optimize across the execution graph, is on our roadmap.

One run = one complete, automated optimization of a single prompt. You provide your starting prompt, an evaluation dataset, and a scoring method. EigenPrompt generates variations, evaluates them, and returns a Pareto-optimal set — the prompts that offer the best trade-offs between cost and quality.

You get an interactive visualization so you can pick the exact trade-off that fits your use case. One credit, one run, one problem solved.

EigenPrompt does not just score every candidate on the same evaluation examples and call it done. Variants are screened on a training subset, then surviving prompts are validated on held-out examples so the final frontier reflects improvements that generalize better.

No evaluation setup is perfect — if the dataset is tiny, noisy, or repeatedly reused without refresh, confidence drops. We try to make those limits visible rather than hide them. Every prompt and evaluation dataset undergoes extensive automated QA as part of an optimization run.

Every dataset is split 80/20 into train and test. Candidate prompts are first screened on the train set, and only those within reach of the Pareto frontier earn an expensive evaluation on the held-out test set. The frontier you pick from is ranked by held-out test score — examples that did not drive the candidate’s generation.

To prevent leakage during search, failure-driven mutation hints and adversarial example generation are built from train failures only; test expected outputs never appear in the meta-prompts that propose new candidates, and a runtime guard rejects any run where the train and test sets would alias. Before optimization starts, automated dataset QA flags duplicate rows (which would inflate scores via implicit reuse) and a preflight pass catches labeling or format mismatches against your evaluator.

Honest limits: the held-out test set is re-scored on every promising iteration, so on small datasets with many iterations some compounding pressure on test scores is unavoidable — it’s a held-out set, not an oracle. We surface both the train and the test score for every candidate, so a large train-vs-test gap is visible rather than hidden. The strongest defense is dataset quality: 50–200 diverse, deduplicated, representative examples beats any algorithmic guard. If your eval set is tiny or skewed, no optimizer can fully compensate.

EigenPrompt currently supports Quantitative evaluation - you provide expected outputs and the optimizer scores prompts by how closely they match. Best for classification, extraction, and structured outputs where you know the right answer.

Qualitative evaluation (LLM-as-a-Judge) is coming soon. It will let one or more LLMs act as judges, guided by your optimization goal rubric - ideal for summarization, tone, creative writing, and reasoning quality where outputs are open-ended.

Rule of thumb: If you can write down the right answer, use Quantitative. If you can only describe what a good answer looks like, Qualitative will be the right fit once available.

It’s the set of prompts where no prompt can be improved in one dimension without getting worse in another. Think of it as the efficiency frontier: every prompt on it is the best available at its price point.

EigenPrompt finds this frontier automatically. Instead of guessing which prompt balances cost and quality, you choose from a menu of proven trade-offs.

LangSmith and PromptLayer are observability and management tools — they help you track, version, and monitor prompts. EigenPrompt is an optimization tool — we automatically find better prompts using multi-objective algorithms.

They show you how your prompts are performing. We find prompts that perform better. They’re complementary — use EigenPrompt to optimize, then deploy and monitor with your existing stack.

Most Standard runs finish in around 10–15 minutes. Exact time depends on your evaluation dataset size, prompt length, batching settings, and chosen optimization preset.

The biggest factor is dataset size — 50–200 examples is a good sweet spot for balancing speed, cost, and statistical confidence. Larger evaluation sets or higher sample counts can increase runtime beyond this range. We also provide Turbo-mode, which trades off some prompt quality and automated QA for speed. Compared to hours or days of manual prompt tuning, a single run gives you a data-backed answer fast.

No — and anyone who claims to guarantee a global optimum over a non-deterministic system isn’t being honest. What we do guarantee: if no run finds an improvement in at least one dimension, no credit is spent.

In practice, we almost always find prompts that beat the baseline on cost, quality, or both. Higher optimization presets (Advanced, Max) explore more of the search space and tend to find larger improvements.

You should build one — it’s the single highest-leverage investment in your LLM pipeline, whether you use EigenPrompt or not. A good eval dataset lets you swap models, compare prompts, and catch regressions with confidence.

That said, we offer synthetic dataset generation so you can get started quickly. It’s useful for exploring the platform, but we always recommend real data for production decisions.

Need help designing an eval? Reach out — we’re happy to advise.

Yes. Saved prompt text, run base prompts, run optimization goals, evaluation dataset base prompts, and evaluation rows are encrypted at rest with application-level AES-256-GCM using per-account HKDF-derived keys. All persisted database data is also encrypted at rest by our infrastructure provider and encrypted in transit via TLS.

Your API keys are sealed separately in a passphrase-protected vault (Argon2id key derivation, AES-256-GCM encryption). They are excluded from run records, redacted from logs, and never sent back to the browser.

Deleted datasets and runs are soft-deleted immediately and hard-purged on a retention schedule. Account deletion purges all associated evaluation data and anonymizes your record.

We never sell your data; we use subprocessors and the LLM providers you authorize to deliver the service. See our Security page for the current subprocessor list.

To improve our optimization algorithms for all users, we process anonymized and aggregated metadata from optimization runs.

What we process: Anonymized statistical patterns, scoring distributions, structural prompt metadata, optimization performance signals, and transient embeddings computed during processing and not retained.

What we never process: Your raw prompts, raw evaluation datasets, API keys, or any data that could identify you or your business.

What we never do: Sell your data, share it with third parties for their own purposes, or make automated decisions about your account based on individual data.

We use anonymized optimization metadata to improve our algorithms under legitimate interests. Your raw prompts, raw evaluation data, and API keys are never used for aggregate learning. You can object at any time in your Privacy Settings.

Yes — EigenPrompt uses a Bring Your Own Key (BYOK) model. You add your API keys, and we handle orchestration securely. That means your prompts and evaluation data are sent only to the model providers you explicitly configure and authorize for the run — not to any other LLM vendors. Your costs go directly to your provider, your usage counts toward your existing rate limits, and performance metrics reflect your real production setup.

You can use different keys for the evaluation model (the one you’ll deploy) and the meta-model (the one that guides optimization). That separation lets you optimize for the model you want to deploy while using the best available model to guide the search.

We support all major providers: OpenAI, Anthropic, Google, Mistral, Cerebras, Groq, and more — including any provider with an OpenAI-compatible API.

You can mix and match providers freely. For example, optimize a prompt for a cheaper production model while using a stronger model to guide the search. The platform is model-agnostic by design.

EigenPrompt is subscription-only. Choose the Evaluation, Pro, or Business plan that matches your monthly credit needs and evaluation dataset size. Enterprise teams can contact us for custom onboarding, procurement, security, and usage terms.

You pay two things: one credit per run (from your subscription), plus the underlying model API costs (billed directly by your provider).

API costs scale roughly linearly with your evaluation dataset size. As a rule of thumb: Advanced mode costs 2–5× Standard, and Max costs 5–10×. You set a maximum budget per run, so costs are always capped.

Yes. You can optimize prompts that solve multiple examples per LLM call, and easily compare batched vs. non-batched performance. Batching often reduces per-example cost, and EigenPrompt helps you quantify that trade-off.

Yes - the JSON match type compares parsed values, so whitespace and object key order are ignored while array order is preserved. JSON Subset match type allows additional fields (e.g. fields for reasoning) which are ignored, and scores continuously based on the proportion of fields that match the expected outputs. Pydantic-based structured outputs (e.g. using `instructor` or `outline`) with validators is not currently supported. We strongly recommend optimising the prompt for structured JSON first, and then adapting them to use validators in order to minimise the number of LLM retries.

Coming soon. We’re building a full API for programmatic prompt optimization — trigger runs, retrieve results, generate synthetic eval data, and integrate optimization into your CI/CD pipeline.

Join the waitlist to be notified when it launches.

Yes. The Tool Call match type optimizes prompts that define their tools in a <tools> block and reply with the tool call as JSON. Scoring is deterministic and graded: selecting the right tool earns half the score, exact per-argument matching earns the rest — so the optimizer improves both tool selection and argument correctness. Your tool definitions are preserved byte-for-byte through every prompt rewrite.

Select Tool Call as the match type in the run wizard — there’s a one-click example to start from. v1 scores one tool call per example; multi-step tool chains are on the roadmap.