01 / Production AI
LLM evaluation & inference
Know where your AI breaks. And what to fix first.
When this is useful
A convincing demo does not tell you how a model behaves across real inputs. Prompt changes, retrieval gaps and tool failures can quietly change the product experience. This check makes those failure modes visible for one agreed workflow.
- Answers vary in quality with no shared acceptance criteria.
- Retrieval misses useful context or introduces irrelevant material.
- Model or prompt changes ship without a regression check.
What you receive
- Rerunnable evaluation + case-level results
- Failure categories + evidence notes
- Next-step proposal + one review call
What’s included
- Review one business task, up to 100 sanitised customer cases and two existing configurations.
- Review a bounded, agreed sample of sanitised inputs, outputs and failure reports.
- Deliver a rerunnable evaluation with explicit criteria and case-level results.
- Analyse prompt, retrieval, tool-use and fallback failures; review inference latency and token usage where existing traces support it.
- Deliver failure categories and a bounded next-step proposal; include one review call and one factual-correction round.
Schedule & agreement
Draft planning assumption: 5–7 business days, starting only after adequate inputs, safe access and a delivery slot are agreed. Commercial scope is subject to review.
Currency, tax treatment and final scope are agreed before work starts. Implementation and ongoing support are scoped separately.
What’s outside the check
- Production changes, full new systems, migrations or stress testing.
- Formal security certification, legal review or model training.
- Ongoing monitoring, unrestricted datasets or guaranteed accuracy.
Access we’ll agree
Read-only architecture and configuration access; sanitised example requests and responses; existing prompts and evaluation results where available. Production credentials and identifiable customer data are not needed for the initial scoping conversation.
From diagnosis to a scoped fix
A follow-on fix might add a regression harness, repair a retrieval stage or implement a specific fallback. Each is scoped separately with an acceptance test and a fixed price.
Questions before you start
Do we need an evaluation set already?
No. The check can establish an initial set from agreed examples and known failures. It is a starting point to maintain, not an exhaustive guarantee.
Can you assess a retrieval-augmented system?
Yes, within one workflow. We can separate retrieval relevance, context quality and generated answer quality so the recommended fix targets the right stage.
Related starting points
Have a specific problem?
Start with a useful question.
Describe the symptom and the outcome. We’ll define the boundary before quoting the work.
Book a consultation ↗Or prepare an enquiry →