AI engineer portfolio

Most AI builds fail in the gap between demo and production. The model works. The latency budget doesn't. The retrieval looks right in isolation; the answer drifts under load. The eval “feels fine” until a regression ships and three customers churn before anyone notices. The work below is what I run on every AI engagement to keep that gap closed.

1. Latency budget, written before the first line of code

For voice especially, latency is product quality. Sub-one-second turns require an explicit budget per subsystem: STT inbound, first-token-to-speech on the LLM, TTS outbound, network and jitter. I write that budget into the scope doc, instrument every leg in traces, and set the P95 alert before voice traffic hits the box. A median that hits 600ms is meaningless if your P95 is two seconds — the call is already broken for the user who matters.

2. Eval loop with golden sets and regression detection

Every task gets a golden set of inputs and expected outputs, scored by either deterministic checks (for structured outputs and tool-calls) or a judge LLM with a calibrated rubric (for free-form text). Evals run on every prompt change in CI. The team sees a diff before the prompt ships, not a churned customer after. Prompts get versioned in code, not pasted into a Notion doc.

3. Tool-calling reliability with structured-output validation

Tool-calls fail in three ways: invalid JSON, schema-valid but semantically wrong arguments, and the model refusing to call the tool at all. Every tool gets a zod schema, retry-with-correction on parse failure, and a fallback chain when the third attempt still fails. The product behavior degrades gracefully; it doesn't 500.

4. RAG architecture, where most teams actually get stuck

RAG done right is mostly retrieval engineering, not LLM engineering. Chunking strategy depends on your data shape — a contract chunked by token count is unusable; a contract chunked by clause is searchable. Hybrid retrieval (BM25 plus embeddings) is the default when keyword recall matters. Reranking with a cross-encoder lifts precision more than swapping embedding models ever will. And every retrieval configuration gets an eval set with answer-faithfulness scoring before it ships.

5. Cost-of-failure tiers and human-in-the-loop gating

Not every AI output should ship without a human. I sort outputs into cost-of-failure tiers — irreversible vs. correctable, public vs. internal, customer-visible vs. operator-visible — and gate the irreversible/public ones behind explicit confirmation or review queues. The model writes the draft; the operator owns the send.

If your build has an AI surface and a deadline, this is the depth you'd be paying for. The discovery call is 30 minutes and free; you'll leave with a written take on whether your architecture survives the gap above.