Engineering

Engineering defaults, and why.

I reach for boring, reliable tools unless there’s a reason not to, design the architecture before writing much code, and build evaluation in from the start rather than bolting it on. Simple architectures, honestly justified, beat complexity theater.

flowchart LR
  U["Users / systems"] --> API["API layer\n(FastAPI)"]
  API --> ORCH["Orchestration\nRAG · agents (LangGraph)"]
  ORCH --> MODELS["Model layer\nfrontier APIs + open-weight\nrouting & fallback"]
  ORCH --> RET["Retrieval\nhybrid search + rerank\nPostgres/pgvector"]
  ORCH -. traces .-> OBS["Observability\nLangfuse · Prometheus · Grafana"]
  API --> DATA[("Systems of record\nPostgres · object storage")]
  EVAL["Evaluation harness\nRAGAS · LLM-as-judge · golden sets"] -. gates .-> ORCH
  subgraph Deploy["Deployment"]
    Docker["Docker / Kubernetes"] --- Cloud["AWS or on-prem / air-gapped"]
  end
  API --- Deploy

Technology & deployment overview — how the layers fit together on a typical production AI system.

Architecture approach

Requirements first. Scope the actual problem and constraints before choosing an approach; sometimes the answer is that AI isn’t the right tool.
Simple before complex. Prefer the smallest architecture that meets the requirement — a single retrieval service over a multi-agent swarm when that’s all it needs.
Security boundaries early. Decide what data can leave, where secrets live, and what a public surface may touch, at design time.
Evaluation from the start. Define how “good” is measured before building, so there’s a gate to ship against.
Deployment & observability as first-class. Containerized, traced, and monitored — not an afterthought.
Human oversight where it matters. Human-in-the-loop gates for consequential or regulated actions.
Cost and latency are requirements, not surprises — measured per stage and per query.

AI engineering

Patterns I use, in context:

Retrieval (RAG). Domain-aware chunking, hybrid search (dense + lexical), reranking, and citation-enforced generation. Postgres/pgvector is a pragmatic default when the corpus fits and the client already runs Postgres.
Agents & workflows. LangGraph-style state machines with explicit tool use, approval gates, and audit logs — over no-code toys — when a task genuinely needs multiple steps.
Structured outputs & tool use. Function calling and schema-validated outputs; standardized tool integration (including MCP-style interfaces) where it reduces glue code.
Model strategy. Frontier APIs (OpenAI, Anthropic) where policy allows; open-weight models (Llama/Mistral-class) deployed privately where data residency demands it; routing and fallback between them for cost and reliability.

MCP appears here as an integration approach I work with, not a shipped credential — see how claims are scoped in the case studies.

Evaluation & reliability

The part that separates a demo from a system you can trust:

Golden evaluation sets built with the domain, per language where relevant.
RAGAS metrics (faithfulness, answer relevance, context precision) and LLM-as-judge with calibration and human spot-checks.
Retrieval metrics (recall@k, MRR) measured before and after reranking.
Regression testing against the golden set on every change, as a release gate.
Error analysis and failure-mode tests (out-of-scope queries, prompt-injection attempts).

Software & infrastructure

Languages/services: Python, FastAPI, background workers, REST APIs.
Data: PostgreSQL (with pgvector), Parquet/S3, Elasticsearch, Kafka and Airflow for pipelines; Pandas/Polars.
Serving/infra: Docker and Kubernetes; AWS (EC2 g5/g6 for GPU serving, ECR); vLLM for open-weight serving; on-prem and air-gapped deployments where required.
Ops: GitHub Actions CI/CD, MLflow / Weights & Biases for lifecycle, Prometheus + Grafana and structured logging for observability.

Quantitative-system engineering

Outside LLMs, my quantitative-trading platform is the clearest evidence of systems engineering under adversarial conditions: statistical validation designed to resist leakage, time-series discipline, a risk engine with a fail-closed kill switch, event-driven and asynchronous execution, reconciliation, and a large automated test suite. It uses the same instincts the AI work does — evaluate honestly, fail safely, make the system observable.

Read the engineering case study →