Case study

Quantitative Trading Research & Execution Platform

Research / R&D Personal R&D. Focuses on methodology, architecture, and execution safety — it does not claim guaranteed returns or proven profitability.

A Python research and execution system for crypto perpetual futures, built around leakage-resistant validation, realistic backtesting, staged promotion, and execution safety. This case study is about methodology and architecture — not trading performance.

  • Python
  • Purged / combinatorial CV
  • Realistic backtesting
  • Risk engine + kill switch
  • Async execution
  • Docker · Prometheus

Evidence: Private repository — 201 modules, ~47,500 LOC · 923 test functions (2,934 parametrized cases) · Wave-29 parity build notes

This is a personal research and engineering project: a Python platform for quantitative research and execution on crypto perpetual-futures markets (Bybit, BTC/USDT). I built it to practice the parts of quantitative systems that are hard to get right — leakage-resistant validation, realistic backtesting, and execution safety — not to sell signals or manage money.

Context

Most retail “trading bots” fail for boring, technical reasons long before markets get a say. This project exists to take those failure modes seriously and build the scaffolding a serious quantitative process needs — the validation, risk, and operational layers — as a personal R&D exercise and a portfolio of systems-engineering practices.

Why it sits on an AI engineer’s site: the instincts it exercises — evaluate honestly, prevent data leakage, fail safely, make the system observable — are exactly the ones I bring to production AI systems. It is systems engineering under adversarial, non-stationary conditions, which is the same discipline good LLM evaluation and reliability work demands.

The problem: why quantitative systems fail technically

Before any question of “does it make money,” these are the ways such systems mislead their own authors:

  • Lookahead leakage — features or labels that peek at the future, inflating backtest results.
  • Overfitting — tuning until a strategy fits noise, then watching it evaporate live.
  • Unrealistic backtests — same-bar fills, ignored fees, no slippage or market impact, no funding.
  • Weak calibration — probabilities that don’t mean what they claim.
  • No backtest/live parity — the live system behaves differently from the one that was validated.
  • Missing safety — no kill switch, no reconciliation, no staged rollout.
  • Poor observability — you can’t see what the system is doing until it’s too late.

The platform is organized around defending against each of these. Solving them does not imply the system is profitable — it means the research is honest about what it measures.

System overview

The system runs the full research-to-execution loop as separable layers, each independently testable and importable:

flowchart TD
  MD["Market data<br/>Bybit perp futures · OHLCV<br/>funding · OI · basis<br/>microstructure (capture)"] --> DQ["Data quality &<br/>causal feature pipeline"]
  DQ --> ES["Event sampling & targets<br/>CUSUM · triple-barrier<br/>meta-labeling · sample weights"]
  ES --> MT["Model training & calibration<br/>GBDT (+ optional TCN)"]
  MT --> VAL["Statistical validation<br/>purged WF · CPCV · PSR/DSR"]
  VAL --> GATE{"Promotion gates"}
  GATE -->|pass| RISK["Risk & portfolio controls"]
  GATE -->|fail| HALT["Reject / halt"]
  RISK --> EXE["Execution<br/>idempotent orders · retries"]
  EXE --> MON["Monitoring · reconciliation<br/>kill switch"]
  MON -. drift / breach .-> HALT
System architecture — market data flows through a causal feature and validation pipeline; only candidates that clear promotion gates reach risk-controlled execution, and monitoring can halt the system.

Research methodology

The validation layer is the heart of the project. It implements techniques from the modern quantitative-finance literature (López de Prado–style methods and reference stacks such as Qlib/mlfinlab), verified against real source modules:

  • Causal features and event sampling. A symmetric CUSUM filter (a change-detector that flags when price has moved enough to be worth acting on) selects events; features are constructed causally (using only past data — no forward peeking), with fractional differentiation, regime/HMM features, cross-asset features, and causal higher-timeframe alignment assembled into one unified panel.
  • Triple-barrier labeling and meta-labeling. Triple-barrier labels each candidate trade by whether it would hit a profit target, a stop-loss, or a time limit first; meta-labeling adds a second model that decides whether to act on the first model’s signal. Forward-volatility targets and sample weights (by label concurrency and average uniqueness) stop overlapping trades from being counted as independent evidence.
  • Purged walk-forward validation and combinatorial purged cross-validation (CPCV) — in plain terms, only ever train on data from safely before each test window, with a gap (“embargo”) at the boundary so information can’t leak across it. This is the single most common way backtests fool their authors.
  • Statistical rigor: probabilistic and deflated Sharpe ratios (PSR/DSR — Sharpe variants that discount for luck and for how many strategies were tried), bootstrap confidence intervals, Monte-Carlo analysis, calibration metrics (Brier score, ECE), and classification metrics (MCC, balanced accuracy).
  • Robustness: parameter perturbation, cost-sensitivity sweeps, turnover analysis, drift detection, and backtest/live parity checks.
flowchart LR
  D["Labeled panel<br/>+ sample weights"] --> PWF["Purged walk-forward<br/>(embargo)"]
  D --> CPCV["Combinatorial purged CV<br/>(many backtest paths)"]
  PWF --> STATS["PSR / DSR · bootstrap CIs<br/>calibration (Brier/ECE)<br/>cost sensitivity · turnover"]
  CPCV --> STATS
  STATS --> GATE{"OOS promotion gate<br/>thresholds"}
  GATE -->|pass| PROMOTE["Promote candidate"]
  GATE -->|fail| REJECT["Reject"]
Validation & promotion flow — a candidate must clear purged walk-forward and CPCV, then statistical thresholds, before it is promotable.

Backtesting realism

The backtester is designed to remove the usual optimistic biases: next-bar execution (fills at the next bar’s open, not the signal bar), explicit fee modeling (maker/taker), slippage and size-aware market impact, and perpetual funding accrual. Crucially, the backtester routes position sizing through the same risk-engine functions the live path uses, so the validated system and the deployed system size positions identically.

Execution & risk

The execution and risk layers are built to fail safely:

  • A fail-closed kill switch (environment flag plus a sentinel file) is the highest-priority gate — when engaged, it blocks new entries while allowing exits and reduce-only orders.
  • Idempotent order submission with deterministic client-order IDs, retry with backoff, and circuit breakers.
  • Partial-fill handling, orphan-position reconciliation (report-only by design), and a local stop monitor.
  • Risk controls: position sizing, regime-dependent leverage, daily/weekly drawdown and exposure limits, and ATR-spike tightening.

Market microstructure

The panel treats order-flow signals as first-class: trade tape, cumulative volume delta, L2 order-book state, and liquidation streams, with sequence-gap invalidation and causal aggregation.

Staged validation & operations

Nothing goes straight to live. A staged-promotion controller moves a strategy through explicit stages with per-stage criteria (duration, trade count, drawdown, failure and orphan limits), and rolls back or halts on breach:

flowchart LR
  DEV["dev"] --> SHADOW["shadow"] --> PAPER["paper"] --> TESTNET["testnet"] --> SMALL["live-small"] --> LIVE["live"]
  SHADOW -. breach .-> HALTED["halted"]
  PAPER -. breach .-> HALTED
  TESTNET -. breach .-> HALTED
  SMALL -. breach .-> HALTED
  LIVE -. breach .-> HALTED
Staged-deployment ladder — a strategy is promoted stage by stage against explicit gates; any hard breach sends it to a terminal halted state.

Operationally, the system ships with Docker Compose (application, Prometheus, Grafana — ports bound to loopback), a Prometheus exporter, structured JSONL logging, secret redaction, and config validation, plus operator, incident-response, and recovery runbooks.

Engineering scale (verified)

Counts generated directly from the repository:

MetricValue
Application modules (tradebot/)~201 Python files
Application code~47,500 lines
Test files73
Test code~14,800 lines
Test functions~923 (≈2,934 parametrized cases)
Subpackages14 concern-scoped (data, alpha, ml, validation, risk, execution, portfolio, regime, runtime, adaptation, monitoring, research, strategy, security)

Validation status

An honest map of what is and isn’t established by the repository:

AreaStatusEvidenceLimitation
ArchitectureDemonstrated14 subpackages; 8 parity layers wiredComponent parity, not a performance result
Data pipelineDemonstrated (with data ceiling)ingestion, data-quality and coverage auditsDeep history is OHLCV/funding/OI/basis; tape/L2/liquidations capture-only
Synthetic pipeline smokeDemonstratedend-to-end pipeline report on a synthetic basket“Not a performance verdict”
Unit / integration testingDemonstrated73 test files; last run 2,901 passed / 33 failed33 failures in async/reconciliation tests (see below)
Statistical-validation infrastructureDemonstrated (as infrastructure)CPCV, PSR/DSR, bootstrap, calibration, drift, gatesExercised on synthetic/offline data — not a real-edge result
Paper / live wiringPartialstaged live-validation controller; default paper modeLive order-routing integration is the next step
Real-market edgeNot demonstratedDeliberately deferred; prior probes ran on an impoverished data slice
Live profitabilityNot demonstratedNo live/production performance evidence exists

Test-suite honesty

The repository contains a large automated test suite. The archived run I’m reporting is 2,901 passed and 33 failed. I am not writing “all tests pass,” because that isn’t true: the 33 failures are concentrated in asynchronous-support and reconciliation tests (test_resilience_extensions.py, test_wave8_daemon_async.py) and are consistent with an async test-plugin/runtime issue rather than a logic defect in the traded path — but they are unresolved in that archived run, and I’d rather show that than hide it.

My contribution

This is a solo personal project: I designed the architecture and wrote the code across all layers. I’m not claiming it is novel research or that any component is uniquely mine — the techniques come from the public quantitative-finance literature and reference stacks — but the implementation, integration, validation harness, and operational scaffolding are my own work.

Limitations

Repository access

The repository is kept private by default — it contains a large, evolving codebase and I’d rather share it deliberately than expose implementation details wholesale. On request I can provide a guided walkthrough, sanitized architecture excerpts, or selected modules (for example, the validation harness) as a focused reference. The counts and claims on this page were generated and verified directly from the source tree.

Future work

Wire the staged controller into a live order-routing runtime end to end; run the validation harness on real (not synthetic) history with the microstructure channels captured over a meaningful window; and only then attempt an honest, cost-aware edge evaluation — reporting whatever it finds, including “no edge.”