benchmarking-&-performance（基准测试和性能）

Skill: Benchmarking & Performance

When to use this skill

• After adding or modifying a strategy
• To validate that a strategy is profitable
• To compare different configurations
• Before going from paper trading to live

Available scripts

Key metrics to monitor

Profitability metrics

Risk metrics

Interpretation

Sharpe Ratio:
  < 0.5  → Bad, don't use
  0.5-1  → Mediocre, needs improvement
  1-2    → Good
  2-3    → Very good
  > 3    → Excellent (or suspicious, check overfitting)

Max Drawdown:
  < 10%  → Conservative
  10-20% → Moderate
  20-30% → Aggressive
  > 30%  → Dangerous

Benchmark commands

Simple benchmark

# Backtest on one symbol
cargo run --bin benchmark -- --symbol AAPL --days 365

# Backtest on multiple symbols
cargo run --bin benchmark -- --symbols "AAPL,GOOGL,MSFT" --days 365

Advanced benchmark

# Parallel mode (multi-core)
cargo run --bin benchmark -- --parallel --symbols "AAPL,GOOGL,MSFT"

# With sequential comparison
cargo run --bin benchmark -- --compare-sequential

# Parameter matrix
cargo run --bin benchmark_matrix

Available scripts

# Stock benchmark
./scripts/benchmark_stocks.sh

# Market regime benchmark
./scripts/run_regime_benchmarks.sh

# Automatic benchmark
./scripts/auto_benchmark.sh

Strategy validation workflow

Step 1: Initial backtest

cargo run --bin benchmark -- --strategy <STRATEGY> --days 365

Verify:

• [ ] Sharpe Ratio > 1.0
• [ ] Max Drawdown < 20%
• [ ] Win Rate consistent with strategy type
• [ ] Profit Factor > 1.5

Step 2: Test on different periods

# Bull period
cargo run --bin benchmark -- --start 2021-01-01 --end 2021-12-31

# Bear period
cargo run --bin benchmark -- --start 2022-01-01 --end 2022-12-31

# Volatile period
cargo run --bin benchmark -- --start 2020-02-01 --end 2020-04-30

The strategy must be profitable (or at least not lose too much) in ALL conditions.

Step 3: Multi-symbol test

cargo run --bin benchmark -- --symbols "AAPL,MSFT,GOOGL,AMZN,META"

Verify result consistency across different assets.

Step 4: Stress test

Test on crash periods:

• COVID crash: February-March 2020
• 2022 Bear market: January-October 2022
• Flash crashes: Verify resilience

Pitfalls to avoid

Overfitting

Symptoms:

• Sharpe Ratio > 3 on backtest
• Performance degrades in live/forward test
• Too many optimized parameters

Solutions:

• Use train/test split
• Test on out-of-sample data
• Prefer simple strategies

Look-ahead bias

Symptom: Using future data in decisions

Solution: Verify indicators only use past data

Survivorship bias

Symptom: Only testing on assets that still exist

Solution: Include delisted assets in backtests

Key files

Checklist before production

• [ ] Positive backtests on 2+ years of data
• [ ] Sharpe Ratio > 1.0 on different periods
• [ ] Acceptable Max Drawdown (< 20% recommended)
• [ ] Tested on bull, bear AND sideways markets
• [ ] No sign of overfitting
• [ ] Paper trading validated for 1+ month