Optimization verdict

Optimization verdict · 2026-08-08

Source docs/OPTIMIZATION_VERDICT_2026-08-08.mdCommit 78387fc55fc76c255f431205d22dc93579836b45SHA-256 e9352bebd7a835f098954970646b9834ad7322822ae5f1b41a7a2382216a9ae7Window Trailing five years; out-of-sample is the last thirty percent of that window.

Can this system be optimized per ticker? Measured answer: no.

Date: 2026-08-08 · Window: trailing 5 years, never more · Cost: 0.20% round trip Reproduce: python3 scripts/drl_tune5.py --n 250 · python3 scripts/drl_cohort.py --min-names 4 --min-trades 30

The question

The owner’s rule is that every ticker’s parameters get optimized on up to five years of data, refreshed every 12 months. This measures what that produces.

Three levels were tested, each with the same grid (96 configs: ATR × stop mode × volume × chop × peak), the same 5-year window, and the same discipline — selection in-sample only, scoring out-of-sample. Choosing the config that scores best out-of-sample and then quoting its out-of-sample score is not validation; it is the same overfit with an extra step, and it is the easiest way to fake this entire exercise.

Level 1 — per ticker, all 191 Tier A names


Beat the global config in-sample	179 / 191 — 94%
Beat it out-of-sample	87 / 191 — 46%
Mean in-sample gain found	+11.06pp
Mean out-of-sample delivered	−2.25pp
Correlation r(IS gain, OOS gain)	−0.052
Median out-of-sample trades per name	3

The optimizer reliably finds +11pp and delivers −2.25pp. And r = −0.05 means the in-sample score — the only thing an optimizer can see — carries no information about what happens next. It is not a weak predictor. It is not a predictor.

Twelve names cleared the +0.5pp adoption bar. Ten of the twelve rest on fewer than 10 out-of-sample trades, and the two that don’t (SMCI, ACN) are losing configurations that lose slightly less than global: SMCI −5.85%, ACN −5.35%. Nothing here is adoptable.

The single cleanest illustration is PLTR: +128.67pp in-sample, −46.94pp out-of-sample.

Level 2 — per cohort (pooled by sector)

If the problem is sample size, pooling similar instruments is the only way to get more trades without breaking the 5-year rule. Semiconductors pool 21 names; Healthcare 20. Median pooled out-of-sample trades: 26, versus 3 per single ticker — an order of magnitude more evidence.

cohort	names	OOS n	IS delta	OOS delta	t
Materials	6	27	+2.72	+4.10	+1.00
Consumer	18	16	+8.43	+3.38	+0.84
Energy	12	27	+3.45	+2.32	+0.82
Internet	7	19	+6.22	+4.26	+0.62
Semis	21	34	+9.77	−0.48	−0.04
Software	13	26	+23.14	−2.13	−0.27
Healthcare	20	79	+1.78	−0.59	−0.31
Crypto	8	22	+15.48	−2.71	−0.46
Tech	9	22	+6.83	−2.94	−0.79

13 cohorts · 6 beat global out-of-sample (46%) · 0 statistically significant.

Same coin flip as per-ticker, with 10× the data. The largest in-sample gains (Software +23.14pp, Crypto +15.48pp, Semis +9.77pp) all produce negative out-of-sample deltas — the signature of a fit, not a finding.

This was the author’s proposed alternative and it failed. Recorded as a negative result rather than quietly dropped.

Level 3 — pooled across the entire book

The ATR sweep in the same session pooled 273 instruments and ~2,000 held-out trades, the maximum sample this program can construct inside 5 years:

ATR	IS expectancy	OOS expectancy
3.0 (shipped)	+1.50%	+3.63%
4.5	+3.99%	+5.19% (best OOS)
8.0	+16.62% (best IS)	−4.78%

Best OOS is ATR 4.5 at +1.57pp over shipped — t = +0.60, not significant. And the in-sample optimum (ATR 8.0) is a catastrophe out-of-sample. Even at maximum sample size, 3.0 and 4.5 are indistinguishable.

Verdict

No level of optimization produces a validated improvement. Not per ticker, not per cohort, not pooled across the whole book. The shipped global config — ATR 3.0, daily close, filters off — is not a placeholder that nobody got round to tuning. It is the answer, and it has now been tested three ways.

What this means for the rules

Keep the global config. Every global-seed row in LEDGER.md is correctly labelled: untuned on purpose, because tuning was tried and measured worse.
The 12-month re-optimization cadence should re-run this verdict, not a parameter search. The question it answers is “has anything changed such that tuning now works”, not “what are this year’s parameters”. If a future run shows cohorts clearing t > 1.5, that is the moment to adopt — not before.
Per-ticker work is qualification, not optimization. “Does the ladder work on this instrument at all” is answerable with 9 trades at the tier level (A/B/C). “What are this instrument’s ideal parameters” is not.
The edge lives in the ladder itself, not in the filter settings. Every filter tested — volume, chop, MACD, breakeven, ratchet, harvest, adaptive ATR, stale exit — is worth between −0.79pp and +0.49pp out-of-sample, i.e. nothing distinguishable. MACD was deleted outright in 06.13.00 on this basis.

The honest caveat

This says the grid contains no findable improvement at this sample size. It does not say no better system exists. A genuinely different mechanism — a different entry rule, a different exit, a regime filter with real economic content — is untested here and is where the remaining upside would have to come from. Turning the same nine knobs harder is the one thing now measured to be worthless.