# The cost of trading across our backtests - specification (cross-test study, RIP Backtest protocol v1.0)

Specification written and **frozen 2026-10-22 (approved by the owner the same day, draft 1 unchanged)**. No cross-test number of this study has been computed: no table of cost shares, no count of sign flips over all tests, no correlation. Once frozen, any change of a rule is documented in the article; bugs may be fixed.

## What kind of study this is
- **Descriptive, not a new backtest.** No strategy is run again, no rule is changed, nothing is optimised. It reads the gross and net results already published for every test and asks one question: **how much of the gross result did our cost model take, and how often did it change the verdict?**
- **No gates, no control, no holdout.** Every number comes from results already on the site; the holdouts of all tests are already used. The article makes no claim of edge.
- **Chosen by the owner** on 2026-10-22, after a proposal to add cross-test research articles ("The Cost of Trading - Across Our Backtests" as the pilot, a cross-market scoreboard second).

## What is already known (disclosed, so the study is not blind)
- Each article states its own gross and net, and the variants labelled **"Costs"** (a positive gross result that becomes a net loss).
- The **gold hub** (gs_data.js, 14 strategies, 97 versions) already counted 42 versions lost to costs, 13 lost before costs, and more than half of the cost losers on 1-, 5- and 15-minute bars. The idea that **faster systems lose more to costs** therefore comes from a result we have seen; it is tested below only on the tests **outside** the gold hub (see Question 3).
- I have read most articles while writing them, so I know roughly that many tests had a positive gross result. I have not added them up.

## Which tests are in (inclusion rules, fixed before counting)
1. **Every published backtest post** in `registry.js` (49 posts written out plus 5 added by loops - noise area on ES/NQ/gold, intraday momentum on ES/NQ - on 2026-10-22), **with every variant published in its gate table**, as `check_site.js` reads them (`orig()`: gross, net, trades, gates). Round-2 variants count as published variants of their test.
2. **Out:**
   - guides and hubs that only re-use other tests (gold hub, profit factor, risk-reward, the findings page, the $100-account study), so nothing is counted twice;
   - buy-and-hold benchmarks, controls and the forward test;
   - variants without both a gross and a net number in the published files (listed by name in the article with the reason).
3. **A variant is counted once**, in the market of its test. A test run on several markets in one post (FX six strategies on four pairs, ADX on four pairs, noise area) counts each pair/market as its own line, as the post's gate table does.
4. **The inventory is printed first** (tests, variants, unit, cost model, excluded variants and why) and checked by me against the registry before any cost number is computed.

## Units: only unit-free measures are compared
Tests are in dollars per contract, R, pips or percent. No result is converted into another unit. Per variant (from the whole-sample `all` numbers the article publishes):
- **gross G**, **net N**, **cost C = G - N** (the total cost our model charged), **trades n**.
- **M1 - cost share** = C / G, only when G > 0: the share of the gross profit the costs took. Values above 100% mean the costs took all of it and more.
- **M2 - verdict class:**
  - *lost before costs*: G <= 0;
  - *lost to costs*: G > 0 and N <= 0 (the protocol's "Costs" condition, with N = 0 counted here);
  - *survived costs*: N > 0 (with no claim that it passed the gates).
- **M3 - cost headroom** = G / C, when G > 0: how many times our cost the gross result could pay before reaching zero. Below 1 = lost to costs.
- **M4 - gates**: how many variants passed all six gates, and for the variants that failed, whether gate 1 (validation PF) or gate 5 (without top 10) would still have failed with zero cost is **not** computed (it would need re-running the trades); only the published gates are used.

## Questions (fixed now; the answers are whatever the numbers say)
1. **How often did the cost model decide the verdict?** Counts and shares of the three M2 classes, per variant and per test (a test counts as "lost to costs" when the majority of its variants are; ties reported as ties).
2. **How much of the gross did costs take?** The median M1 over variants with G > 0, with the 25th and 75th percentiles, overall and by market group (index futures ES/NQ; gold; FX; Bitcoin; SPY). Medians, not means, because M1 explodes when G is near zero.
3. **Did faster systems lose more to costs?** Trades per year per variant (n / years of data of its test) against M1, on variants with G > 0 **outside the gold hub**: Spearman rank correlation, reported with its sample size. Prediction written now (from the gold hub): positive. Read as a description of our sample, not a law: the tests are not a random sample of strategies, and fast and slow tests differ in market and rules too.
4. **How sensitive are the verdicts to the cost level?** With a cost charged per trade, the net at k times our cost is N(k) = G - k C. For k = 0, 0.5, 1, 1.5, 2: the number of variants with N(k) > 0. This says how many verdicts a cheaper or dearer broker would flip, **not** that they would then pass the gates.

## Checks (must pass before the article prints a number)
1. **Recompute from the trade logs** for every test whose trade CSVs exist: gross, net and n summed from the trades must equal the summary files (difference < 1e-6 of the unit; any difference stops the study until explained).
2. **Linearity check for Question 4:** for each test, confirm that the published net is the sum of per-trade returns minus a per-trade cost (true for $, R, pips; for the percent tests check whether the published gross/net are sums or compounded). A test where N(k) = G - kC does not hold exactly is either recomputed per trade from its log at each k, or left out of Question 4 with the reason stated.
3. **Cross-check with the gold hub:** the gold variants of this study must reproduce its counts (42 lost to costs, 13 lost before costs out of 97), or the difference is explained (e.g. variants outside the hub).
4. **Cause labels:** every variant labelled "Costs" in an article must fall in the M2 class "lost to costs"; `check_site.js` already enforces the converse.
5. **An independent second script** computes M1, M2 and the Question 4 counts from the inventory without sharing code with the first, and must give identical numbers.

## What the article may and may not say
- Every number is **"in our N tests"**, never "in trading" or "for retail traders". The sample is the strategies we chose to test, mostly popular rules, so a high share lost to costs says which ideas we tried, as much as anything about costs.
- **Protocol versions differ** (t threshold 2.0 or 2.5, controls improved after 8 Oct 2026); the table marks the test date, and gates are reported as published.
- Cost models are ours, stated per market: ES $30.76, NQ $15.76 a round trip per contract; FX 1 pip; gold $0.30/oz; Bitcoin 0.05% a fill; SPY 0.04%. A different broker changes the numbers; Question 4 shows by how much.
- Every row links to its original test.
- "Survived costs" never becomes "works".

## Output
- `cost_out/cost_inventory.json`, `cost_out/cost_rows.json` (one row per variant), `cost_out/cost_summary.json`, the check results in `cost_selftest_result.txt`.
- A guide article (slug and SEO title chosen when it is written), built from these files with asserts on every number in the text, linked from the findings page and from each test's post.