# Cross-market validation scoreboard - specification (cross-test study, RIP Backtest protocol v1.0)

Specification written and **frozen 2026-10-22 (approved by the owner the same day, draft 1 unchanged)**. No cross-market number of this study has been computed: no transfer rate, no base rate, no rank correlation across markets. Once frozen, any change of a rule is documented on the page; bugs may be fixed.

## What kind of study this is
- **Descriptive, not a new backtest.** Nothing is run again. It reads the results already published for the tests that ran **the same rules on more than one market** and asks: **when a strategy version did well on one market, did the same version do well on another?**
- **No gates, no holdout.** The holdouts are all used. The page makes no claim of edge for any strategy.
- **A permanent page.** When a new test runs an existing rule set on another market, its family is extended in a fixed config file and the page rebuilds; every addition is logged on the page.
- **Chosen by the owner** on 2026-10-22, as the second cross-test project after the trading costs guide.

## What is already known (disclosed, so the study is not blind)
- Several tests were themselves replications and their articles compare with the origin: the FX study (six ES strategies on four pairs, and the findings page section "Did the ES results travel to FX?"), Initial Balance NQ -> ES, Turtle gold -> EUR/USD -> Bitcoin, A/D Bitcoin -> SPY, noise area ES/NQ/gold -> Bitcoin, golden cross 1-hour ES/gold -> EUR/USD.
- I know the verdicts of all these tests (almost all buried) and that only three versions on the site passed every gate (1-hour golden cross long-only on gold, later Inconclusive; the two New York noise area versions on NQ). I have not tabulated any version against its twin on another market.
- **Prediction, written now:** transfer is weak. A version positive after costs on one market will be positive on another only somewhat more often than any version of the same family; rule ordering will agree little across markets; close pairs (ES and NQ) will agree more than distant ones.

## Families (which tests are in)
A **family** is one rule set whose versions were run with **identical rules** on two or more markets, where only what the market forces was changed and declared in the spec before the run: costs, units, tick size, contract value, session clock, data source. Every family is confirmed in the inventory (step 1 of the checks) by quoting the line of its spec that says the rules are unchanged.

**Tier 1 - identical rules** (pooled):

| Family | Markets | Versions per market (provisional) | Tests |
|---|---|---|---|
| ES strategies on FX (RSI(2) A/B and round 2, Bollinger, Donchian, RSI divergence, MACD, Supertrend 60-min with round 2) | ES, EURUSD, GBPUSD, USDJPY, AUDUSD | 24 | the-rsi2-dip-buyer, bollinger-band-reversal, donchian-channel-breakout, rsi-divergence, macd-crossover, supertrend (ST60 versions only), fx-six-strategies-four-pairs |
| Golden cross 50/200, daily | ES, gold | 4 | golden-cross-50-200 |
| Golden cross 50/200, 1-hour | ES, gold, EURUSD | 12 | golden-cross-1-hour, golden-cross-eurusd-1h |
| EMA 9/21, 1-hour | ES, NQ, GC futures | 4 | ema-9-21-crossover-1h |
| Turtle trading, daily | gold, EURUSD, Bitcoin | 4 | turtle-trading-gold-daily, turtle-trading-eurusd-daily, turtle-trading-bitcoin-daily |
| Noise area breakout, exchange sessions | ES, NQ, gold | 6 | noise-area-breakout-es, -nq, -gold |
| Intraday momentum | ES, NQ | 4 | intraday-momentum-es, -nq |
| Initial balance breakout | NQ, ES | 3 | initial-balance-breakout-strategy-nq, initial-balance-breakout-es-futures |
| ADX + DI, 1-hour | EURUSD, GBPUSD, USDJPY, AUDUSD | 2 | adx-indicator-strategy-forex |

**Tier 2 - adapted replications** (reported on their own, never pooled with Tier 1), where a rule had to change for the market:
- Noise area on Bitcoin (UTC day instead of an exchange session, weekends included): NA-BTC-24H-L/-LS against the New York versions of ES, NQ and gold (the 15-minute Bitcoin versions are a different setting and are left out).
- A/D: Bitcoin 4-hour long and short -> SPY daily long only (AD-DIV, AD-MA, AD-CHO against ADS-DIV, ADS-MA, ADS-CHO).

**Out:** tests on one market only, and pairs of tests with the same name but different rules (EMA 9/20 gold vs EMA 9/21; opening range breakout gold vs NQ; CME gap fill on Bitcoin vs the ES gap fill; TDI vs Trading Made Simple, which are one market).

**Variant matching:** versions are matched by their rule id with the market prefix removed (e.g. `ES GH-L-N`, `Gold GH-L-N`, `EURUSD GH-L-N`). A version present on only some markets of its family is listed and left out of the pairs it lacks.

## What is compared (unit-free)
For each version v and market m, from the published results (read as `check_site.js` reads them, as in the cost study):
- **net sign** (positive after costs), **gross sign** (positive before costs), **t** = the whole-sample t-statistic of the net per-trade results, as published in the summary (computed from the trade log where a summary lacks it), **gates passed** (0-6) and **all six passed**.
- No result is converted into another unit; t and signs are comparable across dollars, R, pips and percent.

**Pairs:** every ordered pair of markets (A -> B) within a family, for every matched version. Because a family with many markets and versions would dominate a pooled count (the FX family alone has 20 ordered market pairs of 24 versions), **every pooled rate weighs each family equally** (the mean of the family rates); the plain pooled count is shown beside it.

## Questions (fixed now; the answers are whatever the numbers say)
1. **Transfer of a positive result.** Among (version, A -> B) with net > 0 on A: the share with net > 0 on B (**transfer rate**). Against it, the **base rate**: the share of all versions of the same family with net > 0 on B, whatever they did on A. The **lift** = transfer rate - base rate. Same three numbers before costs (gross > 0) and for "at least 4 of 6 gates on A".
2. **Do the rankings agree?** In families with at least 4 versions, for each market pair, the Spearman correlation of t across the versions. Reported: the median over pairs and the number of pairs with a positive correlation.
3. **Close and distant pairs.** Close: ES-NQ, spot gold-GC futures, and two FX majors. Distant: every other pair. Question 1's lift for each.
4. **Long-only versions.** Long-only versions on markets that rose over the test years (ES, NQ, gold, Bitcoin, SPY) are positive together for the reason that the markets rose. Questions 1 and 3 are reported for long-only and for long-and-short versions separately, with long-and-short as the cleaner test.
5. **The versions that passed every gate.** For each, its twins on the family's other markets: net sign, t, gates passed.

**How surprising is a lift?** For each family, the destination results are shuffled among the family's versions 10,000 times (keeping each market's base rate) and the family-weighted lift recomputed; the share of shuffles with a lift at least as large is reported. Descriptive only: versions of one family share rules and often the same years, so pairs are not independent.

## Checks (must pass before the page prints a number)
1. **Inventory first:** families, markets, matched and unmatched versions, and the quoted "rules unchanged" line of each spec; checked by me against the tests before any comparison is computed.
2. **Numbers:** gross, net and trades from the cost study's inventory (already checked against every trade log); t read from each summary and recomputed from the trade log of every included version (difference < 0.01, else the study stops until explained).
3. **Gates:** gates passed per version equal the published gate tables (`check_site.js` reads the same files).
4. **Independent second script** recomputes the answers to questions 1 to 5 from the inventory file without sharing code, and must give identical numbers.

## What the page may and may not say
- Every number is "in our tests", never "strategies do / do not work across markets". The families are the rules we chose to replicate, often because they looked interesting on the first market.
- **Tier 2 is never pooled** with Tier 1.
- A transfer is not a confirmation: a version positive on two related markets (ES and NQ, same days) is one piece of evidence, not two.
- Positive is not "works": the gates are shown next to every sign.
- Every family links to its tests.

## Output
- `xm_families.json` (the config: families, tests, markets, id mapping), `xm_out/xm_inventory.json`, `xm_out/xm_pairs.json`, `xm_out/xm_summary.json`, `xm_selftest_result.txt`.
- A guide page (slug and SEO title chosen when it is written), rebuilt from these files with asserts on every number in the text, linked from the findings page and from each test in a family.