Does a Strategy Work on Other Markets? 682 Backtest Pairs
Testing a trading strategy on multiple markets is a common robustness check. We ran the same rules, unchanged, on several markets in 9 rule families of our backtests. Here is what a good result on one market said about the same version on another.
Scoreboard · updated with every new cross-market testA good result on one market predicted nothing about the next.
Rule families: 9 · Versions: 228 · Market pairs: 682 · Transfer above the base rate: 0 pts · New backtests: none
Short answer: not in our tests. Across 9 rule families run unchanged on several markets (682 market pairs), a version that made money after costs on one market made money on another 65% of the time, the same as any version of those rules (65%). The three versions that passed all our gates passed on none of their other markets.
00The short answer
- In our tests, a version that made money on one market was no more likely than its family to make money on another. Positive after costs on market A, it was positive on market B 65% of the time; any version of the same rules was positive on B 65% of the time. The difference is 0 pts.
- The order of the versions carried over a little. The versions that did best on one market tended to do somewhat better on another (median rank correlation 0.25, positive in 18 of 24 market pairs), not enough to pick a winner.
- The three versions that passed every gate did not pass on their other markets. Their twins were mostly positive, but none passed more than 3 of the 6 gates, and an adapted version of the noise area rule lost clearly on Bitcoin.
- So a second market is a weak check on its own. Whole families were positive everywhere (the daily golden cross, Turtle trading) or almost nowhere (intraday momentum, ADX), which may reflect the markets’ trends and the costs as much as the rules.
01What we compared
A rule family is one set of rules run with identical rules on two or more markets, where only what the market forces changed (costs, units, tick size, session clock), declared before the run. Every version is matched with its twin: the same rule id on the other market.
- Transfer rate: among versions positive after costs on market A, the share also positive on market B, over every ordered pair of markets in a family.
- Base rate: the share of all versions of the same family positive on market B, whatever they did on A. If a good result on A means anything, the transfer rate should be above the base rate.
- Each family weighs the same, so the FX family (24 versions on five markets, 480 pairs) does not drown the others. The plain pooled count is shown too.
- Unit-free: results in dollars, R, pips and percent are compared only by their sign, by their t-statistic and by the gates passed.
The families, the questions, the checks and our prediction (“transfer is weak; close markets agree more”) were written and frozen before any cross-market number was computed (the specification).
02The scoreboard
Versions positive after costs on each market, and how often a positive result carried over:
| Rule family and tests | Positive after costs, by market | Transfer rate | Base rate | Difference |
|---|---|---|---|---|
| Six ES strategies on four FX pairs The RSI(2) Dip Buyer · Bollinger Band Reversal · Donchian Channel Breakout · RSI Divergence · MACD Crossover · Supertrend · Six Strategies on Four Currency Pairs | ES 15/24, EURUSD 9/24, GBPUSD 5/24, USDJPY 11/24, AUDUSD 5/24 | 36% | 38% | −2 pts |
| Golden cross 50/200, daily Golden Cross (daily) | ES 4/4, gold 4/4 | 100% | 100% | 0 pts |
| Golden cross 50/200, 1-hour Golden Cross (1H) · Golden Cross (EUR/USD 1H) | ES 11/12, gold 8/12, EURUSD 5/12 | 67% | 67% | 0 pts |
| EMA 9/21 crossover, 1-hour EMA 9/21 (1H) | ES 2/4, NQ 4/4, GC futures 2/4 | 63% | 67% | −4 pts |
| Turtle trading, daily Turtle Trading (gold, daily) · Turtle Trading (EUR/USD, daily) · Turtle Trading (Bitcoin, daily) | gold 4/4, EURUSD 4/4, Bitcoin 4/4 | 100% | 100% | 0 pts |
| Noise area breakout, exchange sessions Noise area breakout (ES) · Noise area breakout (NQ) · Noise area breakout (XAU) | ES 2/6, NQ 3/6, gold 2/6 | 71% | 39% | +33 pts |
| Intraday momentum Intraday momentum (ES) · Intraday momentum (NQ) | ES 0/4, NQ 0/4 | – | – | – |
| Initial balance breakout Initial balance breakout (NQ) · Initial balance breakout (ES) | NQ 3/3, ES 2/3 | 80% | 83% | −3 pts |
| ADX + DI, 1-hour ADX + DI (4 forex majors, 1-hour) | EURUSD 0/2, GBPUSD 1/2, USDJPY 1/2, AUDUSD 0/2 | 0% | 25% | −25 pts |
– = no version was positive on any market, so there was nothing to transfer.
Only the noise area breakout carried a positive result over clearly (+33 pts), and that comes from its New York versions. Where every version was positive on every market, a positive result could not predict anything: there was nothing else to be. On the four FX pairs, the ADX rule’s positive results never repeated on another pair.
03All families together
| On market A | Then positive on B | Base rate | Difference | Pairs (plain count) |
|---|---|---|---|---|
| Positive after costs | 65% | 65% | 0 pts | 301 of 682 |
| Positive before costs (outcome: before costs) | 84% | 83% | +1 pts | 503 of 682 |
| 4 or more of the 6 gates passed | 64% | 66% | −2 pts | 54 of 682 |
To see whether 0 pts could come from chance, we shuffled the results among each family’s versions 10,000 times: 16% of the shuffles gave a difference at least as large. There is nothing here to explain. The plain pooled count (50% against 44%) looks better only because the families where most versions were positive also supply most of the pairs that start from a positive result.
04Did the ranking carry over?
For families with at least four versions, we ranked the versions by their t-statistic on each market and correlated the rankings (Spearman), market pair by market pair:
| Rule family | Market pairs | Median rank correlation |
|---|---|---|
| Six ES strategies on four FX pairs | 10 | 0.23 |
| Golden cross 50/200, daily | 1 | 0.80 |
| Golden cross 50/200, 1-hour | 3 | −0.03 |
| EMA 9/21 crossover, 1-hour | 3 | 0.40 |
| Turtle trading, daily | 3 | 0.20 |
| Noise area breakout, exchange sessions | 3 | 0.77 |
| Intraday momentum | 1 | 0.80 |
Across all 24 pairs the median is 0.25. The ordering is not random (positive in 18 of 24), but a correlation of a quarter means the best version on one market is often only middling on the next. On the 1-hour golden cross the order did not carry over at all.
05Close markets, distant markets, and long-only versions
| Pairs | Transfer rate | Base rate | Difference | Pairs (plain count) |
|---|---|---|---|---|
| Close markets (ES and NQ, two FX majors) | 52% | 51% | +1 pts | 112 of 346 |
| Distant markets (all other pairs) | 72% | 69% | +3 pts | 189 of 336 |
| Long-only versions | 75% | 72% | +3 pts | 150 of 274 |
| Long-and-short versions | 56% | 59% | −3 pts | 151 of 408 |
Our prediction that close markets would agree more held for the ranking (median correlation 0.40 on close pairs, 0.20 on distant ones) but not for the sign. Long-only versions carried over slightly (+3 pts), as expected when the markets rose over the test years; the long-and-short versions, the cleaner test, did not (−3 pts).
06The versions that passed every gate, on their other markets
| Passed every gate on | t there | The same version elsewhere |
|---|---|---|
| gold, GH-L-N (golden cross 50/200, 1-hour) | 3.57 | ES: positive after costs, t 0.32, 1 of 6 gates EURUSD: positive after costs, t 0.14, 1 of 6 gates |
| NQ, NY-LS (noise area breakout, exchange sessions) | 2.88 | ES: positive after costs, t 0.92, 2 of 6 gates gold: negative after costs, t −0.14, 2 of 6 gates |
| NQ, NY-L (noise area breakout, exchange sessions) | 2.70 | ES: positive after costs, t 1.26, 2 of 6 gates gold: positive after costs, t 0.77, 3 of 6 gates |
The 1-hour golden cross on gold was later changed to Inconclusive after a data check, and the two New York noise area versions on NQ are in a forward test on new data. On Bitcoin’s 24-hour day, an adapted version of the same noise area rule lost clearly (t −7.95 and −3.24).
07Adapted replications (counted separately)
Two replications had to change a rule for the new market, so they are shown on their own and not pooled with the families above:
| Replication | Each version on each market (sign after costs, t, gates) |
|---|---|
| Noise area breakout: New York session versions and Bitcoin's UTC day Noise area breakout (ES) · Noise area breakout (NQ) · Noise area breakout (XAU) · Noise area breakout (Bitcoin, 24/7) | NY-LS: ES + (t 0.92, 2/6), NQ + (t 2.88, 6/6), gold − (t −0.14, 2/6), Bitcoin − (t −7.95, 1/6) NY-L: ES + (t 1.26, 2/6), NQ + (t 2.70, 6/6), gold + (t 0.77, 3/6), Bitcoin − (t −3.24, 1/6) |
| Accumulation/Distribution: Bitcoin 4-hour and SPY daily Accumulation/Distribution (Bitcoin, 4-hour) · Accumulation/Distribution (SPY, daily) | DIV: Bitcoin + (t 0.98, 2/6), SPY + (t 0.31, 1/6) MA: Bitcoin + (t 0.64, 1/6), SPY + (t 0.91, 2/6) CHO: Bitcoin + (t 0.72, 1/6), SPY + (t 0.74, 2/6) |
The three A/D readings were positive on both Bitcoin and SPY, but none passed more than 2 of the 6 gates on either.
08What this does not say
- It is not every strategy. These are 9 rule families we chose to replicate, often because they looked interesting on the first market.
- Markets are not independent. ES and NQ trade the same days; a version positive on both is closer to one piece of evidence than to two.
- An edge can exist on one market only. A rule that fails elsewhere is not disproved where it worked; it just gets no support from the other market.
- Positive is not proven. The gates are shown with every result.
09How to test a strategy on multiple markets
- Fix the rules and the second market before looking. Choosing the market where it also works is a search, not a test.
- Compare with the family, not with zero. If most variants are positive on the second market, yours being positive says little. Long-only rules on rising markets are the usual trap.
- Test against random entries on each market. Beating the same trades at random times is the stronger check; see how to backtest a trading strategy without fooling yourself.
- Count costs on each market separately. The same rule can pay its costs on futures and not on forex; see how much trading costs ate across our backtests.
- Prefer new data to a new market. A forward test of the same market answers the question a second market only hints at.
FAQCommon questions
Should I test a trading strategy on multiple markets?
Yes, but read the result against the other versions of the same rules, not against zero. In our tests, being positive on a second market happened as often for versions that failed on the first as for versions that succeeded.
Does a strategy that works on one market work on another?
In our 682 market pairs, a positive result on one market raised the chance of a positive result on another by about zero: 65% against a base rate of 65%. The ranking of versions carried over weakly (median rank correlation 0.25).
Do strategies transfer better between similar markets?
Only a little, and only for the ranking: the median rank correlation was 0.40 between close markets (ES and NQ, two FX majors) and 0.20 between distant ones. The share of positive results that carried over was not higher for close markets.
Why can long-only strategies look robust across markets?
Because markets that rose over the test years make most long-only rules positive together. In our tests long-only versions carried over slightly; long-and-short versions, the cleaner test, did not.
What is a better check than a second market?
Comparing the strategy with the same trades at random times on each market, and a forward test on new data. How to backtest without fooling yourself.
10Check our arithmetic
xm_spec.md · xm_families.json
xm_run.js · xm_ref.js · xm_inventory.json · xm_summary.json · check results
11Corrections
None since publication.