How to Test a Trading Rule on Your Own Trades (and Tell It From Luck)

Every trader has a rule they are sure would help. No Fridays. Nothing after the New York lunch. Stop after two losses. Only A+ setups. Most of these rules get adopted on a Sunday evening after a bad week, which means they are built on the five trades you remember most, not on the three hundred in your journal.

You can test a rule before you follow it, on your own trades, in an afternoon. And the test has a second half almost nobody runs: checking whether the improvement is bigger than what luck alone would give you. Without that half, most rules look like they work.

What testing a rule actually means

Testing a rule on your journal is a replay. You go through your past trades in order and mark every trade the rule would have stopped you from taking. Then you compare two curves:

• Your trades: everything you actually took.
• With the rule: the same trades, minus the ones the rule skips.

That is the whole mechanism, and it covers a lot of rules: skip a weekday, skip entries after a time, only take one setup, only take trades with a confluence present, stop for the day after N losses, cap trades per day, stop once the day is down a set amount.

It does not cover rules that change what happens inside a trade. Moving the stop to break-even, taking partials, a wider target or a smaller size all need the price path after your entry, which a journal does not have. Be suspicious of any tool that claims to replay those from a trade list.

The trap: skipping trades always changes the result

Here is the part that makes most rule tests worthless. Take any journal and delete 60 random trades. The curve changes. Sometimes it improves, sometimes the drawdown shrinks, sometimes the win rate goes up a few points. Not because the deleted trades shared anything, but because fewer trades means a different path.

So "my profit went up with the rule" is not evidence. The question is whether your rule did better than skipping the same number of trades at random.

That comparison is easy to run with a computer:

1. Count how many trades your rule skipped, say 65.
2. Pick 65 trades at random, skip them, and record the result.
3. Do it 2,000 times.
4. Count how often random skipping did at least as well as your rule.

If random skipping matches your rule more than about 1 time in 20, your data does not show the rule works. If your rule beats at least 95% of the random versions, the effect is unlikely to be luck in your sample. Statisticians call this a permutation test. Traders can call it the luck check.

Eleven rules on 340 trades

To show what this looks like, we ran eleven common rules through the luck check on the 340-trade sample journal behind the TradingSFX demo. The journal is built for the demo with a few real-world patterns on purpose (a strong morning, a weak late session, one setup that keeps failing), which makes it a good test: a sound method should find what is there and call the rest noise.

Before any rule: +$12,322, +92.2R, max drawdown $5,736, win rate 39.5%.

The percentage is how many of the 2,000 random skips the rule beat, in dollars unless noted.

What the table says

The planted patterns are found. The late session and trading against the higher-timeframe bias were built into this journal as weaknesses, and the two rules that remove them beat chance at 99%. The gold-shorts rule skips only 17 trades and still clears the bar, because those 17 are concentrated losers.

Most popular rules are noise here. No Fridays, max 2 trades a day and stop after 2 losses all made this journal worse. They are not bad rules in general. They are rules this trader has no evidence for.

Dollars and R can disagree. No Wednesdays adds $2,421 but removes 12R. A rule that looks good in money and bad in R is usually telling you about position size on those days, not about the days themselves. Look at both before you decide.

Less drawdown is not automatically a win. Max 1 trade a day cuts the drawdown from $5,736 to $2,523, which feels like progress. But skipping 117 trades at random cuts drawdown too, because fewer trades means a shorter path for losses to stack up on. In R the rule loses 13R, and the luck check calls it noise.

Beating chance is not the same as making more money. Only the best setup earns less than all five setups together, yet it beats 96% of random skips. Removing 242 trades at random costs about $8,800 on average; removing these particular 242 cost $1,365. The kept setup is clearly better than the average trade. Whether you want fewer trades with a smaller drawdown is a decision, and the test shows you its price.

The second trap: trying too many rules

A 95% bar means about 1 random rule in 20 passes by luck. Test twenty rules on the same journal and you should expect one false winner, and it will look exactly like a real one.

Four habits keep that in check:

• Write the rule down before you test it, with the reason you believe it. "I lose focus after 4 pm" is a reason. "Let me try every hour until one works" is a search.
• Test few rules. Three hypotheses with reasons beat thirty without.
• Check both halves of your history. Split the journal in two by date. A real effect usually shows in both; luck usually lives in one.
• Forward test it for a month. Follow the rule live, keep logging, and run the comparison again on trades the test has never seen.

What makes a rule testable

The test can only see what you logged. Three things decide what you can test:

1. The entry time. Without it, no time-of-day rule can be checked. A date alone tells you the weekday, not the session.
2. The setup name, spelled the same way each time. "London OB", "london ob reversal" and "LOB" are three setups to a computer.
3. Both answers to a confluence. If you only ever tick "HTF bias: Yes", there is nothing to compare it with. Log Yes and No, every trade, and the trades without the confluence become your control group.

How to run the test yourself

In a spreadsheet. Export your journal to CSV. Add a column that flags each trade the rule skips (an IF on the weekday, the entry hour or the setup). Sum the P&L and the R with and without the flagged rows, and build both running totals for the curves. The luck check is the hard part: add a column of RAND() values, sort by it, skip the same number of trades from the top, note the result, and repeat. Even 100 repetitions gives you a rough idea of where your rule sits.

In TradingSFX. Strategy Lab on the Coach page does the replay and the luck check in one click, with 2,000 random skips, on your own journal:

• Ready rules: no Fridays, nothing from 16:00, stop after 2 losses a day, max 2 trades a day, plus rules built from your own setups and confluences.
• A builder: days, entry window, symbol, setup, long or short and a confluence value, or a cap on trades, losses or loss per day.
• Typed rules: write the rule in your own words, like "no shorts on gold after 15:00". The coach turns it into a rule on your own symbol and setup names, shows you what it read it as, and refuses rules a journal cannot test, such as break-even stops, instead of guessing.

Every result shows both curves, the numbers side by side in dollars or R, and the luck check. You can pin a comparison to your dashboard, where it updates with every new trade, which makes the month of forward testing automatic.

You can try it without an account in the Coach demo, on the same 340 sample trades as the table above. Strategy Lab is part of the Pro plan, and the AI Trading Coach guide walks through the rest of the Coach page.

The short version

• A rule test is a replay: which of your past trades would the rule have skipped?
• A better curve proves nothing on its own. Compare the rule with skipping the same number of trades at random.
• Look at dollars and R. When they disagree, your position size is part of the story.
• Decide the rule before you test it, test few, and confirm it on trades the test has not seen.

None of this is financial advice: past trades describe what happened, not what will.