Skip to content
TradingTune
Content and Compliance

How To Explain Plan Differences

Shows how to compare two parameter sweep plans by range, step size, history slice and walk-forward split so each difference has one cause.

On this page

Key takeaways

  • Explaining a plan difference means naming the scope, the mechanics and the tooling that separate two sweeps.
  • Two plans compare cleanly only when everything but one swept input is held constant, so a change in the equity curve traces back to one cause.
  • Read the shape of the curve and the cycle count, such as a plateau or a thin sample, rather than picking the highest single result.
  • A stable-looking plateau from one walk-forward slice only shows the parameter was forgiving under one historical regime.

What It Means to Explain Backtest Plan Differences

Explaining a plan difference means describing, in plain terms, how two parameter-sweep or walk-forward configurations differ: which features or inputs each one tests, how many cycles each one runs, how wide a slice of history each one covers, and which optimization tools produced the numbers you are staring at. A backtest is just a simulation of a strategy against historical price data, and a parameter sweep runs that simulation many times while one or more inputs change across a range. Walk-forward testing splits that history into rolling in-sample and out-of-sample windows so a configuration is validated on data it never touched during tuning. The instinct to line two runs up side by side and ask “what actually changed here” is not unique to trading software either; database engineers do the same thing when they compare two query execution plans to see which operations, costs, or access paths shifted between runs, and the underlying discipline, isolate one variable, read the delta, ask why, transfers cleanly.

Three things matter when you explain a difference between sweeps: name the scope (what date range and symbol each run covered), name the mechanics (cycle count, step size, and whether walk-forward splits were used at all), and name the tooling (which optimizer or search method generated the configuration). This is written for a trader who is already running more than one sweep against their own TradingView strategy and needs a repeatable way to describe why run A and run B produced different-looking equity curves, not for someone shopping for a strategy to copy.

Building Two Sweep Plans to Compare Side by Side

Before you can explain what changed between two runs, you need two runs built on purpose, not two runs that happened to come out of the optimizer at different times. A sweep is the set of input values you tell the engine to march through, and a “plan” in this context is nothing more than one fully specified sweep: which parameter or parameters move, what range they cover, what step size separates each tested value, and which slice of price history backs the whole thing. Change any one of those four things and you have a different plan, even if the strategy logic underneath is identical. For a first plan on the Smart Momentum Grid, I typically sweep the lookback, the number of historical bars the strategy looks back over to compute its trend filter, across a range of 20 to 100 bars in steps of 10. For a second plan meant to contrast against it, I hold that lookback fixed at a single value and instead sweep the grid spacing multiplier from 0.5 to 3.0 in steps of 0.25, so the difference between the two plans is legible: one interrogates trend sensitivity, the other interrogates entry density.

The walk-forward split is the second axis that has to be named, not left implicit. Walk-forward testing here means carving history into a rolling in-sample window, the slice the optimizer is allowed to tune against, followed by an out-of-sample window the configuration never sees during that tuning pass, then sliding both windows forward and repeating. A plan using a 12-month in-sample window with a 3-month out-of-sample window is mechanically distinct from a plan using 6 months in-sample and 6 out-of-sample, even with the identical swept parameter, because the second plan is asking a fundamentally different question about how quickly a setting goes stale. This is the same discipline that shows up when engineers line up two query plans and separate the read from the write path before drawing any conclusion about a cost shift, as the Oracle explain-plan comparison guidance lays out when walking through what changes between two execution plans: you cannot compare outcomes honestly until you have pinned down exactly what varies and what stays constant.

A good sweep must not vary two things at once and then blame the result on one of them. If I widen the lookback range and shorten the walk-forward window in the same plan, any shift in cycle behavior could come from either change, and the comparison becomes worthless for explaining anything. When I ran side-by-side sweeps on the grid spacing multiplier last quarter, I kept the lookback locked, the symbol locked, and the date range locked, and only let the step size move, specifically so that whatever shape showed up in the equity curve traced back to one cause. That is also why naming the optimizer or search method matters as part of the plan definition: a full grid search over a range and a randomized search over the same range are two different plans even when the swept parameter and its bounds are word-for-word identical, because they sample the space differently and can surface different regions as interesting. The instinct to hold everything constant except the one variable under test is exactly the practice engineers describe when they compare two runs and treat every unexplained delta as a signal to isolate further, not a conclusion to accept at face value, and it applies just as directly to a parameter sweep on a Pine strategy as it does to a database query plan.

TradingTune for Chrome

Stop tuning your strategy's inputs by hand.

Add to Chrome, it's free

Free tier, no API keys. A free account is required to run. Not affiliated with TradingView, Inc.

How Equity Curve Behavior Shifts Across Sweep Settings

Once a sweep runs, the equity curve does not shift randomly as the swept parameter moves; it tends to shift in one of two recognizable shapes, and learning to name that shape is most of what “explaining” a plan difference means in practice. Sweeping the lookback on the Smart Momentum Grid from the low end of its 20 to 100 bar range toward the high end produces the first shape: the curve gets visibly choppier at the short end, with more direction changes packed into the same stretch of history, and cycle count climbs because a shorter lookback recalculates its trend filter more often and reacts to more of the noise sitting inside each price swing. At the long end of that same range, the curve smooths out and cycle count drops, because a longer lookback averages over more bars and only shifts its read on trend slowly. Neither end is “better” in any scored sense; they are just structurally different responses to the same input, which is exactly the point a structural comparison is built to surface.

The second observation sits on the grid spacing multiplier sweep rather than the lookback sweep, and it is a different shape entirely: a plateau. When I ran the grid spacing multiplier from 0.5 up through 3.0 in quarter-point steps, the equity curve barely moved for a wide middle band of values, roughly 1.25 through 2.0, then changed character sharply outside that band on both sides. Below 1.25 the entry density goes up and the curve develops a saw-toothed look, since the strategy is opening and closing positions on tighter spacing than the underlying price movement really supports. Above 2.0 the entries get sparse enough that whole stretches of the curve go flat, not because nothing happened in the market but because the strategy sat out. That plateau-then-cliff pattern is itself a structural finding: it tells you where the parameter is forgiving and where it is fragile, without needing to attach a single performance number to any of it.

This is the same kind of reading a database engineer does with two execution plans that diverge on a join order or an index choice: the Microsoft documentation on comparing execution plans walks through spotting where operator costs or row estimates shift between two plans rather than reducing the comparison to one plan being simply faster, because the shift itself is the information worth explaining. A sweep comparison works the same way: what I’m reading off the curve is where the shape changes and why, not which single point in the range scored highest, because “highest” on a historical sweep tells you nothing about what a live market will do next.

Cycle count differences carry their own separate signal, distinct from curve shape, and worth naming on its own. A parameter that produces a low cycle count across its whole range is telling you the strategy rarely finds conditions it wants to act on, which changes how much you can trust any conclusion drawn from that plan at all, since a handful of cycles is a thin sample no matter how clean the curve looks between them. This lines up with how pganalyze’s guidance on comparing Postgres explain plans frames the same caution for query plans: a plan that touches very few rows can look deceptively efficient purely because it never got tested against the harder cases, and the comparison only means something once you account for how much work each plan actually did. Reading a sweep the same way, a plan with a thin cycle count across its whole tested range deserves a wider range or a longer history slice before its shape gets trusted, and that judgment call, not a single scored winner, is what the comparison is actually for.

Reading a Labeled Table of Plan Differences

The following table is hypothetical, the founder’s own Smart Momentum Grid, historical and illustrative only.

Parameter Range Step Cycle count Qualitative note
Lookback (Plan A) 20 to 100 bars 10 bars 320 cycles curve choppier at the low end, smooths out toward the high end
Grid spacing multiplier (Plan B) 0.5 to 3.0 0.25 280 cycles plateau from roughly 1.25 to 2.0, sawtooth below it, flat stretches above it
Symbol and timeframe scope (Plan A) single symbol, 4-hour chart n/a n/a scope held narrow on purpose, one instrument only
Symbol and timeframe scope (Plan B) single symbol, 4-hour chart n/a n/a identical scope to Plan A, kept constant so only the swept parameter differs
Walk-forward window (Plan A) 12 months in-sample, 3 months out-of-sample n/a n/a quicker validation cycle, tighter out-of-sample check
Walk-forward window (Plan B) 6 months in-sample, 6 months out-of-sample n/a n/a slower turnover, longer out-of-sample check on each tuned setting

Read this table row by row rather than top to bottom in a hurry, because each row is answering a different question about the same underlying strategy. The first two rows are the swept parameters themselves, and they are the only rows where cycle count means anything, since cycle count is a direct readout of how often the strategy found conditions worth acting on across that particular range. Three hundred and twenty cycles against a lookback sweep and two hundred and eighty against a grid spacing sweep are both healthy sample sizes for drawing a shape conclusion; if either number had come in closer to a few dozen, I would treat the whole row as too thin to trust and widen the range or extend the history before reading anything into the curve.

The scope rows matter precisely because they are boring, and that is the point. Plan A and Plan B share the identical symbol and timeframe on purpose. If those two rows had differed, an equity curve difference between the plans could be explained by the scope change instead of the parameter change, and the whole comparison would collapse into noise. The same principle shows up in how plan-comparison tooling in adjacent fields is documented for a general audience: this Microsoft guide to side-by-side plan comparison walks through presenting two configurations side by side specifically so a reader can spot what actually changed rather than guessing at it, which is the same job this table is doing here.

The walk-forward rows are the ones I would probe further before trusting either plan’s shape completely. A 12-month in-sample window with a 3-month out-of-sample check validates faster but on a shorter forward-looking slice, while the 6-and-6 split trades speed for a longer look at how the tuned setting holds up once it stops seeing fresh data it was calibrated against. Neither split is stated here as the correct one; a practitioner reading this table should notice that the two walk-forward rows are not directly comparable to the parameter rows above them, and should treat any claim about “which plan held up better” as incomplete until the out-of-sample windows themselves are lined up on equal footing.

When a Sweep Plan Looks Fine In-Sample and Falls Apart Out-of-Sample

The clearest overfit trap I have walked into with the Smart Momentum Grid was not a parameter that looked bad and got worse. It was a parameter that looked stable, in exactly the plateau shape described above, and then stopped looking stable the moment the walk-forward window rolled forward onto fresh data. The grid spacing multiplier’s flat middle band from roughly 1.25 to 2.0 held up cleanly across the 12-month in-sample window I first tuned it against, and I want to be specific about why that fooled me rather than just naming the symptom.

The in-sample window I had picked happened to sit inside a stretch where price moved in a fairly narrow, grinding range for months at a time. Under that kind of price behavior, a wide band of grid spacing values all produce roughly the same entry density, because the underlying moves are small and slow enough that spacing between 1.25 and 2.0 barely changes how often the strategy triggers. The plateau was real, but it was a property of that particular slice of history, not a property of the parameter itself. When the out-of-sample window rolled forward into a period with sharper, faster price swings, the same spacing values that had looked interchangeable started producing very different cycle counts: the tighter end of the band clustered entries during the fast moves in a way the in-sample window never showed, and the curve that had looked smooth and forgiving in-sample turned choppy in a way that had nothing to do with the parameter changing and everything to do with the market regime changing underneath it.

The lesson I keep coming back to is that a stable-looking plateau from a single walk-forward slice tells you the parameter was forgiving under one historical regime, never that it is forgiving in general. This is close in spirit to the caution built into how ProjectionLab documents comparing what-if scenarios, where a plan that looks resilient under one set of assumed conditions still needs to be checked against a different set of conditions before anyone treats that resilience as durable. A sweep needs the walk-forward window rolled across more than one historical stretch, ideally spanning both a calm period and a volatile one, before a flat middle band gets read as a genuinely stable region rather than an artifact of whatever slice of history happened to sit in the in-sample window.

A second, smaller version of the same trap shows up with the lookback parameter when the in-sample window happens to end right before a trend reversal. A lookback value tuned to react quickly to that specific reversal will look sharp in-sample and then lag badly on the next reversal the out-of-sample window contains, because the in-sample window rewarded a setting tuned to one transition rather than to transitions generally. Naming a swept input cleanly in code at least keeps the plan itself legible when you go back to check which range was tested and how it was stepped:

Swept Parameter Input Structure. Defines a float input for the grid spacing multiplier with the minimum, maximum, and step that the optimizer marches through during a sweep.

Pine v6
//@version=6
indicator("Swept Parameter Example", overlay=true)
// Input structure for a swept parameter: range and step define the sweep plan.
gridSpacingMultiplier = input.float(1.5, title="Grid Spacing Multiplier",
minval=0.5, maxval=3.0, step=0.25)
lookbackBars = input.int(60, title="Lookback Bars",
minval=20, maxval=100, step=10)
plot(gridSpacingMultiplier, title="Grid Spacing Multiplier")

The Expert Take: What to Compare Before You Trust a Plan

Before treating either plan as robust, compare four things and only four things: the scope each run held constant, the mechanics of the sweep itself, the walk-forward split each configuration used, and whether the shape you are reading (plateau, cliff, smooth taper) survives more than one historical regime. Skip any of those four and you are not comparing plans, you are comparing coincidences that happen to share a chart.

My take is that the walk-forward comparison deserves more scrutiny than the parameter comparison, not less, because a swept parameter’s range and step are visible on the page while a walk-forward window’s regime dependence is invisible until you deliberately go looking for it. Two plans can share an identical lookback range, an identical grid spacing range, and an identical symbol, and still disagree about what “stable” means if one was validated across a single calm stretch and the other rolled its out-of-sample window across a stretch that included a sharp reversal. I recommend treating any plateau found under a single walk-forward slice as provisional, not proven, until it has been checked against at least one different historical window with a visibly different character, since that is the only way to tell a genuinely forgiving parameter from one that only looked forgiving because the market sat still while you were tuning it.

The concrete next step: take one sweep you already ran this week, note which walk-forward window backed it, then re-run the identical parameter range and step against a second historical window drawn from a period with a different character than the first, faster price movement if the first was calm, a longer flat stretch if the first was volatile, and compare cycle count and curve shape across the two windows before comparing anything against the other plan. If the plateau holds across both windows, you have learned something about the parameter. If it collapses in the second window the way the grid spacing multiplier did in ProjectionLab’s documentation on checking a plan against a different set of underlying conditions, you have learned something more valuable: which part of your sweep was reading the market regime instead of the parameter, and that is the finding worth writing down before you touch either plan again.

Closing the Loop on Explaining Plan Differences

Structural comparison beats a single number because a single number cannot tell you where a plan is fragile and where it is forgiving, and that distinction is the entire reason to run a sweep in the first place. Every method in this post, naming scope, naming mechanics, naming the walk-forward split, reading a shape instead of a score, exists to answer one question a bare figure can never answer on its own: under what conditions does this plan hold together, and where does it quietly come apart? A plateau tells you a parameter is tolerant across a range. A cliff tells you exactly where tolerance ends. A thin cycle count tells you the sample is too small to trust either reading yet. None of that comes from picking the highest number in a results column; all of it comes from walking the sweep the way you would walk two execution plans looking for where the operations diverge, which is why the discipline behind comparing execution plans by their operations and access paths rather than a single cost figure transfers so cleanly to a parameter sweep on your own TradingView strategy. The risk this method protects against is concrete: mistaking a stable-looking plateau, tuned against one slice of history, for a parameter that is stable everywhere, when it may just be a parameter that got lucky with the regime it was tested against.

That closes the loop on the promise this post opened with, to explain plan differences in plain terms rather than in scored comparisons. Explaining a plan difference means being able to say, out loud and specifically, what varied between two configurations, why it varied on purpose, and what shape that variation produced in the equity curve and the cycle count, the same habit of isolating one variable and reading the delta that shows up when engineers work through why one query plan’s cost shifted against another. If you take one habit away from all of this, make it the walk-forward check: before you trust any plateau, cliff, or smooth taper your sweep produced, roll the out-of-sample window across a second historical stretch with a genuinely different character than the first, and only call a parameter forgiving once it has survived more than one regime asking the question.

Sources

TradingTune for Chrome

Stop tuning your strategy's inputs by hand.

TradingTune runs systematic parameter searches on your own TradingView strategy, right inside the chart page. You keep the strategy, you read the results.

Free tier, no API keys. A free account is required to run. Not affiliated with TradingView, Inc.

Share this post