In-sample vs out-of-sample
Optimizing on all your data and admiring the result proves nothing: the strategy has seen every bar it is judged on. The fix is simple and non-negotiable. Split the history, tune on one part, and judge on a part the optimizer never touched.
Two datasets, one rule
In-sample data is the period you optimize on. Out-of-sample data is a later period the optimizer never sees. You tune the parameters on the in-sample window until the metrics look good, then you run those exact settings once on the out-of-sample window. If the strategy holds up, you have evidence of a real edge. If it falls apart, you fit noise. The rule that makes it work: the optimizer must never, ever see the out-of-sample data.
TradingTune for Chrome
Stop tuning your strategy's inputs by hand.
Free tier, no API keys. A free account is required to run. Not affiliated with TradingView, Inc.
Choosing the split
A 70/30 or 80/20 split is the usual starting point: optimize on the earlier 70 to 80 percent, hold out the most recent 20 to 30 percent. Keep the out-of-sample period in the future relative to in-sample, never the reverse, so the test mimics how you would actually deploy. Make sure both windows are long enough to produce a meaningful trade count; a holdout with eight trades tells you nothing.
- 70/30 or 80/20, in-sample earlier, out-of-sample later.
- Both windows need enough trades to be meaningful on their own.
- Never tune on the holdout, not even once, not even a little.
Holding out data in TradingView
The cleanest way to hold out data is to gate trading by date with inputs, optimize with the window set to the in-sample period, then move the window to the out-of-sample period and run once without changing any parameter. Because the gate is an input, you never edit the strategy between the two runs, which removes a whole class of accidental leakage.
In-sample / out-of-sample gate (Pine v5)
Pine v5One boolean input flips the strategy between the in-sample and out-of-sample windows without touching any tuned parameter.
//@version=5
strategy("IS / OOS Split", overlay=true)
splitDate = input.time(timestamp("2024-01-01 00:00"), "In-sample ends")
phase = input.string("in_sample", "Phase", options=["in_sample", "out_of_sample"])
inSample = time < splitDate
active = phase == "in_sample" ? inSample : not inSample
longSignal = ta.crossover(ta.sma(close, 20), ta.sma(close, 100))
if longSignal and active
strategy.entry("Long", strategy.long)
if not active
strategy.close_all() Leakage, the silent killer
Leakage is any way information from the out-of-sample period sneaks into your tuning. The obvious form is optimizing on the holdout by accident. The subtle forms are worse: peeking at the holdout result, then going back to tweak the in-sample settings; running the holdout dozens of times and picking the run that happened to work; or using indicators that repaint and quietly use future bars. Each one turns your honest test back into an in-sample fit.
Warning
Peeking counts as leakage
If you look at the out-of-sample result and then change anything and re-tune, the holdout is burned. You now need fresh, unseen data to test again.
Key takeaways
- Tune on in-sample data; judge once on out-of-sample data.
- 70/30 or 80/20, with out-of-sample later than in-sample.
- Gate by a date input so you never edit the strategy between runs.
- Peeking at the holdout and re-tuning is leakage; the test is burned.
Frequently asked questions
What split ratio should I use for in-sample and out-of-sample?
70/30 or 80/20 are good starting points, with the in-sample period earlier and the out-of-sample period later. Both windows must contain enough trades to be meaningful on their own.
What is data leakage in backtesting?
Any way out-of-sample information influences tuning: optimizing on the holdout by accident, peeking at the holdout result then re-tuning, running the holdout many times and cherry-picking, or using repainting indicators that read future bars.
How is out-of-sample testing different from walk-forward?
A single split tests one holdout once. Walk-forward repeats the optimize-then-test cycle across many rolling windows, which is a stronger and more realistic check. See the walk-forward analysis guide.
Related guides
- Beginner
Choosing a backtest date range
Pick a TradingView backtest date range that spans several market regimes, avoids cherry-picking, and yields enough trades to trust before you optimize.
Read the guide - Advanced
Walk-forward analysis
Validate a TradingView strategy with walk-forward analysis: roll optimize-then-test windows forward to prove an edge survives out of sample, repeatedly.
Read the guide - Intermediate
How to avoid overfitting
Optimized parameters can look perfect in-sample and fail live. How to spot overfitting and build robust TradingView strategies that hold up out of sample.
Read the guide
Related reading
- Why Peeking At Holdout Data Weakens The Split
Explains how repeated decisions based on held-back data can turn validation into another tuning step.
- How Out-Of-Sample Tests Help Trend Review
Explains how unseen historical windows can reduce the chance of reading an optimized result too confidently.
- TradingTune blog: TradingView backtesting and optimization
Research notes on testing and optimizing your own TradingView strategies: Pine Script inputs, validation habits, execution costs, and a repeatable workflow.
Put it into practice
Install TradingTune and optimize any TradingView strategy right in your browser. Free tier, no API keys. A free account is required to run.
Add to Chrome, it's free