Guide

What to do when a strategy fails validation

Last reviewed: 15 September 2026·Tradelyze

When a trading strategy fails validation, find the cause before changing anything: the usual causes are a data mismatch, too little evidence, too much tuning, or a position too large for the rules. Fix that cause, then run the test once more. Re-optimizing until a check passes makes overfitting worse, because every extra try is another chance for luck.

New to this? Start with How Tradelyze validates a strategy, which explains every card named on this page.

What does each failing result mean, and what should I do next?

Each validation check in a Tradelyze report ends in a verdict, and each failing verdict points to a different cause. The first two rows below are not verdicts at all: the run never finished, so nothing about the strategy was judged. Find your result, make the one change that addresses its cause, and run again once. The Source column separates what Tradelyze's code does from judgment calls with no published standard behind them.

A trial in that table is one full backtest with one set of settings. The optimizer is the part of Tradelyze that runs many trials and keeps the best. Every step in the table changes one thing and runs once. That discipline is what keeps a second run informative.

What if the verdicts disagree?

When two verdicts in a Tradelyze report disagree, let the more cautious one decide, because each check can see a problem the other cannot. A prop firm card checks the firm's rules on one backtest. The walk-forward badge asks whether the tuning held up on prices it never saw. The robustness verdict asks whether the result depended on luck. Match Rate asks whether any of those numbers describe your TradingView strategy at all.

A Qualifies badge means every rule on a prop firm card passed, and Not Feasible means at least one failed. FRAGILE is Tradelyze's lowest robustness verdict, given to a score under 50. A low Match Rate can sit beside a full report, because Continue anyway and Run Optimization Anyway both optimize the re-run as it is.

A constructed example, not measured data: a strategy's prop firm card reads Qualifies, with a maximum total drawdown of 7.2% against a 10% limit. Its walk-forward badge reads FAIL, with WF Efficiency at 0.21 across two windows that each held more than 40 trades. WF Efficiency is roughly the share of the tuned Sharpe ratio that survived on the test windows, so only about a fifth survived. The rules passed on history the settings were tuned to fit. A challenge trades new prices, so the walk-forward result is the better guide.

The badges, and what each check measures, come from Tradelyze's implementation. Which verdict should limit the decision, and the next steps in the table, are practitioner judgment with no primary source.

Why doesn't my re-run match TradingView, and what should I fix first?

A low Match Rate means Tradelyze's re-run of your script and your TradingView trade list disagree about which trades happened. Match Rate is the share of trades the two lists agree on, shown on the Conversion Result card. Fix a low Match Rate before reading anything else. Every later number, from Best Metrics to the prop firm cards, describes the re-run, not the strategy in your TradingView report.

Check these three causes in order, quickest to rule out first:

1. Chart Timezone. The upload form asks for the zone shown in the bottom-right corner of your TradingView chart, because both exported files are stamped in that zone. A wrong zone shifts every trade time. Tradelyze's own code notes that the same faithful re-run can measure a 100% match against one timezone and about 16% against another. The export guide's timezone section shows which zone to pick.
2. Script defaults. Tradelyze runs your script with its own defaults: initial capital, commission, slippage, order size, pyramiding and input values come from the strategy() declaration and the input() calls. If you changed any of these in TradingView's Properties or Inputs tabs before exporting, write the same values into the script and upload it again. TradingView strategy properties explains what each setting changes.
3. Price data range and timeframe. The OHLCV file holds the open, high, low, close and volume of each bar. It must be for the same instrument and timeframe as the chart you exported from, and cover the same dates. TradingView trades that entered after the file's last bar cannot match anything.

A constructed example, not measured data: a strategy with 212 TradingView trades comes back with a Match Rate of 18%. Review comparison shows the re-run took the same sequence of long and short trades, but each one entered exactly five hours after its TradingView partner. Tradelyze only pairs trades whose entries fall within five minutes of each other, so a constant offset leaves almost nothing matched. A constant offset points to the timezone, not the strategy. Correcting Chart Timezone and submitting again is the fix; retuning would change nothing, because the settings were never the problem.

Continue anyway, on the Optimization stopped card that shows a match percentage, optimizes the re-run as it is. It is reasonable only once Review comparison shows the unmatched trades are explained and minor. After a wrong timezone, changed defaults or a mismatched price file, fix the cause and submit again instead; each submission costs 1 credit. The full list of reasons trades do not match and what Continue anyway does are covered on the TradingView backtest accuracy page.

What should I do when walk-forward fails or shows no verdict?

When walk-forward fails, first find out whether the edge faded or the evidence was missing, because the two need opposite responses. Walk-forward analysis tunes a strategy on one stretch of history, called in-sample, then scores it on the stretch that follows, called out-of-sample, which the tuning never saw. Tradelyze repeats this over 2 windows by default and shows a PASS, FAIL or NO VERDICT badge.

The badge reads PASS only when four conditions all hold. WF Efficiency is above 0.5, and more than 60% of the usable test windows made money. At least one window was usable, and no more than half the windows were excluded. WF Efficiency is the average annualized Sharpe ratio on the test windows divided by the same average on the tuning windows. The Sharpe ratio is average return divided by how much returns swing, and annualized means converted to a yearly rate.

In that constructed result, the tuned edge mostly carried over, yet the badge reads FAIL because one of only two test windows lost money. One window decided the verdict. The right response is more evidence, not re-tuning. Export a longer trade list and price file for the same strategy, or raise Walk-forward windows. Tradelyze allows 2 to 6 windows when walk-forward and robustness settings are enabled for your account. More windows on the same history make each test window shorter, with fewer trades in it, as how many walk-forward windows to run explains.

A different FAIL needs a different response. If WF Efficiency is at or below 0.5 across windows that each had plenty of trades, the tuned edge faded on data the optimizer never saw. The next step is a smaller search, not a bigger one. Fix the inputs you would never change, and narrow the remaining ranges to values you would really trade, using the Fixed, Min, Max and Step settings. Fewer combinations tried make any result that survives stronger evidence. What each WF Efficiency label means is covered in reading the WF Efficiency tile.

NO VERDICT means the walk-forward stage ran but formed no pass or fail for the method it used. It is neither a pass nor a failed check. Read the small label beside the badge first: Re-tuned each window, Fixed settings across periods or One run, split by period. Then read the Per-Window Results table instead of the badge. The two methods answer different questions, set out in re-optimized walk-forward versus parameter stability.

What should I do when a robustness check shows NOT RUN or FRAGILE?

When a robustness check shows NOT RUN, make sure it runs next time; when the verdict is FRAGILE, read which checks failed and fix that cause. The robustness score is one 0–100 number that adds up five stress tests of the tuned result: Monte Carlo, Permutation Test, Parameter Sensitivity, Deflated Sharpe Ratio and Min Backtest Length. Tradelyze's verdict is ROBUST when all five ran and passed with a score of 80 or more. Otherwise it reads ACCEPTABLE at 70 or more, MARGINAL at 50 or more, and FRAGILE below 50.

NOT RUN means a check did not happen, which is different from failing it. The common case is Parameter Sensitivity on a small search. Tradelyze's automatic sensitivity budget is zero when fewer than 50 optimization trials ran. The automatic trial budget is never fewer than 60, so the usual trigger is Optimization trials set to a custom 20, 30 or 40. If walk-forward and robustness settings are enabled for your account, set it to Automatic or to 50 or more. A NOT RUN check leaves the score, and the verdict cannot be ROBUST without it.

Two checks refuse to run on thin data and score zero instead. The Permutation Test needs at least 20 closed trades, and the Deflated Sharpe Ratio needs at least 5. Those minimums only let a check run; they do not make 20 trades enough evidence, as how many trades a backtest needs explains.

FRAGILE means the robustness score is under 50. The total hides which checks failed, so read each row and act on the one that failed:

• Parameter Sensitivity reads Unstable. Nudging the winning settings slightly cost 20% or more of the Sharpe ratio on average, or the original Sharpe ratio was not positive. The result depends on exact values: a spike, not a plateau. Search fewer inputs and prefer sensible round values; see plateau versus spike.
• Permutation Test reads Not Significant. The trades did not clearly beat copies whose wins and losses were flipped at random. With fewer than 20 trades that is a data problem; with many trades, the edge may not be there.
• Monte Carlo shows a Ruin Probability of 20% or more. Redrawn sequences of your own trades broke the firm's total drawdown limit too often. Position size is the first thing to test; see what Ruin Probability measures.
• Deflated Sharpe Ratio reads Not Significant. The Sharpe ratio does not beat what luck would produce from the number of settings tried. Search less.
• Min Backtest Length reads Insufficient. There are not enough years of history for this Sharpe ratio and this many settings tried. Add history, or search less, because Required Years rises as more settings are tried.

A constructed example, not measured data: on one robustness card, Monte Carlo passed and Parameter Sensitivity reads Unstable. The Permutation Test reads Not Significant on 17 trades, and the DSR Value of 0.62 reads Not Significant. Min Backtest Length reads Insufficient, with 4.1 Required Years against 1.5 Available Years. Four of the five rows point to one cause: too few trades and too much searching for the data available. Raising the trial count would make that worse, because the deflated Sharpe ratio and Required Years both grow stricter as more settings are tried. Each row is explained in what each row on the robustness card means.

What should I do when a prop firm card reads Not Feasible?

When a prop firm card reads Not Feasible, read the first failing row of its Rule Results table and fix that rule first. Not Feasible means at least one rule failed on the backtest. Each row shows a Status mark, the Rule, the Actual value the backtest reached, the Limit and a Message. Tradelyze checks maximum daily drawdown, maximum total drawdown, profit target, minimum trading days, consistency and minimum trade count. It does not check evaluation time limits, news-trading rules or weekend-holding rules.

When a drawdown row fails, test a smaller position size before rewriting the strategy's logic. A drawdown is a fall in account value from a high point, and a firm's drawdown limit is a fixed amount while losses grow with size. Trading smaller also shrinks profit, so check every row after the change.

In that constructed case, cutting from 3 contracts to 2 scales both figures by two-thirds: 6.8% becomes about 4.5% and 11% becomes about 7.3%. The drawdown rule now passes and the profit target now fails, so the card still reads Not Feasible. Costs that do not scale with size would move the real figures further. A smaller size fixed the rule it was aimed at and exposed the next one, which is why every row needs reading after each change.

To change size, edit the order size in the script's strategy() declaration, for example default_qty_value, because Tradelyze runs the script's own defaults. Run the strategy again in TradingView so the exported trade list matches, then submit again. How to work out a size from a firm's limits is covered in position sizing for prop firm challenges. The daily and trailing rules are covered in daily loss limit and trailing drawdown.

Two further checks belong here. Tradelyze's firm presets are snapshots of each firm's rules, so compare every limit with the firm's current terms on its own site. And do not loosen a custom rule to make the card read Qualifies: the firm applies its real limit, whatever the card says. How to read the Rule Results table has the details.

When should you abandon a strategy instead of tuning it?

Abandon a strategy when the failures remain after the data problems are fixed and the evidence is as large as you can make it. No published rule says when to give up, so the signals below are practitioner judgment, not a standard:

• The match is fixed and walk-forward still fails after you added history or windows.
• The edge exists only at one exact setting. Parameter Sensitivity reads Unstable, and nudging an input one step in TradingView turns the result into a loss.
• All the history you can get still gives too few trades. The Permutation Test refuses to run below 20 trades, and Min Backtest Length reads Insufficient.
• No position size fits the firm. The only size small enough for the drawdown rules is too small to reach the profit target.
• You have already re-run many times, and the result moves with every change instead of settling.

Dropping a strategy that fails honestly is a result, not wasted work: you found out before paying a challenge fee or risking an account. A strategy that passes is ready for more checking, not for money. The next step is forward testing on new prices, covered in backtest vs live trading, and the whole sequence is in how to validate a trading strategy. Common product questions are collected in the Learn FAQ.

Why is re-optimizing until it passes a trap?

Re-optimizing until a check passes is a trap because it turns the check into part of the search, so the eventual pass is partly luck. A validation check is only useful if the strategy could fail it. If you change settings or ranges after each failure and stop at the first pass, you have searched until the test agreed with you. Statisticians call this the multiple testing problem: the more tests you run, the more likely one passes by chance.

The arithmetic is simple. Suppose each run of a check had a 5% chance of passing a strategy with no real edge, and the runs were independent of each other:

Re-runs on the same data are not independent, so the real figure is different. But the chance of at least one lucky pass can only grow as you add runs, never shrink. The 5% is an assumption chosen for the arithmetic, not a measured property of any Tradelyze check.

Research on backtests points the same way. Bailey, Borwein, López de Prado and Zhu, in Notices of the American Mathematical Society (May 2014), argue that an impressive backtest is easy to produce by trying enough configurations. They add that the more configurations are tried, the more likely the backtest is overfit. Harvey, Liu and Zhu, in the Review of Financial Studies (2016), make a related point. So many strategies have already been tested, they argue, that a new finding should clear a t-statistic of about 3.0 rather than the conventional 2.0. A t-statistic measures how far a result stands out from noise.

The deflated Sharpe ratio, from Bailey and López de Prado (2014), lowers a Sharpe ratio to allow for how many settings were tried before it was picked. Tradelyze's DSR Value and its permutation test both correct for the distinct settings tried within the current optimization run. Neither can see runs you discarded, strategies you abandoned or ranges you changed between runs. A pass found on the sixth attempt is graded as if it were the first; see the deflated Sharpe ratio.

A constructed example, not measured data: run 1 reads FAIL with WF Efficiency 0.31. You narrow two ranges around the settings that did best in the test windows. Run 2 reads FAIL at 0.44, runs 3 to 5 also fail, and run 6 reads PASS at 0.58. Run 6's report shows only its own trials. The five earlier runs, and everything you learned from their test windows, are invisible to every check on it.

The defense is a written record. Before you start, decide how many runs you will allow and what single change each run tests, and keep the run count next to the result you trade. Why the number of attempts matters more than the number of inputs goes further.

Is it wrong to widen a parameter range and run again?

Widening a parameter range once is not wrong when the Best Value sat at the edge of its Range and the wider values are ones you would really trade. A value at the edge suggests the search wanted to go further, a reading with no primary source. It is still another attempt. Set the new range before looking at any result beyond the old edge, run once, and accept the answer. How to read the Parameter Search Space table covers the columns.

Frequently asked questions about a strategy that fails validation

Sources

• David H. Bailey, Jonathan M. Borwein, Marcos López de Prado and Qiji Jim Zhu, Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance, Notices of the American Mathematical Society 61(5), May 2014, pp. 458–471, DOI 10.1090/noti1105. The statement that more configurations tried raise the probability of an overfit backtest is worded most directly in the paper's abstract, which the printed journal version does not carry; the article develops the argument.
• David H. Bailey and Marcos López de Prado, The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-Normality, Journal of Portfolio Management 40(5), 2014, SSRN 2460551.
• Campbell R. Harvey, Yan Liu and Heqing Zhu, … and the Cross-Section of Expected Returns, Review of Financial Studies 29(1), 2016, pp. 5–68 — the argument for a t-statistic hurdle of about 3.0 rather than 2.0.
• Tradelyze implementation, reviewed 15 September 2026: the Optimization stopped card with Continue anyway and Review comparison, and the code note that one faithful re-run can measure 100% against one timezone and about 16% against another; the five-minute entry window for matching trades; the walk-forward PASS, FAIL and NO VERDICT badge, its four pass conditions and the WF Efficiency labels; 2 walk-forward windows by default, adjustable from 2 to 6 when the settings are enabled for an account; Optimization trials of Automatic or 20 to 300 in steps of 10, with the automatic budget never below 60; the automatic Parameter Sensitivity budget of zero below 50 optimization trials; the 20-trade minimum of the permutation test and the 5-trade minimum of the deflated Sharpe ratio; the 20% Ruin Probability limit; the robustness verdict bands; the Qualifies and Not Feasible badges, the rules checked and not checked, and the Rule Results table, with each firm's verdict graded on the selected trial's own trades over the full optimization history; the card headed Optimization stopped — backtesting service unavailable, whose Resubmit opens the Run Optimization dialog, uses no further credit and reuses finished backtests cached for the same UTC day, and the backend note that retrying an aborted or baseline-halted run is free; the Processing is taking longer than expected banner, shown with Cancel & Delete once 5 hours pass without a result; the 1-credit charge for a new submission and for Resubmit Strategy, and the absence of any refund when a strategy is deleted; and the correction of the permutation test and deflated Sharpe ratio for distinct settings tried within one optimization run only.
• Arithmetic used on this page, requiring no source: 1 − 0.95¹⁰ ≈ 0.401, which assumes independent runs and an illustrative 5% false-pass chance; 0.81 ÷ 1.30 ≈ 0.62; 6.8% × 2/3 ≈ 4.5%; 11% × 2/3 ≈ 7.3%. Every worked example on this page is a constructed illustration, not measured data.
• The order in which to work through failures, the next steps in the results table, which verdict limits the decision when two disagree, and the signals for abandoning a strategy are practitioner judgment with no primary source.