Changing a Trading Strategy Without Destroying the Experiment

A strategy cannot be evaluated if it changes inside every result

A trader modifies an entry rule after a loss, widens an exit after a near miss and removes a filter after watching a move they did not take. At month end, they ask whether the strategy works. The record contains several different procedures, but the trader is trying to interpret them as one coherent test.

The central problem is not that a strategy must never change. Conditions, knowledge and objectives can change. The problem is losing track of what was changed and why. If each uncomfortable result quietly creates a new rule, the review cannot distinguish the original idea, the revised idea and the trader’s response to the last emotional event.

Describe the hypothesis before changing the rule

Write what the proposed change is intended to improve. A hypothesis should identify an observable issue rather than a general desire for more wins. Perhaps a rule produces entries outside the intended session, or a recorded cost was omitted from evaluation. The specific concern determines what information the review needs.

Avoid explaining every unwanted result as an obvious flaw after the outcome is known. Some adverse results are compatible with the original plan. Others may reveal an error, a missing assumption or an unsuitable method. Keeping those categories distinct prevents the result itself from becoming the only evidence that the rule should have been different.

Give each version an identity

Record the rule set, the date it began and the reason for the revision. Keep earlier records intact. If historical trades are recalculated under a new rule, label that exercise separately from what was actually done at the time. A revised chart is not a revised history of execution.

Versioning need not be elaborate. A short document can specify the instrument, permitted conditions, entry and exit rules, risk constraints and known limitations. The benefit is interpretability: another review can see which procedure produced which observations. Without that, even a detailed spreadsheet may be a precise record of an undefined activity.

Separate simulated behaviour from actual execution

A historical test can explore a question, but it does not establish what would have happened in real execution. The CFTC warns about limitations of hypothetical trading results, including assumptions about execution and costs. Treat that distinction as part of the method rather than a footnote added after an impressive graph has already created confidence.

Record whether observations come from historical simulation, forward observation, a demonstration environment or actual trading. Do not combine them into a single performance claim. Differences in spreads, slippage, fees, liquidity and behaviour can matter, and their relevance depends on the instrument and trading arrangement. A clean category is more useful than a large total assembled from incompatible evidence.

Change fewer variables when the question permits it

If entries, exits, session, instrument and position sizing all change together, it becomes harder to identify what explains a difference in the record. Sometimes a broad redesign is necessary. Call it a redesign rather than pretending that the effect of one small adjustment has been isolated.

Where possible, define the proposed change and the evaluation criteria before looking at new observations. This does not eliminate uncertainty or make a small sample conclusive. It reduces the opportunity to keep altering the question until the existing data appears to support the answer the trader wanted from the beginning.

Include costs and constraints in the question

A method may look different after realistic costs and execution assumptions are considered. The CFTC’s advisory also highlights that fees and other costs can affect apparent results. Specify what is included and what is missing. Do not describe a test as comprehensive if an important part of the trading arrangement remains unmodelled.

Capacity matters too. A procedure that requires continuous attention may be unsuitable for someone whose work or family commitments prevent it. That is not necessarily a statistical failure of the idea; it is a mismatch between the method and the person attempting to carry it out. The review should make that mismatch visible rather than recording every missed condition as a character flaw.

Decide what would justify another review

Define a review point and the questions it will answer. Do not infer certainty from an arbitrary number of observations or present a threshold as universally sufficient. Different methods and data conditions require different analysis. Where technical or financial decisions exceed your competence, obtain suitable professional input.

Keep the conclusion proportionate. The evidence may justify further investigation, a correction to the record or rejection of a specific assumption. It may not justify a claim that a system is proven. A useful review can end with uncertainty stated clearly, rather than a dramatic conclusion invented to make the time spent testing feel worthwhile.

Preserve the evidence while revising the idea

Before the next change, save the current rules and state the reason for revising them. Keep a distinction between correcting an error, exploring a hypothesis and redesigning the whole method. These are different decisions and should leave different traces in the record.

A trading review becomes stronger when it remains honest about what it cannot establish. Versioning does not remove market risk or promise an edge. It prevents avoidable confusion from being mistaken for insight and gives the trader a clearer account of the experiment they are actually conducting.