In
part one of this article, we looked at key elements of the backtesting
process and discussed some of the differences between “in-sample” and
“out-of-sample” data. We also looked at some of the complexities and
procedures involved when optimizing strategies so that traders can use
these methods to settle on an approach to technical analysis that is
likely to result in long-term profitability.
It is important to read the first part of the article, as there are
some basic steps in the process that must be first understood before
moving on to the final steps. In the next sections, we will look at
correlation as a critical component of the total process.
Defining Correlation
In this context, correlation refers to the performance similarities
that can be found when comparing a technical analysis strategy that is
applied to more than one data set. The results from correlation
measurements will allow traders to spot broad trends and evaluate the
performance of a given strategy over your chosen test period. When
strong correlations can be found between results generated from all data
sets (in-sample data and out-of-sample data), there is an enhanced
probability the system will generate positive results when used in the
following steps (forward testing and in live trading).
There
will be cases where a system is found to be curve-fit to perform
strongly with one of your data sets but these situations fail to meet
the requirement threshold to make it to the next phase of the testing
process. When these situations are seen, it is likely that the system is
over-optimized and much less likely to generate long-term gains when
applied to live trading conditions. High levels of correlation mean that
the system is ready for the next step, which requires further
out-of-sample data testing. This data will form the basis for the next
part of the process, which is forward testing.
System Rules when Forward Testing
Forward testing might sound overly complicated for traders new to the
process. But forward testing is simple paper trading (trades are placed
with a demo account using “virtual” money). With an additional set of
out-of-sample data, forward testing gives traders new information with
which to test a strategy that has already performed well (and shown
strong correlation) in historical back tests. Forward testing simulates
the live trading environment and gives us a better idea of how a
strategy will work when real trades are placed.
This is an
important step of the process because there has been no optimization in
the same way there was with historical price data. Optimization at this
phase of the process would be impossible because there is no way to know
where prices will travel next. This is also why it is important to
place these trades with a demo account first, before moving on to live
trading with real funds. The results (profits and losses) of each
forward testing trade will be recorded, so it is vital that the rules of
your system are rigorously followed. Any changes made (with respect to
the rules of the system) will cloud the data and make it difficult to
accurately assess the viability of the system in achieving consistent
profitability.
Making Valid Assessments
When forward
testing, it might become tempting to cherry-pick some trades that might
look especially attractive and neglect others. There might be instances
where your system send a trading signal but you look at the situation
and avoid the trade because you believe it is unlikely you would
actually take the position with a real account. But what is important to
remember here is that you are not testing your skills as a trader, you
are testing the viability of your proposed system. If the rules of your
system send a signal to execute a trade, that is the trade that should
be placed. This is the only way to properly evaluate your strategy.
So, while the term “forward testing” might seem as though we are
attempting to predict the future, the fact is that we are really only
backtesting with live data, rather than historical price data. Because
of this, the same standard apply when we look to assess the performance
of a proposed system. We are looking for trading results (profits vs.
losses) to show consistent profitability through all phases of the
testing. This includes in-sample data, out-of-sample data, and forward
tested data (or live data). You will also need to see strong
correlations in all phases of your testing. When this is found, you will
know you have a trading system that holds up even in situations that
were not optimized. This means you have found a system that is likely to
perform well in active markets (with real funds).
Conclusion
Backtesting a strategy can give traders some highly valuable
information when looking for new ways to approach the markets. Most of
the latest trading platforms enable traders to conduct these tests, and
here we looked at some of the key elements required when we evaluate
regular trading programs. There are many critics of backtesting, as
people will often argue that historical price data does little to tell
us how markets will behave in the future. While there is some validity
to these arguments, we do have ways of dividing up historical data to
increase the probability that our assessments are accurate.
When we make our determinations using in-sample and out-of-sample data,
traders have an efficient (and relatively easy) way of evaluating the
potential success rate of a technical analysis strategy. These
strategies can also be optimized (in the hope of improving on potential
profit performance) but it is important to retest your approach on
multiple data sets in order to prevent your trading profits from
becoming a “self-fulfilling prophecy” that might not work under live
trading conditions.
As a final step, traders should implement
out-of-sample forward testing, as this puts your strategy up against a
new set of price activity and provides another layer of protection
against potential losses. Once these steps are completed, you are ready
to start actively trading with real money. So, while the general process
cannot guarantee your next trades will be profitable, it does increase
your chances of success when compared to systems that have received no
backtesting. Ideally, you will be looking for strong profit performance
and high levels of correlation between all data samples, as this creates
the best scenario for systems later used in live markets.
No comments:
Post a Comment