Warning: Cannot modify header information - headers already sent by (output started at /data/web/virtuals/188780/virtual/www/index.php:1) in /data/web/virtuals/188780/virtual/www/wp-includes/pluggable.php on line 1219
22. Walk-Forward Analysis - Tool for Robustness Testing | Algotradingacademy.com

GET YOUR FREE WEBINAR NOW

CLICK HERE
22. Walk-Forward Analysis – Tool for Robustness Testing

22. Walk-Forward Analysis – Tool for Robustness Testing

MultiCharts Walk Forward Optimizer (WFO) is an advanced tool for optimising automated trading systems (ATS) which automates the very complex multilevel process of statistical Walk Forward testing of ATS input parameters (Inputs). Where the possibilities of most classical optimisation tests end WFO enables performance of set of walk-forward analyses on optimised In-Sample historical data against the unknown and non-optimized Out-of-Sample historical data. The purpose of these tests is to simulate the unpredictability of live trading. The purpose of this simulation is to determine whether the ATS is likely to be profitable in live trading or not.  We discussed a lot the principle of optimisation of input parameters and Out-of-Sample testing in the last and last but one chapters so we will not return to these issues.

We will concentrate solely on WFO which is a universal tool for ATS robustness testing using the MultiCharts backtesting engine, EasyLanguage, and the advanced backtesting performance reports. In order to help you to fully understand this tool, we must first explain what Walk Forward testing means (Walk Forward tests are the backbone of the Walk Forward Analysis – WFA). To illustrate this concept we drew up the following scheme in which you can see a WFA containing a series of In-Sample and Out-of-Sample Sample historical data with individual Walk Forward tests (8 tests in total).

Walk Forward Analysis

Fig. 1: Walk Forward Analysis

In Fig. 1 you can see an example of WFA for 12 months (each trader can set the historical period by himself, I often use historical data older than 10 years). This WFA includes 8 In-Sample parts (blue fields) and 8 Out-of-Sample parts (green fields). Thus from the fifth month we actually simulate live trading conditions on unknown data.

Specifically how do we proceed in the analysis?

The basic principle is very simple: We perform optimisation tests for all In-Sample runs and after identification of input parameter settings with the highest In-Sample Fitness Function (FF) we apply these settings to the Out-of-Sample data. If you are not acquainted enough with the issue of FF and optimisation testing, please read again the previous chapters which I have already referred to at the beginning.

It is always good to repeat everything on a simple example:

Imagine that you trade via an ATS with two input parameters (Inputs) for a longer and a shorter moving average. For the longer moving average we use for example the range from 50 to 100 with increments of 10 (i.e. 6 combinations) and for the shorter moving period 5-45 also with increments of 10 (i.e. 5 combinations). In total we get: 5 x 6 = 30 possible combinations of the two input parameters. Now imagine that our FF will be the highest Net Profit. In the first In-Sample run, i.e. the first to the fourth month, we found out that the highest net profit achieved the combination 5 for the shorter moving average and 60 for the longer moving average. We will therefore apply the parameter settings 5 and 60 to the first Out-of-Sample testing, i.e. the fifth month.

In the second In-Sample run, i.e. the second to the fifth month, we found out that the highest net profit achieved the combination 15 and 80. We will therefore apply the parameter settings 15 and 80 to the second Out-of-Sample testing, i.e. the sixth month.

We apply the same principle to the next runs (Walk Forward), up to the In-Sample run 8 with application on the last Out-of-Sample data, i.e. the 12th month.

This principle is called WFA “Rolling”. The point is that we divide the historical data into various parts thanks to which we obtain more Out-of-Sample data for evaluating of the ATS´s robustness. Then we evaluate the robustness potential by the predetermined test criteria. We will introduce these test criteria in some of the next chapters. To start off, we just say that the fundamental test criterion for us should be that the Out-of-Sample results were profitable enough and at least half as profitable as the best In-Sample optimised parameters.

All the above findings show that the WFA analysis is the most realistic simulation of the ATS behaviour in live trading. WFA helps us to answer these basic questions:

  • Will the ATS be profitable even after optimisation (the aim of which is to find the most suitable input parameters for live trading)?
  • What performance characteristics should the ATS have in order to have the potential to be profitable on unknown data in live trading?
  • What impact on the ATS´s performance will have a change in trend, volatility, or liquidity in the future?
  • How often should we re-optimise the system´s input parameters (Inputs)?

A much more advanced tool that goes beyond the possibilities of instruments offered by most software trading platforms and which is also included in the MultiCharts platform is the Continuous Walk Forward Analysis. It is a set of many Walk Forward analyses the principle of which we will explain in the next chapter.

Petr Tmej

(c) Algotradingacademy.com

Previous: 21. MultiCharts Helps You with Optimisation

Next: 23. Continous Walk Forward Analysis – Sophisticated Tool for Robustness Testing

Author: Renata Tmejová

http://www.algotradingacademy.com

The trader, the co-founder of QuantOn Solutions hedge fund, the lecturer in one person who has been successfully trading US futures via algorithmic trading systems (ATS) for many years. He is the main “brain” of the team Algotradingacademy.com. Petr´s mission is to provide relevant and necessary knowledge and skills in ATS trading to his clients so that they can become successful traders too. In 2009, Peter graduated from the Technical University of Ostrava, Department of Quality Control. The studies were mainly focused on Probability Theory and Statistical Processes. If you would like to get know more about his background and how the trading influenced his life you can find his story here: https://aostrading.cz/en/petr-tmej/. Peter honestly describes all his trading journey there. Furthermore, he reveals also his past and present trading results. His story can be quite inspiring for you since he began trading with very low initial investment. Nowadays, he can enjoy the success he has already achieved in live trading with his hedge fund QuantOn Solutions.

  • What Do You Need to Know About Maximum Drawdown?

    13.05.2018

    This performance indicator has for investors about the same importance as Total […]

    Read more
  • 29. Diversification as a Basic Assumption of Winning Traders?

    29.04.2018

    If there in the world of trading and investing exists something like […]

    Read more
  • 24. Testing Criteria of Continuous Walk Forward Analysis – MultiCharts

    24.04.2018

    This chapter´s content follows the previous chapter in which we explained what […]

    Read more

CONTACT