Warning: Cannot modify header information - headers already sent by (output started at /data/web/virtuals/188780/virtual/www/index.php:1) in /data/web/virtuals/188780/virtual/www/wp-includes/pluggable.php on line 1219
23. Continuous Walk Forward Analysis - Sophisticated Tool for Robustness Testing | Algotradingacademy.com

GET YOUR FREE WEBINAR NOW

CLICK HERE
23. Continuous Walk Forward Analysis – Sophisticated Tool for Robustness Testing

23. Continuous Walk Forward Analysis – Sophisticated Tool for Robustness Testing

In the previous chapter we explained what the Walk Forward Analysis (WFA) is. Now we are going to add a much more complex analysis – the Continuous Walk Forward Analysis (CWFA). CWFA is a very sophisticated tool that helps us determine whether the automated trading system (ATS) is really robust, i.e. able to generate profits in live trading. CWFA contains a set of many WFAs with different Out-of-Sample periods expressed as percentages and as different numbers of runs. In Fig. 1 you can see a classic example of WFA performed in MultiCharts with default settings. This CWFA includes 30 WFAs with different Out-of-Sample periods (rows) – specifically 10%, 15%, 20%, 25%, and 30% – and with different numbers of runs – specifically 5, 10, 15, 20, 25, and 30 (columns). In Fig. 1 you can see that I chose an example in which all 30 WFAs have successfully passed the analysis. In other words, the system met the test criteria (which we will explain later within our series on CWFA). In this case, the overall results for the given WFA include a box saying “PASS”. Yet in practice we will rather meet with the variant that most of the 30 WFAs will not meet some of the test criteria and in such case we will see a box saying “FAILED” in the WFA´s results – see fig. 2. This is because CWFA is a very demanding robustness test and only few ATSs are robust enough to meet the test criteria. In Fig. 2 you can see a CWFA the results of which clearly show that the ATS is not sufficiently robust for the selected market and timeframe. In contrast, Fig. 1 shows a perfect example of an ATS that is potentially very robust for the selected market and timeframe. It must be said that this is a fairly rare phenomenon. Searching for a truly robust ATOS by CWFA can often be likened to searching for a needle in a haystack (if it was easy then the algorithmic trading could be performed by anyone).

What is the threshold for a robust ATS according to CWFA results? I’m leaving that as an open topic which I will address on our website  in the near future.

Cluster Walk Forward Analysis comprising 30 Walk Forward Analyses all of which passed the demanding test criteria (PASS)

Fig. 1: Continuous Walk Forward Analysis comprising 30 Walk Forward Analyses all of which passed the demanding test criteria (PASS)

Cluster Walk Forward Analysis comprising 30 Walk Forward Analyses 27 of which did not pass the demanding test criteria (FAILED)

Fig. 2: Continuous Walk Forward Analysis comprising 30 Walk Forward Analyses 27 of which did not pass the demanding test criteria (FAILED)

Since we have introduced many new terms, we should graphically show (see Fig. 3) the individual links between Walk Forward tests, Walk Forward analysis, and Continuous Walk Forward analysis. You can see that CWFA includes many Walk Forward tests that are the basis of WFA.

The link between Walk Forward Tests, Walk Forward Analysis and Cluster Walk Forward Analysis

Fig. 3: The link between Walk Forward Tests, Walk Forward Analysis and Continuous Walk Forward Analysis

While one WFA can provide a preliminary and a very limited indication of an ATS´s robustness (one successful WFA may be a coincidence), CWFA is able to prove or disprove (in great detail) the ATS´s validity and robustness with a much higher degree of certainty (30 successful WFAs are not a coincidence but a valid statistical sample). CWFA, which is available in the MultiCharts platform, is unique in its ability to execute a set of WFAs. It helps us to avoid the “Curve-Fitting” of results of the input parameter optimisation. Curve-Fitting is a phenomenon of overoptimisation and overvaluation of results. The problem is that after trying many different combinations we find a combination of input parameter values ​​(Inputs) which captures the past trends very well and do not execute some crucial losing trades which would be executed with another input values. This combination of parameter values generates wonderful and unique results (such as very high net profit with the minimum possible maximum drawdown). Unfortunately, such results are often totally accidental. This means that in live trading the system will most likely have very different results (often literally catastrophic). In order to eliminate Curve Fitting as much as possible we use CWFA which helps us to confirm that the good performance results for a selected In-Sample input parameter setting are not a coincidence. The ATS that succeeds in CWFA according to predetermined test criteria will most likely withstand the constantly changing market conditions thanks to its natural qualities and stability. In Fig. 4 you can see the individual test criteria that we use in CWFA in MultiCharts.

Cluster Walk Forward Analysis Test Criteria in TradeStation

Fig. 4: Continuous Walk Forward Analysis Test Criteria in MultiCharts

We have explained what CWFA is and showed examples of ATS that passed (PASS) and failed (FAILED) the demanding test criteria of individual WFAs. We have also explained the connection of Walk Forward tests, WFA, and CWFA. In the next chapter we will focus on the individual test criteria that must be met in order for the ATS to pass the individual WFA robustness tests within CWFA.

Petr Tmej

(c) Algotradingacademy.com

Previous: 22. Walk-Forward Analysis – Tool for Robustness Testing

Next: 24. Testing Criteria of Continuous Walk Forward Analysis 

Author: Renata Tmejová

http://www.algotradingacademy.com

The trader, the co-founder of QuantOn Solutions hedge fund, the lecturer in one person who has been successfully trading US futures via algorithmic trading systems (ATS) for many years. He is the main “brain” of the team Algotradingacademy.com. Petr´s mission is to provide relevant and necessary knowledge and skills in ATS trading to his clients so that they can become successful traders too. In 2009, Peter graduated from the Technical University of Ostrava, Department of Quality Control. The studies were mainly focused on Probability Theory and Statistical Processes. If you would like to get know more about his background and how the trading influenced his life you can find his story here: https://aostrading.cz/en/petr-tmej/. Peter honestly describes all his trading journey there. Furthermore, he reveals also his past and present trading results. His story can be quite inspiring for you since he began trading with very low initial investment. Nowadays, he can enjoy the success he has already achieved in live trading with his hedge fund QuantOn Solutions.

  • What Do You Need to Know About Maximum Drawdown?

    13.05.2018

    This performance indicator has for investors about the same importance as Total […]

    Read more
  • 29. Diversification as a Basic Assumption of Winning Traders?

    29.04.2018

    If there in the world of trading and investing exists something like […]

    Read more
  • 24. Testing Criteria of Continuous Walk Forward Analysis – MultiCharts

    24.04.2018

    This chapter´s content follows the previous chapter in which we explained what […]

    Read more

CONTACT