Implementing your strategy as code(custom formula) in Amibroker is not complicated but how could we verify that our strategy is actually working and profitable? The answer is you have to test your strategy against historical data. There are many providers which you can get KLSE end of day (EOD) historical data from. You should make sure the data is adjusted for bonus, split, consolidate and special large dividend.
The process of testing your strategy against historical data is called back testing. From the testing you could fine tune your code and parameters to improve the performance of the system. This is called optimization. However, over optimization will cause your system to only perform well on the data which you have tested against with. This is because the data is static and you optimize your code to fit the data. This issue is called curve fitting. Most probably in live trading, your system will not perform well as the result from your back testing. The solve this issue I will split my data to in sample data and out of sample data. The concept is simple, you do the back testing optimization against one set of data (in sample data) and then verify the performance of your system by testing against another set of data (out of sample data). You have done the right optimization if the performance of your system is consistent for in sample and out of sample data. The process of optimizing your system against in sample data and verify its result against out of sample data is also known as walk forward analysis. Amibroker support optimization and walk forward analysis out of the box.
Now I will use my strategy to explain about the back testing process above so that you could have better idea. Assuming you have implemented the volume spike strategy as I outlined in previous post and you have 20 years historical EOD data imported to Amibroker. Let us refer back to the setup of volume spike strategy, in the step to identify volume spike, we have parameter n as how many time the spike volume is greater than average volume. Assuming we set this parameter to 1.5 initially. Before back testing, we split our historical EOD as below:
- In sample - EOD from 1/1/1995 to 31/12/2004
- Out of sample - EOD from 1/1/2005 to 31/12/2014
Note that this is just a simplified example to explain the concept, in real back testing, we could split EOD to multiple sets of in sample and out of sample data.
Now, we can start testing our system against in sample data. Let us assume we only focus on annual return for now as performance indicator of your trading system (I will explain later about what aspects we should look at to determine the performance of a system in my next post, apparently, annual return should not be the only consideration). After back testing, we found that the annual return is 15% which we are not happy with. We should do another back test against out of sample data as a benchmark too, assuming that the annual return which we get is consistent which is also around 15%.
Now we will try to optimize this parameter by trying different value, let us say when we set this value to 5 and we notice that the back test result has been improved to 25% which meets our expectation. Then, we will need to verify our optimization result against out of sample data. For instance, the annual profit from back test against out of sample data is 16%. Obviously, the result is far behind our expectation, hence we haven't identified the optimum parameter. We should go back to optimized the parameter again and repeat the step of optimizing against in sample data and verifying against out of sample data. Assuming finally we have identified parameter n of value 4 which gives a us 26.5% return from in sample data and 25% return from out of sample data. From this result it appears that the annual return from out of sample date has been significantly improved from the benchmark. We could conclude that we have found a optimum value for parameter n which most likely will work well too in live trading.
The following is the summary process of back testing
No comments:
Post a Comment