It is a complex process from developing an excellent system of programmatic trading to obtaining a certain profit from using it, and there are many problems in the process. For example, there are often investors who are very confident in the profitability of the strategy before actually using the trading strategy, because the historical test yield curve of the strategy is smoothed upwards. After the real world, the capital curve is curved downward, unsatisfactory.
The design process of a programmatic trading system consists of two parts, both of which can lead to over-configuration. The first part of the design of the trading system is to form a complete system of trading rules. The formation of trading rules generally has two approaches: a top-down approach to summarize rules based on long-term observation of market trends, and then to form quantitative trading strategies based on the rules, which requires a long-term accumulation of trading experience.
The goal of designing a trading system is to make a profit in the real world of the future rather than to pursue a beautiful historical test curve. Over-fitting a trading system is a beautiful trap. How do we escape this trap? We think that we can start from the formation of trading rules and the development of trading systems in two main ways. Modern mathematics of financial market data analysis shows that the time price sequence consists of two parts: the first part is a definite item from which more and more certain laws can be found; the second part is a random item, the rules of uncertainty are predictable, the occurrence of a phenomenon is only a probability.
First, increase the sample capacity of historical test data to avoid too many trades. If the historical test data is small, while the designed system performs well in the sample, testing for shorter periods of time is unconvincing and the future performance of the system is difficult to predict.
Second, when testing, the sample of data to be tested is divided into in-sample and out-sample, and when designing the system, the in-sample data is used, and then the resulting system is tested with out-sample data. If the effect is greatly reduced, then such a system is most likely to be suitable.
Thirdly, a system with too many core parameters is a system with many degrees of freedom, which always produces a beautiful system after optimizing several parameters, but the reliability of such a system is questionable.
Fourth, when optimizing the parameters of the system, we need to look at parameters near the optimal parameters. If the performance of the nearest parameters system is significantly different from the performance of the optimal parameters, then this optimal parameter may be the result of an over-simplification, mathematically known as singularity solving, which is unstable. If the characteristics of the market change slightly, the optimal parameter may become the worst parameter.
Fifth, apply the trading system to other varieties and observe its effectiveness. Versatile trading systems are rare, but systems that perform well in one variety can at least be profitable in another. If it is not profitable in another variety, attention should be paid to its effectiveness in the process of using the system, i.e. whether it is too well suited to the particular market of a particular variety.