Backtesting Strategies, Platform Selection & Programming
Backtesting Strategies:
Even if we manage to avoid all the common pitfalls outlined earlier and there are enough trades to ensure statistical significance of the backtest, the predictive power of any backtest rests on the central assumption that the statistical properties of the price series are unchanging, so that the trading rules that were profitable in the past will be profitable in the future. This assumption is, of course, invalidated often in varying degrees: A country’s economic prospect changes, a company’s management changes, and a financial market’s structure changes. In the past decade in the United States, we have witnessed numerous instances of the last category of changes.
Decimalization of U.S. stock quotes on April 9, 2001. (Prior to this date, U.S. stocks were quoted in one-eighth or one-sixteenth of a penny.) This caused bid-ask spreads to decrease, but also caused the “displayed liquidity” at the best bid and ask prices to decrease (Arnuk and Saluzzi, 2012). This in turn caused profitability of many statistical arbitrage strategies to decrease while increasing the profitability of many high-frequency strategies.
The 2008 financial crisis that induced a subsequent 50 percent collapse of average daily trading volumes (Durden, 2012). Retail trading and ownership of common stock is particularly reduced. This has led to decreasing average volatility of the markets, but with increasing frequency of sudden outbursts such as that which occurred during the flash crash in May 2010 and the U.S. federal debt credit rating downgrade in August 2011. The overall effect has been a general decrease in profits for mean-reverting strategies, which thrive on a high but constant level of volatility.
- The same 2008 financial crisis, which also initiated a multiyear bear market in momentum strategies.
- The SEC’s Regulation NMS implemented in July 2007, which also contributed to the drastic decrease in the average trade sizes and the obsolescence of the NYSE block trade (Arnuk and Saluzzi, 2012).
- The removal of the old uptick rule for short sales in June 2007 and the reinstatement of the new Alternative Uptick Rule in 2010.
Strategies that performed superbly prior to each of these “regime shifts” may stop performing and vice versa. Backtests done using data prior to such regime shifts may be quite worthless, while backtests done using recent data may be no more indicative of future profits if and when a future regime shift is to occur. The general point of this is that algorithmic trading is not just about algorithms, programming, and mathematics: An awareness of such fundamental market and economic issues is also needed to inform us on whether a backtest is predictive and will continue to be predictive.
Platform Selection:
Software companies have worked very hard to provide traders with a wide variety of backtesting and automated execution platforms that cater to every possible level of programming skills. We are faced with two basic choices when it comes to deciding on a trading platform:
- Buying a special-purpose backtesting and execution platform, and implementing your strategy using that platform’s special-purpose graphical user interface (GUI) or programming language.
- Writing your own backtest and execution program in a generic programming language such as C++, either in a completely standalone manner with piecemeal purchases of software libraries to make the task easier or within an integrated development environment (IDE) that comes with a comprehensive library catering to algorithmic trading.
Programming Skill:
The only option if you lack programming skills is to use a special-purpose trading platform. These systems provide a graphical “drag and drop” user interface for creating a trading strategy, relieving the user of the need to learn a programming language. Deltix and Progress Apama are two examples of these products. The range of methods you can develop using these GUIs, in my experience, can be very limited. In the long run, it is far more effective to learn a programming language so that you can describe your approach.
Traders possessing the next level of programming skill should consider implementing both backtesting and automated execution using one of the scripting languages. These languages do not require compilation, and you can instantly see the results the moment you finish typing in the mathematical or logical expressions. Many traders’ favorite backtesting platform, Microsoft Excel, perhaps used in conjunction with Visual Basic (VB) macros, belongs to this category. But it is actually quite hard to build a reasonably complicated strategy in Excel, and even harder to debug it. Excel also is not a particularly high-performance language, so if your strategy is very computationally intensive, it is not going to work. If you use Excel for automated executions, you may find that you have to use DDE links provided by your brokerage for market data updates, and you will likely need to add Visual.
Basic macros to handle more complicated trading logic, which is quite inefficient. (However, see Following for an Excel like trading platform that is supercharged for efficiency.)
The FXone Automated Execution Platform is Excel on steroids.
An Excel-like currency trading tool called FXone uses C++ as its high-performance programming language rather than Visual Basic macros for its computational engine. Every tick (a tick in the FX example is a new quote) causes all the values in all of the spreadsheet’s cells to be recalculated, making it a real tick-driven application.Additionally, it contains an internal cache for real-time data, which eliminates the need for duplicate subscriptions of the same data by allowing cells that need the same data to compute to simply retrieve it from the cache. Additionally, it is a platform that is truly multithreaded on two levels. Initially, concurrent order submission and market data updates are possible for various strategies defined on separate Excel workbooks. Second, many cells in a single worksheet can receive updates and respond to new information at concurrent times. It means that other cells can still react to a new tick by, for example, placing an order, even if the calculation in one cell takes a very long time. Figure 1 displays an FXone screenshot.
Backtesting and Automated Execution
Many special-purpose trading platforms, including QuantHouse and RTD Tango and the aforementioned Deltix and Progress Apama, also include ways for coding a strategy with their own proprietary programming languages, which are usually quite simple and easy to learn, maybe as easy as Visual Basic. Aside from the institutional platforms mentioned here, many retail traders are familiar with MetaTrader, NinjaTrader, Trading Blox, or TradeStation Easy Language. I have not tried all of these platforms personally, but I have a lingering suspicion that despite the apparent ease of use and other advantages I mention later, they all in some way place some limitations on the type of strategies that can be backtested and executed.
Requiring just slightly more skills than programming in VB, traders will find the scripting languages of MATLAB, R, and Python off er vastly more ease of debugging, much greater flexibility in the type of strategies that can be backtested, and higher efficiency in backtesting large data sets. These are what we call “REPL” languages. REPL is programmer-speak for “Read-Eval Print-Loop.” That is, you can type in a mathematical expression, and the program will immediately evaluate it and print out the answer, and get ready for you to input the next expression. It works exactly like a handheld calculator, but better: You can also save all these expressions in a file, and have the program automatically execute them sequentially. The syntax of these languages is designed to be more intuitive and easier to understand than conventional programming languages such as C++ and much more flexible in terms of the type of variables that can be used in a program. Scalars, arrays, and strings are all basically dealt with using a similar syntax and passed along to functions in the same way.
MATLAB can also utilize Java, C++, or C# libraries or application programming interfaces (APIs) and call functions implemented in those libraries or APIs. This allows MATLAB to take advantage of the more efficient implementations in those conventional languages when a task is particularly computationally intensive. Also, there are far more libraries and APIs that are written in those conventional languages than those written in MATLAB, R, or Python, so this feature is often essential.
Many algorithmic traders are aware that MATLAB, R, and Python are excellent languages for backtesting. But less well known is the fact that they can be turned into execution platforms as well with the addition of some toolboxes. Most brokerages have APIs written in Java, C++, or C#; and, as I said earlier, MATLAB can call functions in APIs written in such languages, though it does take some familiarity with these languages to know how to call these functions. If you would prefer a solution that obviates making “foreign-language” API calls in MATLAB, there are a number of commercial products available. MATLAB’s own Data feed Toolbox can send orders to Trading Technologies’ X_TRADER. To connect MATLAB to Interactive Brokers, undocumentedmatlab.com has developed an API called IB-MATLAB. Another vendor, www.exchangeapi.com, has a similar API called quant2ib, as well as one called quant2tt for connecting MATLAB to Trading Technologies. For other brokerages, www. pracplay.com offers a bridge from MATLAB or R to 15 or more brokers for a monthly fee. A free, open-source MATLAB API for connecting to Interactive Brokers was developed by Jev Kuznetsov and is available for download from MATLAB Central’s File Exchange. Meanwhile, the MATFIX software from agoratron.com lets your MATLAB program send orders using the Financial Information exchange (FIX) protocol to brokers or exchanges. You can also use MATLAB to call the Java or .NET functions in QuickFIX, an open source FIX engine (Kozola, 2012). For Python users, the free, open-source software IbPy will connect your Python trading program to Interactive Brokers. While these add-ons to MATLAB and Python make it possible to connect to a broker, they nevertheless do not shield you from all the complexity of such connections. And, more important, it is cumbersome to use the same program for both backtesting and execution.
If you are a hard-core programmer, you will, of course, have no problem backtesting and automating execution directly in the most flexible, most efficient, and most robust of programming languages, such as aforementioned trio of Java, C++, or C#. As I said earlier, all brokerages or exchanges that cater to algorithmic traders provide APIs in one or more of these languages, or they allow you to submit orders using the FIX messages, which in turn can be created and transmitted using a program written in one of these languages. (For example, QuickFIX, mentioned previously, is available in C++, C#, VB, Python, and Ruby.) But even here the software industry has come to make our strategy implementation easier and more robust by providing IDEs designed just for backtesting. In fact, many of the special-purpose trading platforms (Deltix, Progress Apama, QuantHouse, RTD Tango, etc.) include ways for coding strategies using general-purpose, advanced programming languages that make them resemble IDEs. There are also free, open source class libraries or IDEs that I describe in the next blog.
Read Also: When You Should Avoid Backtesting a Trading Strategy
