Notice:
  The views and information expressed in this document reflect the opinions and experience of the author Robert C. Pelletier.  Neither CSI nor the author undertake or intend to provide tax advice or trading advice in any market or endorse any outside individual or firm.  All recommendations are provided for their informational value only.  Readers should consult competent financial advisors or outside counsel before making any software purchase or investment decision.  CSI does not stand behind or endorse the products of any outside firms.

Copyright (c) 1999 Commodity Systems Inc. (CSI).  All rights are reserved.

Holiday Schedule Beyond 2000i and Trading System Certification

    CSI will be closed for voice communication on Friday, December 24 for the Christmas holiday and Friday December 31 for the New Years holiday. U.S. exchanges will be closed, but data from those exchanges that remain open will be available according to the current Data Release Schedule. The CSI host computer will be accessible as usual throughout the holiday weekends.

 Beyond 2000i and Trading System Certification

    Bill Cruz has told me many times that he considers CSI to be a competitor to Omega Research.® For years, I did not agree with this and considered that we offered largely complementary services: CSI supplying the data to investors and Omega supplying software for a specific type of analysis. Omega's entrée into the data business through HistoryBank.com and the expanding scope of Unfair Advantage, including the Trading System Performance EvaluatorTM (TSPE) may have made him a prophet -- of sorts.

Database Non-Competition

    Although Omega may consider themselves competitors to CSI in the database business, they are very far from our equals. Acquiring, cleaning and supplying data is our lifeblood and our passion at CSI. For very nearly three decades, we have honed this business into an art and a science and can confidently say that our staff includes several experts in the very specific niche of investment database management. Scores of major corporations and thousands of individual investors around the world buy our data to power their analysis because they know it is well worth the modest expense. Does anyone believe that Omega can compete with our service by giving away the free Historybank.com database with their software?

    For those who don't know, HistoryBank.com is Omega's private label data repository. In a recent independent study by Futures Magazine, HistoryBank.com was shown to have an error and omission rate equivalent to one full month of defects (21.83) per year of data per symbol. The same independent study also showed that by substituting CSI data, simulated profits were 50% higher than those derived from HistoryBank.com. The result could have been the opposite, but truth and realism are key. A flawed database is a two-edged sword: it might influence you to jump into an unwise trade, or it might sideline you just when the markets present outstanding opportunities. Worse yet, it leaves you with an uneasiness about all your trading decisions. Does anyone think such a handicap is worth the savings?

Differing Views on Analysis Software

    Although Unfair Advantage includes a growing list of potent analytical studies that give TradeStation® a run for its money, we offer a unique program that is designed to complement the market analysis done by almost any third-party program. Our Trading System Performance Evaluator (TSPE) does not generate trading signals or suggest market posture. Instead, it certifies the results derived from your trading system and helps you understand your prospects for sustained success. 

    By analyzing the profit and loss record (P&L) of any simulation exercise, TSPE helps investors avoid ill-conceived trading systems that have little hope of producing regular profits. TSPE is a Monte Carlo simulation, which, through the assumption of trade independence, computes the capital requirement necessary to achieve a predetermined goal at a specified level of confidence. 

Is there a Winning System Out There?

    In preparation for this article, we analyzed several of the TradeStation studies with TSPE, searching long and hard to find one that showed a positive outlook. We finally gave up and used another program to illustrate the TSPE results. For this example, it should suffice to say that we analyzed a profitable two-parameter trading system that experienced only nine trades. As we'll explain below, the limited number of trades coupled with the two controlling parameters produced expected results. Although this result was better than everything we tried with TradeStation, and in spite of rosy performance information, the trading system certification was only fair. Please see the chart below for details.
 

A key display from TSPE offers the following:

BANKRUPTCY - The top curve in the above chart shows the chance (vertical scale) that drawdown will be any given number of dollars (the horizontal scale) or less. Saying it in another way, given that we started with the horizontal scale value, there is a (100 minus the vertical scale reading) chance that drawdown will exceed the dollar value reading of the horizontal scale.

CAPITAL PRESERVATION - The middle curve shows the probability (vertical scale) of recovering the original capital stake (horizontal scale). Saying it in another way, given that we started with the horizontal scale reading, there is a vertical scale chance that the account holds a horizontal scale balance or more.

GOAL ACHIEVEMENT - The vertical scale shows the probability that a given goal achievement can be met with original capital intact (horizontal scale) given that original capital began with the same dollar value shown on the (horizontal scale).

The curves shown in the above chart are presented with TSPE for any given exercise. Study the charts carefully before deciding to risk your valuable capital in pursuit of a predetermined goal. 
 

TSPE and TradeStation

    Using Omega Research's software, traders subject a given market to one of many possible studies such as moving average crossover, stochastics, RSI, and others. Once the choice has been made, stop-loss limits can be imposed, objectives can be targeted, and the paper trading commences. TradeStation supplies a "Performance Summary" that reports the maximum number of consecutive losses, the percentage of losses, the average profit, the average loss, the largest winning trade, etc., etc. A quick look at a Performance Summary reveals several assumptions that may be mitigated through TSPE. 

    One important Performance Summary statistic is the number of consecutive losing trades that occurred in the actual or simulated past. If it is small and the overall profits and losses appear to alternate, then you may be in for an unpleasant result when you attempt to repeat that situation in real life. To find out how the system might play out, TSPE will simulate the trading system that derived the P&L string. Over and over again, it randomly selects (with replacement) from that P&L set. Don't be surprised if it reveals that the same set of outcomes may not be so opportune in practice, causing excessive drawdown and possible account bankruptcy.

Understanding the Discrepancies

    When TradeStation's Performance Summary suggests a profitable system, be aware that a winning simulation could easily deliver a substantially negative result in actual trading. Here's why: Every trading system is controlled by one or more parameters that, in a very real way, fit the data to the chosen system under study. Matching data with a technical remedy could force a non-repeatable circumstance because you, the developer, would allow the historical data to be analyzed against your personal study rules. Your choice of market or system could be based solely upon simulated performance results, which will pose a problem when excessive parameter control is innocently introduced.

    A moving average crossover study requires two parameters: a short term moving average and a longer term moving average. Two-parameter systems are less likely than one-parameter systems to return a profit in real life. A less constrained one-parameter system example could be a breakout system, which will execute trades into an opposing long or short posture when the market moves out of a trading range. Your parameter consumption quantity reveals much about how your P&L string was prepared, so the mathematics can assess the validity of the profits and losses. In general, the fewer parameters, the better the anticipated result because TSPE imposes a disclosed penalty for excessive parameters.

    The quantity of trades in a P&L input is very important in a TSPE evaluation, where a large quantity of trades restricts the degradation to actual experience. Any system with many trades that shows a favorable outcome in the Performance Summary will also show favorable results with TSPE, even with moderate freedom loss. This is because many, many consistent profits significantly add to the credibility of a systematic approach and repeatability becomes statistically possible. More typical situations involving smaller sample sizes are not as reliable because a small sample suggests a less likely chance of repetition. A small sample size was the downfall in our example study. Although this may, indeed, be a profitable system, the limited amount of statistical data makes it impossible to certify that profitability will be sustainable.

    In a TradeStation Performance Summary, the quantity and percentage of winning trades must be viewed with a wary eye. Any trade ending with a breakeven result is counted as a profit! While every trader desires to achieve consistent winning performance, introducing faulty input through false assumptions is not helpful. The foregoing is not a criticism of Omega's software. We are simply suggesting that TSPE does much more. It will help keep you from misinterpreting the details by requiring an unbiased account of each trade.

    Two important inputs to Omega's profit simulation are the commission charge and the slippage "tax". The default response to both prompts is zero. When these defaults are used, many marginal systems would, due to that liberty, be cast in an undeservedly favorable light. If commission and slippage charges are understated with TradeStation, TSPE will accept additional dollar costs of commission and slippage that will approximate a more realistic assessment of those charges.

    By randomizing trade sequence, compensating for parameter control, factoring in the quantity of trades and taking commission and slippage charges into account, TSPE goes way beyond the information provided by the Performance Summary. It may certify your efforts and confirm that you have found a sound trading system, or it may alert you to rethink and reevaluate your technical approach. TSPE is a reliable safety net for all technical investors, even customers of our newfound competitor Omega Research. It takes only a few moments to run the program and reduce your likelihood of unfortunate surprises, so no one should commence trading without it
 

TSPE is provided free to paying UA subscribers.

TSPE was a popular product for past non-Windows® oriented CSI users and has saved many a speculator from entering the market unwisely. The new version in Unfair Advantage goes a step further by providing a realistic drawdown assessment that helps to inform on the likelihood of achieved performance.