Notice:
  The views and information expressed in this document reflect the opinions and experience of the author Robert C. Pelletier.  Neither CSI nor the author undertake or intend to provide tax advice or trading advice in any market or endorse any outside individual or firm.  All recommendations are provided for their informational value only.  Readers should consult competent financial advisors or outside counsel before making any software purchase or investment decision.  CSI does not stand behind or endorse the products of any outside firms.

Copyright (c) 2000 Commodity Systems Inc. (CSI).  All rights are reserved.

Optimization, Verification, Certification and other Perils A Message from UA Developer Steven Davis COT Survey Update

    For some stock and commodity investors, the trading arena is a place where serious plans are executed and monitored. For others, it is a virtual gambling hall: each trade is a new roll of the dice. The CSI customer tends to be of the more serious ilk, and thankfully so, for only through careful analysis, planning and execution can the trader sustain continued capital growth. 

    The art of developing a reliable and profitable trading system requires the use of powerful tools. These are either built into the developer's algorithm or, in the case of those programs that leave that assignment to the trader, a language is provided for developing a private technique. Over the years we have added many features and tools to help our users make the most of their investment research, and, like most trading system developers, we have done so with good intent. However, even the most skillful market technician must recognize that system optimization and certification present perils of their own.

The Developer's Algorithm

    Some trading systems incorporate simple algorithms, while others use in-depth methods to accomplish a given result. We have found that systems which make full econometric use of causal or related markets to predict any one dependent market are likely to do a better job than an approach which may suggest that past prices of a given market can predict that market's future performance. Moving average crossover systems and stochastics are examples of price-determines-price systems. UA's Multi-Market AnalyzerTM is a good example of a multi-variable econometric system. All of these are available through Unfair Advantage® (UA). Other developmental approaches including genetic algorithms and neural networks can be found in CSI-compatible software programs.

Developing Your Own Technique

    The business of selling tools to conveniently build and process your own trading method has been popular for many years, although at CSI, we are just now in the process of releasing our first offering as part of UA's enhanced Stock ScannerTM. Much like similar tools on the market, our system-writing software and the easy-to-use language come with all the necessary programming tools. System-writing software invites the user to improve upon a set of supplied tools that can help fashion your own system. In other words, you receive the necessary software, modify it to add your own wisdom, then try it out in the market.

    By developing your own system, you can take charge of your own financial destiny. Of course, this is a noble pursuit, so long as you take every result with that proverbial "grain of salt."  Discovering an approach that has consistently performed well in hindsight is absolutely no guarantee that it will work far into the future. 

Certification and Verification

    Certification and verification are important and very necessary steps that are required before trading begins. This is as true for a system you just invented as it is for any pre-packaged trading system you may have purchased!  Fortunately, there are ways to analytically verify and certify a trading system that will do well in actual performance. Unfair Advantage offers fully licensed users software called the Trading System Performance EvaluatorT (TSPE) for proving a system before it is applied to the market.

    There are some pitfalls to using TSPE's Monte Carlo simulation tool however, and they stem from over-optimized (substantially curve-fit) results. TSPE applies a parameter tax constraint that compensates for the degrees of freedom lost through optimization, but this constraint cannot always be properly applied. For example, TSPE should not be used to certify a trading plan for a particular stock when the given stock ranks considerably above all other stocks subjected to the same approach. Many degrees of freedom may be lost when thousands of stocks are screened with the same basic procedure, but they cannot be correctly quantified in TSPE. 

    When used properly, TSPE offers a couple of important statistics. One is the likelihood that a process can be repeated with favorable results. The other is an assessment of the capital necessary to fund the trading activity. Both of these results are derived from the simulation exercise that TSPE goes through to analyze the user's simulated or actual trading record. Of the two results supplied, the likelihood (probability) that you will succeed with a given level of capital stake is probably most important. Be aware that one minus the success probability provided is the chance that you will not succeed with the level of capital given. No matter how sure you are that results will be favorable, there will always be a small chance that you will fail. The trader should focus on this before embarking upon any trading exercise.

    When stocks are screened for the best performer from among thousands (as described above) special considerations apply. In general, the more stocks considered, the less reliable will be the future profits of the best performer identified through optimization or a simple sorting of each stocks performance rating. This is especially true if results of the analysis prove unfavorable for most of the stocks in the screened list. No trading system should be considered unless it shows profitable scenarios for a significant majority of the markets studied.

    As has always been our practice with Unfair Advantage, our analysis tools and system certification software are offered as free supplements to the CSI data feed. We hope that they help you to trade successfully and with confidence. Whether you use CSI market analysis products or those purchased from a third-party vendor, always remember that nothing can eliminate all risks associated with market investment. Well used analysis tools can, however, make trading less risky than a roll of the dice.
 

Bob Pelletier

    TCSI is committed to offering our customers an increasingly comprehensive list of services, with scheduled enhancements ranging from delivery of stock industry groups and fundamental data to option factsheet corrections and more. Unfortunately, a limitation in UA version 1.73.2 and earlier versions does not allow for the necessary expansion of data fields to accommodate these enhancements. Our goals could not be completely fulfilled with this limitation in place, so we removed it from UA version 1.74.0 (released in February 1999) and all subsequent versions.

    At this time, over 90% of our customers are using UA versions which are not affected by this limitation. We believe this is a good time to upgrade the remaining UA customers so we can add to the transmitted data and improve service for all. To this end, we plan to send a current CD-ROM to each subscriber using versions prior to 1.74.0. If you received an upgrade CD, you were asked to install either version 1.76.1 or version 2.1.4. Version 1.76.1 looks much like the version to which you are accustomed. Many enhancements await users of 2.1.4, which has a new user interface with expanded functionality. Version 2.1.4, for example, allows you to see COT data and to make automatic currency exchange conversions on displayed charts.

    Our current plans are to begin transmitting new data types on August 23rd. This will give the affected subscribers approximately one month to upgrade and resolve any migration problems. After that time, UA versions 1.73.2 and earlier will no longer be supported and will not function. If you have not installed UA v.1.76.1 or above, please do so right away.

    We apologize for the inconvenience, but it is necessary for us to expand and improve our service to you.

    The CFTC is still collecting public input about possible changes in the reporting frequency of Commitments of Traders data. We published their survey in the July CSI Technical Journal, along with the comments and recommendations of Steve Briese, editor of the Bullish Review*. Mr. Briese is the supplier of CSI's enhanced COT data feed, and is unquestionably an expert in the use of this data. Thanks to everyone who responded to the CFTC survey and to Steve for his insights into what changes will be most helpful in preserving this valuable data for traders. If you have not yet responded to the survey, please review Mr. Briese's recommendations and then fill out the survey on the CFTC's website at: http://www.cftc.gov/dea/survey/cot%20survey.htm.

Thank you!

* Due to an editing error, we failed to credit Steve Briese and the Bullish Review (issue #279) for the COT Survey article reprinted in the July Technical Journal. We regret and apologize for the oversight.