[Click here to return to journal index]
January 2003
CSI Technical Journal
Volume XXI, Number 1
Déjà vu? The Return of PDI
2003 -- This New Year puts us firmly into the 21st century, with much uncharted water ahead. Before looking forward, let us first reflect on the words of Patrick Henry, who, more than two centuries ago, said: "I know of no way of judging the future but by the past." Perhaps an even better New Year's reflection for traders came in 1984 from John Naisbitt who, in his book "Megatrends: The New Direction Transforming Our Lives," said, "The most reliable way to anticipate the future is by understanding the present." We market analysts tend to meld the wisdom of both these noted Americans by studying the past and present in our attempts to forecast the future.
When I look back on the humble pre-beginning of CSI, the most interesting thing about the early 1960s era is the fact that we managed as well as we did without personal computers. I can see now that there was actually no good way to capture daily data reports in the late 1960s. We did it anyway. I owned a GE Teletype terminal that I operated in my home, with which I sent data to friends and associates who wanted to follow the futures markets. At 110 baud (110 characters per second), the transmissions were pitifully slow, but at the time it seemed like lightning speed.
As far as I know, I was the first vendor in the U.S. to distribute market data (via acoustic coupler) to individual traders. I remember one early competitor who briefly followed my lead, but who soon disappeared from the horizon. The science of technical analysis didn't find a strong following until the late 1970s when microcomputers became widely available. Many of our early customers were serviced on Radio Shack's TRS 80 computers, and later on Commodore 64s and Apple IIs. Still later came the Texas Instruments MiniComputer, and finally (still) the PCs.
We mid-20th century pioneers in computerized market analysis put up with the hassles and sought ways to overcome many challenges because we had found a good way to learn and profit from the markets. There was no doubt in our minds that our efforts improved our chances of earning money off the futures markets that were designed to maintain a stable food supply. What an opportunity!
My analysis told me that the future directional movement of futures markets was heavily influenced by the combined and associated effects of two very important statistics that were, and still are, often overlooked and neglected by traders. These statistics involve daily volume and/or open interest reports. Combining volume and open interest market reports with price can help you forecast future price movement by measuring the significance of daily price fluctuations.
I believed then and continue to believe now that futures traders have a significant edge over stock-only traders because futures markets report open interest information. This critical data set tends to define the number of players who will eventually vacate their positions. Keeping tabs on the build-up of open interest during a future contract’s life helps the trader to understand the makeup of the other market players.
Way back in the '60s we came to understand that when you subtract the daily trading volume from the open interest reports, you are left with a volume reference that reflects the interests of those who remain in the market for the given day. The difference represents the so-called day-trader influence. The growth in open interest from day to day represents the new “blood” entering the market, and this happens to be a reliable indicator for measuring the quality of new trader presence.
The movement of these important measures of trading activity will likely help identify the market’s future direction. Volume, for example, has traditionally been considered to reveal the significance of price action. Standard market interpretations tell us that when prices advance on heavy volume, the tendency for the market is to sustain the price advance. A significant open interest jump that is associated with rising price is likewise a bullish event. A price decline on heavy volume or open interest, on the other hand, may forecast a further drop in price. Conversely, any price movement that doesn't have high volume and open interest lacks support and is likely to fizzle. Volume and open interest movement together with price will generally affect future price depending upon the magnitude and direction of price, volume or open interest movement.
Over a decade ago, CSI introduced a study as part of our QuickTrieve® software system that used price, volume and open interest as market predictors. This study, known as the Probable Direction Index™ (PDI™ for short) was hugely popular among CSI users. Many complained that the study was not carried forward to Unfair Advantage.® PDI was the study of choice for many who reported good success by simply basing their trading on PDI and associated studies.
The above charts (from QuickTrieve) show the daily market prices of Soybean futures and the PDI graphic for the same series. The PDI graphic shows an oscillating index and two horizontal lines of significance, which can be considered buy and sell signals. The user is allowed to set the thresholds for each market. See the caption for the charts on page 3 for more on interpretation.
The charts on these pages were made by QuickTrieve's PDI study. They demonstrate the study's predictive oscillator.
PDI is a probability reading derived from the relative daily movement of price, volume and open interest. Although designed to study these three components, traders found that PDI could work for stocks by focusing on the subset of price and volume data. Larger PDI readings (close to 1.0) indicated the market might move upward in the future, and lower relative readings suggested the market might move lower. Because of the way in which the PDI index was calculated over time, there was a significant tendency for its accuracy to improve as more and more data points were provided.
In this sense, one could argue that the study possessed heuristic attributes. PDI was not like most popular studies, RSI or MACD, for example, because those studies will return the same resulting values and form when given a small subset of a given time series. The PDI statistics actually reflected a learning potential derived from the market’s past behavior. Weights applied to volume, open interest and price varied over time as PDI learned from the predictive evidence of the data supplied.
On the other hand, PDI was a calculation that viewed an ongoing short-term segment of the historical past. The relatively short time frame was meant to keep your perspective as current as possible without overlooking history's lessons. Anyone viewing a long-term chart may be fooled by an overly broad perspective, which could lead you to remain in a market as it slowly moves against you. If you limit your perspective to the relative merits of the short term, you won't be so likely to hang on to a losing situation. In the event of short-term persistent losses, PDI would quickly shift its emphasis to price only and follow the market in the prevailing direction.
One interpretation of PDI signals is when the upper horizontal line (0.90) is touched by the oscillator, a buying mode begins; when the lower horizontal line (0.10) is touched, a selling mode begins. Using this procedure, one would remain in a position until the opposite position is indicated. Mid-trade movement of PDI can be watched, but is not considered significant until the opposite threshold is touched.
Dr. A. Elder, in his book "Trading for a Living" (John Wiley & Sons, 1993) made an apt analogy, saying that a frog who jumped into a pot of boiling water would jump right out, but a frog who was placed in a pot of tepid water would stay in the pot as it was heated, eventually allowing itself to be boiled alive. Some stock market traders have been boiled alive over the past couple of years because they didn't jump out of the proverbial pot. Since PDI’s design may help such traders stay focused on immediate conditions and avoid painful and persistent losses, we have decided to reintroduce the program.
In next month's CSI Technical Journal, we hope to announce that a new PDI study is ready for release as a DLL within Unfair Advantage. The new PDI may be a little different from the original, but the concepts behind the study will be the same. Re-engineering PDI will be my first project for the new year, followed shortly by a fresh look at PDI's sister study called CSI Stop.™ That study reveals the price level where the forward price dynamics of a particular market are likely to change direction. In the QuickTrieve days, traders told us that PDI and CSI Stop, when used together, contributed positively to their trading success. It is our goal to make both of these programs available to Unfair Advantage subscribers early this year.
We are beginning 2003 by revisiting some remarkable products from our past that deserve their places in this 21st century. We hope that you will enjoy and benefit from them.
Bob Pelletier
Tech Talk
Each month in this column, the CSI technical support staff addresses issues that may be of concern to many subscribers though this question-and-answer forum. This month they discuss using an FTP download as an alternative to UA's API, compatibility with MetaStock software, reporting program errors and speeding updates on an older computer through adjustments to the DMA setting.
Q. I use Unfair Advantage for data retrieval and nothing more, having developed my own fully automated analysis system. I would prefer to avoid the approach used by UA's API for processing data. Are there any other options for retrieving data without going through UA?
A. Ideally, UA's Application Programmer's Interface (API) would meet your needs, as it does for many of our customers. We are currently making improvements that we hope will increase its appeal to you. In the meantime, or as another alternative, you might consider switching to the File Transfer Protocol (FTP) data transmissions that our largest commercial customers use. These transmissions, which provide a straight data download, can be initiated by either a CSI staff member or by a call from your computer. The transmission time is typically 7:30 PM ET for the complete database.
Rates for the FTP service are significantly higher than for a standard UA account, as this is considered a commercial service, but the benefits are worth the price in certain situations. If you are interested, please contact CSI's commercial sales representative at 800-274-4727 or 561-392-8663.
Q. I use MetaStock® and have recently had problems with spikes in my data. The puzzling thing is that these apparent errors are only in my MetaStock charts, not the charts made by UA. The folks at MetaStock tell me it's a compatibility problem. Please advise.
A. The spikes you have reported may indeed be a compatibility problem between MetaStock and UA's CSIM-format files. MetaStock and Unfair Advantage are compatible so long as UA writes its export files to the MS format. UA employs a DLL provided by the makers of MetaStock to write those files, which are not adversely affected when they are modified by the MetaStock software. Exporting to MS format is a simple process that can be arranged when you create or modify your UA portfolios or through a menu selection to write the files on demand.
There have recently been changes to the way MetaStock handles data that have reduced MetaStock's compatibility with the CSIM format (also an export option within UA). We recommend that all MetaStock users stop using CSIM format and use exclusively MS format files with MetaStock. CSI has an agreement with the makers of MetaStock whereby they have provided us with the necessary tools to assure compatibility and we have agreed not to modify their format. We intend to continue this relationship for the benefit of our many mutual customers. We regret any inconvenience caused by the transition.
Incidentally, spikes did erroneously appear in the UA database for stock symbols T and AMCC on 12/6. These stocks both declared dividends and had reverse splits on the same day, which we mistakenly processed out of sequence. This rare occurrence led us to change our adjustment procedure, so the problem won't be repeated. The spikes were remotely corrected through UA.
Q. I'm running an older version of UA and think I've found a bug in the program. What should I do?
A. Please call or e-mail to tell us the specific problem you're encountering. Chances are good that it has already been corrected in the current version of UA (v 2.6). We recommend that every subscriber maintain an up-to-date copy of UA so that you can benefit from ongoing product enhancements and development.
Q. My computer isn't what you'd call the latest and the greatest, but it works for me in all areas except one; distribution of my downloads from CSI take over an hour. I tried the suggestions listed in the 12/02 CSI Technical Journal to speed things up, but they didn't help much. Any other ideas?
A. Since you have tried our non-hardware related suggestions for speed improvement, it seems likely that your distribution time is being hampered by a sheer lack of power in your older computer. You might get some relief by enabling the IDE DMA bus-mastering mode of your computer's Microsoft operating system. (Not applicable for Windows 95 or for computers manufactured before 1996.) When the DMA (Direct Memory Access) switch is set, information passes to your drive in 512-byte groups instead of four-byte groups, which can result in significant increases in speed.
We have used the DMA mode successfully on an older computer here at CSI, and in doing so, decreased the download time from one hour to 12 minutes, an improvement of 80%. This improvement is still not within the optimal time of one or two minutes.
Switching to the DMA bus-mastering mode is not without drawbacks. Undesirable results can include (but are not limited to) an increased possibility of corrupting UA data files and damaging your operating system to the point that it must be reinstalled. Any changes to the DMA setting should be preceded by a full system backup and should only be undertaken at your own risk.
For information on and warnings about setting the IDE DMA bus-mastering mode, please see the following articles in Microsoft's customer support library. Feel free to contact CSI's technical support staff if additional assistance is needed.
Windows NT 4.0: http://support.microsoft.com/support/kb/articles/Q158/8/73.ASP
Windows 9x and Me: http://support.microsoft.com/support/kb/articles/Q258/7/57.ASP" target=_blank>http://support.microsoft.com/support/kb/articles/Q258/7/57.ASP
An article on the kind of problems that can be encountered: http://support.microsoft.com/support/kb/articles/Q190/6/30.ASP
Other suggestions are to make sure you are running the latest version of Unfair Advantage and to be sure your portfolios do not include any delisted stocks.
Notice:
The views and information expressed in this document reflect the opinions and experience of the author Robert C. Pelletier. Neither CSI nor the author undertake or intend to provide tax advice or trading advice in any market or endorse any outside individual or firm. All recommendations are provided for their informational value only. Readers should consult competent financial advisors or outside counsel before making any software purchase or investment decision. CSI does not stand behind or endorse the products of any outside firms.
Website Links: This CSI website and Unfair Advantage system contain hyperlinks and automatic links to websites operated by parties other than CSI. No association with or endorsement of third-party websites should be inferred through these links. They are provided for your convenience and reference only. CSI is not responsible for and has no control over the content of third-party websites.
All information and data on this website are © Copyright 2002-2003 by Commodity Systems Inc.(CSI) All rights are reserved.
800-274-4727 | 561-392-8663 | 561-392-1379 (Fax)