Issue 54, Summer/Fall 2000

INTRODUCTION

Basics of Candlestick Charting Techniques

Candlestick charts are the most popular and the oldest form of technical analysis in Japan, dating back almost 300 years. They are constructed very much like the Open-High-Low-Close bar charts that most of us use everyday, but with one difference. A “real body,” a box instead of a line as in a bar chart, is drawn between the opening and closing prices. The box is colored black when the closing price is lower than the opening price and colored white if the close is higher than the open. With the colors of the real bodies adding a new dimension to the charts, one can spot the changes in market sentiments at a glance – bullish when the bodies are white, and bearish when black. The lines extending to the high and to the low remain intact. The part of the line between the real body and the high is called the “upper shadow,” while the part between the real body and the low is termed the “lower shadow.”

The strength of candlestick charting comes from the fact that it adds an array of patterns to the technical “toolbox,” without taking anything away. Chart readers can draw trendlines, apply computer indicators, and find formations such as ascending triangles and head-and-shoulders on candlestick charts as easily as they can on bar charts. Let us now examine some candlestick patterns, their implications, and the rationale behind them.

A bearish engulfing pattern is formed when a black real body engulfs the prior day’s white real body. As the name implies, it is a signal for a top reversal of the proceeding uptrend. The rationale behind a bearish engulfing pattern is straightforward. A white candlestick is normal within an uptrend as the bulls continue to enjoy their success. An engulfing black candlestick the next day would mean that the open price of the next day is higher than the close of the first day, signaling possible continuation of the rally. However, the bulls of the first day turn into losers as the price closes lower than the first day’s open. Such a shift in sentiment should be alarming to the bulls and signals a possible top.

A morning star is comprised of three candles: a long black candlestick, followed by a small real body that gaps under that black candlestick, followed by a long white real body. The first black candlestick is normal within a downtrend. The  subsequent small real body, whose close is not far off the open, is the first warning sign as the bears were not able to move the price much lower as they did the first day. The third day marks a comeback by the bulls, completing the “morning star,” a bottom reversal pattern.

There is really no need for me to cover too many candlestick patterns here. The examples are given merely to illustrate the fact that most candlestick patterns are nothing more than collections of up to three sets of openhigh-low-close prices and their relative positions to the others.

Importance of Size and Locations of Candle Patterns

The other important points to keep in mind are the sizes of the candlesticks in the patterns and the locations of the patterns within recent trading range of the price. Let us consider a stock that trades around $60. Scenario One: On the first day, it opened at $60 and closed at $63.75. On the next day, it opened at $64 and closed at $58.75. Those two days’ trading constitutes a bearish engulfing pattern. This pattern should be considered meaningful since a roughly $4 run-up followed by a $5+ pullback is of considerable impact on a $60 stock. Scenario Two: The same stock opened at $60 and closed at $61 on the first day, and opened at $61.25 and closed at $59.875 the next day. Those two days’ trading did indeed still constitute a bearish engulfing pattern, but the effect of the pattern would not be considered as meaningful. A $1 fluctuation for a $60 stock is pretty much a non-event. It should be evident that the sizes of the candlesticks within a pattern do matter as much as the pattern itself.

We should now look at the importance of the location of the pattern relative to its trading range. Let us assume that the aforementioned $60 stock has been trading between $45 and $55 for five months, broke out to new high, and ended the last two trending days as described in Scenario One, one could speculate that support at $55 probably will be tested in the near future. If the stock, however, has been trading between $58 and $64 for five months, tested the support at $58, rallied up to $63.75, just slipped back to $59.875, the bearish engulfing pattern described in Scenario One should have little meaning. The validity of this pattern is limited here since there was not much of a uptrend proceeding the pattern and the downside risk to $58 is only $1.875 away.

From these comparisons, it should be obvious that the usefulness of candlestick patterns rely on: 1) The size of the candlestick components – real bodies, upper and lower shadows; 2) The relative positions among themselves – gapping from one another, overlapping. After all, that is how patterns are defined; and 3) The patterns’ locations relative to the previous periods’ trading range and 4) The size of the trading range itself. In order to find an effective candlestick pattern, these points should all be considered integral parts of the pattern definition.

[Author’s Note: Although volume and open interest accompanying the candlestick patterns could be used as confirmation, the author has decided not to include either as part of the pattern definition for two reasons: 1) A stock can rally or decline without increasing volume. Volume tends to be more evident around structural breakouts and breakdowns, but not always so around early reversal points. This is especially true for thinlytraded stocks where the price can move either way quickly without much volume. Depending on volume as confirmation is impractical at times. 2) The pattern the author plans to find should be rather universal just like the other candlestick patterns. Shooting stars are not unique to crude oil futures; nor is the bearish engulfing pattern only designed for Microsoft stock. Since some market data, such as spot currencies and interest rates, do not contain volume nor open interest information, a candlestick pattern with volume or open interest as an integral part of it will not be universally useful.]

Basics of Genetic Algorithm – “Survival of the Fittest”

“Survival of the Fittest.” Darwin’s theory of evolution remains one of the scientific theories that has had the most profound impact on humankind to date. In his theory, Darwin proposed that species evolve through natural selection, that is, species’ chance of existence and their ability to procreate depend on their ability to adapt to their natural habitat. Since only the organisms fit for their natural environment survive, only the genes they carry survive. During the reproductive process, the next generation of organisms are created from the genes drawn from the surviving, and hopefully superior, gene pool. The new generation of organisms should, in theory, be even more adaptive to their environment. As some of the offspring produced will certainly be more adaptive to the environment than their peers and therefore survive, the natural selection process repeats itself, and again only “superior” genes will be left in the gene pool. As the process is repeated generations after generations, nature will preserve only the “fittest” genes and dispose of the “inferior” ones. It should be noted that during the process, the genes sometimes will experience certain degrees of mutation that might create combinations of genes never seen in previous generations. Mutation actually brings about a more diversified pool of genes, perhaps creating even more adaptive organisms than otherwise possible. At times in nature, an array of species evolve from the same origin, with each of them as fit for its habitat as the others in its own right.

So, what is “genetic algorithm?” In plain English, genetic algorithm is a computer program’s way of finding solutions to a problem by the process of eliminating poor ones and improving on the better ones, mimicking what nature does. In constructing a genetic algorithm, one would start out by defining the problem to be solved in order to decide on the evaluation procedure, imitating the process of natural selection. Try a few possible solutions to a problem. Rank them based on their performance after applying the evaluation process. Keep only the top few solutions and let them reproduce, or mix elements of the top solutions to come up with new ones. The new solutions are then, in turn, evaluated. After a few iterations, or generations, the best solutions will prevail.

For a better explanation, we should now try a fun, practical exercise. Let us consider the process of finding that perfect recipe for a margarita. We start out with six randomly mixed glasses; record the proportion between tequila and the lime juice and taste them. The evaluation process here is your friends’ reactions. They agreed that two glasses were found okay, three of them so-so, and one which yielded the response, “My dog is a better bartender than you.” You then mix five more glasses using proportions somewhere between those found in two okay glasses, and throw in one wildcard, a randomly mixed sample. Now let them try picking the two best ones again. After you get your friends stone-drunk after repeating this process 20 times, you will have yourself two nice glasses of margarita. Most importantly, those two glasses should be of similar proportion of lime juice and tequila; that is, the solution to this problem converged.

Let us now review our margarita experiment. The goal was to find the best tasting margaritas, and therefore the way to evaluate them was to taste them and rate them. Assuming that you have an objective way to total your friends’ opinions of the samples, the ones that survived by this “somewhat natural selection” – the okay glasses – will get to “reproduce.” The wildcard thrown in represents the mutated one. Just like mutations’ importance in nature as a way of injecting new genes into the gene pool and bringing about a more diverse array of species, artificial mutations are very important in opening up more possibilities in the range of solutions for the problem we intend to solve.

In a more involved problem with more variables, the procedure should be repeated many more than 20 times for any half-way decent solutions to prevail. The more iterations the program perform, the more optimal the solution should be. In fact, if the surviving solutions do not resemble each other at all, they are probably far from optimal and require more time to evolve. What we are searching for here are “sharks.” Sharks, one of the fastest species in water, have been swimming in the ocean for millions of years. For generations, the faster ones survived by getting to their food faster in a feeding frenzy. Only the “fast” genes are left after all these years. As sharks’ efficient hydrodynamic lines became “perfect,” all of them started to “look alike.” (At least to most of us.)

End Note: To explain the concept of genetic algorithm in an academic manner, I will turn to the principles set forth by John Holland, who pioneered genetic algorithms in 1975:

1. Evolution operates on encodings of biological entities, rather than on the entities themselves.
2. Nature tends to make more descendants of chromosomes that are more fit.
3. Variation is introduced when reproduction occurs.
4. Nature has no memory. [Author: Evolution has no intelligence built in. Nature does not learn from previous results and failures; it just selects and reproduces.]

Now the steps of the algorithm:

1. Reproduction occurs.
2. Possible modification of children occurs.
3. The children undergo evaluation by the user-supplied evaluation function.
4. Room is made for the children by discarding members of the population of chromosomes. (Most likely the weakest population members.)

Definition of the Project’s Intent: Applying Genetic Algorithm to Identify Useful Candlestick Reversal Patterns

Let me now define the problem I intend to solve in this study. I intend to find one candlestick pattern that has good predictability in spotting near-term gains in future price by using the bond futures contract as my testing environment. Since I have cited sharks as the marvelous products of natural selection, I would like to call the organisms that should evolve through my study “candlesharks.”

The candlesharks will have genes that tell them when to signal potential gains, or “eat” if one would parallel them to the real sharks. The first generation candlesharks should be pretty “dumb” and not really know when or when not to “eat” – maybe some are eating all the time while some simply do not move at all. As some of them over ate or starved to death, the smarter ones knew how to “eat right,” correctly spotting potential for profit. As only the smart ones survive, they begin to preserve only the smart genes. As these smarter ones mate and reproduce, some of the next generation may contain, by chance, some even better combination of genes, resulting in even smarter candlesharks. As the process continues, the candlesharks should evolve to be pretty smart eaters. Once in a while, some genes will mutate, creating candlesharks unlike their parents. Whether the newly injected genes will be included in the gene pool will depend on the success of these mutated creatures in adapting to their environment.

One thing that should be pointed out is that the late generation candlesharks will probably swim better in “bond futures pool” better than in a “spot gold pool” or “equity market pool” which they have never been in before. “Survival of the fittest” is more like “survival of the curve-fittest” here. It should be understood that the candlestick pattern found here has evolved within the “bond” environment and thus is best-fit for it. If thrown into the “spot gold pool,” these candlesharks might “die” like the dinosaurs did when the cold wind blew as the Ice Age hit them so unexpectedly, as one of the many theories goes. As in many cases in life, the finer a design with one purpose in mind, whether natural or artificial, the less adaptive it will be when used for other purposes. For example, a 16-gauge wire stripper, while great with 16-gauge wires, will probably do a lousy job stripping 12-gauge wires even when compared to a basic pair of scissors, which can strip any wire though slowly.

BUILDING A SUITABLE ENVIRONMENT FOR EVOLUTION

Defining the Genes

As described in the previous sections, there are, but not limited to, four major deciding factors of the significance of an occurrence of a particular candlestick pattern. They include the size of the recent trading range, current position within that range, relative position of the candlesticks to each other, and the sizes of the those candlesticks. To successfully survive in the “bond futures pool,” it must be in the genes of the candlesharks to be able to distinguish variations in these environmental parameters. I have therefore designed a candleshark to possess the listed genes in Exhibit 1. Genes within Chromosome C-2, for example, tells the candleshark the range of position and size of the candlestick of two days ago should be within, combined with other chromosome’s parameters, before giving a bullish signal. Actually, think of all these genes as simply tandem series of on-off switches for a candleshark to decide to give a bullish signal or not.

All the genes defined here come in pairs. The first, with suffix “-m”, tells the candleshark the minimum of the range in question. The second, with suffix “+”, signifies the width of the range. Here is one example. If RB-m of C2 is -24 and RB+ of C2 is 16, the candleshark will only give a bullish signal when the candlestick of two trading days ago has a black real body sized between 8 (-24 + 16 = -8) ticks and 24 ticks, or the contract closed between _ to _ lower than it opened. Please keep in mind that C2 only contributes to a part of the decision-making process for the candleshark. Even if the real body of two days ago fits the criteria, the other criteria have to be met as well before the candleshark will actually give a bullish signal.

The number of digits needed within each gene can be calculated. The size of a real body for a bond contract can not be larger than 96 ticks since the daily trading limit is set to three points. A seven-bit binary number is capable of handing numbers up to 128 and therefore needed for a gene like RB-m of C2. The 40-day trading could be no larger than 96 ticks times 40, or 3840 ticks. (Besides the fact that it seems very unlikely that the bond contract would go up, or down, 3 points day after day for 40 days.) A 12-bit number, capable of handing a decimal number up to 4096 is needed here.

As the reader might have noticed after referencing Exhibit 1, a number of trading ranges are used. As previously mentioned, the size of the recent trading range and the candle pattern’s position within the range are crucial elements that might make or break the pattern. How does one define “recent” though? It then seemed obvious to me that a multitude of day ranges is needed here, much like the popular practice of using a number of moving averages to access the crosscurrents of shorter and longer-term trends. I have chosen to include 5-day, 10-day, 20-day and 40-day trading ranges, which are more or less one, two, four, and eight trading weeks, in my study. The advantage of using a multitude of day ranges could be demonstrated using two examples. Let us say a morning star, a bullish candlestick reversal pattern, was found near when the price is at the bottom of both the five-day trading range and the 40-day trading range, as it would if it was just making a new reaction low. A morning star around this level is probably less useful since that pattern is more indicative of a short-term bounce. While this morning star, a one-day pattern, could be signaling a possible turn of the trend of the last five days, it is unconvincing that this one-day pattern could signal an end to the trend that lasted at least 40 days.

Let us now say that the same morning star was found near the bottom of the five-day trading range, but near the top of the 40-day trading range, as the price of a stock would if it experienced a shortterm pullback after breaking out of a longer-term trading range. This morning star now could be a signal for the investor to go long the stock. The morning stars in both examples could be of the same size, both found near the bottom of the five-day trading range, but would have significantly different implications just because they appear at the different points of the 40-day trading range. It should be clear now that the inclusion of multiple trading ranges in the decision-making process could be very beneficial.

Since each gene is a binary number, a string of 0’s and 1’s, each will have a MSB (most significant bit, the leftmost digit, like 3 in 39854) and LSB (least significant bit, the rightmost digit.) Since the MSB obviously has more impact on the candlesharks’ behavior, the common state of the more significant bits among the candlesharks are what we should closely examine after the program has let the candlesharks breed for a while. The candlesharks’ main features should be similar after a while. That is, if we do have a nice batch of candlesharks to harvest, they should all have the same sets of 0’s and 1’s among the more significant bits within each gene. The 0’s and 1’s among the trailing bits are less significant by comparison, much like the curvature of an athlete’s forehead should have less to do with his speed than his torso structure. The trailing bits are in the genes and do make a difference, but are basically not significant enough for us to worry about.

When we are ready to harvest our candleshark catches as the performance improvement from one generation to the next decreased to a very small level, we could reverse-engineer the gene to find the criteria that trigger their bullish signals. For instance, if all eight candlesharks have “-00011” as the first five digits in RB-m of C2 (that means RB-m is between -00011000 and -00011111, or -24 and -31) and “00000” as the first five digits of RB+ of C2 (that means RB+ is between 0000000 and 0000011, or 0 and 3), these candlesharks would only give bullish signals when the real body of two days ago is black, and between size of -21 and -31. (The minimum = -31 + 0 = -31; the maximum = -24 + 3 = -21)

Defining the Evaluation Process

I have weighed several methods to evaluate the performance of each of the organisms. First of all, a suitable method has to be of short-term nature since the pattern in question is basically formed in three days. It is very unlikely that, for instance, a morning star formed in three days that occurred twenty days ago has much influence on current price. Secondly, the upward price movement that comes after the pattern has to be greater than the downward movement, at least on the average. It should be realized that no matter how useful a candlestick pattern, or any technical tool, may be, there will be a time that it would not have predictive ability, or even give a downright wrong signal. It then seems reasonable that including total downward moves is essential in the evaluation process. That is, we would like to find a pattern that is not only right and right enough most of the time, but one that will not take anyone to the cleaners when it is wrong.

What I have decided on is the average of the maximum potential gain less the maximum potential loss. First, we find the maximum potential gain by totaling all the differences between the highest of the high prices reached by the contract within the next five trading days from the closing price when the pattern gave the signal. We then find the maximum potential loss by totaling all the differences between the lowest of the low prices reached by the contract within the next five trading days from the closing price when the pattern gave the signal. The maximum loss is subtracted from the maximum gain, and is then divided by the number of signals generated. This ratio is the average of the maximum potential gain less the maximum potential loss I am looking for.

Defining the Reproductive Process

We desire enough permutations of the parents’ genes to create diversity among the offspring. Yet, too many offspring in each generation would greatly increase the processing time needed to evaluate the performance of all the offspring. After a fair amount of contemplating, I believe that eight offspring from two parents is adequate. A trial run shows that the evaluation of eight organisms with 1,500 days worth of data requires roughly four minutes of processing time on my personal computer. That equals 1,080 generations after three straight days of processing. Since a large number of generations might be required for the effective evolution, eight organisms per generation would have to do. Besides, allowing only two out of eight offspring to survive is a stringent enough elimination process. Many large mammals have fewer offspring in their life times.

The second crucial element of the reproduction is the introduction of mutation. Under the principles set forth by John Holland, mutations come in two modes: binary mutation, the replacement of bits on a chromosome with randomly generated bits, and one-point crossover, the swapping of genetic material on the children at a randomly selected point. I have favored a higher level of mutation since it would introduce more diverse gene sequences into the gene pool more quickly. Both the binary mutation level and one-point crossover level have been set to 0.1, or 10% of the time.

EVALUATION OF RESULTS

Preliminary Results and Modifications

As some people might have suspected, what I thought was a wonderful study had a pretty rocky start. The final version of the gene definition, as the readers know it, is actually the third revision. A large number of genes slows down the processing time dramatically. Having too many trading ranges defined also proved to be a waste of time.

Choosing the right gene pool to start with, much to my surprise, was in fact quite tricky. I first wrote a small routine using the Visual Basic Macro in Excel to randomly generate eight candlesharks. These candlesharks did indeed reproduce and the evolution program, also written in Visual Basic, performed its duty and evaluated each one of them. After testing the programs and becoming convinced of their ability to perform their functions, I let the program run overnight, evolving 50 generations. What I found the next morning were eight candlesharks that did absolutely nothing. None of them gave any signal. As they all performed equally poorly, the two selected to reproduce were basically chosen arbitrarily. What I had come up with were the equivalents of the species that would have been extinct in nature.

The next logical step then was to use two organisms that would give me signals all the time and to write a small routine to generate six offspring from them. The eight of them would then be my starting point. This proved to be much more effective. Since the first batch of candlesharks did indeed provide me more signals than I needed, their offspring could only give an equal amount or less signals. As some of the children became more selective, their performance did improve. After viewing the printed results of the first seven or so generations, I was glad to see the gradual, yet steady improvement in the ability to find bullish patterns. I again let them grow overnight.

What I found the next morning was a collection of candlesharks that gave either one or two signals. Each of the signals was wonderful, pointing to large gains without much risk. As it turned out, a few of them were simply pointing to the same date. These candlesharks basically curve-fitted themselves just to pick up the day followed by the best five-day gain in the data series. They were again useless.

It then occurred to me that I had to set a minimum number of signals per organism that the candlesharks would have to meet before the program would consider evaluating them as the breeding ones. Since I was using 1,500 days, or roughly six years, worth of data, the minimum of 20 signals, or a little over three a year, should suffice.

I am glad to report that things started to look up afterward. I have also decided that running the program with smaller iteration could be beneficial. I began running only five generations each time so that I could observe the progress and make any necessary adjustment. It was not until that most things had been fine-tuned that I began my overnight number-crunching again.

Finding Useful Gene Sequences

I first wrote the program with the intention to let the candlesharks evolve and hoped to see them converge into a batch of similarlyshaped creatures. In other words, I was looking for a group of consistent performers. It so happened that in the evolution program, I had coded a line to print out the gene pool with the evaluation results after every generation. This feature was first put in place as a way to monitor the time needed for each generation, and to ensure that the reproduction process was performed correctly. As it turned out, this feature had an extra benefit.

Looking through pages and pages of printouts, I every so often spotted one candleshark with excellent performance. Since a genetic algorithm is based on the principle that nature has no memory, this candleshark’s excellent gene sequence could not be preserved. The only traces of the sequence is in its children which might not be as good a predictor as it was. This does occur in nature as well. Einstein’s being an incredible physicist did not imply that his children would have been, too. Even if some of his genes would live on within his children, none might be as great as he was. A number of Johann Sebastian Bach’s sons were outstanding composers, but none as great as he was.

With the printouts in my hands, though, I could reconstruct that one unique gene sequence. I could examine the organism by itself and let it reproduce several times to see if it could be improved upon. Better yet, I could find two great performing candlesharks who did not have to be of the same generation, and let them mate, something impossible in nature. Imagine the possibility of seeing the children of J.S. Bach and Clara Schumann if they were ever married, or turning Da Vinci into a female to mate with Picasso. One might hesitate doing so in nature, but I face no moral dilemma moving a few 0’s and 1’s all over the place.

Evaluating Gene Sequences for Useful Patterns

Running the program from different starting points yielded varying results. One of the runs that intrigued me the most was one that ended with at least five or six organisms out of the last three generations with similar if not identical performance numbers. Upon more careful inspection of the signals they generated and referencing candlestick charts for the bond futures contract, one pattern seemed prominent. (For a sample of the signal results, see Exhibit 6.) The first day of the pattern shows a large black candlestick, usually with modestly sized upper and lower shadows. The second candlestick is a slightly smaller black one with more pronounced upper and lower shadows. The last candlestick is a small real body, usually white or at times doji, with upper and lower shadows of considerable sizes as well. Very importantly, all three real bodies rarely, if at all, overlap one another.

Let us examine a few of these patterns. All the charts included here are those of bond perpetual futures contract, a weighted moving average of the prices of the current contract and those of the forward contracts. The perpetual contract, versus the continuous contract, is ideal for long-term study of futures since it includes more than one active contract and eliminates price gaps around contract month switching dates that have plagued the continuous contracts.

As one can see by examining the sample charts, the last candlestick within the pattern is always near the bottom of the five-day trading range. This observation does follow the fact that the two preceding days are both down, by definition of this pattern. I hope that some readers will find this pattern useful in making their trading and investment decisions. I myself now have another tool in my “technical tool belt,” and I am planning on constantly re-evaluating the validity of this pattern by looking for it in the bond contract’s future price activities.

FUTURE POSSIBILITIES 

While this study has some interesting results, the possibilities are endless. This study was set to find candlestick patterns that would identify possible up moves in bond prices in the next five days. The next study could be set to find a complete entry/exit trading system with both long and short positions. A different version of the devised program could be used to find an optimal combination of technical indicators and parameters for a particular security instrument. In fact, the use of genetic algorithms as a way to optimize neural networks has been widespread.

Looking not so far out, one could even use the existing program to find other candlestick patterns. By simply changing the gene pool one starts with, very different organisms from the one we found might evolve. Much like different species in nature, of which each excels in its habitat and its own way, different candlestick patterns can be found to be effective under different situations. Change the gene pool a little and let the computer go at it. One day, your machine might find something to surprise you. 

As the computers become faster and faster, one day I should be able to include a large number of organisms. Using a multitude of selection/evaluation criteria, several species might evolve at the same time, just like an ecosystem in nature. I might be able to find, after numerous iterations, one pattern particularly good for a fiveday forecast while another one is found to have an excellent oneday forecasting ability. The possibilities are endless. 

EXHIBITS

BIBLIOGRAPHY

• Nison, Steve. Japanese Candlestick Charting Techniques: A Contemporary Guide to the Ancient Investment Technique of the Far East. New York, N.Y.: New York Institute of Finance, 1991.
• Deboeck, J. Guido, Editor. Trading on the Edge: Neural, Genetic and Fuzzy Systems for Chaotic Financial Markets. New York, N.Y.: John Wiley & Sons, Inc., 1994. Chapter 8 by Laurence Davis. Genetic Algorithm and Financial Applications.
• Note: All gene definition as described in Exhibit V-1 and the Excel Macro program Evolution written in Visual Basic are of original work, and therefore no further reference is given. The author drew on the knowledge and experience as an electrical engineering and computer science major during undergraduate years and seven years as an programmer/analyst to develop the program.