Technically Speaking, September, 2006

From the Editor’s Desk

This month we introduce Contributing Editor Molly Schilling, an MTA affiliate. In this issue, you’ll find Part 1 of her In-Depth interview with Josh Rosen. Molly did an outstanding job providing insight into the mind and work of a New York Stock Exchange Specialist with Kellogg Group. As a market maker on the floor of the NYSE, Josh is one of the people responsible for ensuring traders can trade. What and how he thinks may lead to insights that help us trade better.

During the May Seminar in New York I had the chance to ask Josh why the floor at every exchange is always gunning for my stops. He had a very Michael Corleone-like expression as he told me it was nothing personal, just business. Stops tend to be clustered and it’s his role to provide liquidity. He must assume that we want trades to occur at the stop prices, and he is only doing his job when he allows me to exit my long at the low of the day. He also pointed out that in the very short term, stop points usually represent good buy or sell prices. When you think about it, stops are support and resistance and as traders, we expect bounces at these levels.

From that conversation, I learned to place wider stops. The losses are bigger when they are hit, but they are hit less often. Overall, that short talk with Josh has helped me to become a little more profitable. And, conversations like that represent the largest part of the value of my MTA membership.

In coming months, Molly will be offering more insights into the minds that make the markets. Hopefully we can all learn a little from each of her skillful interviews. As always, I hope you learn a great deal from this issue of Technically Speaking.

Sincerely,

Mike Carr, CMT
Editor, Technically Speaking

What's Inside...

From the President’s Desk

Phil Roth, CMT

Greetings Members and Affiliates:

August is normally the slowest month of the year in the financial markets, and 2006 was...

In Depth Part I: Josh Rosen Interview

Josh Rosen, CMT

Molly Schilling

An Interview with Josh Rosen, CMT, a New York Stock Exchange Specialist with Kellogg Group, conducted by Molly Schilling.

NOTE:...

New York Chapter Report

David Keller, CMT

The members and affiliates of the New York Chapter have used 2006 to review a variety of technical approaches to...

Understanding Neural Networks

Adapted from “Developing Neural Models with Tradecision,” available at www.tradecision.com.

Even the simplest definitions of neural networks can be confusing....

From the President’s Desk

Greetings Members and Affiliates:

August is normally the slowest month of the year in the financial markets, and 2006 was no exception (for equities anyway, my area of specialization). I did manage to take some vacation, but things did not slow down much for the MTA.

The Market Technicians Association is a year-round business, and very different from when it began operation in the early 1970’s. We did not even have one paid employee at the beginning; we now have a full-time staff of five. As I write this message the position of Executive Director has not been refilled (our staff will increase back to six employees when it is), but we have made an offer to an excellent candidate, and I expect to be able to report to you next month that the new ED is on board. In the meantime, I have been functioning as ED, as well as President, and it has been business pretty much as usual.

The final preparations for the Long Term Planning Meeting on September 9 have been made and the meeting will have taken place by the time you read this. The committee will report to you next month on the outcome of the meeting. The direction of the MTA for the next few years will be the result. We will have discussed such issues as growth, the relationship of the MTA with other professional organizations, the management of the CMT certification program, and seminars and meetings, among other important topics.

The planning for our Mid-Winter Retreat in Florida is ongoing. It will be held at the Eden Roc Renaissance Resort & Spa in Miami Beach, Florida from January 18 to January 20, 2007. The format will be a bit different from last year. We will start with a Walk-About to get acquainted with one another. Facilitators will direct discussions on topics of interest to producers and users of technical analysis. We plan to cover all the markets (equities, fixed income, futures, commodities, currencies, options, etc.) in a fashion that should be useful to analysts, money managers, and traders. If you have any thoughts about potential topics or moderators, please let me know, and we will try to incorporate your wishes.

I visited Delphi on my vacation in Greece last month. Having been a market forecaster for over 40 years, I wanted to get some new and different insights from the Oracle. While she was not there, her spirit was. Here is her sibylic prophesy: “There will be another great confrontation between the Bulls and the Bears; the winner will be the purest of heart.” Hey, she said it; I’m just the reporter.

Sincerely,

Phil Roth, CMT
President

Contributor(s)

Phil Roth, CMT

Philip J. Roth, who holds the Chartered Market Technician designation, was the Chief Technical Market Analyst at Miller Tabak + Co. from 2001 until April 2012. Phil was a Wall Street professional for 46 years, and has been in the industry for...

In Depth Part I: Josh Rosen Interview

An Interview with Josh Rosen, CMT, a New York Stock Exchange Specialist with Kellogg Group, conducted by Molly Schilling.

NOTE: This interview will be printed in two parts in the September and October issues of Technically Speaking. Josh Rosen blogs daily at TheTrendline.blogspot.com.

Molly Schilling: So, Josh, here we are at the famous Bobby Van Restaurant on Wall Street after a difficult trading day (June 22, 2006). You work as a Specialist on the floor at the New York Stock Exchange — what a challenging job…What was it like today?

Josh Rosen: It was very interesting today – it seems that a lot of things are shifting right now in the market in general…Old adages aren’t necessarily working…The summer market was supposed to slow up — light trading and low volatility — we’ve seen low volatility in the summer for the past three years.

But, all of a sudden, we have a situation where the VIX is starting to pop up to its highest levels since 2003. And then we have a day like yesterday– I think it was five-to-one volume and three-to-one on the breadth numbers as far as stocks up versus stocks down. That gives a clear signal to jump on board — but today proved them very wrong.

Volatility is starting to explode. But traditional momentum indicators and oscillators weren’t pointing to that. We’re seeing a different situation — a situation that involves waves of volatility.

MS: A new paradigm?

JR: Yes, a new paradigm. Now I need to be prepared for much more volatility — I need to revisit all of my core ideas because, for example, the volatility necessary to retest those May highswould be explosive — also, the volatility would have to trend to the up-side… and volatility doesn’t tend to explode to the up-side as much as it does to the down-side.

MS: What kind of technical analysis are you using?

JR: I use chart analysis and candlesticks, a couple of different indicators, and I use Elliott Wave Formations — and some days I can’t believe how wonderfully and beautiful everything is working out from a technical viewpoint…

MS: Like a peak experience.

JR: Yes. And then there are other days when I have absolutely no clue as to what’s going on. Nothing seems to work. Luckily, I have incredible people working with me, trading with me. And yesterday was one of those bad days…I just said to myself at one point – “you know what — they got me… I just don’t know today.” And I stepped back — and I’m convinced that it’s important to have the ability within myself to do that.

But when its working, I automatically know where I’m right and where I’m wrong…all the techniques are working, and it’s just a big whammy – I mean, I don’t know how else to put it. There are days like that where it’s just a peak experience — where everything seems to be clicking in a daily framework, a weekly framework and an intermarket framework as well. It’s like everything comes together — all the hard work pays off. Even the intermarket work just makes a lot of sense, interest rate sensitive or not. And then there are those days when its not working – and that’s when you need to step away and say – I just don’t have it right now…being able to do that makes all the difference – ultimately, its not about the days where it’s all going well for you.

It’s when you can’t figure it out, and you can step back – that makes all the difference…and its the hardest thing to do…you’ve done all this hard work; you’ve spent hours and hours researching; you’ve come up with an incredible plan…and it doesn’t work out. How can you justify stepping away from that?

MS: It’s really hard…

JR: Nassim Taleb wrote a book called Fooled By Randomness, in which he says that its easier for a trader to divorce his wife than to divorce a position, and it is. You get married, as it were, to something that you put so much effort and time into… not to mention the ego investment! Psychology really interests me — it’s one of the things I really like to study and learn about when it comes to trading. I’m psychologically motivated; I’m motivated by the philosophy of trading, and the discipline that I think every trader works on for his entire life.

MS: When you get into the idea of the ego and the ego of the trader, is that a personal challenge for you – to understand and work on?

JR: It is. I think that you need to really look at yourself – I mean, everybody wants to say that they’re a disciplined trader who uses their stops and understands when they’re wrong…and looks for places when and where they’re wrong. But I think that over the years I’ve studied myself and how I trade, and I continue to err in the same way I have in the past. Traders are always looking for great reasons to get into something and rarely look for ways to get out – to exit.

Volumes have been written on the subject of ego in trading, but in the end, it seems to be all about experience. When you look at the people that have been in this business for a long time they tend to be people who learn about themselves. Trading will teach you a lot about who you are. I used to think it was devotion to trying to get away from your emotions… But as human beings we’re wired with emotions. So now, I try to understand the emotional aspects – that we are emotional beings – wired to be fearful, even panic-stricken — and understand when that’s happening and be prepared for it. The traders who tend to talk about this are superstar traders — Jim Rogers, George Soros…

MS: …Always working at developing self-understanding, self-control..?

JR: Absolutely.

MS: Interesting…my biggest weakness is patience…

JR: I’m full of good quotes and ideas about the market…You always grab something when you read and it kind of stays with you…And I read once that if you plan to be a market professional or trader for a living, you should expect it to be an extremely boring profession sprinkled with moments of panic.

MS: Moments of panic?

JR: Panic. Now if you explain that to someone in the profession they would say – no way. But this is not a profession of constant activity. What we really should be doing is doing our homework, our research, putting our trades on, understanding our ins and outs, our reasons where we’re wrong…and then we should be going for a nice walk.

MS: Interesting….

JR: And letting what happens happen. But we sit in front of machines constantly and we are constantly driven by what we hear in the media, what we hear by market pundits. You know, there’s nothing worse than being handcuffed to somebody else’s analysis, because you don’t know when they change their mind. You don’t know why they did what they did. You don’t know why they came up with their notions or if and when those are wrong?

So I try to do as much of my own homework as I can, and look at things with as critical an eye as I can. I like to look at analysts that don’t trade at all. Because they’re the first ones to admit that once you start trading a whole new game – many other elements — come into the picture…

MS: So, what is your job really like? What’s a day like…what do you do?…What time do you get up in the morning?

JR: I’m up at 6 a.m. and at Kellogg by 7:15 or 7:30. But I start the previous evening, looking at hundreds of charts – at sectors, at industries. So, when I wake up in the morning I revisit those charts in the morning. I check the news, and I often read a magazine or a book on the way to work. When I get there, I look at plenty of financial websites and world market news on the Internet.

MS: You live and work in New York City, so do you take the train to work?

JR: Yes, the NYC subway.

MS: I noticed that you have The Economist magazine with you…

JR: Yes.

MS: …And that’s a more heady, thoughtful, philosophical kind of –

JR: Absolutely. I really enjoy the editorials in The Economist. It gives a really bipartisan look at the global financial, economic and political markets and interactions. Everybody understands what the motivations of market news can be. It’s extremely scary how you can dissect the headlines, and how there’s so much underneath the headlines…

MS: You mean, they can be trying to force a direction one way or the other.

JR: Absolutely. They’re just so biased that it’s unreadable sometimes.

MS: Interesting…

JR: They come up with reasons for things… and as a technical analyst, I don’t always think those reasons reflect technical, fundamental, or economic sensibilities. I, frankly, enjoy the psychology of the market, and how the individual person is a player in these markets — it’s really not the news but our reaction to the news that moves markets…We’ve all seen how the news can send the market up or down 100 points…its because people always want some kind of a reason – it makes them sleep well at night. And it’s easy to rush to a publication or an internet site and find a reason why, for example, the bond market sold off 20 points…The problem is, that reason often doesn’t make good enough sense.

MS: Right, it doesn’t cloud your –

JR: Well, no, and I also look at it sometimes with just a crooked eye and make sure I know what’s going on, but I also want to take it with a grain of salt.

MS: So, you’re saying that you rely more on the technical reasons for things, rather than the narrative reasons, or storytelling, that newspapers come up with to explain things?

JR: Absolutely. I will take clear price movement any day over a news story. You know, Bob Prechter put out an educational piece on investor perception and news stories. He said “A genie is going to tell you the news for tomorrow, and you get to invest. But you have to hold the position for at least 24 hours once the genie tells you.” Essentially, he shows how people can be scared out of their positions, because most people can’t hold the position for 24 hours. But if you are following technicals, you would be much more apt to hold onto the position in spite of the news.

MS: So if you’d been following the technicals – as opposed to the news headlines –

JR: You wouldn’t necessarily be swayed by somebody’s opinion of the economic condition of the United States of America. I do enjoy reading various economists – if you pay attention to the household, it tends to throw a light on world economy and debt levels. If a household continually gets more credit cards, and keeps borrowing, and borrowing and borrowing, that type of leverage can continue to snowball.

MS: So, you pay attention to the financial ecology out there…

JR: I do — It helps me to know that markets have gone through similar periods in the past and have remained strong. In the end, though, I always look at how price is behaving. The market, itself, will eventually tell me what’s going on… I will let price and momentum and those indicators really steer the ship.

That being said, on a daily basis on the floor — what really steers the ship is order fl ow. As a technical analyst I look at what will happen in the next few days or weeks. As a Specialist – a trader on the floor I look at order flow for each stock as well as the different sectors and industries that my stocks are following.

Down there on the floor, I have the franchise in certain stocks; and my job first and foremost is to create a two-sided market — an auction market — to create a liquid market in the absence of other buyers and sellers, and to cushion the market on the way up by adding liquidity, and on the way down by buying stock. I don’t buy stock on the way up, I don’t sell stock on the way down. I don’t move markets. I bring the buyers and sellers together. I handle institutional and retail order flow for others, and I act as their agent.

One of my most important tasks is being a principal — which means adding capital to the marketplace in the absence of other buyers and sellers — and that happens quite often. We stabilize the market down — buy on the way down, sell on the way up.

How can technical analysis help…or fundamental or economical analysis? It makes us more knowledgeable. If everything that I’m seeing relates to a gold rally, I can prepare myself for a volatile day — more aggressive on the way down with the buying – and in creating long positions going up. I use technical analysis to give me a broad framework of the market, and I let the order flow on the floor tell me what to do.

MS: Now, order fl ow comes to you over a computer…?

JR: Yes…all the volume trades on the floor of The New York Stock Exchange –through brokers who handle large retail and institutional order flow, and through an inter-market trading system with secondary markets. And, through my electronic data display book located at my trading post. My volume tends to relate to overall volume in the market. I’ll know in my fingers whether I’m having a busy or a quiet day…

MS: How is the fl ow back and forth for you between electronic interaction and broker interaction? Does it become confusing?

JR: I work with a trading assistant who is extremely proficient. He can work the electronic book if I’m busy with the brokers. What’s going on in that book? Bids are canceling, offers are coming in, — market orders, limit orders, stop orders. Some of these guys that are down on the floor have been there for five, 10, 15, 20 years, and they don’t miss a beat — listening to everything that’s going on in the crowd, everything that’s hitting the tape, watching everything that is input… Every trade that hits the floor is printed through the display book.

MS: So this is supreme multitasking…

JR: Absolutely. You have to know everything that’s going on around you. You have to hear everything…Because a broker can walk into my area and yell “buy 50,000” of XX. How is that going to happen? It’s going to happen because we are paying attention…But, more importantly, all that means absolutely nothing without the most cardinal rule on the floor which is your word is your bond. You wouldn’t make it if you didn’t play by the rules. If a rumor got out – watch out for this guy – you wouldn’t survive on the floor.

MS: So there’s a lot of trust…

JR: When a broker says he’s buying, and I say that I am selling, and that’s inputted, then we go ahead and input our side, and they input their side, and that should all be meeting up overnight — and that is what we call comparing and clearing. Do miscommunications occasionally happen? Yes, but there’s a big difference between “things happening” and not standing by your word…and that just doesn’t happen on the floor. There’s a lot of integrity on the floor.

There are times when the entire floor is watching…If something is happening, everyone’s going to want to know what’s going on. When huge crowds gather, the big guns step in…people with the respect of the brokerage community. Every specialist firm has qualified people to handle big situations. There’s never a situation where a Specialist gets overwhelmed.

MS: You’re suggesting that there’s a kind of a trusting ethic down there. Is it a good feeling to walk onto the floor in the morning?

JR: I feel good about it; I enjoy what I do. I enjoy studying the markets, I enjoy trading. There are some people on the floor that I absolutely can’t wait to see, to see how their night was and spend some time talking to them, and they’re a joy to be around. Even though I’ve been doing this for a while, I started on the AMEX in 1997, traded there for about three years; traded on The New York for a total of about four years; traded upstairs for a while – I still have been there for just a fraction of the time that a lot of people have been down there.

And like anything else, the more time you spend down there, the more operations you’re involved with, the more situations you’re in, the more you meet people. And I’ve been lucky enough to meet a lot of people. But like I said, especially for the specialists, we stand at a post all day, you don’t move around as much as some of the broker’s community does who’s handling a lot of different orders in a lot of different places. So, I’m sometimes limited to my area.

You know, it’s funny, but there are a lot of rooms on the floor – main rooms, and side rooms, and they’re all doing the same thing, all trading stocks. You could be in the main room for four years and then move into another room, and all of a sudden you see people you haven’t seen for years. So it’s a large community, there are an awful lot of people walking on the floor as far as The New York Stock Exchange surveillance, administration, supervisors, brokers, different specialists and clerks. It’s just like any other major business.

But there is a sense, no matter what, that if a lady or a gentleman walks into your crowd with a badge on and is a licensed holder on the floor of The New York Stock Exchange, there’s no question that they are a person of integrity and the relationship is already bound by that. We’ve been through the same examinations, rigorous examinations, we all understand the rules and regulations, we all understand what it means to bid an offer and to be party to a bound trade, so that’s never a question.

MS: Interesting. So from your experience there’s a very high ethic operating all the time.

JR: Has to be, and I think it’s incredible and it’s wonderful that we get to work in an environment where that’s so prominent. But I think it needs to be in any corporation, in any business. But in our business we have a fail-safe: if you don’t exercise that type of integrity, that type of character then you’ll be found out immediately.

MS: That’s great, so you like your job.

JR: I do, I like trading …

MS: Thanks, Josh, we look forward to continuing this conversation in Part II of our interview with you, next month in Technically Speaking

Contributor(s)

Josh Rosen, CMT

Josh Rosen, CMT is a Senior Domestic Trader at Driehaus Capital Management and Driehaus Securities.

Molly Schilling

Molly Schilling is an independent trader and freelance writer. Molly has been a member of the MTA since 2005.

New York Chapter Report

The members and affiliates of the New York Chapter have used 2006 to review a variety of technical approaches to market analysis. In one of the heaviest attended sessions in recent memory, Arch Crawford spoke in May about astro-finance and its implications for equities and commodities. As Arch started in the field with the original technical analysis team at Merrill Lynch, his experience and history in the markets caused us all to pay special attention to his comments. In June, Tom Keene shared insight gained from his daily appearances on Bloomberg Television and Radio. He reflected on the relationships of technical analysis to fundamental, political, and economicevents. Next, Jeff Weiss stressed the importance of using trend lines and moving averages from the monthly charts as a foundation for any investment strategy. Dr. Marvin Appel took the stage in August, and demonstrated how a basic trend-following strategy with indexes and ETFs could prove quite profitable in the long-term.

Looking forward to the rest of the year, we will continue our exploration of technical strategies. Joerg Schroader spoke to our chapter in November 2000 on his uses of Elliott Wave. In September, he will return to evaluate the performance of his strategy since then, and present some new proprietary algorithms he has developed. Connie Brown led an informative discussion on oscillators at the 2005 MTA Education Seminar in New York. She has agreed to attend our October meeting to lead a lively discussion on the changing face of technical analysis. In November, David Aronson will present some topics from his upcoming book on applying the scientific method to trading strategies. We will finish out the year with Donn Fishbein, whose work with neural networks should give attendees plenty to consider.

I encourage all members and affiliates in the tri-state area to attend these upcoming meetings. These sessions are designed to be interactive, so the more people that attend, the more effective the conversations. Also, remember that one of the most important benefits of this organization is the ability to meet with and network with other technical analysts and traders. Only by actively participating in these meetings can you help to further the understanding of technical analysis in our industry. If you are located outside the New York region, please let me know if you’re ever traveling to the area. At the very least, I would be happy to introduce you to our city! I’ll look forward to seeing you all at the mid-winter retreat in Miami. Regards, Dave.

David Keller, CMT
Chair, New York Region

Contributor(s)

David Keller, CMT

David Keller, CMT is President and Chief Strategist at Sierra Alpha Research LLC, a boutique investment research and consulting firm focused on helping investors manage risk through market awareness. He combines the strengths of technical analysis, behavioral finance, and data visualization to empower...

Understanding Neural Networks

Adapted from “Developing Neural Models with Tradecision,” available at www.tradecision.com.

Even the simplest definitions of neural networks can be confusing. For example, “Neural networks learn to associate values in the output column with the underlying patterns they detect within historical input data.” A neural network is trained on past data to find patterns (inter-dependencies) between the input features and desired target. After an artificial neural network is trained, it uses historically detected patterns to make future forecasts. When you update the data in an already trained neural network or feed new data into it, the network starts looking for patterns that are similar to those it has already found to be associated with certain price movements. After that, it makes a forecast based on the existing similarities with the known price patterns. Neural networks are not a technical indicator, fundamental valuation, regression algorithm or magic super-technology.

The main advantage of neural networks is their ability to find subtle non-linear interdependencies and detect patterns within these dependencies, which are otherwise almost impossible to detect visually or using traditional statistical approaches. Neural networks are truly one of the best methodologies ever invented for the prediction and forecasting of historical time series data.

To make neural networks work for you, you must prepare input data that contains variables that will allow detecting underlying patterns. The rest of the tasks involved in making an accurate prediction (i.e. preprocessing, architecture selection, intelligent control over network training, and so on) will be performed by the software. As a practical guideline, do not select only technical indicators to describe a symbol’s price history when you know that your stock is driven primarily by fundamental valuation ratios or factors. Additionally, do not use neural networks with stocks that show changes in their behavior and dramatic changes in the related influencing factors.

In general, a neural network consists of an input layer with input units, one (or several) hidden layers with hidden units, and an output layer with one (or several) output units. The underlying neural network architecture determines how the network units are connected and how information is processed in each unit. The training algorithm determines how the network weights (values assigned to each interconnection between the units) are calculated during the network training. Preprocessing and partitioning methods define how the data will be converted into a form suitable for neural networks. And, finally, “generalization improvement methods” will define how to improve the network’s forecasting ability. We do not provide herein the formulas and other technical details of these methods.

To “learn” the input data variables, the neural net finds patterns by tuning its weights (and architecture) during the training phase. To tune its weights and architecture, neural network compares its output with the correct value (the actual value from the dataset) during each iteration, and adjusts its structure in such a way that it rewards itself for becoming more precise in predicting the output. In this way, the network trains to the point when further adjustments stop producing lower error rates and the generalization ability can no longer be improved. If there is not enough data for finding the majority of the underlying patterns, or if the old patterns are now obsolete and they will not work with the new data due to a changing market environment, the network will not be able to produce good forecasts. However, the following should be mentioned: neural networks excel in their robustness, pattern recognition abilities and capacity to produce good forecasts for new data that has never been analyzed before.

Using a Neural Network

The process of using a neural network for forecasting can be divided into two stages:

Data preparation and network training using historical data;
Forecasting with a trained network using new data (i.e. Real-time stream or EOD update).

The first stage can be divided into several tasks:

Input dataset analysis: data type definition (i.e. numerical, categorical, text, data/time, and so on), adequate handling of missing values and outliers, feature input(s) selection and noise reduction.
Data preprocessing: transferring source data into a form suitable for neural network management and utilization.
Network design: selection of network architecture, the training method and training parameters.
Network training.
Performance testing.
Network improvement(s) based on actual test results: the network can be re-trained with other inputs, architecture, training method(s) or parameters.

Using a ready network is a less difficult and time-consuming process than its preparation. A data set that has been preprocessed according to the same principles and methods as during the first stage is fed into the network. After getting an input signal, the network generates an output signal, being the forecasted value after post-processing. It needs to be mentioned that network preparation and training, generally, require a significant amount of computer/CPU cycle time. The quality of the resulting forecast depends strongly on the training data selected, representation of input features, preprocessing, as well as on the architecture and the training algorithm’s fine-tuning. In any case, the amount of the training data should be sufficient to allow solving a problem of higher complexity.

Selecting Target

When you create a model, you need to select a market parameter (data column) that should be predicted by the model. This parameter is called target. It may be the Percent change, Close, Open or any of the available technical indicators such as EMA or RSI. Selecting a technical indicator as the target is also helpful, especially if you intend to forecast the market volatility, typical price, moving averages, price momentum or market strength. You can also create a composite indicator to use smoothed or de-trended representation of the market. The High or Low values contain a lot of noisy movements and outliers. Therefore, a model targeted toward forecasting High or Low would be of rather poor quality.

Unfortunately, price data and most indicators contain noise in their values. This is one of the reasons that contribute to the poor model performance. To reduce the noise effect, the end-user can employ a moving average of the Open, Close or indicator as the model target. When employing the target of a moving average of price or a technical indicator, you should keep the Period parameter of the moving average lower than the model Lookahead parameter.

Selecting the Right Amount of Input Data

The key to success in neural network prediction is choosing the correct combination of predictive inputs. Selecting those variables that are to be included in your neural network is a crucial task which determines whether or not your neural network will work properly. Too many inputs will cause the net to over-train. The net will learn the training data very well, but it will perform poorly on out- of- sample examples (your walk-forward data set). For the majority of cases the number of inputs that should be used is between 3 and 10.

The correct input feature set depends on the trading instrument. In the majority of cases, you should use the moving averages or Change/ %Change and not Open, High, Low, Close or Volume as inputs. Try to find inputs that characterize the stock from different points of view so that the neural net will have less difficulty in modeling. For example, chose technical indicators from different categories (i.e. volatility indicators, volume indicators, verticality indicators, velocity indicators, cyclical indicators) and hybrids thereof. In many cases, Momentum indicators, such as Stochastics, PercentR, Williams AccumulationDistribution, CCI and RSI are good choices. Other adequate choices include Volume indicators, such as Accumulation-Distribution, MFI, and OBV.

Selecting appropriate inputs is the most important task the user is faced with. Inputs are the only kind of knowledge about the market that will be provided and made available to the neural network to enable it to determine the best model on which the future forecasts will be based. All the time series data which as you believe significantly affects the value of the target column or represents the current market situation should be used as input data.

For example, to forecast what the closing price will be three days from now, you could train the model using the price data, lags and other indicators as the inputs and closing prices as the target. This includes the price data (open, high, low, close and volume), technical indicators (moving averages, lags, MACD, RSI, PercentR, and so on), market indices (S&P500, Dow Jones Industrial, NASDAQ, and others), fundamental data (Interest Rates, Unemployment Rates, Consumer Price Indexes, and so on), as well as the relevant correlated stocks and other information, representing the market conditions.

Irrelevant or insignificant input selection may deteriorate the model’s performance. Additionally, the more inputs you have, the more historical data you need to provide to train your model. With more historical data, you will need more time to train your model, running a greater risk of curve-fitting or over-optimization. Curve-fitting occurs when a model simply memorizes the training data, which results in the model’s lower generalization ability and a larger forecasting error.

For example, you should not add Momentum if you are forecasting in a market exhibiting a sideways pattern. In most cases, this will have a minute impact on your model or may even worsen it. However, do not forget to include oscillators (such as Stochastic or RSI), as they perform well on sideway markets.

If the number of price patterns in your historical data is too small, your neural network will not have enough information about the market to train itself correctly. For daily bars, it is recommended that one have at least four (better 6+) years of price data (1000-1500+ daily bars) for adequate network training, and at least six (better 12+) months for walk-forward testing. Providing too much data can increase a neural model’s training period but it will not improve the quality of the forecasting. Additionally, the old detected price patterns will not be valid for the current market situation. Therefore, to reduce the input dataset you need to remove the oldest data.

Selecting Model Test Period (Walk-forward Test)

The current model coupled with your model-based strategy should be tested on out-of-sample data (walk-forward period). Out-of-sample walk-forward testing will achieve the results that simulate real-world trading using this model. The more data you select for the testing, the more optimal is the number of the patterns that are made available for testing the model, and, therefore, the more confident you can feel about your newly derived test results. At the same time, the more data you select for the testing, the fewer price patterns are available for the model’s training. With less data, the model training will be less efficient. It is recommended that one have at least 100 bars of data for testing.

Preventing Curve-Fitting

The biggest mistake made by users of artificial neural networks is that they tend to over-train their networks. Curve-fitting is a dangerous shortcoming of neural networks, resulting from over-training. When a network over-trains, it does not “learn” price patterns or develop the ability to generalize, but simply memorizes historical price data without learning any internal dependencies.

An over-trained network can produce good results with a training set, but performs unexpectedly or poorly when used with new data.

Technical Analysis vs. Neural Nets

Neural network approaches are quite different from the classical technical analysis techniques and, therefore, require time and effort on the part of the trader who wants to master them. Neural networks can process huge amounts of data and detect trends and underlying price patterns that cannot be easily seen by humans while observing charts or discovered using traditional technical analysis methods.

Neural networks learn from price data and discover complex non-linear dynamic patterns in price movements. Unlike the traditional indicators, neural networks have the ability to learn from the data itself. Neural networks learn the patterns of price data. Routine standard analysis techniques usually assign a model form to data and then test it to see if the data fits the assigned structure.

A typical trading system often works for a short while until the market situation changes and/or the system performance deteriorates. A neural network-based trading system can be retrained every time you integrate or update your data. This allows the system to adapt to new market conditions.

Some General Aspects in Application of Neural Networks

Your main task is to select the correct or proper inputs. The rest of the tasks, such as preprocessing, architecture selection, GA optimization, training and others will all be performed automatically by software.
Don’t rely on a single model. Build several models based on different ideas, i.e. whether the trend will continue after a certain condition is met, using different types of market phases, and so on. Combine these models in one neurostrategy. Direct price forecasting works mainly for sideway markets, and often with momentum indicators only.
Do not select multiple inputs that contain the same basic underlying information, such as the Close and the exponential moving average of the close. It is much better to expose a neural network to some different aspects of the market.
Remember that a neural network analyzes only one row at a time, so if you want the network to consider yesterday’s Close value, add a lagged closing value as an additional input. A neural network uses one row at a time to adjust its weights. After the adjustment is made, the network “forgets” about this row and proceeds to the next one.
It’s often beneficial to add market and industry indices.
Use only walk-forward test figures to analyze your model. Do not use the model in trading until your strategy performance report shows good figures for the walk-forward data range.
Add indicator-based filters to your neural strategy. Sometimes, simple techniques can significantly improve advanced mathematical modeling.
Prepare to spend significant time on building your neural model.
Do not stick to just one favorite stock. It is easy to build a good model for some trading instruments, and, oftentimes, it is almost impossible to model other symbols. In this case, you have not simply taken into account those influential factors that are necessary to build adequate models for these relatively “unpredictable” stocks.

Introduction to Genetic Algorithm

Another AI (Artificial Intelligence) technique that has proven performance gains and excels in optimization tasks is Genetic Algorithms, are search algorithms based on the mechanics of natural selection and natural genetics. They combine the survival of the fittest rule with a structured yet randomized information exchange. The method uses terms accepted in genetics, such as fitness, population, generation, mutation, gene, and so on.

In contrast to random search methods (such as, for example, x Carlo) genetic algorithms are not a simple random walk. These algorithms efficiently use historical information to speculate on new search points with expected improved performance. Their goal is forming or finding a population of trading strategies that will have the best fitness level, or, in other words, the best optimization criteria values.

Genetic algorithms possess the best characteristics of the other optimization methods, such as robustness and fast convergence, which does not depend on any of the optimization criteria (for instance, on smoothness).

Although genetic algorithms are much faster when a large number of possible options are searched, they are slower than exhaustive search as far as a small search space is concerned. Genetic algorithms are useful in maximizing or minimizing an objective function within a set of constraints. They are especially efficient when the relations are non-linear and/or discontinuous. This approach is also used to optimize neural model inputs and their parameters.