So over the past 12 months I have noticed a degradation in the models I use - wasn’t sure why and was too tied up at work to really investigate closely. So I recently re-ran the ranking performance tools on the ranking systems I use, split this up into first 20 buckets, then 100, and looked performance over the last month/6 month/12 months. What became obvious very quickly was that the problem was not the buy/sell rules but the underlying ranking system. After spending a lot of time testing my rankings a few years ago, I thought they would hold up over time (yeah, I know… ). What became clear to me is that the ranking position (#1, #2, etc) was far less important than the rank performance (last bucket, one month period).
Is there any way to incorporate ranking performance (bucket, timeframe, benchmark) into a model for buy/sell rules?
What I really want to do is if the ranking performance is worse than the benchmark (for the given time period), then shut down the model.
Thanks,
I don’t know of a way to do this. But I would strongly caution you against abandoning a ranking system purely on the basis of underperformance over a short period of time. There is no such thing as a ranking system that always works. Even the very best ranking system will underperform over certain periods.
Try this. Take a ranking system that you have a great deal of faith in, one that still works for you. When you do a performance test of it, use the button by “Chart Type” that says “Performance” rather than the default (“Annualized Returns”). Download the result–it’ll be a CSV file. In another column, calculate the returns of the top bucket minus the bottom bucket, or the top bucket minus a middle bucket. The formula might look something like this: =L16/L14-C16/C14, depending on which columns you’re comparing and how many weeks or months you want to look at. Go down the list and look for negative numbers. You’ll find them. Sometimes you’ll find them lasting a lot more than a month. Every time a negative number comes up, that represents the top bucket underperforming the bottom or middle bucket.
Yuval, thanks for the feedback. I did look at performance charts as well as the performance "buckets, but did not download. I will do that and take a look as you advised.
I wasn’t considering abandoning the ranking systems. I have used benchmark performances (S&P, sectors, etc) as part of the “buy” rules, so if the market is heading in the wrong direction, go flat. What I was looking to do is add a buy (or) sell side rule to go flat if the ranking performance drops below a threshold.
FYI: My membership level gives me data from 2011, so I cant see how the ranking systems performed during the 2007 to 2009 period, but outside of that, they have been acceptable. My problem, but something I also need to consider.