| Index | Recent Threads | Who's Online | User List | Search |
|
|
 |
Portfolio123 » List all forums » Forum: General Comments » Thread: BLOG: A Change-Of-Pace Growth Strategy |
|
Total posts in this thread: 36
|
[Request a Feature]
[Post new Thread] [Add To My Favorites] [Watch this Thread] |
| Author |
|
|
Stittsville123
Advanced Member
|
Josh - I agree with your statement(s) about more factors being better than fewer. I think I made a post or two about that a long time ago on P123. What I said was "in this environment" it is better to keep it simple. The rest of your post hurts my brain too much so I'll stop there. Steve |
||
|
|
grokkalot
Advanced Member
|
We're all writing comments without an editor, so it's understood that every thought may not come across clearly, and I'd be happy to elaborate on any particular point. Part of what I was trying to say is that folks are giving a lot of opinions based on I-don't-know-what principles about how to be robust in the face of a prediction problem where the future may well not look like the sample data. That's an inherently ill-posed problem, but it's still better to be explicit about what sort of logic stands behind what somebody is saying. To that end, I didn't (so far) catch the idea behind your thought about the interaction between the current environment and model simplicity. But my remarks on simple vs. complex models above were responding to a common theme in many comments at P123; some of them, if I recall, are in the introductory materials for the site. |
||
|
|
Stittsville123
Advanced Member
|
"I didn't (so far) catch the idea behind your thought about the interaction between the current environment and model simplicity." Skipping the statistical explanations - the idea behind large ranking systems (at least for me) is that if one factor falls off it's horse then the remainder of the factors/formulas will hold up the performance. I buy into this reasoning. However, what we have here isn't one factor falling off it's horse. We have the horses riding the jockeys. When you don't understand the market behavior the obvious thing to do is fall back to something simple that you feel confident in. That and hedge to the maximum. Enterprise Value / EBITDA has been used by stock analysts for a long, long time and from what I can tell this formula (and variants such as the one I used) is head and shoulders above any other factors or formulas around. That combined with short interest makes a powerful combination. I will demonstrate these statements in another post when I have the time. Q1) Is a strategy that backtests well against 5 years of data likely to perform better than a strategy that backtests well against 1 year of data? OK - I've given the Advil some time to circulate in my system now. A1B) The answer is that the strategy that backtests well in BOTH 1 year and 5 year data is better than either of the two options offered. And it should backtest well in a variety of sectors and markets. Steve |
||
|
|
Stittsville123
Advanced Member
|
"Results of top 5 of 200 buckets (Aug 2007-present)" Brian - Everyone has their own stress testing ideas that they believe in. I personally have issues with analyzing 200 buckets particularly on 1 year data. For starters you have no visibility into what is actually in the buckets. (You may think that I have lost it but) I don't fully comprehend how the stocks are assigned to buckets. Someone correct me if I'm wrong but the $3 minimum price and some volume filter are post-ranking. Also all stocks with same rank will be put in one bucket and the next bucket down will have fewer stocks. Therefore it is a big assumption that there will be x number of stocks in any given bucket. Also, by slicing that fine (2oo buckets) I suspect you are too much into the trees and not looking at the forest. My approach is exactly the opposite. I reduce the number of buckets. Then the bigger picture is more clear (to me). I often use 4 buckets to get a better assessment of how a ranking system really works. Then I run the ranking system on each sector to see the performance. Below is ET run from July 28th 2007 to present, one year conveniently starting with the stock market turmoil. I generally like to see that the top bucket (25%) outperforms better than the bottom bucket (25%). For the one year period this was true in all cases except for the energy sector. Anyone looking for a long/short strategy could do very well with this ranking system. Using 25% top and bottom gives lots of headroom for applying additional rules or filters. You can also see from these graphs that the conservative sectors that investors tend to like during a recession did OK - consumer staples, health and utilities. Steve. ---------------------------------------- 
Sectors.gif
(39376 bytes)
(Download Count: 79)
|
||
|
|
mgerstein
Advanced Member
|
Q: Is a strategy that backtests well in a 5 year test likely to work better than a strategy that testes well in a 1 year period? A: Maybe. Maybe not. It depends on (a) what one is looking for (how long one wishes to wait before seeing performance come into line with expectations), and (b) the nature of the sample period. For example, if one wants to achieve performance right now, in the current environment and if one assumes the current conditions will persist, I would not want to go back more than 2 years and may be happy with something less. A five-year test would include a lot of data from a period when energy prices, financials,and housing weren't as troublesome as they are now. It would also include the time when these were hot-button issues. It would be an awkward mix. This is where the art of system design competes with the science. The above paragraph reflects the science. The art comes in determining whether the recent conditions are really in the process of changing. If so, a 1-2 year test could turno out to give misleading signals. Maybe now is the time to shift to a longer test period,or even a custom eriod (say 2004-05). And, of course, we always have to stay alert for new conditions unlike anything seen in the historic data. That's why we need to keep sight of the normative (what common sense tells us should work) instead of going too far with the empirical (what we see actually had worked); i.e. the dangers of curve fitting. Q: How many buckets and how concerned should we be with rank order among buckets? In my opinion, 10 is as far as I'd want to go, and I often like to see 5-bucket tests (which I always look at, even if I don't use those charts for blog screen shots). To me, it's simply a matter of how far one wants to push the principles of probability. In a 4,000 stock universe, for example, it's one thing to say stocks 1-400 should, on average, outperform stocks 401-800. But it's a very different matter to suggest that stock # 187 should outperform stock # 188, or even that stocks 1-20 (the top bucket in a 200-bucket system) should outperform stocks 21-40, or even that stocks 1-40 should outperform stocks 41-80. In a 200-bucket test, it's not reasonable to expect all the buckets to falll into place when considering performance rank order. It's a lot more reasonable in a 5 or 10 bucket test to want to aim for that. |
||
|
|
grokkalot
Advanced Member
|
"Skipping the statistical explanations - the idea behind large ranking systems (at least for me) is that if one factor falls off it's horse then the remainder of the factors/formulas will hold up the performance. I buy into this reasoning." That's basically the same as the statistical idea. If a ranking involves a lot of different factors with smaller weights, then it is statistically more robust to variation in the efficacy of individual factors. "However, what we have here isn't one factor falling off it's horse. We have the horses riding the jockeys. When you don't understand the market behavior the obvious thing to do is fall back to something simple that you feel confident in." I agree with the general idea, though not with the specific recommendation. One of the most important ingredients for investment success seems to be to have confidence in one's positions up until the point that evidence shows your idea was wrong (and then exit quickly). "That and hedge to the maximum." Finding a hedge that you have confidence will work as a hedge is easier said than done. "Enterprise Value / EBITDA has been used by stock analysts for a long, long time and from what I can tell this formula (and variants such as the one I used) is head and shoulders above any other factors or formulas around. That combined with short interest makes a powerful combination. I will demonstrate these statements in another post when I have the time." Look forward to it (I also make use of versions of EV and EBITDA, as discussed elsewhere). One problem with this value approach in the recent market is that the most undervalued stocks are typically small, and one a small stock starts in a downtrend the recent market behaves as if it doesn't believe in any of the numbers going forward. "A1B) The answer is that the strategy that backtests well in BOTH 1 year and 5 year data is better than either of the two options offered. And it should backtest well in a variety of sectors and markets." Everyone would agree that other things being equal that is better, but I thought the recent topic was quite focused on the question of whether it is better to give up some recent performance in backtesting for a better longer term track record. Probably everyone would also say it is advisable for some values of recent and longer term (again, other things being equal). So it all boils down to how to quantify various evidence for relevance. My ideal is probably to due some sort of exponential weighting (value of evidence decreases as time goes back) and do a meta study to figure out a good value for the exponent. |
||
|
|
grokkalot
Advanced Member
|
Strongly disagree with what is being posted about the number of buckets to use. In the first place, the optimal number of buckets relative to any statistical definition of bias/variance depends on the size of the universe being used to fill the buckets. Clearly, it doesn't make any sense to debate about 10 vs. 20 vs. 200 vs. 1000 buckets in the abstract - almost any definition of optimality depends on the number of data points going into each bucket. To go much beyond that, we need to know something about how the bucket bar graph is going to be used/understood/evaluated. If someone plans to be fully invested at all times holding exactly N more or less equally sized positions, then maybe it makes sense for them to pick a bucket size so that their portfolio always fits comfortably within the top bucket. But that's not how I work. In terms of pure information extraction, and the way *I* use the graphs I can say this: if I am planning to use a ranking to pick N stocks, where N is always less than or roughly equal to the number of stocks in the smallest bucket granularity I can create on P123 (based on 200 buckets), then I'm mainly going to look at the finest granularity so long as the number of stocks per bucket is at least 10. So If my universe is 8000 stocks, I'm pretty much always going to mainly use 200 buckets. If my universe was just the S&P 500, I wouldn't want more than 50 buckets. But if I did use 50 buckets for the S&P 500 (10 per bucket) then my evaluation function for the quality of a ranking would certainly/obviously look at more than just the avg. performance for the top bucket. If any advocates for smaller number of buckets actually disagree with the above, then I'd like to ask them to propose some concrete examples for discussion that meet the following criteria: They are comparing two ranking systems using some explicit ranking evaluation criteria, using a smaller number of buckets leads them to prefer ranking R1, using a larger number leads them to prefer ranking R2, and they have good reasons to believe that R1 is a better choice than R2. By looking at one or more concrete examples like the above, we could better figure out if the disagreement is about how to use the bucket graphs, about which ranking is truly better in the example, or something else (e.g. some sort of idea about robustness). |
||
|
|
Stittsville123
Advanced Member
|
Josh - I have confidence in my hedging strategy. Ultrashort ETFs have worked quite well for me in the past. The trick is to select an Ultrashort ETF that is a good match for one's particular holdings. " ...hold a strategy until evidence proves your idea was wrong. .." I did see a study some time ago that indicated the most successful investors are those that have a low tolerance for losing positions (dump quickly). Every one has their own trading style. I am definitely a shorter term trader with a low tolerance for things that aren't working. I am fully convinced that the logic behind need for long term strategy originated with financial advisors who receive large commissions. The only way they can justify their large commissions is to make a case for long term strategies. The specific K.I.S.S. recommendation was based on the fact that many of the stock factors are not working well. I'm not sure it makes sense to use a complicated ranking system when many factors are standing on their ears. If it was just one factor then I would agree that sticking with one's system is a good idea. For me short term is more important than longer term - I prefer to ride the current trend. But it all depends on what the trend is. I wouldn't bet against P/S Ratio for example even if it isn't working correctly for a period of time. Steve |
||
|
|
olikea
Advanced Member
|
And I strongly agree with what is being posted about the number of buckets to use.
OK. A little background. When I first joined p123 in 2006, I created my first ranking system based on what I knew. I started a paper portfolio set to autotrade, you can find it here. Naturally, I was dissapointed by the real time performance, so I tried to figure out why it was failing. In my opinion, explaining the poor performance is crucial to understanding and developing systems in the future. One thing I noticed, while the portfolio failed to perform in real time, the ranking system, at a 20 bucket resolution worked fine in the period after it was developed; the top bucket performed best. My portfolio contained 20 stocks. Since each bucket contains considerably more than that, I hypothesised that a portfolio with more stocks would have performed better out of sample, and this was true. This is all discussed in a post called "Review of my first generation ranking" **** Onto ranking systems. If you consider the returns from stocks, they are highly variable. In fact, you could say they are noisy. The more stocks you collect together, the more the stock specific noise cancels out. If you imagine a factor such as EV/EBITDA to be like a signal, what you want is to attain a high signal-to-noise ratio. You want to do this both in backtesting (to ensure you have found a real relationship) and in real time (to ensure you enjoy the benefits of the signal). The only way to do this is through a large number of stocks or a large number of trades. I made a post "Benefits of diversification" which shows the results of ranking "nothing". This is like a scientific control experiment, and you see the variablility of the buckets increases dramatically as the number of buckets is increased. This supports my original statement, that adding more stocks reduces the noise. I now optimise based on 20 buckets. Optimising to 200 buckets is not a good idea in my opinion, because you are likely to be optimising to noise, and if you run a small portfolio in real time, noise is likely to decimate the returns. However, since having written the prior posts, I have made some further discoveries: Microcap stocks (mktcap < $200m) are the greatest source of this noise, and if you include them, then the minimum diversification level is 50 stocks. If you exclude them, I believe 20 becomes acceptable. Originally I was suggesting far more stocks, such as 200, but I now believe that is not necessary provided the ranking system is diversified - namely that it contains several unrelated factors. My original first generation ranking was not diversified, so if a single factor goes through a "factor reversal" then you will be staring down the gun of some unpleasant losses. ---------------------------------------- [Edit 1 times, last edit by olikea at Aug 1, 2008 10:39:05 AM] |
|||||
|
|
grokkalot
Advanced Member
|
"I am definitely a shorter term trader with a low tolerance for things that aren't working.... The specific K.I.S.S. recommendation was based on the fact that many of the stock factors are not working well. I'm not sure it makes sense to use a complicated ranking system when many factors are standing on their ears. If it was just one factor then I would agree that sticking with one's system is a good idea. For me short term is more important than longer term - I prefer to ride the current trend. But it all depends on what the trend is. I wouldn't bet against P/S Ratio for example even if it isn't working correctly for a period of time." If TA/trend factors are going to play an important role in one's trading then I don't see a good reason not to backtest performance using those factors in one's ranking system - whether you decide if you want to backtest for the last five years or the last five weeks, it seems to me that would still be true. |
||
|
|
[Show Thread Printable Version] [Post new Thread] |
