| Index | Recent Threads | Who's Online | User List | Search |
|
|
 |
Portfolio123 » List all forums » Forum: General Comments » Thread: BLOG: A Change-Of-Pace Growth Strategy |
|
Total posts in this thread: 36
|
[Request a Feature]
[Post new Thread] [Add To My Favorites] [Watch this Thread] |
| Author |
|
|
Stittsville123
Advanced Member
|
Josh - I don't understand your point. The only way to do a meaningful long term backtest on short term trends is to have a dynamically altered ranking system. We don't have that capability as far as I know. ---------------------------------------- [Edit 1 times, last edit by Stittsville123 at Aug 1, 2008 11:14:01 AM] |
||
|
|
grokkalot
Advanced Member
|
Oliver, your post above, like almost all of your posts, is super interesting. But I just don't see the connections that you do between the evidence you are presenting and the argument you are making. I can't approach the familiarity that you have with your ranking systems and sims in a short time, but here is how I understood your example: You started with a ranking system in 2006/2007 that had very good backtested performance (in fact I see that the 5 year raw backtest performance including all the small illiquid stocks is still good - http://www.portfolio123.com/rank_perf_show.jsp?StartDate=07%2F26%2F03&EndDate=07%2F26%2F08&freq=1&univ_list=0&naselect=0&no_buckets=200&min_price=1.0§or=0&chartType=bar&width=500&height=500&generate=Show+Graph&universe_val=All+Stocks§or_val=ALL ) But you found that the performance of a 20 stock portfolio based on that system was poor in 2007. However, you found that as you expanded the portfolio to 50, 100, 200, etc. stocks the performance of the portfolio (absent transaction and slippage costs) converged to the performance of the screen that you expected, which was still pretty decent in 2007. Is the above a fair summary so far? If so, what I'm not getting is the big leap from there to the argument that it would have somehow been better to look at the performance of the original ranking system using only 20 bins. It sounds like you must be saying that you should have selected some different ranking system which you would have evaluated more favorably if you just looked at 20 bins. Is that right? Can I get more detail about what that other system is and how you think the two systems should be compared using both 20 and 200 bins? Of course it is true that an average of 100 things has lower variability than average of 20 things, and that an average of 20 things with low variability has overall lower variability than an average of 20 things with higher variability, but I don't see where those facts directly relate to the argument about whether to look at 200 bins or 20 bins or whatever when examining the ranking performance. |
||
|
|
grokkalot
Advanced Member
|
Many P123 ranking systems include trend and other TA factors. They work with systems that only need daily H/L/C/Vol info. They don't work with day trading systems. If you meant above that you focus on intra-day price trends and what's working intra-day then I simply misunderstood you. And if you were actually talking about performance trends for stock factors rather than price trends of stocks, then again I misunderstood what you meant. But conventional price momentum/pullback factors work fine in screens and many people use them. |
|||
|
|
olikea
Advanced Member
|
I basically created further ranking systems based on 20 buckets, for the more conservative portfolios, e.g. Fourth Generation. Interestingly, at a 200 bucket resolution, 4th gen is less good than 1st gen, but at a 20 bucket res it is better. I must admit, the temptation to "really go for it" has been there, so my playground is a series of ranking systems called "Extreme ranking", where caution is thrown to the wind. However, they are highly speculative and only meant for "play money". The diversified approach works well with more conservative portfolios. ---------------------------------------- 
extremrev3.png
(19349 bytes)
(Download Count: 88)
|
|||
|
|
DennyHalwes
Advanced Member UNITED STATES Joined: Apr 28, 2004 Posts: 1532 Status: Offline |
All, I have to agree with Josh on the discussion of number of buckets to use. I have been a proponent of following many independent Ranking Systems using 3 to 5 stocks in each. I bring to the argument the example of Filip’s SuperValue Ranking System, one of the most powerful public systems. The first chart below shows his system’s performance using 200 buckets, over the full database, pre-filtered with AvgDailyTot(60) > 200000 & Close(0) > 1. It is rebalanced weekly, and I use Minimum Price = 1 so that there is no post filtering. With the pre-filter there are a little over 4000 stocks evaluated, or about 20 stocks in each bucket. This give about 380 weeks X 20 stocks = 7600 individual performance data points to calculate the annual return for each bucket. I would argue, that is sufficient to be statistically adequate with minimum data mining. Looking at the chart, I have to ask, why would anyone want to buy the stocks in the second highest bucket when there is a probability of a 35% difference between it and the highest bucket? You would not even know that this difference exists if you tested with 100 or fewer buckets. By using 3 to 5 stocks it is easy to stay in the top bucket even with additional buy rules. This ranking System, when re-run from 06/30/07 to 12/29/07, yields a -10% return in the top bucket. That’s not much of a loss considering the upside potential. Re-run since 12/29/07 yields a 22% return, and re-run over the last year yielded a 6% return. Since the system was created over 2 years ago this is out of sample results. There have been arguments in the Forum that taking such a fine cut of a systems performance only creates “noise” in the results, that the performance cannot be achieved, and going foward would likely produce results that were worse that the top 5% bucket. Hogwash! Re-running the system with 20 buckets over the last year yields a -6.2% return in the top 5% bucket. I have been using Filip’s system since 07/19/06 and I have had great success with it. The second chart below is a Sim of a Port that I have been following since September ’07. This is a 5 stock Port rebalanced daily using some Market timing to avoid some of the drawdowns during corrections. It only holds a stock if it stays in the top bucket (well duh). This Sim had a 19% drawdown in December ’07, and is up 56% over the last year, far out performing the market. At some point in time the economy and market will improve. By following this Ranking System I’ll be ready! Denny  ----------------------------------------
Filip's SuperValue Performance.png
(12864 bytes)
(Download Count: 86)
----------------------------------------
Sim of Filip's SuperValue Port.png
(7982 bytes)
(Download Count: 86)
---------------------------------------- "The significant problems we face cannot be solved at the same level of thinking that we were at when we created them". Albert Einstein ---------------------------------------- [Edit 1 times, last edit by DennyHalwes at Aug 1, 2008 12:05:18 PM] |
||
|
|
Stittsville123
Advanced Member
|
Josh - by trend I mean playing a high P/S ratio because that is working as opposed to a low P/S ratio that worked well longer term (just an example). I'm not referring to TA trend versus pullback. I am fully aware of these sorts of systems. Denny / Josh - There are two separate issues here. One is ranking system development/evaluation. The second is trading system development. As far as trading system development goes there are a lot of personal choices to be made and varied development strategies. Some people prefer multiple small trading systems while others prefer one large trading system. If a large number of small trading systems is what turns you on then go for it. As far as ranking system development and evaluation goes I don't see the relevance of using a large number of buckets. You want to have a robust ranking system that generally increases %profit as you increase the bucket number. Reducing the number of buckets improves the signal to noise ratio (as Oliver said). Now why do you want to work with noise? Steve ---------------------------------------- [Edit 1 times, last edit by Stittsville123 at Aug 1, 2008 12:04:44 PM] |
||
|
|
DennyHalwes
Advanced Member UNITED STATES Joined: Apr 28, 2004 Posts: 1532 Status: Offline |
Oliver, OK, how DO you achieve 190% in the top bucket? Steve, Yes, increasing the number of buckets does increase the noise. However, if you look at my first chart, I contend that the noise is the +- variation from 1 bucket to the next. That averages less than 5% and has a maximum (ignoring the top bucket) of about 15%. Therefore, with 7600 individual performance data points / bucket, I would expect the top bucket to have a minimum statistical return of about 80%. The key to my argument is that if you didn’t look at 200 buckets you would not know that the top bucket performance existed. With this knowledge, you might be encouraged (within your personal choices) to develop trading systems that try to take advantage of the top bucket performance, and your system might not be quite as likely to hold stocks with much lower rank value. Denny ---------------------------------------- "The significant problems we face cannot be solved at the same level of thinking that we were at when we created them". Albert Einstein |
||
|
|
olikea
Advanced Member
|
The problem is that this is *simulated* performance created with the benefit of hindsight. Denny, I looked at your profile and note that the "Filip's Super Value" ranking system has a "last updated date" of 2006-07-19. Now, I don't know when this ranking system was created, but it must have been before this. Therefore, anything post 2006-07-19 can be treated as being "out of sample", and anything created afterwards as being "in sample". I decided to create a series of simulations, first using the "in sample" period, then the "out of sample" period, using my usual minimum liquid screen and a simple rank based exit. I tested this using a number of different positions, ranging from 3 stocks to 100 stocks. The exit rank was varied in order to keep the turnover about the same in each case. Here are the results (click on the links to find the simulations): 3stks: In sample : AR 123.17% DD: -33.08%Out of sample AR: 22.78%, DD: -44.21% 5stks: In sample : AR: 123.86% DD: -24.65 % Out of sample AR: 8.13% DD: -42.78 % 10stks: In sample : AR 127.73%, DD: -24.77 % Out of sample AR: 15.95%, DD: -32.78 % 20stks: In sample : AR 107.11%, DD: -23.66 % Out of sample AR: 12.97%, DD: -29.06 % 50stks: In sample : AR: 80.93%, DD: -19.85 % Out of sample AR: 23.19%, DD: -22.92 % 100stks: In sample : AR: 62.68%, DD: -20.79 %Out of sample AR: 22.81%, DD: -21.99 % 200stks: In Sample: AR: 51.27%, DD: -22.17 %Out of sample AR: 17.31%, DD: -23.37 % ************************** I think the numbers speak for themselves. Where you see "out of sample", read "what would have really happened if you had put your money into it". Simply chosing the portfolio with the best AR is not necessarily the way to go. Notice also, how the real time drawdowns go progessively from a totally unnacceptable 45%+ to a much more reasonable 22% for the 50 stock portfolio. Based on the above considerations, I consider the 50 stock portfolio to be an optimal size, getting the tradeoff right between noise and pushing up the ranks. |
|||
|
|
grokkalot
Advanced Member
|
Oliver, as you know, ranking performance in the top buckets can sometimes be dominated by totally illiquid and otherwise defective issues that one could never trade. And this can be a big problem when going from ranking performance to simulation performance. So as an advocate for getting as much info out of the ranking system development as possible, I'm in favor of somehow adding to the ranking to whatever extent is possible the screening factors you would want to use in your buy/sell rules. So I've modified your "first generation" and "fourth generation" systems with an extra screening category called "OkayToBuy" that checks whether a give stock has some liquidity, is current in its filings, and trades over $1.10. The modified copies are here: http://www.portfolio123.com/rank_details2.jsp?rankid=68205 http://www.portfolio123.com/rank_details2.jsp?rankid=68208 Then I look at the trailing 5 year performance of both rankings for all stocks using weekly rebalancing and the Fourth generation clearly looks a lot better. If you want I can focus on some other period (2006?) if you think that would change the story and make the first generation look better there. So my conclusion from this exercise was that the problem you ran into was not caused by using too many bins to look at the ranking but rather by having the ranking's top bins not reflect the stocks you would actually buy. |
||
|
|
olikea
Advanced Member
|
It is based on a pullback approach. I may be willing to consider a trade. Just as an addendum to my prior post: I noticed that a lot of the out of sample trouble comes from the microcap stocks. So here is Filip's Super value (20 stock) excluding everything with a market cap less than $200m: In Sample AR: 58.36%, DD: 23.93 % Out of sample: AR: 22.29%, DD: -21.32 % By excluding the really small stocks ( < $200m) the backtested performance is severely effected. However, the real time performance is much better. To be honest, I don't know to what extent this is because microcaps are relatively unstable (and cannot be relied upon) or because they have been severely beaten up over the past couple of years, therefore excluding them is good. However, I am leaning towards the former. I have some microcap-only (defined 50-350m) ports that have been doing well recently, but these are with 50 stocks. So micros are ok, but you need that diversification in my opinion. |
|||
|
|
[Show Thread Printable Version] [Post new Thread] |
Re: Number of buckets
