My third generation ranking system

By analysing the different stock factors in different areas, similar to the techniques used by Dan, I have developed my third generation ranking system, the main focus has been to keep the results robust and use a number of different unrelated factors in each area, while keeping the number of factors limited and only using the most potent ones.

Here it is:

http://www.portfolio123.com/rank_details2.jsp?rankid=52503

To verify I have a 100 stock sim based on it:

http://www.portfolio123.com/port_summary.jsp?portid=314289

Comments/improvements welcome!

I rerun the sim introducing 0.25% slippage and $5 commissions on a 10 stocks portfolio. Still very good results, CAGR 45%, Sharpe 1.32: http://www.portfolio123.com/port_summary.jsp?portid=314609.

IMO the most interesting indication is the 63% winners. It’s rare to see such a high percentage in a 10 stocks portfolio. I run the sim with the same commission/slippage scheme from 3/31/2001 to 3/31/2003, CAGR 47%, Sharpe 1.46, Overall Winners 60.87%.

I don’t know. Ran your sim, probtrader but get different feeling about this one and think the returns are skewed to the earlier years, which mkes the overall return look good.

10/09/04 - 10/06/07
Total Return 118.58%
Benchmark Return 39.48%
Active Return 79.10%
Annualized Return 29.77%
Annual Turnover 195.55%
Max Drawdown -25.29%
Overall Winners (38/70) 54.29%
Sharpe Ratio 0.79
Correlation with S&P 500 0.54

03/31/01 - 10/09/04
Total Return 387.82%
Benchmark Return -3.29%
Active Return 391.11%
Annualized Return 56.74%
Annual Turnover 187.52%
Max Drawdown -26.12%
Overall Winners (52/78) 66.67%
Sharpe Ratio 1.89
Correlation with S&P 500 0.50

I still think it is amazing that Olikea’s original 100 stock port does so well. That means the ranking system looks to be very good indeed. Now it’s just a question of tweaking the buy/sell rules.

Just on a point about the slippage used here - it is 0.25% in probtrader’s sim, which I think is way too high.

olikea:

You have created another interesting ranking system. Since it has so many factors and they are unequally weighted I gave it some extra testing to see if I could get it to “break”. The results are excellent overall.

Ranking Performance tests

All ranking performance tests used my custom universe screen “Liquid All Cap” = (mktcap>25) and ( (price>3 and avgvol>50,000) OR (price>10 and avgvol>10,000) ) and noOTC and noADR
The Liquid ALL Cap screen gives a custom universe of about 3,800 stocks for rank testing.
The ranking test looked very good with each bar higher than the one on its left, especially for the final few bars.
I retested for other cap sizes (keeping the same liquidity screen).
Small cap (mktcap 25-1000) about 2,000 stocks - gave good results
Tiny cap (mktcap 25-350) about 1,000 stocks - gave good results
Even Large cap (mktcap >1000) about 1,000 stocks - gave reasonably good results (annual gain about 30%). Often P123 ranking systems do not work well for large caps, but yours does.

Then I retested for 3 time periods using the 3,800 liquid all cap screen:
March 2001 - March 2002 (bear market): WOW! the 199th and 200th buckets (ie the top 40 stocks) averaged 60% for this period. virtually all the other buckets averaged out to breakeven or worse. WOW!
April2002-March2003 (raging bull market): almost all buckets showed great gains so the top buckets did not standout even though they did quite well in annual gains.
April2003-present (regular bull with occasional pullbacks): good results.

What really makes this ranking standout is how well it did during the tough bear period of March2001-March2003.

Olikea, as you know I am very cautious (perhaps overly), so I am still hesitant about a ranking system like this that has so many factors and the factors have varying degrees of weight. Just so great a possibility for unintended curve fitting on this “in sample” data history of March2001-present. I would love to see how this ranking system performs on “out of sample” data, so I am keenly awaiting the release of P123’s extended data history. I will be especially interested to see if it can handle the “bust” period from March 2000-March 2001 and 1998 which had the short lived crash in the 2nd half of the year.

One final observation: when running simulations (I used your liquidity filters rather than the ones I stated above for my ranking tests) the simulations suggest the system is better for medium and large portfolios (20 to 100 stocks) than it is for small ones. For a 5 stock simulation, the gain is 10% less than larger simulations and the equity curve is basically flat (with a lot of bumps) for the past 2 years. A 5 stock portfolio would be very painful to trade. Although the 10 stock simulation is not as bad as the 5, the 10 stock does not give any extra yearly gains compared to a 20, 50, or 100 stock simulation. So this ranking system is unusual since most P123 rankings I have seen give increased gains as the portfolio size is reduced. You have a very interesting ranking system

Thanks again for sharing.

The ranking system also gives good results on each individual sector. Looks good!

Yes, I run a sim for the last 2 years only with these results:

Total Return 37.05% Benchmark Return 32.56% Active Return 4.49% Annualized Return 17.08% Annual Turnover 209.49% Max Drawdown -30.29% Overall Winners (29/54) 53.70% Sharpe Ratio 0.39 Correlation with S&P 500 0.61 It looks like the system hasn’t performed as well more recently.
A note about the 10/9/2004 cutoff: the bull market starting in March 2003 was extremely good for small caps with the Russell 2000 gaining around 58% from 3/31/2003 to 10/9/2004. Including this period skew results for almost any small cap portfolio.

Probtrader - I understand, but where does it say small cap?


Picture 33.png

jpkernot, I was assuming the sim was picking mostly small cap stocks. Thank you for pointing this out. However, returns for the midcap index during the period 3/31/2003 to 10/9/2004 were still 45% ex-dividends:
http://finance.yahoo.com/q/hp?s=^MID&a=02&b=31&c=2003&d=09&e=10&f=2004&g=m

Thanks for the comments so far:-

I have been trying to focus a bit more on building a ranking system that is logical and robust, rather than going for out-and-out performance. I have seen many sims feature out-of-sample performance that quite frankly has no relation to their in sample performance, and this seems to be strongly correlated to the number of stocks they hold - I very much doubt the triple digit gains seen in some 3 stock sims will be replicated in the future.

The more stocks, the harder it is to curve fit.

Ultimately, we are searching for alpha - returns above the benchmark. Some factors provide a source of alpha, and depending on how these are combined, it is possible to achieve synergy and gain further increases in alpha.

One issue I have, is that factors that provided alpha in the past, may fail to do in the future, or may not do for extended periods of time, such as value factors which have clearly been lagging for the past few months, with value-only portfolios doing poorly.

Therefore, having different unrelated sources of alpha is a good thing because only one has to be a source of alpha in the future for your portfolio to have positive alpha.

In each area I will discuss the sources of alpha:

Valuation: - this is obviously the classical version. While most people focus on earnings yield, while this is important it is important to look at cash flow and sales too. Earnings without cash flow are frankly “fake”. The price in relation to the top line has also been found to be a very important factor, as discovered by O’Shaugnessy. This is interesting - investors can’t “eat” sales. Nevertheless, sales are probably the best economic measure of the size of the business, and arguably they are a more stable measure than earnings. Finally the PE based on next FY projection is important, if only to make sure that the current low valution ratios are not caused by a temporary spike in earnings. Beyond the next FY the accuracy of estimates decreases geometrically, and are probably useless. That is why no other projected figures are used.

I believe the most recent up to date figures (quarterly) are important. This goes somewhat against the Graham approach of averaging earnings, because that ignores the fact that earnings can and do trend higher and lower, the most recent figures are the closest to what is about to be seen. While on their own the quarterly figures do not seem to add more alpha than the TTM figures, when combined with momentum factors the level of synergy is a lot higher, possibly due to the effect of earnings momentum.

Technical - trade with the trend, your trend is your friend, all cliches, but thats because frankly it works. The first thing I used to look at was relative strength, price now vs price 6months or 12 months ago. I noticed, however, that short term RS seems to have little alpha, and very short term RS is a contrary indicator - interesting. A short term price spike is not bullish - probably because it attracts a lot of speculators. Long term investors are less likely to trade on the trend and trade on fundamentals instead. This is probably why a longer term trend is a good indicator - the smart money is moving in based on improving fundamentals. Therefore I use two factors. I use the daily sharpe over the past 6 months instead of 6 month RS because it seems to add more alpha. This may be because the sharpe function punishes volatility - and that makes sense. A very volatile stock can easily make it into the top decline of RS through randomness alone, not because of a true trend.

To guard against recent price spikes, the difference between the 50 day and 200 day simple moving average is used as well. This ensures the higher prices has persisted over a long period of time, not caused by simple speculation, and reduces the risk of simply buying an overbought security.

Short interest - This is perhaps a more unsual source of alpha. One interesting thing is that following average analysts recommendations does not provide any alpha at all, but following the shorts does. I guess it pays to look at opinions where people have put their money where their mouth is! A while ago I read a paper showing that shorts were “smart money”, more sophisticated than your average investor. It pays to see what they they are up to, and there is a stong correlation between the level of short interest and the future performance. The more the short interest, the worse the performance. A good amount of recent short covering is bullish too. So both factors are considered, the amount of short interest, as a % of shares outstanding, and the 1month percent change.

Efficiency - Warren Buffett has often commented that “ROE” is everything. He believes in investing in high ROE companies. Many people think he is a value investor, but I think he is really a high ROE investor. It has worked for him, he talks about how companies which require little assets to operate (media businesses etc.) can easily grow their earnings year after year just by raising prices.

Now I like to tweak this a bit, and look at ROA instead. The problem with equity is that its “assets-debt”, so you can have an inflated ROE because of high debt. Of course, Buffett famously prefers companies with little or no debt, so for him ROA=ROE. But since I have found that debt/equity ratios do not appear to be a source of alpha either way, I am indifferent to that ratio. If you test ROA by itself, it is clearly a source of alpha.

Interestingly, related to ROA is the Asset Turnover. Sales/assets. Interestingly it does seem to be an even stronger source of alpha. Again the “you can’t eat sales” arguement spings to mind, but then again, this may be due to the relative stability of sales compared to earnings (on which ROA is based).

There are other possible measures such as ROI ROC etc. etc. but I think there is little point in putting in extra overlapping factors, just stick to one.

Interestingly, traditional things such as margins are not included as they do not appear to be a signifcant source of alpha!

Growth - it appears that long term growth rates provide little alpha. This is possibly because they are already priced in, possibly because they are irrelevant to future growth rates. However, short term growth rates, specifically year-over-year do provide a strong source of alpha. Comparing the most recent quarter to the prior quarter is less indicative than comparing to the quarter one year ago. I think this is because seasonal effects may cause earnings spikes that distort the true growth trend.
I am interestested in why this is a source of alpha. I believe it is because earnings do have momentum, trends in the underlying economy and business do continue over the medium term, and this is not fully priced in to the market - future positive earnings surprises drive the stock higher.

I look at EPS change, cash flow change, and sales growth over the year, and also the improvement in ROA vs. the long term average. This is probably important because it indicates the company is not having to invest heavily in assets to achieve this growth, this means more returns for shareholders.

Financial Strength - this is born out of “debt is bad” mantra. The problem is there seems to be little evidence of underperformance of companies with high debt-to-equity ratios etc. and even interest coverage seems to have little effect either way. The only factor I can find that does seem to provide convincing alpha is the coverage of short term liabilities, or EBITA/Current Liabilities. Probably companies that are headed for a hard place start off with a swelling in current liabilities, unable to keep the lid on expenditures. Perhaps the reason long term debt is less important is because - being long term - if the company has survived so far it will survive in the future. The very occasional bankruptcy has little effect on alpha.


So there you go, this is a ranking system built from the ground up designed to capture as many different sources of alpha as possible, and not focus too much on a single source, such as value. Because of the broad spread, it means that in any single area, it is unlikely to get the top ranked stocks, and this means it doesn’t have the fireworks performance of some other sims. However, my belief is that it is more robust to some of these sources of alpha from stopping. if you run a deep value portfolio, or a high momentum portfolio, then you are essentially betting on value, or betting on momentum. This is a way to get a more diversified bet, a more conservative portfolio that is more likely to give positive alpha under all market conditions.

Have you tried this with only 50 stocks? My current membership level doesn’t support 100 stocks.

Thanks,

Kit

http://www.portfolio123.com/port_summary.jsp?portid=314980

If it works on 100 and didn’t work on 50 i’d can it immediately!

Fama and French recently published a paper on the impact of historical Price/Book to value stocks returns (historical P/B mean reversion). The study is confirmed by James Montier, the author of the Behavioural Investing blog. James adds that dividends growth is one of the most significant factors contributing to value stocks returns.

The Source of Value
The Anatomy of Value and Growth Stocks

P123 has a number of 5 year valuation ratios, including Book Value per Share. Reversion to the mean could be predicted with a formula like BVPSA / BVPS5YAvg, the lower the better, meaning book value should increase.

In my own studies I’ve found that one of the most reliable factors is shares supply. I use a formula that compares TTM number of shares to the previous year in some of my systems.

Tried to run it with my usual qualifiers Marketcap >=150, Price >=5, AvgVol (20)>=200000 on “All 123stocks” List.
Test period : last 2 years.
Capital 20.000 USD, 20 Stocks.
I included transaction fees of 1ct./share and a slippage of 0.5%

The result is worse then what the SP500 did over the last two years, with much greater volatility. Although the system started out well in the first 6 months.
But it lost all its gains ( +45% ) and went into the ( -10% ) red in the may / june crash 06. Since then it has only slowly recovered and did worse then the market.

Think throwing darts on a list of SP 1500 Stocks would have probably yielded similar or better results - sorry - but that seems to be far from being a robust system if a single, short downturn can knock it out. Think it has had it’s big time in the years before 2006.

Here the stats : of the 2 yr. test.
Total Market Value (inc. Cash) $ 23,474.76
Cash $ 4.11
Number of Positions 5
Total Return 17.37%
Benchmark Return 32.62%
Active Return -15.24%
Annualized Return 8.35%
Annual Turnover 209.93%
Max Drawdown -40.92%
Overall Winners (11/28) 39.29%
Sharpe Ratio 0.08
Correlation with S&P 500 0.50

and here for a 1yr test :
Total Market Value (inc. Cash) $ 22,054.89
Cash $ 28.49
Number of Positions 5
Total Return 10.27%
Benchmark Return 14.60%
Active Return -4.33%
Annualized Return 10.28%
Annual Turnover 226.94%
Max Drawdown -19.83%
Overall Winners (5/17) 29.41%
Sharpe Ratio 0.19
Correlation with S&P 500 0.65

regards
Stefan

Privateer,
Two points:
First, I think you are setting the hurdle rate way too high on your sim. For instance, the slippage of 0.5% effectively removes 1% per closed trade. Why 0.5%? Do you really think that your trade will have a market impact of 0.5%? remember you are already using next open price for the trade.
Next, you are using a very small capital sum with a large number of stocks (20). Once again, a sure way to have the commissions and slippage eat into your performance.
But, point taken about lacklustre recent performance, which I raised and has been raised again.
I don’t think this ranking system is finished though.

olikea:
Thank you for the rank. It seem very stable from 2002 to 2007. The question is did you use all the data from 2002 to 2007 to optimize it? Is there a out of sample period? Thanks.

Hi jpkernot,

No - don’t think that my trade has this impact of .5%, but as we all know, the price of monday morning open is rarely the same as the one you’ll get when your trade is filled.

In addition, market makers and specialists have to live from something, so the invented what they call “Bid/Ask spread”. Maybe not an issue with MSFT, but many less liquid stocks show spreads larger than 0.5% . Actually, when you trade here in Europe, slippage is much larger than 0.5%/tradeside.

As it has been discussed in countless other threads here and in many other trading software forums, a slippage of .5% per tradeside is nothing unusual but quite the rule unless you’re trading QQQQ, SPY or any of the other high volume ( thick ) stocks.

You might have also noticed, that I used a commission of 1 ct/Share , which is already on the low side, while the original system didn’t use neither of both - not very realistic either, don’t you think ?

20 Stocks - well what’s the problem with having 20 Stocks in a 20K Portfolio - same story as having 100 Stocks in a 100K portfolio or not :wink: ?

With commissions as low as 1 USD/trade( IB ) we’re talking of 0.1% per trade. But the main reason was to see, how the system performs with a diversiefied, but smaller account. I’m always on the quest to find a robust mechanical system which works under most market conditions and with mid and large cap stocks. Still very difficult, I believe. Especially, when you look at the %age of winners in the 1 yr. sim.

regards and good luck

Er, maybe not. Your sim is already assuming that you are buying ‘at the open’. The only real issues you will have in trading is (a) market impact, which is too small to be significant is your case as the $$$ amounts traded are too low, and (b) slippage which occurs as the result of a trade on a Nasday or OTC stock. your fills will be pretty much ‘at the open’ on any listed stock. I trade a lot of NYSE listed stocks and I get the open 99.5% of the time.

There is a thread somehwere which talks about this and I think it was Denny who mentioned that we should be using smaller slippage % now as the P123 team implemented a facility to simulate using buys on ‘tomorrow’s open’ rather than ‘last close’. Obviously a sim using last close would need a higher slippage factor.

I think there is still a bit of confusion in the forums regading slippage.

And here really is the point. You’re not really comparing like with like here. Your sim with commissions, slippage and the like is not really comparable to Olikea’s which has no slippage and no commissions. He likes to do that as it shows the raw value of what the ranking and buy sell rules can achieve. Running $20k on this sim will not have the same results as running $200k.

Last point about % winners. I truly believe that it is comforting to have more winners than losers by a siginificant margin but this statistic is to some extent illusory. Remember the 80:20 rule about where profits come from! My current ports (all profitable) show winning percentages (in order of total return) of

61, 31, 52, 47, 53, 54, 44

Maybe there is a point that one can make here about instituting some kind of standardised ‘test rules’ so that we can adequately compare sims? It seems that a lot of effort is going in to proving or disproving a sim by taking into account various trading styles. Any ideas anyone?

JP -

I understand that you should get the opening price on NYSE at the opening. Does this hold true if you are trading odd lots (i.e. not 100’s)?

Steve

Steve, I am not 100% sure, but the specialist has to balance the opening and sell orders, either from public orders or from his own book. Therefore, the specialist may well be selling stock to satisfy buying demand on the open, against the trend. I assume that hold true for odd lots although there are special rules for dealing with odd lots which I think hold for dealing away from the opening price.

Hi,

Most of the time I would agree more with privateer that the slippage and trading costs are underestimated. On a small cap port I have never been able to get .5% even on a sim with fills at the open. However in this case with 50 stocks in which if it seems most are listed or have enough liquidty if not, Ibelieve the slippage could be lowered. However to put trading costs at zero is unrealistic. I also would like to know if odd lot listed stocks recieve an open fill. I always assumed they would but maybe not.

Thanks everyone for the input.

Jbarnh