Theoretically, if I run a ranking performance test with 20 buckets, the annualized return of the top bucket should be the same as if I run a screen with one rule: rating(“name of ranking system”) > 95. But they’re totally different. If rating < 5 it should match the bottom bucket, but again it’s totally different. My universes, holding periods, and slippage percentages are the same. So what accounts for the big differences? I’ve tried this with different systems and different universes, and the results of the ranking performance test are ALWAYS significantly higher.
A few thoughts for your consideration:
Try rating(“name”)>=95. By using >95, you are leaving out some stocks that the ranking performance test would put in the top 5% bucket.
The ranking performance run that you compare to should have no filters applied after ranking. Using a minimum price, for example, will filter out stocks that could appear by using the Rating function. It is possible, but I haven’t tested it, that having a non-zero minimum price in the rank performance and the same price minimum in the screen will give comparable sets of stock. Same reasoning applies to sector filtering.
The screen should use Next Close pricing and zero slippage to match what the ranking performance test uses, just for test comparison.
To get an actual list of stocks in each group of the ranking performance test for each rebalance, you can tell it to Save Log, thanks to Marco’s recent addition, then view the log. It will show the stocks in each bucket at each rebalance, their starting and ending prices and individual returns. It will list a max of 30 stocks in each bucket, so I suggest using 200 buckets instead of 20 (depending on the size of your universe) and a screen comparison using Rating>=99.5 for this check. You might want to cut the backtest period down to one or two rebalancing periods to reduce the log size. Then you can compare stock for stock in the screen’s rebalance logs.
Also, the screen will have dividends but I don’t think the ranking will account for them.
Thanks. I didn’t realize the ranking system uses close prices. I think I may have mismatched the periods. At any rate, it works now.
According to the rank test report, dividends are included. I suppose it uses the dividend adjusted database of prices that screens use.
Correct, ranking uses fully adjusted price for divs&splits. Except if you have a minimum price , in which case it evaluates point in time.