We have removed all limits to rolling backtests in screens for all users because we felt that users shouldn’t be limited when using this very powerful tool. Until now, most users were limited to a certain number of samples. No longer.
For those unfamiliar with it, open any screen and click on the “Rolling Backtest” tab, fill in the blanks, and run it. It’s a great way to test the “robustness” of a screen or ranking system. I like to use a ten-year lookback period, a three-month or six-month holding period, and a frequency of “Every Week,” but you can play around with whatever suits you best.
I don’t see how that would be possible. Let’s say you’re running a three-month rolling backtest of a screen, with weekly frequency. Alpha, beta, Sortino, Sharpe–none of those can be measured for three-month holding periods. One could calculate the alpha and beta for the whole rolling backtest by plotting all the results against the benchmark results and measuring the slope and intercept, and then annualizing the results for the alpha. But that would give you very different numbers from those provided from a regular screen backtest, depending on the length of the holding period, and it would be, in a sense, a perversion of alpha and beta as commonly understood. Risk statistics were designed for normal equity curves, not rolling backtests. I think the performance data we do give–return, bench, excess, min, max, and std dev–are very valuable, especially when broken down for up and down markets.
No, I’m afraid I’ve been told it wouldn’t make sense to include books. I think you just have to run the rolling backtest on the various components of the book.
I think rolling backtests are different for each person (port). If you rebalance weekly with the sell rule RankPos > x with x stocks in the port it is different than if you rebalance yearly.
Being mostly in the rebalance weekly camp myself I see less benefit (there can still be a benefit) in slicing and dicing the same results over and over again with a rolling backtest. Clearly, one has to be careful about the statistics while doing this. You cannot have the same results added into the data multiple times and have valid statistical results. Yea, you can get a number but Yuval is correct. It is not a valid number.
Just as you would not put the same person, multiple times, into the data to see if a new medication is effective. Well you could, but if you submitted it to the FDA you would go to jail. And the FDA regulators understand statistics enough to have a good reason for not liking that.
I really have not used rolling backtests much (see above) so I am not really in a position to discuss rolling backtest for ports with difference rebalances, longer holding periods etc. But at a minimum, I appreciate Yuval’s understanding that some statistics just do not work for some situations.