the high performance of your top bucket suggests that the ranking factors were ideal for 2009-2014, but perhaps not before or after.
The only way to answer that is to upgrade so you get access back to 1999. I usually test a ranking system for two time intervals: 1999-end 2006 and 2007-today. Performance should be approximately the same in both periods to qualify as a good ranking system.
Also, it is important to consider what buy rule you have. A simple ranked-based rule is probably best, or otherwise none at all, such as my 7sisters KISS R2G.
If you want real help, then give a link to your public system so people can see what is going on without playing 20 questions.
No system will work as well as your top quantile in all market regimes. The high returns in your system do not come smoothly like a bond coupon. The Annualized Returns histogram is nice but the better analysis is done with the downloads of the Performance line plots.
With respect to the micro cap space you are targeting, the weaker performance is a combination of decreasing risk preference and illiquidity that are the flip side of a small cap bull market.
Also, the further you get into the tail of a universe by holding only a few stocks, the higher your volatility is likely to be.
It would be better - but not perfect - to use the S&P 500 Eq weight index. But you’re right that the equal weight simulation mirrors the S&P 500 Eq weight index. I’ll need to think about the meaning of the custom series curve. I’m thinking about it as a cap-weighted index. Maybe that’s wrong.
I have narrowed my universe with 700 stocks by following
expert Mgerstein method apply the screen logic in universe, don’t keep plain white paper on the universe.
I need to wait for 6 to 12 months performance rather than 3 months performance.
I have run the ranking for Summer months; it stays relatively good.
I welcome other suggestions for testing this ranking system.