Reverse Engg doesn't map to Performance results (possible bug?)

Several discrepencies when i ran Reverse Engg across 20 different factors spanning Value, Momentum, Sentiment, Quality and others, using 4week holding period and >10% outperform S&P.

  • The same set of 4-5 factors are relevant (i.e. >65 rank+ with < 20 std-dev) for different universes (Mid-cap, Small-cap, etc) and for different time-periods between 2010-2015. This is really encouraging, but a little suspicious. These are Yield5Y, 2 different Sentiment factors, Surprise, Sales10YGr, etc.
  • However, when i run performance with (1) individual factors one-at-a-time (2) combining these 4-5 factors (equal-weighted) (3) combining these 4-5 factors (rank-weighted), none of the ranking systems beat S&P500 in any of the 3 configurations.

Are we sure there’s no bug? Has anyone used Reverse Engg successfully to generate a ranking system without excessive trial-n-error. (Excessive trial-n-error only leads to curve-fitting, which doesn’t perform walk-forward.)

thanks
Danny