Thanks to P123 user Jim Rinne, last week we realized that FactSet was dating their historical estimates to the market day prior to the estimates’ actual effective date. This had a significant lookahead effect: backtests were reflecting data that had not yet been released.
We have now fixed this behavior by lagging all historical FactSet estimate data by one day, effective this afternoon.
Please also note that estimates are resolved from the daily observation date instead of the weekly one (contrary to what was supposed in a recent thread). So if you run something midweek (screen, daily screen backtest, or daily simulation), this change will still have an impact on your backtest.
This change will not affect current data. The nightly updates of current estimates will be no different than they were.
This change will affect almost all simulations and screen backtests that rely on estimates. Performance of those backtests will likely diminish. In addition, simulation rankings and transactions will likely more closely resemble those of identical live strategies.
Please let us know if you have any questions or concerns about this change.
If estimates are in the rank it will be a weekly thing correct? As suggested by the image where Force Weekly Ranks is forced to be Yes for the sim: “Strategy Simulations can only rank based on weekly data”
But I can put estimates into the buy or sell rules or into the universe and that will be daily? This is how I read your quote but I may not understand.
Also I am wondering how “Average Analyst Recommendation Score” is affected by all of this. Specifically, is it lagged now and can it be used daily if it is in the sim or universe? Also if it is not lagged, should it be lagged?
Maybe my sim could sell a ticker during the week with the buy/sell rules if earnings estimates are revised lower or Analyst Recommendations change during the week? That might be a good thing.
I have no question that this is an improvement and I am just looking to improve my sim further or understand it better.
Jim, First of all thank you a ton for pointing out that error and p123 for fixing it!!!
“Maybe my sim could sell a ticker during the week with the buy/sell rules if learnings estimates are revised lower or Analyst Recommendations change during the week? That might be a good thing.”
Jim, absolutely! That would be great. From what I understand is, that this is not the case in sims (even if run daily) but on ports if you rebalance them manually they catch the daily eps revisions (P123 PLEASE Correct, if not the case)
If that is so, than my ask to p123 is to let daily sims also capture the daily update of earnings revisions, so that there is no difference between
the port (that is rebalanced daily manually) and a sim that runs daily (but so far can only capture the weekly update).
I am sure that would improve performance quite a bit.
My Flagship Strat dropped from 76% / -35% DD to 64% / -48% DD. Would be interesting how other users using earnings estimates are doing regarding this so I get a benchmark.
Will work on my sims to improve them with the no look ahead bias and keep you posted!
So still 15% (which is still huge!) better then my sims without earnings estimates in buy and sell rules (best ones around 50% / 50% DD).
This is entirely correct. Ranks are only once a week, but buy and sell rules can be evaluated daily. You can specify which day of the week you want the ranking to be done, unless you force weekly ranks, in which case it will always be the weekend.
Analyst recommendations function like estimates; they are also being lagged in backtests.
