IMPORTANT: upcoming changes to our estimates

Dear All,

As you may know, our data comes from one company Standard & Poors, but in reality it’s two companies: Compustat & CapitalIQ. We rely first & foremost on Compustat: if there’s no data in Compustat we do not care what’s in CapitalIQ. But there are many little nuances we’ve been learning to go across from Compustat to grab estimate data. Below please find changes we have done and are planning. For clarification “the past” and “the future” are in relation to a particular point in time (the as-of date)

1. Current estimates and revisions (the future)

Here we need to figure out what the current and next year is for a company, and extract the corresponding estimates (analysts report figures for many years and quarter, but most people just look at these: currY/Q and nextY/Q). To do this we use Compustat as the reference. If the last filing is Q3 of 2015, then the CurrY for estimates is 2015, NextY is 2016, CurrQ is 4-2105, NextQ is 1-2016. This keeps things aligned between Compustat and CapitalIq.

However it’s not ideal. Estimates are much more fluid. The “switch” from what is a CurrY and what is CurrQ should mirror the analysts, not the fundamentals. When a company reports, analysts react and adjust their estimates. If compustat takes time to process the filings some of the estimate ratios will be stale. More precisely: the CurrQ and CurrY will point to a past filing (according to the SEC) and NextQ and NextY are pointing the quarter in progress.

We are contemplating changing this and use CapitalIQ to determine what is the current Q and current Y, regardless of what Compustat has. This change should not be a major change if you are targeting large caps since large caps are processed on the same day by Compustat. So , for example, I don’t think the #SPEPSCURRY series will be affected much. But it may have a noticeable effect for small caps that can take days on the Compustat side to process.

2. Surprises/actuals/last estimates (the past)

What the company did vs what the analysts thought. We’ve corrected some long standing issues with surprises. The main problem there had to do with CapIq doing things different than Compustat, ADR ratios, and a new feature they released to go from one side to the other. Overall we don’t think this should have a major impact as relatively few companies are affected.

NOTE: Surprises DO NOT have the synchronization problem mention above. They are independent of Compustat.

Conclusion
We’ll discuss more internally about making the changes for (1): if having the fundamentals being in synch with the analysts is more important than having more timely estimates. They will never be both, so we need to decide what’s best for us.

Thanks

Marco,

I am most interested in up to date information for my port. That is real money!!!.

Your points about the sims being less important that the OOS are probably correct. Especially, those of us who have been running ports for a while but perhaps Smart Alpha too?

Thank you.

Thanks for the update Marco, I would certainly prefer to have the most up to date data too, from capital iq in this case. Also, in general it should be better since it is more likely that the data is being used in that way in the portfolios, instead of being used in tandem with fundamental data.

If during this process, you could also add NTM (next twelve months) estimates, available in capital iq that would be great, since I think this is a better measure as NextY can be very close or far away depending on the fiscal year.

Also, if capiq is faster at updating fundamental data in small caps, why isnt this data source being used instead of compustat going forward? it should be beneficial to live portfolios. or is it a licensing issue?

Yes but lets not get carried away. Estimates are very fluid, update constantly, and plenty of specialized high frequency shops dissect every little bit of real time information. P123 is not for that, and hyper sensitivity can cause knee jerk reactions. A multi-factor approach is fine with daily updates , even if they lag a bit.

Furthermore , there are other inherent, unsolvable problems with hyper sensitivity to news. For example PE, which is Price / Trailing 12 month EPS. The problem is that is uses Price (the most hyper active item out there) for numerator and SEC filings for denominator. It is very possible for PE to be calculated using a price that has reacted to a release, but use stale 12 months for the denominator. The “true” PE will either be too high or too low. Nothing can be done for that.

In other words, P123 is not for designed for real time events trading. And it will never be.

Isn’t Compustat based on IBES and CapitalIQ is their own version after buying some platform. I remember not being able to reconcile the two datasets completely when I had access to both.

I care more about daily fundamental data updates, then European data, then TRADE for Canada.

Marco - It isn’t so much about designing for real-time events. There is an inherent issue with digital technology (Compustat or any other service). There are delays between when news hits the street and when it is captured, analyzed, processed and sent to P123. Although you said before it was 1 day delay for large companies and 1 week for small companies, it seems to me it is more like 1 week to 1 month.

This delay is problematic because the stock prices have already moved based on information long before we can assimilate the info at P123. During this gap in time stocks that have taken a big hit due to bad news will look great from a fundamental perspective and there is a very good chance we will be buying stocks that look great to us but are really on their way down. Actually more than a very good chance, a very great chance.

One way of approaching this is to use a momentum factor. i.e. if the stock price falls immediately before the P123 buy recommendation comes up then you should reject the recommendation based on the assumption that the valuation is no longer correct. This may explain why momentum “works” to some extent for small caps as the gap between analog news and digitally processed news is larger than for largecaps. However, I don’t like this as a solution because fundamentals aside, technically buying on dips in general tends to outperform buying on short term bullishness. This runs counter to attempts to close the gap using momentum.

I see analyst estimates as the alternative. They tend to come out fast and can potentially give P123 a better option for sorting out the information while in no-man’s land. This can be readily seen in backtest simulation simply by using a 4 week change in recommendation. So I would stress that getting the info as fast as possible is important and good for P123. Otherwise, deprecate this factor. It is just a landmine as it is. The same applies to the new sales estimates factors.

Also, have you considered comparing what Capital-IQ is delivering (quality and timeliness) versus Zacks estimates? Maybe it all comes from the same source, I’m not sure, but you can get the Zacks estimates at a reasonably low price through Quandl.

Also, don’t forget the S&P 500 EPS timing uses the estimates I believe. P123 initially took up the charge on that particular market timing model. There are a lot of R2G models using the S&P 500 EPS estimates based on what P123 provided as an example.

Take care
Steve

Steve

Anyone who has tested it has shown the following:

For sims that rebalance every Monday, buying on Tuesday will cost you 20% annualized or more. Test it yourself. But if the signal is missed the first Monday, it is a week until you rebalance again!!!

Think what happens if you are buying more than a week after the signal because you did not get the signal last week!!!

I think this is a big part of why out-of-sample performance is not as good as in-sample. Right now, the delay is more evident for ports than for sims.

This is a real problem, now, for people who rebalance every week.

However, because of the mechanical newsreaders (and other high-frequency traders) this problem will be getting worse in the future for P123 users.

These are Aurelaruel’s (and others) results: [url=https://www.portfolio123.com/mvnforum/viewthread_thread,6871#33787]https://www.portfolio123.com/mvnforum/viewthread_thread,6871#33787[/url]

Aurelaurel’s results copied from above post:

"This method does allow you to see the impact of not getting filled on Monday for a high turnover R2G model.
As Olikea pointed out the decline in perf is correlated with the distance between the trade and the recommendation and the turnover of the strategy.
This is what I get for my “Trading 5 stocks” R2G:
Mon= 122%
Tue= 97%
Wed= 86%
Thu=81%
Fri= 65%

It’s vital to get filled the day of the recommendation !!" (Still Aurelaurel)

I replace what Aurelaurel says with: It’s vital to get filled the next Monday after the signal !!

I would prefer analysts estimates as soon as possible in the database, one key factor of my 5 Stock Models Performance is earnings momentum
based on analysts estimates…

Regards

Andreas

I don’t like having the estimates out of sync with fundamental data, but I also don’t like the idea of withholding the newest analyst estimates, so I suppose I vote for getting the most current analyst estimates. (What about listing “NA” for fundamental data that has been filed but is not yet available in P123? I assume you know this when currQ/Y for CapitalIQ does not match Compustat.)

Thank you for bringing this to our attention.

-debbie

I think it’s better for p123 deliver the data to user asap. It’s designer’s responsibility to learn how to use data properly. I think people are saying they don’t like p123 make “designer decision” for designers.

More recent is better. But will p123 be able to recreate the entire historical series so that the historical data available for backtesting is consistent with the proposed change? If not, perhaps it is better to create the new series but leave the old intact?

Hugh

In my opinion, for now I will stay away from adding any factors in the “ESTIMATES” category into my Buy Rules and Ranking System. If I have any, I will remove and re-run the simulations. I wonder if any items in the “FUNDAMENTAL” category possibly be affected by this lagging issue :slight_smile: ?

All:

I am happy with my Sim’s. I don’t want them to “drift” because of P123 tinkering. Leave everything as is. If you, Marco, think some new particular version is the next best thing since sliced bread then by all means start a new series. I’m O.K. with that, but please don’t destroy my alpha. That is why I’m here. I still have painful memories of the data vendor switch which cost me a lot of money.

Bill

Strader sounds like his ports might be doing as well as his sims if he likes his sims.

For the rest of us, why would would we be happy with our sims? Why would we want to live in a fantasy world of the past with results that would not have been achievable at the time. But now the results in the sim can be achieved in the ports and are realistic.

Strader, maybe you meant to say you like your ports. But they will actually perform better. As good as the sims you said you like.

J:

 I've been here for something like 9 years.  In general, I haven't had a lot of problems with Port's underperforming Sim's.  There is variation, sometimes out-performance, sometimes under-performance, but overall they've been tolerably close.  One notable exception was just before the vendor switch over when the underlying data (or processing thereof) changed and caused a lot of my Port's to under-perform. Recently, there has been some divergence between my paper Port's and their live cousins.  This may well be transitory.  If not, maybe overexposure? (R2G, hedgies, etc.)

FWIW:  I try to develop for robustness; I use market timing; I give the equity curve scrutiny.

Bill

The market timing data at P123 is up-to-date. Perhaps that is why your ports do similar to your sims.

For most of us not doing the above has not been the problem. The emphasis on curve-fitting and robustness was just people’s attempt to explain the difference between sims and ports. There may be some problems with overfitting.

But the biggest problem has been that sims and ports at P123 are fundamentally different. What is done in the sims could only have been done with information provided outside of P123 when we rebalance our ports. We would have had to get earning estimates revision data from another provider because some of the recent data is not now available in the P123 database for use when we rebalance. It is usable later–for the sims–as if we would have had the data available at the time of rebalance. I think it is actually in P123’s database but just not usable when we rebalance: Marco correct me if I am wrong on this. Now we would have to pay a provider to have access to that information when we rebalance

Marco is just offering to unleash the full power of P123 and make that information usable when we rebalance.

Me, I will take him up on that offer with much appreciation.

Jrinne,

My understanding is that sims and rebalancings are using the same data, PIT, at the moment. Its just that sims only have it available weekly and use the saturday update, while ports/rebalancings can use the data as of that day, which is later not available for sims. Both are currently using compustat data to source these, which is a bit slower to update in some cases vs capiq, even though both are S&P providers. The difference going forward would be that estimates data would update faster than it currently does for small caps. For sim/backtest data, I would imagine the data would be adjusted to use historical capiq data as well to be consistent, but I am not sure about this, or if it will only affect data going forward.

It would be great if Marco could clarify this point as it is very important.

Thanks!

Marco (iavanti),

First thank you so much. We are learning about this because you pointed out, in one of your posts, that the Estimates data was not getting paired to the fundamental data at all or very poorly. Marco has to do something and he is sharing some of his decision process with us. We would know none of this without you. And there would still be the problems you noticed too.

Second, to be clear, the data can be considered PIT because the estimates data is publicly available but as Marco says some of it is stale when it becomes usable to us.

Marco has said:

“…some of the estimate ratios will be stale.”

“…NextQ and NextY are pointing (to) the quarter in progress.” (sometimes)

Marco has said this is true for the ports and I think we can accept this.

One question that may not be clear to some is this: “Do the sims reflect what would have been possible when the port was rebalanced using only the data available to us through P123 alone?” It is clear to me that the data is less stale in the sims and probably NextQ and NextY are pointing to the correct quarter in the sims. So, if that is correct, the sims do not reflect what was possible with just a port rebalance alone. If that is the case, it would not just be understandable why the sims do better than the ports: it would be expected.

Maybe Marco will expand on this question. But as far as voting, you have already made the only rational choice: fix the issues to the extent possible.

Not sure where the confusion is. I’ll rephrase it.

Analysts don’t have a concept of CurrentQ, NextQ, etc, they think in quarters and years, and react to reports. Their estimates look like: ABC will do this in Q1,Q2,Q3,Q4 , and it will do this for 2015,2016,2017. Depending on company/analysts they may have estimates for 5 years and future 8 quarters.

So , for simplicity, an artificial Current Q/Y and Next Q/Y “pointer” is created that each points to one of those estimates. If company ABC reported Q1,Q2, then the CurrentQ should point to Q3/2015, NextQ to Q4/2015, and so on.

When ABC reports Q3 analyst will revise their estimates for the future Q4 2015, Q1 2016 so on. If Compustat is late in processing Q3, then we, P123, keep CurrentQ pointing to Q3 and NextQ to Q4. The NextQ will still reflect the latest revisions for Q4 made by analysts in response to the report. Likewise we will still capture all revisions to 2015 & 2016 since that pointer has not moved yet.

So the only things that are a bit stale while Compustat is processing filings is CurrentQ and CurrentY (but only when Q4 is reported). So if you use them both the overall impact should be minimal, and mainly for small companies

Hope that helps.

Marco,

I get that. I do not think this is trivial. And when you say “(but only when Q4 is reported)” you mean for that example. It is happening every quarter for many companies. The impact is not trivial in my ports. Maybe you can trust me on that or I can show you if you want.

Then there is still iavanti’s question as to whether this is corrected in the sims.

The sims do select different stocks than the ports (over the same time period) without question. It can be that 40% of the stocks are different for a port that rebalances weekly. The sim seems to be buying the stocks with the larger and more recent revisions. Trivial?

I think this is so non-trivial that we are considering changing the backtesting methods and removing the sims altogether from Smart Alpha: so the discrepancy is not so evident.

This has been true all along. I just didn’t know it could be improved. Thank you for letting us know.