Short Interest Path Forward

It was mentioned that Portfolio123 was considering alternative providers of short interest data because Factset does not provide that. As it has been declared that Portfolio123 is switching to Factset with the old licenses no longer in effect later this month, I am wondering if there will be a solution or if we will have to disregard models that rely heavily on that sentiment factor. I have tried share turnover and while it does help sentiment it is a far cry from the power of short interest.

Thanks,

Jeff

fwiw, If the licensing the most current short interest data is price prohibitive, I think a lagged or out-of-date short interest factor might be useful for how many use it if it’s more economical to acquire. Delayed data is not as good as the freshest data in my testing, but it’s still of value. I use a very simple ranking of short interest as a rank from low% to high% and most companies don’t swing widely from month to month.

I don’t know how others feel about it, but wanted to mention this consideration if the licensing economics are an issue. I would find even delayed short-interest data of use for current day screening.

Was shortsqueeze.com mentioned as a possibility? Has anyone used their data before? It looks like you can get current and 2 years of historical short data for $40/month at least individually.

I agree

I think that delayed data would be greatly disadvantageous to subscribers. It means we would be trading on yesterday’s news. In addition, what kind of adjustments would we need to make to existing models to allow for lags.

Given the quality of its current offerings, I don’t think this would be a good precedent for p123 to make.

Surely short interest data is available somewhere at a reasonable cost.

Hugh

Any update from p123 staff on this item? Should we start thinking about alternative routes?

I am disappointed in this transition. I understand the reasons for transitioning to Factset but it seems that some critical pieces were not planned for. Now it seems we are paying the same but ending up with less actionable information. This is not just on short interest, but in other areas as well. As I understand the previous provider calculated a lot of values that may not have been explicitly reported. Factset not so much. It is really unfortunate and I have been thinking about whether I want to continue to subscribe especially if the data is no longer available to support my models.

We will be providing short interest but haven’t yet figured out how and whether there will be an additional cost.

In terms of “less actionable information,” the switch to FactSet is enabling us to offer: a) better coverage, with over a thousand additional stocks added to All Fundamentals, including some of the largest companies in the world, which were missing from Compustat’s coverage; b) tons of additional estimates data; c) a much improved global classification system.

Regarding this: “As I understand the previous provider calculated a lot of values that may not have been explicitly reported. Factset not so much.” Both providers calculate a lot of values that are not explicitly reported, but they do so in very different ways. And the way Compustat calculated, for example, CostG for banks and insurance companies really makes no sense. They were treating that number entirely differently for those two industries but pretending that they were standardizing it, and they were giving us numbers for CashEquiv that were radically different from those they put up on their Capital IQ site.

In addition, the transition has enabled us to find a lot of bugs that nobody had noticed earlier, including discrepancies in how we were treating interim preliminaries and in how companies that report semiannually were flagged and how their numbers were presented. Those have mostly been fixed now, though additional fixes are still to come.

I’m sorry that the transition has been a rocky process. You’re right that it hasn’t been smooth sailing or pain-free. But please read once again the document I have posted here, outlining all the changes as fairly as possible. https://docs.google.com/document/d/17Q0cVnGdGwKGEXFZOuwf297itKdqUC_KGN2UHcMTdYg/edit?usp=sharing

That’s great news!

Speaking as a software engineer, under these very challenging circumstances, I am quite pleased with the job that Yuval, Paul, and all the engineers are doing for the transition. Very few could do this better.

In addition, I hope that once the transition period is over, we can more fully utilize the advantages of Factset. In particular, I am looking forward to international stocks and more productive analyst estimates.

[quote]

Completely agree. I know deadline is approaching and nerves are frayed, but P123 staff have done a very admirable job.

Another benefit is that the peace of mind in knowing that my model advantages weren’t just data mining unique idiosyncratic features of the Compustat data, which had always been a nagging concern. I’m able to satisfactorily replicate results fairly closely now between two different data providers.

On what date are we losing the Short Interest data?

ALso, is it possible for me to upload the data myself and use it in my live strategy? I have my own access.

You are losing it on July 1st as everyone is going on Factset at that moment.

You can get custom time series to use with your strategies but not a full database of short interest.

Sure you can import your own data . See Research → Imported Stock Factors . It’s relatively new feature and probably needs a few tweaks to allow massive imports like a point in time factor for the entire universe. But we welcome questions and suggestions.

I’m trying to keep S&P short interest data since it looks like we will maintain a relationship with them to support users that want S&P data. It’s a negotiation in progress. It’s also a litmus test on how much they appreciate us, and our joint venture :slight_smile: If it doesn’t work out we have other options but it’s a bit costly and requires integration. In either case I expect that there will be ways to avoid sudden interruptions.

Marco - is S&P short interest the only variable not available though FactSet?

I know short interest is the most widely used but if there are other is there a list that we can refer to?

Thank you.

Yes, short interest (and derived) is the only snag we hit. Everything else we have replacements.

FYI: I just condensed PIT data of the entire universe (one factor) into just 2.9 MB.

The trick?
Step one: Symbols and dates as row and column headers respectively. That created a 28.4 MB csv file.
Step two: Compress the csv into a zip file. Result? 14.8 million records (1,119 weeks by 13,217 symbols) into 2.9 million bytes.

Sounds great! This is a necessity.