Index | Recent Threads | Who's Online | Search

Posts: 71    Pages: 8    Prev 1 2 3 4 5 6 7 8 Next
Last Post
This topic has been viewed 4169 times and has 70 replies
jmh
Re: New data provider! Global stock data!

I second gs3. Great idea.

Jerome

Oct 11, 2019 5:54:32 PM       
Jrinne
Re: New data provider! Global stock data!

When was the P123 method developed? 2004? Before?

It remains a good system. But it is also true that there have been advancements in the last decade-and-a-half. At least FactSet thinks so.

FactSet:

"Instantly interact with data from the Marketplace in a fully hosted environment that includes industry standard databases, programming languages (Python and R), and data visualization tools."

Note the adjective "industry standard" applies to programming languages Python and R in this sentence.

And this:

"….integrates advanced automated machine learning technology allowing you to build, deploy, monitor, and manage sophisticated machine learning models quickly and easily."

I know P123 works largely based on out-of-sample data since I joined.

If I have to start from scratch I would like to have access to some modern tools (if possible) for validation of a new system.

FactSet finds some of the more modern methods valuable for advertisements, if nothing else. I certainly would not turn off any developments since 2004 based on the philosophical beliefs of one or two P123 members alone.

I cannot imagine that they would charge too much for the Open Source Python. But if they do charge a lot extra, there is the API, perhaps. I would like to be offered these options at a reasonable price.

Anyway, it won’t be 2004 forever. I do think that ranking will always be valuable because it is a HUGELY SUCCESSFUL WAY OF REDUCING THE NOISE IN THE DATA so I do not think it will be hard to catch up to the year 2020.

This could be an opportunity for all of use to learn a new trick or two.

-Jim

From time to time you will encounter Luddites, who are beyond redemption.
--de Prado, Marcos López on the topic of machine learning for financial applications

Oct 11, 2019 6:35:00 PM       
Edit 6 times, last edit by Jrinne at Oct 11, 2019 6:56:59 PM
Jrinne
Re: New data provider! Global stock data!

As per Wikipedia: " In 2008, FactSet bought a copy of Thomson's fundamentals database,[27] securing permanent access to global financial data going back to 1980."

Looks like pretty serious (seriously good) data.

I do not know if an API would be possible: I do not know how FactSet operates but I get that SP500, ultimately, will not give you access to any tools that work. Proof that P123 does work in its present form.

Please, just look at some way to have the data interact with a Python program at some point—even if none of the output can be downloaded. Just if you can as you look at any options and finalize any negotiations.

There are a lot of Python programs and any general opinions about Python will be wrong about some of the specific programs--without getting into a philosophical debate.

Note: Pandas in Python was developed by AQR Capital Management specifically for Quantitative Investing. Something that FactSet seems to recognize when they call the program an "Industry Standard." Theoretically, there could be something there for someone. If not now, maybe someone could (just theoretically) develop something useful in the future. Maybe even someone at P123.

I do get that I am hoping that FactSet is not like the SP500 (and my banker) who only lends someone an umbrella when the sun is shining. When it cannot really be used effectively.

More about their data:

"The company receives data from providers such as Barra, Dow Jones, Russell, and Lipper. Since contractual relationships with third-party vendors can be terminated with one year's notice, the company tries to maintain relationships with at least two vendors for each type of data.[25] Recently the company has attempted to maintain or increase its available content by building its own databases or by acquiring content providers."

And about their size (potential scope):

For 2014, FactSet's 36th year of operation, the company recorded its 34th consecutive year of revenue growth.[23]

For the fiscal year ended August 31, 2014, FactSet revenues increased to $920 million, growing 7.00% year on year.

-Jim

From time to time you will encounter Luddites, who are beyond redemption.
--de Prado, Marcos López on the topic of machine learning for financial applications

Oct 12, 2019 7:23:38 AM       
Edit 7 times, last edit by Jrinne at Oct 12, 2019 10:14:15 AM
Jrinne
Re: New data provider! Global stock data!

Hmmmm…..

Looks like someone at FactSet already found some of the things that can be documented to work with Python.

Guess I was not the first to discover it which makes sense because it is pretty obvious really. Since it turns out not to be such a big secret and since those reading this will probably never use it anyway…..Plus, everyone’s implementation (and control of capital) will be a little different (and limited) should anyone at P123 ever use this.

IN AGGREGATE (including investors outside of P123) THIS IS MAY BE WHY P123’s TECHNOLOGY IS NOT WORKING AS WELL FOR US IN 2019. I can show some academic articles that suggest this.

From FactSet’s site. I could be more specific, uhh after a letter of intent and an NDA.

Just to summarize: It won’t be 2004 forever AND WITH CHANGE COMES OPPORTUNITY.

Attachment Screen Shot 2019-10-12 at 9.02.50 AM.png (949672 bytes) (Download count: 213)


From time to time you will encounter Luddites, who are beyond redemption.
--de Prado, Marcos López on the topic of machine learning for financial applications

Oct 12, 2019 8:07:22 AM       
Edit 9 times, last edit by Jrinne at Oct 12, 2019 8:39:19 AM
U5YJ52993752gH7p5fx8
Re: New data provider! Global stock data!

Yuval,

Is Mexico one of the supported countries?

I Started Out with Nuthin. Still Got Most of It Left. - SeaSick Steve

Oct 12, 2019 8:18:08 AM       
philjoe
Re: New data provider! Global stock data!

HOLY CRAP THIS IS AWESOME

Oct 12, 2019 2:25:22 PM       
yuvaltaylor
Re: New data provider! Global stock data!


Please, just look at some way to have the data interact with a Python program at some point—even if none of the output can be downloaded.


Python is fully compatible with CSV files, and since all our downloadable results are in CSV format, I'm not sure what the problem is. Perhaps you can put this in a feature request, or at least start a new thread with specific, concrete examples? I don't think this particular thread is a good place to discuss Python interactivity. Please also specify which limitations of our output prevent it from meshing smoothly with Python-coded programs, what sorts of Python-coded programs you would write that would use this data, and whether your request would need to be modified in order to apply to other programming languages.

In general, P123 is trying to serve investors, not programmers. But if there are enough programmers like you who find our output frustrating in some way, we could consider use cases.

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Oct 12, 2019 5:43:12 PM       
Jrinne
Re: New data provider! Global stock data!

See feature suggestion as requested.

From time to time you will encounter Luddites, who are beyond redemption.
--de Prado, Marcos López on the topic of machine learning for financial applications

Oct 12, 2019 6:17:15 PM       
Edit 3 times, last edit by Jrinne at Oct 12, 2019 7:07:02 PM
jmh
Re: New data provider! Global stock data!

P123,

when I look at the link that Wwasilev posted on the Python thread (https://www.quantopian.com/docs/data-reference/factset_fundamentals), I get some concern on the PIT quality of FacSet.

Here is an extract from the web page ->

"Point-In-Time
Starting in June 2018, FactSet fundamentals data is collected and surfaced in a point-in-time fashion. This corresponds to when Quantopian started downloading and storing FactSet fundamentals on a nightly basis. Prior to June 2018, FactSet fundamentals data is surfaced in simulations based on the report date provided by the vendor..."

Any thoughts?

Thank you,

Jerome

Oct 13, 2019 2:37:44 PM       
marco
Re: New data provider! Global stock data!

If I recall it's the way Factset deals with prelim data. I think they process a press release and fill in, for example , half the line items. When the actual filing is processed the prelim data is overwritten and there's no way to know which items were not present. My guess is that Quantopian is keeping both versions of the statement, similar to what Compustat does, to be more "precise".

These incomplete prelim statements are the reason why we have fallback mechanisms in P123 . It's all very complicated and avoids NA's, but is it really more "precise"? In between prelims and final filings it's quite easy to end up with a mix of lineitems, some from the current prelim quarter , some from the previous final one. Then you have SEC filings which everyone can see at the same time but standardized data is made available from data providers at different times depending on their processes.

And then of course you have ticker action reacting while the press release is being read.

So what's all this "precision" for?

Rest assured we'll think about it hard. We'll use all our experience to figure out what's best for *our* purposes. Maybe avoid buying or selling a stock if it's in the prelim period might be the right answer all along -- even with Compustat data.

Portfolio123 Staff.

Oct 13, 2019 8:35:40 PM       
Posts: 71    Pages: 8    Prev 1 2 3 4 5 6 7 8 Next
 Last Post