Index | Recent Threads | Who's Online | Search

Posts: 82    Pages: 9    Prev 1 2 3 4 5 6 7 8 9 Next
Last Post
This topic has been viewed 4534 times and has 81 replies
marco
Re: NEW: Data Miner App & P123 API -- v1.0 (beta)

Let's put an example. I would like to see the result of 20 different rolling backtets every 4 weeks. Like the result that I can download with the Rolling Backtest in the screener, if I select Holding period & rebalance period of 4 weeks.


Isn't the RollingScreen operation what you want? Below is image of the output of the RollingScreen sample . You can save the result and view in excel (or copy paste directly). The sample I used is in the dropbox shared folder and is called

"RollingScreen - Testing momentum ranking with different rules.yaml"

There are 5 different versions of the backtest that share the same period, 1 year holding, shifted 4 weeks, for a total of 127 periods of 365 days.

This takes one click to run using DataMiner . The only difference between this and the website output is that we only show the average row of the periods for each backtest. So 5 rows in the output. Missing are the row for each period, and the averages for Up / Down markets. But we can easily add this detail if needed.

Also please note the the RollingScreen operation has two extra columns not present in the website Last13AvgRet and Last65AvgRet . These represent the average return of the most recent 13 & 65 periods. There's no way to configure this at the moment.

thx

Attachment Capture.PNG (193906 bytes) (Download count: 132)


Portfolio123 Staff.

Sep 15, 2020 10:41:52 PM       
Edit 1 times, last edit by marco at Sep 15, 2020 10:46:36 PM
Jrinne
Re: NEW: Data Miner App & P123 API -- v1.0 (beta)

Let's put an example. I would like to see the result of 20 different rolling backtets every 4 weeks. Like the result that I can download with the Rolling Backtest in the screener, if I select Holding period & rebalance period of 4 weeks.


Isn't the RollingScreen operation what you want? Below is image of the output of the RollingScreen sample . You can save the result and view in excel (or copy paste directly). The sample I used is in the dropbox shared folder and is called

"RollingScreen - Testing momentum ranking with different rules.yaml"

There are 5 different versions of the backtest that share the same period, 1 year holding, shifted 4 weeks, for a total of 127 periods of 365 days.

This takes one click to run using DataMiner . The only difference between this and the website output is that we only show the average row of the periods for each backtest. So 5 rows in the output. Missing are the row for each period, and the averages for Up / Down markets. But we can easily add this detail if needed.

Also please note the the RollingScreen operation has two extra columns not present in the website Last13AvgRet and Last65AvgRet . These represent the average return of the most recent 13 & 65 periods. There's no way to configure this at the moment.

thx


Marco,

Thank you for your post. Is there data in this download that can be manipulated to get new information and develop a model in Python?

It is excellent that you and Yuval are addressing a user's interest in rolling backtests. This is very much appreciated.

Still, I do not think you are going to attract a massive group of new users by duplicating what is available on P123 as far as rolling backtests.

Quantopian, to some extent, tapped into a market of machine leaners. But they really have not done it well and they have not taken over the world yet. I think they have not address the use of fundamentals (P123’s strength) at all. But I think they have proven that there is a market.

I think P123 can not only attract some Quantopian users but tap into a larger group. A serious mainstream group:

Like people at UC Berkeley’s Master of FINANCIAL ENGINEERING Program: LINK

Kaggle and other machine learners address a broader spectrum of problems but I think they use the same techniques. Some of them use money and might want to use their skills to make some (money) at P123. I think they are a pretty large group.

Rolling backtests are fine. And again, thank you for addressing this for people who are interested in rolling backtests.

But I would think about calling a professor and maybe even paying a graduate student from a FINANCIAL ENGINEERING department to see what they would want. See if you could attract them (and people like them) to P123.

Truly just a marketing suggestion. I am very happy with the present services at P123.

Best,

Jim

From time to time you will encounter Luddites, who are beyond redemption.
--de Prado, Marcos López on the topic of machine learning for financial applications

Sep 16, 2020 5:56:02 AM       
Edit 8 times, last edit by Jrinne at Sep 16, 2020 8:03:08 AM
ivillalongabarreiro
Re: NEW: Data Miner App & P123 API -- v1.0 (beta)

Thank You, Yuval and Marco.

Indeed, Marco, what I want is the row for each period, as I can get in the website.

For me, that provides multiple operations and I can use the data for other ML processes.

If I understood correctly, this will be available in the future. But I think that it could be one of the most powerful features of the data miner.

otherwise, I cannot see how a ranking system has evolved in different periods, except if I create a new different iteration for each period, which in the case of months we are talking about 250 and around 1100 weeks. Too many interatios to be written manually.

Maybe there is a way to create a script in a text editor to create it automatically. Let's see... XD

Anyway, thank you very much, and please consider to include the CSVs with all the rows for the rolling backtests. That would be really nice :)

Sep 17, 2020 5:51:50 AM       
piard2
Re: NEW: Data Miner App & P123 API -- v1.0 (beta)


Quantopian, to some extent, tapped into a market of machine leaners. But they really have not done it well and they have not taken over the world yet. I think they have not address the use of fundamentals (P123’s strength) at all


Hello Jim and everyone reading this thread. A precision... In theory Quantopian is already more powerful than P123 to build fundamental models, with the condition of being comfortable in their coding framework. They have both US Factset and Morningstar fundamental data, you can process them and even mix them in any way allowed by Numpy, Pandas, Zipline, statistical functions. In fact you can build and mix filtering rules and ranking systems based on fundamentals and technicals like in P123, in a more flexible way (I translated 2 screens and a ranking system from P123 to Quantopian, just to assess feasability). The framework also allows to set simulation parameters (transaction costs, position sizing, etc...)
However, practically the free version is unusable for anything else than very basic models, because backtests are way too long (probably of a 100x magnitude compared to P123). Moreover, in the free version data periods are limited and you can't create and import your own packages (no code reusability). The pro version has all the data history, is likely faster, and probably allows code reusability, but it's too expensive for me.

As for their ML features, the whitelisted packages are limited, but I have made a few tests of market regime detection models with random forests and bagging SVMs, technically it works and I can get metrics from the tests (I didn't find any actionable alpha yet).

Sep 19, 2020 3:53:05 AM       
Edit 5 times, last edit by piard2 at Sep 19, 2020 4:40:33 AM
Jrinne
Re: NEW: Data Miner App & P123 API -- v1.0 (beta)

Federic,

Thank you.

You are a better programmer than I am. That is just a fact and I mean that sincerely. That is pertinent here because I seem to be able to do stuff with Quantopian’s technical data. But I just can’t get the clean wonderful fundamental data that the P123 team provides over at Quantopian.

I watched P123 build the data for FactSet (something else I cannot do as a non-programmer). Near as I can tell there is real value there.

But once the data is arranged properly, I have no problem using Boosting, developing a neural net or using other ML methods. And they do work, plain and simple.

Seriously, does anyone really believe that P123’s linear weighting of ranks works but nothing else does? Like magic, the only useful method that has ever been discovered is in just one place for retail investors and at a reasonable price (except at Renaissance Technologies I guess). You have to be kidding me. That is not how the world works.

IMHO, P123 is sooooo close to having something that is vastly superior to Quantopian.

But at a minimum they would have to have an easy download of the new overnight data in the morning to make predictions and to allow SORTING ALL OF THE PREDICTIONS FOR ALL OF THE STOCKS IN THE UNIVERSE to find the best 5, 10,…or 25 stocks to buy.

And honestly, if P123 wants to develop a market I would not limit the downloads to professional programmers in the top percentile of Python literacy using DataMiner on a windows machine (just me). I think it is a fact that even the best programmers are not getting what they want with DataMiner 1.0.

Maybe DataMiner is the best way. But I would replace "Samples" with "Templates" and make it very user friendly. Friendly enough that an undergraduate using SPSS can get the data she wants without calling Aaron. And if someone does get Aaron’s help that it is needed just once and that a series of templates are available afterward negating the need for any further calls.

Actually, I would scrap DataMiner if it was me.

But whatever P123 does is fine with me. I have not been sitting idly-by waiting for this. I use other sites now. And I have found ways to use ML without having to have every stock in the universe in the sample.

I truly can do without any of this. And probably, I am better off if P123 does not attract a bunch of machine learners to compete with my models anyway.

If P123 develops new abilities I will use them so I am perhaps indifferent. Indifferent except that I would like P123 to survive.

Did I mention that I watched P123 build the FactSet data and I think there is real value there? I appreciate the work that P123 has done.

Wishing everyone the best,

Jim

From time to time you will encounter Luddites, who are beyond redemption.
--de Prado, Marcos López on the topic of machine learning for financial applications

Sep 19, 2020 5:54:33 AM       
Edit 7 times, last edit by Jrinne at Sep 19, 2020 11:33:08 AM
piard2
Re: NEW: Data Miner App & P123 API -- v1.0 (beta)


But once the data is arranged properly, I have no problem using Boosting, developing a neural net or using other ML methods. And they do work, plain and simple.


The value added by the model designer in ML is precisely in the data wrangling process before choosing and tuning an algo. Choosing and tuning an algo will be more and more automated in AutoML frameworks. Arranging data properly is the biggest part of the job and we can't expect to have it done completely. Some visual tools allow to make data wrangling pipelines without programming. Azure ML studio looks promising for that, but it can't compete yet with a python/pandas programmer.

Sep 21, 2020 5:35:46 AM       
Edit 4 times, last edit by piard2 at Sep 21, 2020 6:12:54 AM
Jrinne
Re: NEW: Data Miner App & P123 API -- v1.0 (beta)


But once the data is arranged properly, I have no problem using Boosting, developing a neural net or using other ML methods. And they do work, plain and simple.


The value added by the model designer in ML is precisely in the data wrangling process before choosing and tuning an algo. Choosing and tuning an algo will be more and more automated in AutoML frameworks. Arranging data properly is the biggest part of the job and we can't expect to have it done completely. Some visual tools allow to make data wrangling pipelines without programming. Azure ML studio looks promising for that, but it can't compete yet with a python/pandas programmer.

Frederic,

I agree and I may need to check out Azure ML. Thank you for the tip.

But P123’s tools are pretty powerful. Not that I recommend always using a screwdriver when you need a hammer (which I may be doing here).

But, if it does not work in a sim or a rank performance test do not expect a miracle in that random forest you visit. This is not the story of Little Red Riding Hood and magic in some random forest. I get that as well as anybody.

P123, thank you VERY MUCH for the rank performance tests, easy downloads from the rank performance tests, sims etc. The tools go a long way toward data preparation. Not to mention everything you do with the data before I see it as a rank.

Actually, let me be the first to say (again) that a lot of people use Excel Spreadsheet downloads, panels (e.g., fundamental analysis) etc without ever programing a Random Forest and they do very well. And I am not sure I actually need anything more than the excellent service P123 already provides.

But I am not P123’s entire (potential) market either. As I said, there are a lot of successful methods at P123 and elsewhere.

Best,

Jim

From time to time you will encounter Luddites, who are beyond redemption.
--de Prado, Marcos López on the topic of machine learning for financial applications

Sep 21, 2020 9:07:43 AM       
Edit 6 times, last edit by Jrinne at Sep 21, 2020 12:24:17 PM
philjoe
Re: NEW: Data Miner App & P123 API -- v1.0 (beta)

Is there a schedule as to when the Data Miner will be updated to make it usable?

Sep 21, 2020 12:45:29 PM       
yuvaltaylor
Re: NEW: Data Miner App & P123 API -- v1.0 (beta)

Is there a schedule as to when the Data Miner will be updated to make it usable?


No. As I'm sure you know by now, we rarely set firm dates for improvements and new features. I'm working on getting this out as fast as we can, but if I gave you an estimate, I would likely be proven wrong.

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Sep 21, 2020 9:58:00 PM       
Jrinne
Re: NEW: Data Miner App & P123 API -- v1.0 (beta)

Is there a schedule as to when the Data Miner will be updated to make it usable?


No. As I'm sure you know by now, we rarely set firm dates for improvements and new features. I'm working on getting this out as fast as we can, but if I gave you an estimate, I would likely be proven wrong.


Yuval,

You should take all of the time you need to get this right, IMHO.

You said you are not a programmer or at least have not used Python much. If you make it so that you can use this easily, I am sure it will be a great addition to P123.

Whatever your preferences may be, I can say with certainty that I would like to do as little data wrangling with Python as possible.

But I think you should make it even easier than what both of use could use. I think you should make it so that an undergraduate who has not had much programming training (not yet anyway) could download the data she needs (into a csv file) and plug it into SPSS easily without calling anyone. Even on her MacBook (pretty common even for advanced programmers and for undergraduate students).

Didn't Marc say he uses a MacBook? Marc is actually a very high-level user—even of statistical methods. It would not hurt if you keep this easy for him (and users like him). For me too (I am not picking on Marc but using him as an example of someone who should be able to use this).

BTW, perhaps the majority of Kaggle competitors use MacBooks. For sure, a lot do.

I am pretty sure you will get some additional users with this--if the data wrangling is kept to a minimum. Marc can probably find a windows machine (or use Bootcamp) if necessary. I can anyway.

Does P123 want to market only to Python users? SPSS is better for many things including multiple regression (Econometrics)j, Principle Component Analysis, Factor Analysis Bootstrapping etc.

Me. I would use Python (TensorFlow) for neural nets and SPSS for factor analysis.

Specifically, you should think about making it such that someone could download a CSV file and immediately upload that file into SPSS without any thought (and certainly without any Python data wrangling). Perhaps even on a MacBook

That should be your standard if you want to attract new members, IMHO.

Best,

Jim

From time to time you will encounter Luddites, who are beyond redemption.
--de Prado, Marcos López on the topic of machine learning for financial applications

Sep 22, 2020 5:51:13 AM       
Edit 9 times, last edit by Jrinne at Sep 22, 2020 6:33:59 AM
Posts: 82    Pages: 9    Prev 1 2 3 4 5 6 7 8 9 Next
 Last Post