Index | Recent Threads | Who's Online | Search

Posts: 28    Pages: 3    Prev 1 2 3 Next
Last Post
New Thread
This topic has been viewed 958 times and has 27 replies
InspectorSector
smile Re: Question for Yuval

For example, they spent decades worrying about local minima (a potential concern for the boosting Steve Auger uses) that GENERALLY DO NOT EXIST IN MORE THAN 3 DIMENSIONS. Instead, there are "saddle points." Decades wasted by the mathematicians because their intuitive understand of this is no better than ours.

Every tool has advantages and limitations. The key to success is to understand the limitations and use accordingly. And Jim, I think in one dimension (UP) so the 3 dimension minima that you describe is not a problem for me.

Jan 9, 2021 10:13:29 AM       
Jrinne
Re: Question for Yuval

For example, they spent decades worrying about local minima (a potential concern for the boosting Steve Auger uses) that GENERALLY DO NOT EXIST IN MORE THAN 3 DIMENSIONS. Instead, there are "saddle points." Decades wasted by the mathematicians because their intuitive understand of this is no better than ours.

Just to be clear this is a good thing for boosting. I was not trying to be critical of any method.

Also, the only difference between P123 classic and boosting is P123 classic assumes a "flat" hyperplane and boosting allows for things to be flat or a bit of a curve: a manifold. Clearly the same factors can be used for either one.

Oh yea, and pick your poison: manual optimization to find the hyperplane or let a computer do it but learn Python first to find the manifold.

Steve has picked a good tool IMHO. But if your data fits a hyperplane pretty well then P123 classic is probably your best tool. Again just in my opinion. If someone has other reasons to like P123 classic (or boosting) my only recommendation would be to keep using it.

The only point of my posts is that P123 classic is already doing a lot of pretty advanced stuff. Often without the user fully appreciating it.

From time to time you will encounter Luddites, who are beyond redemption.
--de Prado, Marcos López on the topic of machine learning for financial applications

Jan 9, 2021 3:47:31 PM       
Edit 8 times, last edit by Jrinne at Jan 9, 2021 4:36:43 PM
yuvaltaylor
Re: Question for Yuval

The first two pages of the ranking system tutorial discuss this odd blending of weights.
https://www.portfolio123.com/doc/side_help_item.jsp?id=29

But its not clear why it does this instead of using a simple weighted average.
Is there some advantage to calculating it this way?

Tony


Yes, that's the whole point of composite nodes. If you used a simple weighted average, then putting something in a composite node would make no difference at all.

There is an argument for using composite nodes. It goes as follows. You group like factors together. Then you can get a company that is strong in general in each of the factor groups. If you don't group them together, a company can be strong in various discrete factors but be weak in the group as a whole.

There is an even stronger argument for using composite nodes, and it goes as follows. Let's take ROE. Using DuPont analysis, you can break down ROE into three ratios: income to sales (net profit margin), sales to assets (asset turnover), and assets to equity. Now you can rank all those separately in a composite node and you get a quite different ranking (and arguably a more meaningful one) than if you simply ranked companies by their ROE. But if you didn't use a composite node, you would lose sight of ROE altogether as those three factors would just get mixed in with a bunch of unrelated ones.

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Jan 9, 2021 6:21:33 PM       
yuvaltaylor
Re: Question for Yuval

I am trying to wrap my head around this, I understand a composite node is calculated different than a conditional node. Just to be clear for example is using a conditional node with 4 factors calculated the same as using a stock formula or stock factor with 4 separate factors?

Mike, have you read this? https://www.portfolio123.com/doc/side_help_item.jsp?id=29

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Jan 9, 2021 6:25:10 PM       
InspectorSector
Re: Question for Yuval

Steve has picked a good tool IMHO. But if your data fits a hyperplane pretty well then P123 classic is probably your best tool.

Jim - you have been posting about XGBoost for the last two years. Why don't you stop while you are still ahead :-)

Jan 9, 2021 7:26:57 PM       
Jrinne
Re: Question for Yuval

Steve has picked a good tool IMHO. But if your data fits a hyperplane pretty well then P123 classic is probably your best tool.

Jim - you have been posting about XGBoost for the last two years. Why don't you stop while you are still ahead :-)


Steve,

It’s not a competition among methods and a rational person could end up using more than one method. .

In fact I am using a method not discussed in this thread at all now. Actually a couple not discussed in this thread.

P123 classic is pretty amazing really. And this is a thread about P123 classic and composite nodes. Perhaps I should not have mentioned boosting. I did because I think it is the same topic. They are both just ways of mapping out a flat or not-so-flat manifold in hyperspace.

The topic of composite nodes is an interesting topic.

For the record I have used factor analysis as the basis for determining what factors to put into composite nodes and to determine the weights of the nodes and factors.

So I posted with one perspective on how composite nodes can be used.

It worked and made me money. I’m not going to pretend that didn’t happen just because I am a big fan of boosting too.

My apologies to anyone if I promoted one method too much in this thread about composite nodes or tried to discourage anyone from using something that is working for them or that they want to investigate further.

Best,

Jim

From time to time you will encounter Luddites, who are beyond redemption.
--de Prado, Marcos López on the topic of machine learning for financial applications

Jan 9, 2021 9:07:48 PM       
InspectorSector
Re: Question for Yuval

Jim - you are a very smart guy and most of your posts go over most people's heads, certainly mine at least. But the message you are conveying (in my interpretation) is that Marco has wasted time and resources updating the API and dataminer because P123 proper is already superior. Please choose your wording carefully.

So what I would like to say here is that XGBoost is a means to an end, it is not the end. I have ideas of replacing it with my own home-brew ML algorithm that writes back into P123 a Ranking System. The ML algorithm will embrace some of the concepts in XGBoost but will not be decision tree based, and will be easily mapped into an RS. The algo could be along the lines of what I already do with the ranking system optimizer, which is at the heart of Inspector Sector's Cloud Computing.

The API opens new doors for a vast array of applications, not just XGBoost. Thank you Marco/P123 for making improvements to the API and dataminer. I am sure that Jim has great ideas for P123 and I am just as sure that those ideas can be magnified externally using Python.

Jan 10, 2021 7:42:23 AM       
Jrinne
Re: Question for Yuval

Jim - you are a very smart guy and most of your posts go over most people's heads, certainly mine at least. But the message you are conveying (in my interpretation) is that Marco has wasted time and resources updating the API and dataminer because P123 proper is already superior. Please choose your wording carefully.

So what I would like to say here is that XGBoost is a means to an end, it is not the end. I have ideas of replacing it with my own home-brew ML algorithm that writes back into P123 a Ranking System. The ML algorithm will embrace some of the concepts in XGBoost but will not be decision tree based, and will be easily mapped into an RS. The algo could be along the lines of what I already do with the ranking system optimizer, which is at the heart of Inspector Sector's Cloud Computing.

The API opens new doors for a vast array of applications, not just XGBoost. Thank you Marco/P123 for making improvements to the API and dataminer. I am sure that Jim has great ideas for P123 and I am just as sure that those ideas can be magnified externally using Python.

Steve,

My apologies for not being clear about my opinions on boosting in this thread. I just get tired of people—including me when I do it—saying their way is the only way to do it. Especially when the thread is just about composite nodes. I was trying to keep my opinions and biases out of this thread and that my have been perceived as a shift in my opinion. What I should have done is not mention boosting at all in this thread. My apologies to everyone for not doing this.

So here is my personal opinion about boosting which I do not think will contradict anything I have said before. My apologies if this is slightly nuanced.

1) Boosting is a non-linear method.

2) P123 classic is a linear method in that it uses constants for the weights of factors and nodes.

3) Boosting being a non-linear method will handle non-linear data better than P123 classic as a general rule.

4) Most (but not all) financial data is non-linear ESPECIALLY WHEN YOU START CONVERTING THE INPUTS OR PREDICTORS TO RANKS.

Therefore, boosting should be the better method for most financial data, especially when you are using ranks as inputs. And in backtesting I have found this to be the case so far.

Steve, as you know I am the one who introduced you to XGBoost while we were working on some things with TensorFlow. So obviously I like boosting (or I would not have recommended it). This has not changed.

Perhaps we could move any further discussion about boosting to another thread. Composite nodes is an important topic and it deserves its own thread. Yuval has some interesting, important and useful ideas on this topic and I would like to give him (and others) room to express them here.

Jim

From time to time you will encounter Luddites, who are beyond redemption.
--de Prado, Marcos López on the topic of machine learning for financial applications

Jan 10, 2021 9:09:39 AM       
Edit 14 times, last edit by Jrinne at Jan 10, 2021 10:02:42 AM
InspectorSector
Re: Question for Yuval

Steve, as you know I am the one who introduced you to XGBoost while we were working on some things with TensorFlow. So obviously I like boosting (or I would not have recommended it). This has not changed.

Now that is the Jim I know and love!

Attached is the ranking system for Inspector Sector Cloud Computing. It is actually six ranking systems in parallel, each very optimized on its own. Within the individual nodes, you can see that a conditional node is used for bull and bear markets. The condition that determines bull versus bear is when the SKYY ETF moves above or below a moving average.

Conditional nodes can work very well for this sort of application. I recommend making the condition time period based, not factor based. The latter becomes very confusing.

Attachment SKYY.gif (20648 bytes) (Download count: 51)


Jan 10, 2021 9:52:13 AM       
abwillingham
Re: Question for Yuval


Conditional nodes can work very well for this sort of application. I recommend making the condition time period based, not factor based. The latter becomes very confusing.


When you say “time period based”, what do you mean exactly?

Jan 10, 2021 10:26:47 AM       
Posts: 28    Pages: 3    Prev 1 2 3 Next
 Last Post