All,
Most of our factors interact.
Does momentum work better with growth stocks or with value stocks?
If a stock ranks high on Value, Quality, Growth and Momentum is it a good stock or have you just diluted everything? Found a stock that is bound to be mediocre? Not that I know the answer to this.
The EASIEST way to find a system that IS GUARANTEED TO WORK would be to find some factors that are proven to work out-of-sample. And if those factor did not interact just pick a stock that has the greatest number of positive factors that you have identified.
I have been thinking about this in the context of Naive Bayes. This sounds complex but the main point is that Naive Bayes assumes there are no interactions among factors.
Examples of methods looking to manage (and benefit from) interactions include: endless sims, spreadsheets for multiple universes (as some do), Random Forests, most ML/AI in general (which I have done).
In other words, most of the things most of us do at P123. But, as I said, this does not apply to Naive Bayes.
I think the surest way to get out of the P123 overfitting trap is to switch to a classification problem, and find a few factors that are independent and proven out-of-sample. Buy stocks with those factors. Simple.
You could use Naive Bayes, to be fancy, but if the factors were few-enough you would just find stocks with all of the factors you have identified as proven to work out-of-sample.
The trick is being sure that the factors do not interact. Or if they do, that they do not interact in a negative way. But it would be NAIVE (as the name Naive Bayes implies) to assume there are no interactions.
So how can these interactions be managed? How do you identify negative interactions? Would it be as simple as removing factors that are negatively correlated? Not quite I think, but that might be a start.
Any thoughts appreciated.
Jim