During world war two they developed a written test of mechanical ability. It had a correlation or around .4 with the job success of mechanics. Then they added a reading comprehension test. When they penalized people with high reading skills the test went from a correlation of .4 to .8. People like myself with little mechanical comprehension were formerly able to get high scores based on reading ability. Once the factor of high scores through reading ability was eliminated it was much better at picking mechanics. In this procedure reading was a moderator variable.
It sounds like you’re describing a value trap. Passes all the “cheap” tests, but security is doing well on the “cheap” test for all the wrong reasons. Filtering deep value is not my thing - but that’s what it sounds like.
There certainly are! Take, for example, P/E. Low P/E stocks may (or may not) tend to outperform high P/E stocks, but the correlation is very tenuous. Add the moderator variable of accruals, penalizing low P/E stocks with high accruals, and the outperformance will be far higher. There are probably a half-dozen other examples I could come up with . . .
Yuval, didn’t you write a piece on seekingalpha about “helper” factors that do just this? Factors that aren’t very useful by themselves, but when combined with another factor create enhanced result?
I think so and I think Yuval has a good example. This is also a favorite topic of Marc’s. Marc puts factors together that interact in a favorable way.
But does being able to read make you a bad mechanic? Or are questions related to reading just diluting the effect of the questions more related to mechanical ability on the test?
Maybe the test was finding some budding novelists along with some good tank mechanics. Not a bad thing but maybe the budding novelist should be put to work writing recruitment brochures (or those mechanics manuals that nobody reads). And a few great mechanics were probably better at math than reading and did not excel on the test. The opportunity to use their skills was missed.
But mechanical abilities may not be correlated with or be affected by reading abilities. So it may not be a true moderator variable.
I think your example is more like picking a period for momentum that has no effect—a noise factor. Put enough weighT on the 9 month momentum (pick your own period for no effect) and you will be diluting the effect of an important factor in your system.
I think you are getting at the essence of making a good ranking system. You want to minimized the fitting of noise factors (also called overfitting) while finding as many of the true (and helpful) moderating factors as possible. Easy to say—harder to do.
Moderator variables - absolutely as Yuval and Jim describe, and ranking system performance tests are a good way to look for them.
But to be fair, questions arise for me regarding the measurement of correlation between reading comprehension and the job success of mechanics. Was it actually measured or assumed? How did they measure job success, by performance reports that could be skewed by dissatisfied intellectuals who were fully capable but wanted a different job? Knowing that the results of that correlation would affect the placement of personnel in service, and their likely future satisfaction with their assignment, also means there were potentially a priori assumptions used, rather than evidence, to categorize candidates.
But it probably did prevent a few instances of future gynecologists performing jeep engine overhauls through tailpipes!
The phrase moderator variables is new but the concept is at the core of stock selection. I’d phraseit as ranking within an economically-financially reasonable sub-sample. Others have talked about the value trap and that is the classic example.
You may want to check out the p123 virtual course in strategy design, which explains how different types of variables interact. It’s not yet visible in the Help area of the beta version of the new site (I’m looking into that), but you can find it all on out presnet site here:
Problem: “The use of age and BMI in occupational settings as determinants of fitness for duty has been avoided owing to concerns related to the Americans with Disabilities Act.35-37” I imagine (without proof) that unions may have a say in this too.
Solution: “However, push-up capacity is a functional test.”
Why this works: If you are pushing 300 pounds with each pushup (high BMI) you cannot do as many pushups.
So, the authors are essentially advocating using pushups to measure the moderator variables of BMI and age (for legal, union, etc, reasons).
Maybe just to save on health insurance costs: in the most egregious cases.
But you can’t take it with you. So, the practical question remains: will I live longer if I practice pushups every morning or should I concentrate my efforts on losing weight (reducing my BMI)?
Passes all the “cheap” tests, but security is doing well on the “cheap” test for all the wrong reasons. Filtering deep value is not my thing - but that’s what it sounds like.