Index  | Recent Threads  | Who's Online  | Search
Search  

Total posts in this thread: 22
Posts: 22   Pages: 3    1 2 3 Next
  Last Post
Post new Thread
Author
Previous Thread This topic has been viewed 1142 times and has 21 replies Next Thread
intvalue
Advanced Member



UNITED STATES
Joined: Mar 5, 2009
Post Count: 270
Status: Offline
WHICH ANOMALIES ARE LEGIT? Reply to this Post  Reply with Quote 

P123 users create models based on their own, or academically sourced ideas that they hope are profitable out of sample. Designers who use factors based on their own backtests face the challenge of determining whether or not the productive factors are truly legitimate. Obviously, there is no way to have their conclusions peer-reviewed, which is a requirement for real science. For this reason, many turn to academic research for ideas, but is that research peer reviewed? Many may be surprised to learn that, unlike traditional scientific research it almost always is not, and therefore faces the same challenge.

Last week, researchers at Ohio State published a new study on the NBER website ("Replicating Anomalies," Hou, Zhang, Xue) which may be of interest to the P123 community. The study examines the reliability of 447 different anomalies uncovered by quantitative finance academics and other researchers since the 1980s.

The authors found that more than eight out of 10 anomalies vanish when rigorous tests are applied. Among those failing to reach statistical significance: one anomaly recently set out by the godfathers of quantitative finance, Nobel-winning economist Eugene Fama and his colleague Kenneth French.

EXCERPTS
"We replicate the entire anomalies literature in finance and accounting by compiling a largest-to-date data library that contains 447 anomaly variables. 286 anomalies (64%) are insignificant at the conventional 5% level. Imposing the cutoff t-value of three raises the number of insignificance to 380 (85%). Even for the 161 significant anomalies, their magnitudes are often much lower than reported."

"[The list of anomalies] includes 57, 68, 38, 79, 103, and 102 variables from the momentum, value-versus-growth, investment, profitability, intangibles, and trading frictions categories, respectively. To control for microcaps that are smaller than the 20th percentile of market equity for New York Stock Exchange (NYSE) stocks, we form testing deciles with NYSE breakpoints and value-weighted returns. We treat an anomaly as a replication success if the average return of its high-minus-low decile is significant at the 5% level (t >= 1.96). Our results indicate widespread p-hacking in the anomalies literature."

EXAMPLES OF FAILED ANOMALIES
• "Out of 447 anomalies, 286 (64%) are insignificant at the 5% level. Imposing the cutoff t-value of three raises the number of insignificant anomalies further to 380 (85%)."
• "The biggest casualty is the liquidity literature. In the trading frictions category that contains mostly liquidity variables, 95 out of 102 variables (93%) are insignificant."
• "The distress anomaly is virtually nonexistent in our replication. The Campbell-Hilscher-Szilagyi (2008) failure probability, the O-score and Z-score studied in Dichev (1998), and the Avramov-Chordia-Jostova-Philipov (2009) credit rating all produce insignificant average return spreads."
• "Even for significant anomalies, their magnitudes are often much lower than originally reported. Prominent examples include the Jegadeesh-Titman (1993) price momentum; the Lakonishok-Shleifer-Vishny (1994) cash flow-to-price; the Sloan (1996) operating accruals; the Chan-Jegadeesh-Lakonishok (1996) earnings momentum, formed on standardized unexpected earnings, abnormal returns around earnings announcements, and revisions in analysts’ earnings forecasts; the Cohen-Frazzini (2008) customer momentum; and the Cooper-Gulen-Schill (2008) asset growth."

SOME TAKE-AWAYS
Market anomalies which passed the new study’s tests for statistical significance included several of the biggest. Cheap stocks indeed beat expensive ones; share prices have momentum; companies that invest a lot underperform, and quality of earnings matters. However, 85% of anomalies did not pass the test, including many that are well-known and frequently referenced by the p123 community. Serious p123 model designers may wish to download and review this research paper ($5 from NBER or SSRN) for insights into factors that remain durable under rigorous, unbiased testing by peers, which is a fundamental tenet of scientific legitimacy.

Chris
----------------------------------------
Edit 4 times, last edit by intvalue at May 14, 2017 4:32:01 PM
May 13, 2017 7:20:17 PM Show Printable Version of Post     Link   Report threatening or abusive post: please login first   Go to top 
Chipper6
Advanced Member



UNITED STATES
Joined: Dec 13, 2010
Post Count: 1105
Status: Offline
Re: WHICH ANOMALIES ARE LEGIT? Reply to this Post  Reply with Quote 

Nice paper Chris, thanks.

My first impressions:

He tested every "known" (i.e. published) anomaly. He used market cap weights and measured performance of tenth decile minus first decile. This means that his results are directly applicable to us who use equal weights.

Stock price reaction to earnings announcements should be simple enough for Portfolio123. Unfortunately and surprisingly, they dropped the ball on this one. There seems no way for us to accurately model this, despite the fact that we seem to have the data.
----------------------------------------
Edit 1 times, last edit by Chipper6 at May 14, 2017 12:35:36 AM
May 13, 2017 9:18:24 PM Show Printable Version of Post     Link   Report threatening or abusive post: please login first   Go to top 
primus
Advanced Member
Member's Avatar

UNITED STATES
Joined: Aug 9, 2013
Post Count: 404
Status: Offline
Re: WHICH ANOMALIES ARE LEGIT? Reply to this Post  Reply with Quote 

Chris,

Lovely post, Chris. You greatly added to my Mother's Day reading, much to the chagrin of mothers of the world.

First impressions:
- Not all that surprised by paper's conclusion, but mildly feeling vindicated.
- Also, says "Our results indicate widespread p-hacking in the anomalies literature." Uses p-value of .03. ... teeheehee

Chaim,

Stock price reaction to earnings announcements should be simple enough for Portfolio123. Unfortunately and surprisingly, they dropped the ball on this one. There seems no way for us to accurately model this, despite the fact that we seem to have the data.


Seem like we should be able to model this. Can you explain where the difficulty lies?
----------------------------------------
"The world is. The world is. Love and life are deep maybe as his eyes are wide." - Rush, "Tom Sawyer"
http://the-world-is.com/blog
May 14, 2017 2:04:47 AM Show Printable Version of Post     Link   Report threatening or abusive post: please login first   Go to top 
Jrinne
Advanced Member
Member's Avatar

UNITED STATES
Joined: Feb 23, 2013
Post Count: 1748
Status: Offline
Re: WHICH ANOMALIES ARE LEGIT? Reply to this Post  Reply with Quote 

Nice to see some of you are developing an appreciation for statistics!!! At least when someone else has done it.

The only thing better than this paper would be to find that your own studies confirm these studies and that the anomalies are working for you. Then you could be really sure that you are on solid ground—and get rid of the trash once and for all.

Not that I am done or that I have even gotten a good start yet. And it is true: P123 is a statistics platform with backtests and rank performance tests often telling you all you need to know without a formal t-statistic or p-value.

You can do some things that aren't in this paper. You can add-in slippage up front. Do you really want to compare the upper decile to the lower decile? Why not look at what you are really interest in: the upper decile compared to the benchmark.

P123 in not just a statistics platform: P123 IS A GREAT STATISTICS PLATFORM USING THE SAME DATA SOURCE AS THIS PAPER!

Thanks Chris.

-Jim
----------------------------------------
"Financial markets are noisy. You'll do a lot better fitting the noise than you will fitting the signal (in-sample)"
—Yaser Abu-Mostafa
----------------------------------------
Edit 16 times, last edit by Jrinne at May 14, 2017 7:31:29 AM
May 14, 2017 6:02:28 AM Show Printable Version of Post     Link   Report threatening or abusive post: please login first   Go to top 
Chipper6
Advanced Member



UNITED STATES
Joined: Dec 13, 2010
Post Count: 1105
Status: Offline
Re: WHICH ANOMALIES ARE LEGIT? Reply to this Post  Reply with Quote 

David (primus),

To model stock price reaction to earnings announcements, we would do something like this:
Close(BarsSinceAnnouncementQ0 - 3) / Close(BarsSinceAnnouncementQ0 + 1)
But we are missing a variable. How do we estimate the number of trading days since the earnings announcement?

Seem like we should be able to model this. Can you explain where the difficulty lies?

What's your formula?
----------------------------------------
Edit 4 times, last edit by Chipper6 at May 14, 2017 3:56:04 PM
May 14, 2017 11:54:12 AM Show Printable Version of Post     Link   Report threatening or abusive post: please login first   Go to top 
davidbv
Advanced Member
Member's Avatar

UNITED STATES
Joined: Feb 24, 2013
Post Count: 1099
Status: Offline
Re: WHICH ANOMALIES ARE LEGIT? Reply to this Post  Reply with Quote 

Not that I am paranoid about such things, but what if these researchers for the National Bureau of Economic Research wanted to 'prove' that, quote, "In all, capital markets are more efficient than previously recognized."
If one does not verify their findings, then it would be more fuel to the fire that if you are beating the market, you must be doing something 'odd' ,ie, illegal...
Not that I am paranoid...
----------------------------------------
David
May 14, 2017 2:14:35 PM Show Printable Version of Post     Link   Report threatening or abusive post: please login first   Go to top 
judgetrade
Advanced Member
Member's Avatar

GERMANY
Joined: Apr 21, 2010
Post Count: 639
Status: Offline
Re: WHICH ANOMALIES ARE LEGIT? Reply to this Post  Reply with Quote 

If you got 100 billion hedge and there is almost no way you beat the sp500 long term. Buffet gave away a lot of bets to big
hedge fund managers, but they did not take them or lost not beating the sp500.

If you have a port of 1 Million there are a lot of "anomalies", a lot more then they find, because they will make an Assumtion
on liquidity and port size and transaction costs and slippage that is based on a bigger then 1 Million port.

With a small port like a million, Size (lower market cap), low vol, value, momentum (though only slightly weighted, 10% is enough) do
work just fine and there will be other niches as well with that kind of small port size...
Regards

Andreas
May 14, 2017 3:06:33 PM Show Printable Version of Post     Link   Report threatening or abusive post: please login first   Go to top 
messier11
Advanced Member



UNITED STATES
Joined: Jun 16, 2009
Post Count: 41
Status: Offline
smile Re: WHICH ANOMALIES ARE LEGIT? Reply to this Post  Reply with Quote 

Like typical academic fools, Hou, Zhang, and Xue publish their results. That is great for science, stupid in finance.

The most important thing I have learned in 25 years in markets is that if you tell people about something, it won't work anymore.

If you don't have an edge, you won't win as a trader. If you give away your edge, you no longer have an edge.

I you don't believe what I just said, then you should quit trading and go work in academia :-)
----------------------------------------
Edit 1 times, last edit by messier11 at May 15, 2017 9:10:26 PM
May 14, 2017 9:41:28 PM Show Printable Version of Post     Link   Report threatening or abusive post: please login first   Go to top 
messier11
Advanced Member



UNITED STATES
Joined: Jun 16, 2009
Post Count: 41
Status: Offline
Re: WHICH ANOMALIES ARE LEGIT? Reply to this Post  Reply with Quote 

Buffett makes the S&P bet.... but a lot of people beat the S&P for Sharpe ratio and if you manage money carefully, that can be the most important stat.
May 14, 2017 10:00:21 PM Show Printable Version of Post     Link   Report threatening or abusive post: please login first   Go to top 
primus
Advanced Member
Member's Avatar

UNITED STATES
Joined: Aug 9, 2013
Post Count: 404
Status: Offline
Re: WHICH ANOMALIES ARE LEGIT? Reply to this Post  Reply with Quote 

How close does "LatestActualDays" get you?

I am looking at 1-month drift vs. standardized unexpected earnings.
----------------------------------------
"The world is. The world is. Love and life are deep maybe as his eyes are wide." - Rush, "Tom Sawyer"
http://the-world-is.com/blog
May 15, 2017 3:06:50 AM Show Printable Version of Post     Link   Report threatening or abusive post: please login first   Go to top 
Posts: 22   Pages: 3    1 2 3 Next
 Last Post
Show Printable Version of Thread