Index | Recent Threads | Who's Online | Search

Posts: 42    Pages: 5    Prev 1 2 3 4 5 Next
Last Post
New Thread
This topic has been viewed 1645 times and has 41 replies
Jrinne
Re: Are we over fitting?

Steve,

You make an excellent point(s). Too many good points to comment on all of them. But I think I am restating one of your excellent points when I phrase it this way:

If there are only a few gems out there you are going to have to sort through a lot of sand to find one of them.

If it is an efficient market or a pretty-efficient market then you should expect nothing from your first backtest whether it is optimized or not. You are going to have to backtest more than once to find a truly good strategy especially if you are trying to find something that hasn't already been published.

I do not claim to know--or to be able to articulate in one post if I did--how to balance the need for a lot of tests with the problems or overfitting.

Mine is a slightly different point. It helps if the factors that are overfit are not harmful when I fail to find the proper balance-like the first image that I burrowed from the article.

Great theory, "and yet it moves."
-Quote attributed to Galileo Galilei (1564-1642) gets my personal award for the best real-world use of an indirect proof or reductio ad absurdum.
`

Sep 9, 2016 9:40:06 AM       
Edit 9 times, last edit by Jrinne at Sep 9, 2016 10:48:03 AM
MisterChang
Re: Are we over fitting?

All my early models were horribly overfit. I'm convinced some/many SA models are overfit due to fantastical historic simulation and overly-precise and overly-mathematical description from the designer.

More and more, I ask myself "what are the characteristics of the stocks I want to own in this strategy" and pay less and less attention to the historic simulation.

Sep 9, 2016 9:59:12 AM       
yuvaltaylor
Re: Are we over fitting?

Yuval - everything you say is right on, but there are some issues with "examine every measure to make sure it makes logical financial sense". The problem is that what makes financial sense doesn't consider market behavior, what makes sense may still result in significant losses. That is why we try to capture market behavior the best we can through optimization.

Steve


Very good point, Steve. But does "market behavior" have predictive power the way that "financial sense" does? After all, the whole point of backtesting and optimization is to create models that will survive out of sample. "Market behavior" has to be just as explainable as "financial sense" for it to have any out-of-sample power, and in addition there should be a good reason for it to persist.

Let's take two examples of market behavior. One: stocks with a high short ratio tend to perform worse, on the whole, than stocks with a low short ratio. That, to me, is not only explainable, but likely to persist. You're going to find some wonderfully profitable counterexamples of short squeezes, but as a rule of thumb, the short ratio is a rather nice measure of market sentiment, and there's no good reason why that would suddenly reverse. Two: stocks whose price has risen over the last month are more likely to fall over the next week than to continue rising due to short-term mean reversion. This is also explainable by market behavior: investors overreact to news and are then corrected by the market. This has remained constant for 90-odd years now, according to a study I read. But is it likely to persist? Perhaps in the short-term, but given how differently information is assimilated today than how it used to be, that variable is rapidly changing. I would not put my faith in it.

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Sep 9, 2016 10:15:04 AM       
Edit 1 times, last edit by yuvaltaylor at Sep 9, 2016 10:17:34 AM
davidbv
Re: Are we over fitting?

More quants being hired. From todays Bloomberg:
http://www.bloomberg.com/news/articles/2016-0...-battle-to-recruit-quants

David

Sep 9, 2016 10:58:50 AM       
yuvaltaylor
Re: Are we over fitting?

Are all these quants just trying to perfect market timing or are they stock picking?

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Sep 9, 2016 11:42:35 AM       
wwasilev
Re: Are we over fitting?

Market timing, stock picking, data mining? I don't know, but here's a recent python meetup posting;


Financial Engineer (Boston)

(Manager – ___ ___)



The Expertise We’re Looking For

· Ph.D. degree in Electrical Engineering, Computer Science, Computational Finance, Applied Mathematics or other relevant quantitative discipline

· Series 57 and CFA charter holder would be an asset

The Purpose of Your Role

Financial Engineering team provides quantitative expertise to the design of next-generation institutional-grade trading algorithms and products in equity and fixed-income market. We are a member of Fidelity Centralized Electronic Trading within Fidelity Capital Markets. We work with clients to improve their trading strategy, educate them about market structure and trading tools, and provide consultation to help them solve their unique problems.



The Skills You Bring

· You use KDB/Q, Python or R to see and understand large amount of data

· Your ability to use in-depth analytical knowledge to let data tell stories

· You are able to build mathematical models for prediction and optimize performance

· You are eager to learn new things, share ideas freely and create positive work environment

· You have excellent presentation skill to influence product directions and work with clients

The Value You Deliver

· Creating predictive volume, liquidity, price, interest rate and volatility models to improve trading performance

· Building tools, prototypes and simulation engines to streamline processes and test new ideas

· Dissecting and comparing trading performance of algorithms, venues and traders

· Promoting culture of innovation and thought leadership

· Influencing product directions for the benefit of our clients

How Your Work Impacts the Organization

Fidelity Centralized Electronic Trading aspires to be the most influential provider of electronic trading solutions in the marketplace. We accomplish this by protecting our clients’ interests and focusing on innovative research, products and services that empower Fidelity’s partners and clients to achieve their trading objectives. We receive trading commissions from clients when they use our products, and lower transaction cost for Fidelity’s clients.



Company Overview

At Fidelity, we are focused on making our financial expertise broadly accessible and effective in helping people live the lives they want. We are a privately held company that places a high degree of value in creating and nurturing a work environment that attracts the best talent and reflects our commitment to our associates. For information about working at Fidelity, visit FidelityCareers.com



Fidelity Investments is an equal opportunity employer.



I redacted the contact information. If anyone wants that please let me know.

Walter

Sep 9, 2016 11:57:32 AM       
Edit 2 times, last edit by wwasilev at Sep 9, 2016 1:29:17 PM
InspectorSector
Re: Are we over fitting?

does "market behavior" have predictive power the way that "financial sense" does? After all, the whole point of backtesting and optimization is to create models that will survive out of sample. "Market behavior" has to be just as explainable as "financial sense" for it to have any out-of-sample power, and in addition there should be a good reason for it to persist. Let's take two examples of market behavior. One: stocks with a high short ratio tend to perform worse, on the whole, than stocks with a low short ratio. That, to me, is not only explainable, but likely to persist. You're going to find some wonderfully profitable counterexamples of short squeezes, but as a rule of thumb, the short ratio is a rather nice measure of market sentiment, and there's no good reason why that would suddenly reverse.


Yuval - so does the Short Interest have predictive power? Or is it the tenth fundamental factor that you tested but the first one that actually backtests the way you would think it should? When you really think about it, with all the supercomputers out there and hungry hedge funds, why would there be any advantage with short interest? It is a bit too simple and easy to exploit in this very sophisticated world. BTW SiRatio is one of my favorite factors, I've used it for a long time. I use it because it works, not because someone with a theory says it should :-)

Steve

Sep 9, 2016 1:16:27 PM       
InspectorSector
Re: Are we over fitting?

Are all these quants just trying to perfect market timing or are they stock picking?


They are making a lot of money playing with numbers. Where do I sign up? Oh I forgot, I don't have a PhD :-)

Sep 9, 2016 1:32:20 PM       
judgetrade
Re: Are we over fitting?

Gee, is it really that hard?
Value, Momentum and Size and low liquidity and diversification works since 1870 and it will work in the future.
Why?
Because it is mentally hard to buy a micro cap stock (where you need a GTC LIMIT Price to not let explode
slippage, its 10 times easier to use a market order on FB),
that has shown momentum (für example hard to buy an ATH, "Oh my good, this has run much too far, I can not buy this")), has value (hard to detect and also hard to understand to buy the right value and not to buy a falling knife (e.g. combine only with momentum, size and be on the
save side),
has growing EPS Momentum.

Very hard to bare the vola of this kind of stocks and only 0.1% of all investors want to manage a 100 Stock Portfolio
with regular DDs of over 20%.

You can only be successfull, it you do things, that are hard and counterintuitive to do and go the extra mile (100 Stocks for example) that
almost nobody goes. And use your competetive advantage nobody talks and writes about: the relative small size of your portfolio.

Map this criteria to the science of Fama and Frenh and you know what to do:

Implement the method and train you mind to trust a 100% counterintuative investment, trading
strategy and keep it KISS. Let us just do it and stop overcomplicating things.

Regards

Andreas

Best Regards
Andreas

Sep 10, 2016 9:32:40 AM       
WERNER
Re: Are we over fitting?

Hello Andreas,

WISE words ! But stock market afficionados are "fiddlers". We keep fiddling at the edges to "smooth out" the equity curve, which is creating the problems we want to solve. KISS and "keeping the rough edges" are basic principles and counter-intuitive. Therefore, nobody likes them.

Just like you, I think sticking to these principles will let us mine gold if we can tolerate "feeling terrible" when executing a strategy as you outlined above.

Werner

Sep 10, 2016 10:07:21 AM       
Posts: 42    Pages: 5    Prev 1 2 3 4 5 Next
 Last Post