Weird but very good performance on a sim, looking for thoughts

Hello to all of you guys.

I would like to comment a sim with all of you. The result or the return it’s pretty good, the sim and it’s commands are extremely simple, too much to me, and I do not know how must I take it.

I rank the sim just by P/E, the lower the better, take 100 stocks, rebalance twice a year, and buy just with the Sharpe ratio. There is no more.

I attached the files here, you can have a look at them.

Then, I would like to know your opinion;

Is this “system” reliable?

Can we expected a similar future performance?

Is that a mistake that comes from data?

What do you think about that?

Thanks to all.




Well, I would say yes and no. It just had roughly 45% drawdown so that’s a pretty massive tumble. Excluding 2018, it looks about 13.5%. Assuming you are investing in illiquid stocks with a value tilt that might take in the good and the bad? Maybe. Without further analysis though - it is hard to tell how much risk you are holding. Also, are there any liquidity requirements? Does this include stocks under 10 cents and trading $1,000 per day? Because then the 13.5% isn’t realistic due to compounding needed in tiny stocks. And if you can only trade $100 a day in these stocks, transaction costs and commissions would eat 10% of your capital per trade.

So without knowing anything more - I would say that yes, you can get 13.5% annually by making some risky bets and realizing that you can lose 50% in your first year when the market goes down 10%.

I would be inclined to throw in a liquidity filter, limit short interest, make sure earnings are positive for the past 5 years and at least 1 analyst per stock. See where that takes you. You can still trade twice a year but maybe you can eliminate some potential volatility.

Monday33, I would recommend you read Marc Gerstein’s tutorial series on how to create models from fundamenatl data.
it is here:
https://www.portfolio123.com/doc/side_help_item.jsp?id=200

He explains how you should interpret P/E, how it is a good metric and how it can mislead you. I can give you my own opinion but he has done a much better job of it.

the long and short of it is: don’t just use P/E (along with just Sharpe). It can lead you astray.

And I think what Kurtis says about liquidity is probably equally important.

What Marc has to say is very important. But let me use my own language (a little bit).

First, everyone would have to agree that you have not overfit your model I would think. After all you have only one variable and and have not limited your universe at all. And you are using a rational factor.

Marc has much say about overfitting. But it can be easily summarized to: use rational factors (which you are doing).

But once you are committed to using rational factors then much of what Marc has to say regards the problems of UNDER-FITTING. He uses a pretty much equivalent term of misspecification. BTW, Marc understands the mathematics of all of this much better than he likes to admit.

There is another related concept that Marc likes to discuss. That is interaction of the variables. His models do well in large part (I think) because he used factors that interact with each other in a positive way. He uses ideas that are well developed from his finance degrees and his experience. But he also confirms that there is some interactions with backtesting, I think. The use of rational factors, factors that interact in a rational manner and limited backtesting reduces the overfitting of his models: again this is my perception and Marc will probably state it differently (assuming I have not missing the mark entirely).

So I think you may be good to go once you have addressed any liquidity concerns (to your satisfaction). The low turnover rate does make this less of a concern compared to high turnover models. I would also start the model on different dates and/or use rolling backtests. The 6 month rebalance limits the number of data points in your study and you want to make sure that you have not picked a lucky start date.

But if you are looking for the very best model, you are probably under-fit, or MISSPECIFIED (to use Marc’s parlance). Probably more importantly (IMHO), you could at least look at Marc’s ideas on factors that interact in a positive way.

FWIW.

-Jim

I have replicated the same simulation with a liquidity rule and the performance falls by half.


Hi Hemmerling.

When you said “limit short interest”, what amount of short interest do you recommend?

Thanks.

Hi David.

To be honest I just read the Portfolio 123 manual, not the Marc Gerstein’s tutorial series. Now it’s my duty for the days ahead. :wink:

Thanks!

I mean, just play around a little. Under 10% or under 5% should be fine. Just remove stuff that seems like it could have higher volatility risk going forward.

This is a very simple example of adding a few basic filters to your idea. It still trades 100 lowest PE stocks. It re-constitutes every 52 weeks.


Wow!!! Thanks a lot Hemmerling. You help me a lot to understand what happens.

Thanks to all of you too, Ivillalongabarroeiro, Jim and David.

Hi again Hemmerling.

I tried the changes you suggested on the sim, and the results are pretty different than yours, far worse indeed.

Then I tried in the screener, as you did, and also the results are different, but more closer to yours.

Do you know why??

Maybe it’s because the ranking system??

I do not find another explanation…




and the summary from the sim:


You should probably add the word ‘true’ in your sell rules so that it dumps all the stocks after 52 weeks and buys new ones. It doesn’t look like this is has ever sold anything.

It may not matter but I use the All Fundamentals Universe. Although it really shouldn’t matter based on the rule requiring positive EPS.

Forgot to mention - set it to ‘allow immediate buyback’ in the Rebalance tab.

Done, now it matches perfectly to your sim. But it was about the capital requirements, I add one 0 to the starting capital, and also I changed the Ranking method, before of that we have almost 2 points of divergence.

Thanks a lot for your help.