Hi all, I’m a new user in my trial period. I’m only about 5 days into my trial period, but I am very seriously considering subscribing. This is such a powerful platform built to fit the way I think about investments and modeling. I’m hoping you’ll share your perspective as a user.
Q: During your time working on portfolio123, what are your biggest learnings? What has surprised you? What has disappointed you? What do you wish you knew as a new user that you didn’t?
(I am an individual investor; decently versed in quantitative investing paradigms and processes (mostly using AAII stock data and doing the ranking and calculation for value/quality/momentum/growth/etc in Excel); I particularly find backtesting and portfolio construction of this site powerful and enticing - almost seems too easy).
After way too much time I learned that you can create a simulated book and then add any designer model to it.
If I knew that on day one, I probably would have just subscribed to a few of them and not spent so much time
on my own models.
There are a lot of smart people doing good work, it will take some time before you can do better.
P123 has been a life time experiance to me, finally programmed my trading system that I always wanted to have.
took me some time, but it was worth it! You will learn faster then anywhere else, people are very constructive here,
a great comunity!
I agree with Andreas. Before I came to P123 I tried a number of other services and couldn’t find anything that worked for me. I began using P123 about two years ago. Before I started, I had done nothing but lose money in stocks. And it took me a while before I realized the power of ranking systems. Since I started using ranking in November 2015, I’ve made close to 85% on my own investments, and am due to double my money by August if things keep going the way they’re going. P123 has completely changed my life.
I used to use AAII stock software until I discovered Portfolio 123 which is so much better for testing ideas. There are several different ways people develop models using Portfolio 123. Here is mine.
Translate my stock selection idea into Ranking Factors.
Test the ranking system on a small stock universe like R2000 but also the R1000. Usually the method is more profitable in the small stock universe. But I like to see R1000 results and NYSE universes and NASDAQ to confirm that the method is “robust”. (Actually I now have used Portfolio 123 to make custom universes of about 700 stocks for micro, small, mid, and large cap stocks. Also, I combine the micro and small universes and the split then into even and odd ID numbers to test for robustness. But you don’t need to make your own custom universes at the start.)
Examine how the ranking system performs in up and down markets. I’ve got a subscription level that let’s me test from 1999 on so I can see two bear periods as well as sideways and bull periods. Five years (2012-2017), in my opinion, is not enough because there are no bear periods.
Retest the same system by varying the weights given to the factors in it. I not only look for the “best” results but also try to find where it might break down. This is part of “stress testing”.
Ranking Performance testing is very fast so I can test many variations in a reasonable period of time. Once I get a ranking system I like, then I test it more using the Portfolio Simulation tool where one can test out buy and sell rules.
Thank you, I’m not sure I understand, but I’ll investigate that. I’ve only simulated one model so far to understand how it works with the custom buy/sell rules. Are you saying you take the base model and modify the buy/sell/stop loss rules in portfolio sim?
Thanks Andreas, I’m pretty sure I’m going to subscribe. It’s expensive for me, but I think from the trial I can tell it’s the kind of tool that I’d benefit from.
Actually as I’ve been familiarizing myself with port123 I’ve been having a similar thought regarding step 1, particularly because I find I have to be careful about applying ranking systems to screens, because they bring their own alpha independent of an idea being tested. I’m finding I have to be careful with them during the evaluation phase. But once an idea holds promise, apply a ranking system, and usually whatever idea being tested gets better ;-).
So as you mention, why not just try to convert every promising screen idea into a ranking system for quick application? I like that process. It’s quick and powerful. I’m just thinking I have to be careful about how I use it because it can make understanding results muddier.
Is there a way to compare the correlations of various ranking systems, screens in port123? (Are they all kindof correlated feeding on the same thing, or is there a way to combine systems that are less correlated?)
Also, I definitely want to test in both up and down mkt. I agree that 5 yrs of a bull mkt is not enough. Many would argue that 17 yrs might not enough data, but at least that gives up and down mkts. I think to portfolio test back to 1999 I’d need to subscribe to the 3rd level. Is just being able to backtest to 2000 good enough for you, or does portfolio testing back to 2000 add alot? It’s a lot of money for me and that’s one of the things I’m considering. I think I need to step up to the 3rd tier with extended portfolio testing at some point, but not sure if I know enough to take advantage of it yet.
Again thanks for the thoughts and insights. I appreciate it. I’ll look back to this
I was at the Screener level for almost two years. Backtesting using screens and rolling backtests going back to 1999 and ranking systems and simulations going back to 2010 was good enough for me. I only upped my subscription to Designer level recently. It’s great, but it’s not essential, in my opinion. Others might differ.
You have to be careful with something like that. Such a list is presumably made with current information in mind, so any backtesting could be invalid.
For example, it would not be a good idea to test an idea on the S&P 500 stocks, using the current list of them, because you wouldn’t have known what that list would have been 10 years ago. It would be a big help if I knew which stocks would be in the S&P 500 index 10 or 20 years from now.
Here is my opinion which only applies to my me but it works.
OS is all that matters and the more the better.
My goal is only to beat the market by a few percent with minimal drawdown if that is your goal it’s very achievable. If you want 45% it’s hard.
Make sure you diversify and use the book it makes it very easy. Diversification is another opinion of mine but you need stocks and bonds.
The next crash will come so you need to invest some time in understanding market timing and make sure it’s not optimized. My returns are less with it but I don’t want to lose 55%.
Read the forums there is so much good information.
Finally P123 is the best I look at professionals managing money and I know lot’s of them and I just shake my head and they do the same but I know my returns beat theirs and thats all that matters.
Mark, thanks for the comments and tips. Appreciated. What does “OS” mean below? (EDIT: I just realized OS probably means out of sample)
And the below is part of my goal also.
I certainly don’t expect anything close to 45%. If I can generate some consistent alpha with similar drawdown risk as the overall market, I’ll be fine. In current market valuations I’m particularly cognizant of downside and risk/reward.
Part of my concern right now is I’m currently moving funds to international exposure via ETFs due to relative valuations vs. US mkt, and I don’t really know how to go about international investing in international stocks, much less apply systematic process to it, so I utilize ETF and sometimes ADR. Not that I expect the US market to decline, but I like valuations elsewhere and will likely keep gradually moving out of the US market that p123 addresses.
Re your bond comment. In the coming days I’m planning to work a bit with ETF models similar to what Gary Antonacci describes in Dual Momentum. Bonds are an area I’ve neglected, but I’m learning. Do you group preferreds into your thinking about bonds also, or do you view as separate category?
Yuval, btw, nice article on your blog about ranking systems and complexity. I’ve been wondering some of the same. I think O’Shaugnnessy advocates for using lots of proxies for factors to help even out returns when certain factors go in and out of style, and others like Wes Gray advocate trying to find the best single factor (I think his research indicated EBIT/EV is best value proxy, and his efforts to combine it with other factors tended to dilute the effect.)
I love how port123 allows us to test many of these ideas quickly. I’m learning things here every day, sometimes in conflict with things I’ve learned and/or read over the years, sometimes in support, and sometimes murky. But in any case, I so appreciate the empirical nature of it all.
Mgerstein comments on any questions on this forum.
Following his liquid COST FREE DESIGNER model can obtain your objective with liquid universe SP500
better than bench mark and low draw down, 100+% return in last 4 years out of sample.
My comment about bonds was more about using them in a book with your stocks. My book has minimum 33% bonds when markets are good and it can be as high as 67% when markets are bad. I use Tom’s MXB US bond hedge. There are not many R2G that focus on only on bonds. In my other models I am just using TLT. It has gotten a lot of bad press so read the forums and see if you are comfortable with your bond selections as they are not risk free.
My main piece of advice is that I would give my newbie self is you can’t follow your models in the real world, they’re just meaningless datapoints on your computer screens. You can backtest a model dating back to 1999 and come up some beatiful convexed upward curved smoothed out over the course of 18 years, and you think this is just easy money. How could anyone fail? Drilldown deeper into the curve and really get a grasp on what kind of short term volatility is involved and ask yourself how likely you are to stick with it and see the rewards. Even when your models do perform well out of sample, it’s not always easy to stick with them.
What has made be better at sticking with them?
Reading people like Wes Gray at Alpha Architect, Meb Faber, James O’Shaughnessy or Joel Greenblatt who use a similar approach to investing. Even if I don’t always agree with their approaches or results, just listening to their frame of mind and why they use this approach and why they believe it works. It’s like Seth Klarmann and Warren Buffett couldn’t ride out the bad periods if they didn’t have a deeply ingrained understanding of the thought process as to why long term value investing works over the long haul.
Really getting to know your models, the factors involved and how they work on a gut level. This is generally regarded as a necessary step to avoid curve fitting, but as importantly it’s necessary to give you the conviction that there’s a method to this and why it works when there is really money on the line.
Understand the kind of volatility that you’re in for. P123 does a really good job of showing you the type of variance of outcomes you can expect based on your backtests over a variety of periods. You’re going to really need to know this by heart, because the question “Is this normal downside or underperformance, or is my model actually broken?” is a question that you’re going to ask yourself a lot. When I was starting out, I tended to overvalue alpha and undervalue potential sharp dropdowns in backtests. Utilize the book tool to help smooth out returns with a variety of diversified approaches and asset classes. If you have some 5 stock microcap port that’s returning some ridiculous 90% annual alpha every year, even assuming it’s not curve fitted all to hell and works out of sample, it’s meaningless if you don’t have the stomach to stick with it.
Most of all, I would just stress how fun this is. In my personal budget, I have my P123 subscription expenses categorized under “Entertainment”. Aside from financial gain, it’s given me countless hours of discovery and enjoyment.
Yeah, this is the key, it really is. The first live model I traded had a sharp drawdown, something like 15%, a few weeks in. I panicked and liquidated. In hindsight, looking more closely at the backtest, this had happened dozens of times. But when you’re living through it day to day it’s gut wrenching.
The next time this happened it was easier to deal with, and now after a bit more than a year, it’s pretty much routine.
The funny thing is, I knew all this beforehand. Yet when it happened, reason flew out the window. I am not entirely sure you can be prepared.
Further to the previous posts, if I were you, I would run rolling tests on my models to get a somewhat more statistica/probabilistic l feel of how your model could under perform either with absolute drawdowns or in comparison to your benchmark (as excess returns) .
For example, all models will have periods of under performance against SPY. If you run a 1year/1 week rolling test using SPY as your benchmark, you will see that it happens (maybe a lot).
You will then be better prepared when it does happen to know that it could have happened in the past (and there is nothing wrong with that).
