ETF Ranking Systems

Hey all, I admit that I’ve never traded ETF’s and have never made a ranking system for them. If anyone would like to show there’s off I would really like to see what some people have found that works. Do you find them to have more predictability or less over stock ranking and what kind of performance have you seen while doing backtesting?

I’d like to learn something and I’m sure if I do others will as well :slight_smile:

Hi Tony - Here is a simple system.

http://www.portfolio123.com/port_summary.jsp?portid=847823

It is essentially the IVY Tactical Asset Allocation. (You can find a description of the IVY portfolio on the web.)

I have made a modification to select two asset classes using a ranking system based on 3, 6 9, 12 month performance.

http://www.portfolio123.com/port_summary.jsp?portid=847826

Both ports are giving quite smooth performance so far. Please make any modifications to these public.

Thanks
Steve

Try this one: http://www.portfolio123.com/port_summary.jsp?portid=875845

Added TLT ETF iShares 20 yr Treasury
Reduced the length of the SMA
Added requirement that a buy ETF is not down more than 5% short term

Denny :sunglasses:

Thanks for this nice sim Denny.

A couple of points:

  • most of these ETFs do not extend back to 2001. In fact DBC only goes back to ~2006+. Therfore you should only backtest 5 years; or find ETFs that have a longer history. I think the latter would be difficult. I have often thought I could find stocks (companies) that mimick the ETFs but I don’t have enough time to research this. Technically, even 5 years backtest is too long given the ranking system uses one year of data in the formula.

  • TLT is highly correlated to IEF except more volatile. I believe adding TLT smooths the equity curve because there is more fixed income in the port. It is certainly not an independent asset class.

  • I hate to deviate from the well tested TAA formula of 1M versus 10M average i.e. SMA(21) > SMA(210). Your formulae might be overtuning OR on the otherhand maybe they will outperform in the future…

Thanks again for sharing this,
Steve

Hi,
I didn’t see it mentioned but Denny’s sim is weekly, vs. 4 week rebalancing for the original. If you test the original 4 times shifting the start date a week each time, you will find results vary considerably. (Over 5 years I got annualized returns ranging from 3.6 to 8.3%). The longer hold time has some benefits and is required in some accounts with trading restrictions, but it is more subject to sharp market pullbacks (depending on the start date) and for that reason I prefer the weekly approach.

TLT adds some advantages because it has been in a bull market, but what I think is even more important is to give the strategy safe alternatives during adverse market conditions. TLT and IEF both generally perform well when equities do not, but in my version I replaced IEF with SHY - a short term bond fund for better diversification and safety. In my rotation strategies I always include enough bond funds so that the strategy can hold all bonds if that is what is performing best.

1m vs. 10m is good, but I have seen short term and long term momentum can work well also. This may be better done in the ranking. Turning off both momentum buy rules and switching to the price uptrend conservative ranking, which not only includes long and short term momentum, but volatility as well, provides better results over the past 5 years.

http://www.portfolio123.com/port_summary.jsp?portid=878779

P123 has data on all funds since about April 2006, so any testing requiring data earlier than that is incomplete.

Don

Steve,
There were enough ETFs in the ticker list that had data back to the beginning to be fully invested except for when the Sim was invested in IEF. I noticed from the charts of your Sim that when the Sim went to IEF it only held 1 ETF and was 50% in cash. So I added TLT because I wanted the Sim to be fully invested, but in a similar equity class as IEF. That allowed the Sim to stay fully invested all but a few short periods.

I feel that the SMA(210) is an awful long time to hold an ETF if the market is trending down. I tried SMA(150) & SMA(100) as quick tests and SMA(150) was a significant improvement so I used it.

I added the buy rule; SMA(5) > SMA(21) * 0.95, to force the Sim to buy ETFs that had not lost more than 5% average over the last month. That rule has improved many of my ETF Sims although it only made a small improvement to this Sim. It has a bigger affect on more aggressive Sims.

Denny :sunglasses:

Don,

Turning back on my 2 buy rules in your Sim improves both the annual return and the max drawdown for the 5 year period and the 11 year period.

Denny :sunglasses:

Thanks for all of the good points Denny and Don -

This is becoming more involved than I had anticipated and I probably should have explained my original thinking.

I started with the IVY portfolio. There are several references here:

http://dshort.com/articles/2009/ivy-portfolio-update.html

My thinking was that I would start with this portfolio and then see what improvements I could make. BTW - the 1Month moving average > 10Month moving average has been backtested over the various asset classes for several decades. Starting in 2006 is “out of sample”.

So I created the 5 ETF portfolio using the same ETFs as the IVY portfolio with one exception. VEU only has a history back to March 8, 2007. I substituted EFA which goes back to August 27, 2001. Whether it is the best substitution is the subject for another debate. The simulation with weekly rebalance is here:

http://www.portfolio123.com/port_summary.jsp?portid=879356

Annual Return: 4.95%
% Drawdown: 13.30%

This is with 0.25% slippage. So this sim is not pretty… Yes it has fairly sizeable drawdowns… BUT the moving average filter SMA(21) > SMA(210) did its job. Taking out the moving average filter I got this simulation:

http://www.portfolio123.com/port_summary.jsp?portid=879357

Annual Return: 3.9%
% Drawdown: 47.01%

The drawdown was significantly reduced using the MA filter (from 47% to 13.3%) with less capital deployed. This result is absolutely consistent with the decades worth of backtest results.

My thought was to try to increase profitability by using a ranking system to select the best ETFs.

Now new variables have been introduced… changes to the moving averages and adding another ETF. These are all interesting changes and I appreciate these mod’s. My only concern is not having the decades worth of backtest to support the changes to the moving average filter.

More considerations:

The start dates for the five ETFs are:
VTI June 16 2001
VNQ Oct 1, 2006
IEF July 31, 2002
EFA Aug 27, 2001
DBC Feb 6, 2007

Asuming that one year of data is used in the ranking system or buy/sell rules then by rights the sim shouldn’t start before Feb 6, 2008. So I am cheating by using a five year sim - DBC is not included for the first year. (But let’s ignore that :slight_smile:

Now if I add 0.25% slippage into Denny’s sim and add TLT into mine I get the following results:

Dennys weekly rebalance: 57.38% Total Return

Mine with 4 week rebalance:
#1 91.28% Total Return (5 yr)
#2 79.44% Total Return (5 yr shifted one week)
#3 49.66% Total Return (5 yr shifted two weeks)
#4 55.53% Total Return (5 yr shifted three weeks)

With regards to Don’s simulation - it does provide better results but I would have to go back to square 1 and test all assumptions over again. Things like using SHY instead of IEF, etc…

Again - thanks for the suggestions!
Steve

Steve,

I’m not sure what you did to my Sim when you added the 0.25% slippage, but when I added it I got an annual return = 11.07%; Max drawdown = 15.13%; Total return = 214.84%; 81% winners.
You can see my Sim here.

Anyway, I agree with your concern about deviating to far from the well tested prior IVY approach.

Denny :sunglasses:

Denny - It’s because you are testing from 2001. I am testing 5 years.
Steve

Hi,
With all due respect to Mebane Faber’s work, which I follow quite closely, his team has turned in a very unimpressive performance with their ETF: GTAA. Since inception returns for the GTAA ETF were: AR -1.7%, StdDev 9.4%, and MaxDD -14.4%. At the same time my live (non P123) rotation strategy that I’ve been trading for years has returned: AR 13.5%, StDev 10.2%, MaxDD -6.8%, and that is with a limited set of funds and 30 day trading restrictions. Personally I’ve found that including many funds in a rotation strategy can hurt returns, and I think that may be the issue with GTAA. Maybe he should have stuck with the simpler approach.

While I regularly get ideas from others, I generally test for myself. I have done my own testing on moving averages on multiple markets going back as far as the 1920’s and on rotation strategies going back over 20 years. While monthly averages were suprisingly effective, and Oliver’s buy above 50 or 200 day moving average approach works very well, there are no magic numbers. Momentum works. The research generally says lookback timeframes from 3 to 12 months are predictive one month forward. Sometimes some numbers work better, sometimes other numbers work better. I resolve such issues through diversification (of assets and strategies) more than trying to worry about the optimum approach or testing.

The OP asked about ETF rankings and Steve provided a simple and excellent example of one. Denny and I offered some examples that showcase some minor variations. I’ve found this to be one of P123 biggest strengths: discussions and sharing of various trading approaches. In any case, I wouldn’t recommend that anyone apply someone else’s trading rules without testing it for themselves and judging it by their own criteria. I take what I can from others’ work, including the IVY portfolio, but I don’t limit myself to that. I also include what I’ve learned in my own research, experience, and testing and build my strategies from that.

Don

Don - congratulations on your many years of outstanding returns. I am where you were umpteen years ago. I’m looking for a good system but 5 years backtest isn’t sufficient for me. I have also found that adding too many securities into the mix reduces performance. I’m not really sure how the GTAA compares to the IVY portfolio but was surprised to see allocations such as below:

The actively managed Cambria Global Tactical ETF (NYSEArca: GTAA) weights U.S. fixed-income at 59%, international fixed-income 5%, international emerging markets 2%, U.S. stock 13%, commodities 1%, U.S. Real Estate 11%, cash 8% and currencies 7%.

I have been looking a long time for extended series of historical data for the main asset classes, or if you wish asset class categories (for the more picky readers). I had prevously found monthly data but wasn’t end of month. I think it was a monthly average price.

Please let me know if you have a source of historical data that I can get hold of and play with in EXCEL. Then I can take the results and model the last 5 years in P123.

Thanks
Steve

I really appreciate the chance to see the different approaches to ETF ranking - ETFs are a new way of thinking for me.

One thing I never liked about the Ivy Portfolio was limiting it to 5 (or if you read the book, fewer than 5) ETFs. In creating my version, I decided to use 5 stocks for the US market and 5 REITs for the real estate portion of the strategy. This works well for me since it stays in the realm of stocks, not ETFs.

However, creating the ETF variant for Countries and Commodities has not been as successful. Using some of the ideas presented here, I came up with the following sim:

http://www.portfolio123.com/port_summary.jsp?portid=879091

Any suggestions/feedback are welcome.

By the way, I do not commit 20% of the strategy to Fixed Income - just 25% to the other 4 components and let Fixed Income be the default when not invested.

Jim

Hi,
Jim, excellent sim. It’s performance relies in part on strong performance from the commodity funds over the time frame, as running the sim with a 1 year rebalance still gives returns of almost 10%/year. It is also different from the others mentioned earlier in that it has a shorter hold time and tends to buy reversals, rather than trends, but that can be a very effective approach.

Steve,
I’ll follow up with you directly. What I will say here is that I simply downloaded data from Yahoo and tested in a spreadsheet. I had 36 funds with most going all the way back to 1989, and I tested thru 2008. In one set of testing, I ran 368 rotation test cases plus a buy and hold SP500, buy and hold a diversified 75/25 portfolio, and market timing the 75/25 (think I used the 10 month moving average, of the market, not the individual funds). The advantages of momentum is shown by sorting the results by risk adjusted return (AR/((StdDev-MaxDD)/2) and absolute return. The percentiles are below, where 100 was best, 0 was worst.

B&H MT: Risk adjusted 56, absolute 7
B&H Diversified: Risk adjusted 6, absolute 0 (2nd to last)
SP500: Risk adjusted 0 (last), absolute 0 (last)

In these tests I rebalanced monthly, but I varied the lookback period, the method for measuring momentum, the number of funds to hold, how aggressively to sell, and tested with and without market timing. That ALL cases beat the SP500 and perform so well relative to a diversified portfolio really sold me on the approach.

Overall results are below.

Don


RotationResults.PNG

Jim - I had a look at your sim. One of the issues I have with what you are doing is that there are at least 37 funds that meet your buy criteria.

When I looked at your realized trades it was apparent that there is a tendency for the system to select all of one fund type. For example, for the 5 trades closed on Feb 13, 2011, all five were bond funds.

This is not to say what you are doing won’t work, but there is no diversification…

Steve

Hi Don:

I appreciate your comments about asset rotation and testing. I very much like to do my own testing to confirm ideas. I find this helps me stay the course when an approach has one of its draw downs.

I’ve got a few questions about your table of test results:

  1. To what does the “avg top 100” refer? Is that a benchmark you constructed by averaging the top 100 etfs selected in highsight? Or is that one of your ETF selection methods? Or is the top 100 best runs in the 368 tests done?

  2. Simiarly to what does “avg All” refer? To the average of all 368 of the tests?

Thanks,
Brian

Hi Brian,
Avg all is the average of all tests, even the buy and hold (it was easier to do)

Avg Top 100 is the average of the top 100 sorted on risk adjusted returns, which is Annual Return / avg(StdDev, MaxDD). I think this is useful because some of the parameter selections I made were not very realistic, and with the hindsight of testing I saw advantages in some approaches over others. No guarantee that these advantages will hold up in the future, but that’s how I would bet, and so I think this avg is something of a benchmark for the rotation approach during this timeframe.

Don

Jim,

I feel in trading a Port of your Sim it will be difficult to get the performance the Sim shows. 2 reasons:

  1. The slippage is very low at 0.025%. For a liquidity of $5 Mil/day I have found my real slippage to be slightly > 0.1%. To be a little more conservative I run 0.2% slippage with $5 Mil/day & 0.1% with $10 Mil/day.
  2. The average gain/trade is only 0.81% (from the statistics, Trading Page), and any additional slippage will eat most of that up. I don’t consider trading a Sim that doesn’t show > 2% gain/trade using a conservative slippage.

Here is another ETF Sim to consider that has decent performance: ETF Rotation - Basic 2 Stks, 1 Wk, 10 Mil, AR=18%, MDD=18%, 65% winners

This Sim trades 2 long only ETFs, $10 Mil/day liquidity, rebalances weekly, and it uses 3 market timing rules.
It has an annual return of 18%, a Max drawdown of 18%, total return of 544% and 65% winners. The average gain/stock is 5% and the average days held is 70.

Denny :sunglasses:

Don, Steve and Denny - thanks for the feedback. All good and things I need to address.

I wasn’t trying reversals per se, just trying to avoid Faber’s use of DBC for trending. I generally use the weekly refresh to weed out stocks dropping in rank, but the turnover here is huge. Is there a way to check without the high refresh?

I looked at the bond fund periods as a signal to go to cash. The screen version of this turns out to be an interesting market timing mechanism. If I cannot find a mechanism to limit the commkodities to 2 or 3 per type, I was planning to create a list. Is it possible to detect the shift to bonds in a sim and buy nothing?

Because my stock sims generate few trades and have high average gain/trade, I have been pretty sloppy in examining slippage. Something I intend to change.

Again, thanks for the help.

Jim

Hi,
Regarding turnover. There is generally a tradeoff between performance and turnover. In my experience higher performing strategies tend to have higher turnover, and so trading costs have a much bigger impact. I’m not sure which ranking system you are referring to but simply holding to a lower ranking will lower turnover and likely decrease performance but also trading costs.

To identify when bonds are top performing, perhaps you could use forder to sort all ETF’s, select the top 1, and check it to see if it is a bond fund. I expect going to cash will underperform going to bonds.

Don