Assessing R2G Model Performance Before Subscription

I just wanted to point out some of my insights on the R2G models I see out there…

First, you should be skeptical of all claimed performance results, especially if the simulation starts at a time when the S&P 500 was trading near record lows. Usually the worst time to enter the market is right before it corrects downwards or while it is correcting downwards.

Second, you need to read the trading stats of the model. Here is an example:

Liquidity
(bottom 20%) $1,124,148
Min Stock Price (at purchase) $1.96
Max Profit Contribution Single Stock 9.60%

Trading Stats
Trading Costs / Curr Mkt Value 5.41%
Average Days Held 38
Annualized Turnover 893.38%
Average Return 7.26%
Average Return Winners 16.72%
Average Return Losers -9.28%
Winners 63.60%

This particular model’s simulation shows an annualized return of 75% with an alpha around 60%! Your first thoughts on seeing that performance are, how do I sign up?! But, take a closer look at the trading stats…

Max Profit Contribution Single Stock - 9.60%

This means that one stock returned almost 10% of all of the performance for the whole simulation! It was a lucky pick, and likely won’t be repeated.

Average Return Winners - 16.72%

This means that for all positions which closed out winning (in the money), they only averaged 16.72%. In other words, if you were to continue this model’s performance indefinitely, you would tend to get an average 16.72% return upon closing out all winning stock picks. Conversely, closing out losers would tend to net you the -9.28% return.

Winners - 63.6%

This is good for this simulation, but does it consistently post this high of winning percent? We need additional data from the designer to know this. If 63% of picks win and you get an average return of 16.72% per winning pick, your returns are going to be skewed more towards an average return around 7% overall, after accounting for the negative return stocks. (/begin_edit Edit: This is not quite right; see below. Basically, I forgot to account for the fact the higher turnover rate means you return 7% per stock trade more quickly than once per year. However, it also does not seem to factor in commissions at this time as it forced me to set my model’s commissions to 0 before I could launch it. See below for example of how that impacts returns. /end_edit) This is shown by the stat:

Average Return - 7.26%

This is important as it implies that the real performance of this model is actually around 7.3% return. Thus, this model is a bit misleading in terms of performance. This smacks of either accidental or intentional bias in the simulation.

Finally, look at the turnover rate:

Annualized Turnover - 893.38%

WOW! That means a TON of commission fees! Some of us pay $9 per trade and only start with $5,000 working capital. If I tried this model with those commissions and that working capital, it would probably quickly return -100%.

Compare this to my as-yet-unavailable R2G model (Low Turnover Value Trader - available 17 days from the time of this post) results which show an alpha around 17% (vs this one’s claimed 60%):


I have attached this image to the post, too, just in case it goes missing or won’t load for someone.

First thing you might notice is that Annualized Turnover is only 70.55%. That means it buys and holds stocks so there are very few commission fees involved.

Second, of the trades completed, 64.15% were winning trades.

Third, I tested this model across hundreds of start dates, including some of the absolute worst times to enter the market in the past 15 years. In almost every single case, it had higher than 60% winning trades, and in the few cases it had less, it was still over 50% and returned positive results which beat the index. Most of the time, winning trades exceeded 70%.

Fourth, you can see in the image that the average return on winning trades completed was a whopping 39.28% (yet my alpha is only around 17%). Further, the average loss on losing trades was only -17.28% with the average return for all winning and losing trades combined maintaining a whopping 19% over the 14.5 year run!

Next, you can see that of all stocks still in the portfolio, 95% of them are winning!

In fact, the winners currently in the portfolio have unrealized gains of $25,442 while the losers currently in the portfolio have unrealized losses of only $204.86! You would have to have a pretty bad trading day for that portfolio to turn into a loss.


LowTurnoverValueTrader.png

I took a look at your model and was a little surprised to see that you have revised it on the same day you posted it. One of the problems with doing that is that the current graph and chart are for the original model and only the out of sample trades of the revised rules will gradually change the Stats. Had you thought about deleting the model and re-posting the revised model so all the graphs and stats will reflect the current model?

Denny :sunglasses:

Thank you for your input on how one should interpret the Simulation stats but I would like to hear from P123 on how this should be done. How these are defined and derived by P123 may be different than your assumptions. I think what you are bringing up is very important hence my interest in P123’s input.

You are right, it was a good idea to remake it. The stats changed fairly considerably (in a positive way).

Liquidity Stats GLOSSARY ?
Liquidity
(bottom 20%) $203,361
Min Stock Price (at purchase) $1.25
Max Profit Contribution Single Stock 5.74%

Trading Stats
Trading Costs / Curr Mkt Value 1.81%
Average Days Held 328
Annualized Turnover 59.03%
Average Return 25.77%
Average Return Winners 47.97%
Average Return Losers -15.84%
Winners 65.22%

Liquidity is higher, max profit per single stock is lower, annualized turnover is a fair bit lower, average return losers is lower, average return is a lot higher. Unfortunately, now it will be 19 days from now before it is available. :frowning:


You got the average returns concept wrong. If the model says 75% it’s 75%, not 7%!

To go from average return per trade to CAGR you need to take into account the number of trades per year. If you make 7% per trade and have 8 trades a year your return is (amount invested*1.007)^8 minus the slippage and commission fees which by the way are taken into account by the R2G algo.

P123 is run by professionals, provided such a gross mistake in the stats had existed it wouldn’t have survived very long.

Best,

@davidbv: I agree that P123 should formally define how stats are calculated. I have an issue with some of the portfolio simulation trade summary data. See;

https://www.portfolio123.com/mvnforum/viewthread_thread,7623#38649

Yes, you are right, I forgot to account for the number of trades per year times the average return per trade. However, the R2G model, as far as I understand, does not take into account the commissions for a particular turnover. To launch a R2G model, you must set the commission to 0. Given a high turnover and a low average return per trade combined with low starting capital and a standard commission around $9 - $10 per trade, the overall performance is very far from real world results. If P123 does indeed have some form of commission calculated into the R2G model results, it would be good to have that listed in the information about the model. I tried launching my model based on a $8.95 commission per trade, but it forced me to set it to 0. I am not as worried about slippage calculations because it seemed to me to be estimating near the high end of what one could achieve using limit orders.

Unfortunately, not everyone has $100,000 to invest, which is what it seems most people tested their models with. If you hold 20 stocks in a $100k portfolio, that is $5k per stock so a $8.95 commission is 0.179% of the value. A 7% average return works just fine with a high turnover if round-trip commissions are only one third of one percent or less. However, using a $5,000 portfolio, gives you $250 per stock pick and that $8.95 is much more damaging to your overall return, especially in a high turnover environment.

$250 - $8.95 = $241.05
($241.05 * 1.07) - $8.95 = $257.9235 - $8.95 = $248.9735 (net loss)

This effectively turns the 75% annualized return into a negative return.

On the other hand, a low turnover model like mine which returns an average return of 25.77% makes serious headway:

$250 - $8.95 = $241.05
($241.05 * 1.2577) - $8.95 = $303.17 - $8.95 = $294.22

I don’t understand why the R2G models force you to set commission to 0. It should factor in a $10 commission to be safe.

Better yet, it should allow people to plug their own starting capital and commission rate into the model and test it to find out what it would return with their own personal situation taken into account.

And, as others have stated, I firmly believe P123 needs to be more specific about the performance characteristics and what they imply / how to interpret them. As a software developer, I would be happy to help them resolve these issues.

A $10 fixed commission becomes irrelevant after 15 years of 20% returns. R2G uses variable slippage in lieu of fixed commissions. It’s defined in the glossary although the exact formula is not shown.

Ib commissions are $1 minimum. Also p123 provides the platform and it is up to individual r2go developers to provide information to potential subs.there are some very good presentations and there are others that provide virtually no information.

Yes, commissions can destroy returns if it is very high turnover and you are investing very small amounts. Especially if the first few months/years are unluckly and generate negative returns: combined with commissions this can take a small dollar amount down to 0. But if you are lucky and have positive returns that overcome the commission, or if you invest enough so that commissions are negligible, then you are ok.

Marco, Variable Slippage is defined somewhere in the forum:

A slippage factor is applied to account for a possible higher/lower price for the transaction. The slippage percentage is calculated for every transaction based on this algorithm:

  1. The 10 day average of the daily traded $-amount is calculated (price*volume).

  2. The slippage is set according to where the average falls in this table:
    Price*Volume
    0-$50K… 5%
    $50K-$100K… 1.50%
    $100K-$350K… 0.75%
    $350K-$1M… 0.50%
    $1M-$5M… 0.25%
    $5M+…0.10%

  3. Add (1/ Price)% to the result from Step 2.

For example, Step 3 would add 1% to the slippage if the stock trades at $1, 0.1% if it trades at $10, etc.
For a stock bought at $50.27 with an average daily traded amount > $5M, the slippage percentage would be (0.10 + 1/50.27) = 0.120% of $50.27, amounting to about 6 cents per share. So the average price per share recorded as paid for this transaction is $50.33.

The Glossary needs to be updated by P123 with the definition of terms, how it is calculated and an example of its use in layman’s language. If I was a newbie R2G user and read this thread (without keeping up with all the embedded edits), I would conclude that many of these models, especially the high turnover ones, are misleading and could cause one to quickly lose money. For instance, how is Average return calculated and how should it be used. From what I can tell, the author of this thread made a gross mistake in its use and no one really corrected it. Maybe everyone is too nice but this poorly serves the layman user base.

David,

I do not think Aurel could have been any more clear.

Jim, you are right. He did say something. But the Glossary should have this as an official p123 statement.

egable,

what a coincidence, that you picked for your “insights”, that is, your praises, your new R2G? [;)]

It is doubtful as every model, which does not have a sufficiently long out-of-sample period to confirm the good backtest performance.

It is risky to start with less than $ 10,000 (that is position value less than $ 500) and trading fees near $ 10, as you propose: that means initial trading costs of 4-8 % of position value. There only remains the hope, that out-of-sample performance will reach near 26 % per position as well and that you do not just catch a drawdown at your portfolio start.

The backtest performance of 23 or 25 % is not impressive for a $ 203,000 liquidity model, not even with 20 positions.

Matthias

J3th,

You’re going to have that issue no matter how you go about it. $10 000 is not a lot of money to be starting a 20stk port. You can hardly blame P123 for that. If you take such a small position per stock then of course trading fees are going to eat deeply into your P/L. That doesn’t change the model performance if the backtest is to demonstrate compounding over time. It still takes trading costs into consideration but you should look at what the starting date and sum is at the time of the simulated inception. As for choosing a 25% returning model with 20 positions … well that’s up to you. It might not suit your needs but I’m pretty sure it could suit the needs of someone investing 100 000 into it. As for expected results, I air on the cautious side when evaluating simulated performance. I always ask the question of, would I still invest in this if AR was half and DD was double? If the answer is yes, then I might take the chance. Remember if we had the NEXT 10yrs of data then it’s likely this model would be slightly or drastically different, but since we can’t see the future we have to assume that our assumptions now are not optimal for the next 10yrs.

In smaller accounts with a set $7-10 cost per trade one should consider using either exclusively or partially the commission free etfs that are available at most brokers until ones account is large enough to overcome the potential drag from transaction costs.

Scott