Marco/P123 Staff - I have two identical sims that give different results. They can be found here:
https://www.portfolio123.com/port_summary.jsp?portid=1349571
https://www.portfolio123.com/port_summary.jsp?portid=1349458
I believe they are identical in every respect. One sim was copied from one account to another, then back again.
I need to get this sorted out fairly fast.
Thank you
Steve
Both custom universe and ranking system where updated/modified today. Univ at 8:48AM, ranking at 2:30PM. Unfortunately we have no history of the changes.
Marco - I just reran both sims, they both call up the same custom universe and ranking systems. I’ve been through a complete debug cycle as far as I can go. It appears that the sim that was copied from one account to another was corrupted, perhaps an uninitialized variable.
As a secondary issue, I’ve noticed that there is a bug when copying sims and ports from one account to another through the group module. It doesn’t work anymore when you want the program to copy the universe/ranking system. The ranking system ends up being unrecognized as some numeric ID.
Steve
Gues you re-run it. They are identical now…
Examining each individual transaction yesterday I had spotted a one stock difference after a few identical rebalances, which I was going to look at. A one stock difference in a 5 stock sim can certainly explain a big difference.
Can you reproduce the problem? Thanks
Marco - Thanks for looking at it. I found the problem.
It appears that there is an inconsistency between the ranking method called up by the sim and what is displayed/used.
You can repeat the problem as follows (I don’t think it matters what sim/ranking system you use):
Click on the rerun button for the sim. Go to the Universe & Ranking system page. Set the Ranking Method to “Use Ranking System Default”. Check the ranking method listed in the box below the selection option. In the case of my sim, the ranking method in the box below says “Ranking Method: Percentile NAs Neutral”. Now run the sim. You get results “A”.
Now lets repeat.
Click on the rerun button for the sim. Go to the Universe & Ranking system page. Set the Ranking Method to “Percentile NAs Neutral”. Now run the sim. You get results “B”. The results are different than the first run using Ranking System Default.
Again
Select the Ranking Method “Percentile NAs Negative”. Now run the sim. You get results “A”, same as the first (default) selection.
Conclusion: The ranking system has a ranking method of “Percentile NAs Neutral”. However, when you manually set the ranking method to neutral, the results of the sim are something different than when you manually set to default. When you examine the “Trading System” summary the Ranking Method displays “Percentile NAs Neutral” in both cases. However, in one case the ranking method is default and in the other the selection is neutral.
The default selection appears to work correctly, or at least I get the best results. But the default ranking method appears to be the same as “Percentile NAs Negative”.
The symptoms are as you observed after several rebalances the buy/sells start to disagree.
Steve
It looks like the default selection is always selecting negative N/As, regardless of what the ranking system default is.
Steve
Please make sure this doesn’t fall through the cracks.
Steve
Has this issue been logged at least?
Problem
If the sim/port Ranking Method is set to “Use Ranking System Default”, then the sim/port will always use Negative N/As, regardless of what the default RS setting is. The trading system summary will indicate Neutral N/As if the ranking system default is neutral, thus what is shown in the summary (Neutral N/As) is not what is actually used (Negative N/As).
Someone will eventually lose money because of this.
Steve
Steve,
In addition to posting the issue here, you may want to email Marco or Paul directly; marco@portfolio123.com, paul@portfolio123.com.
bump