Index | Recent Threads | Who's Online | Search

Posts: 13    Pages: 2    1 2 Next
Last Post
This topic has been viewed 2078 times and has 12 replies
marco
IMPORTANT: SP estimate series were re-built due to recent fix

Dear All,

We've rebult the "Fed Model" series: #SPEPSCY, #SPEPSNY , etc due to recent fix discussed here

You can see the image below for an idea on the magnitude of the changes. It should not affect your systems much since the overall trends are the same. For example in one of my systems I entered a hedge one week earlier with these new values

Very sorry for the inconvenience

NOTE: please notice that in the most recent data there are no differences since the bug fix had a compounding effect the older the point in time date.

Attachment SPEPCCY.PNG (16191 bytes) (Download count: 175)


Portfolio123 Staff.

Oct 29, 2014 4:54:51 PM       
Edit 3 times, last edit by marco at Oct 29, 2014 5:32:48 PM
Chipper6
Re: IMPORTANT: SP estimate series were re-built due to recent fix

The link https://ww.portfolio123.com/mvnforum/viewthread_thread,8134 is not working for me. It should be http://www.portfolio123.com/mvnforum/viewthread_thread,8134

Lakewealth Capital

Oct 29, 2014 5:28:39 PM       
marco
Re: IMPORTANT: SP estimate series were re-built due to recent fix

Thanks!

Portfolio123 Staff.

Oct 29, 2014 5:33:08 PM       
RJJ
Re: IMPORTANT: SP estimate series were re-built due to recent fix

How can we be informed of the "exact" before and after changes in performance when a change / "fix" is made? When we subscribe to models we are under the assumption the data is correct. Shouldn't we be informed as to exactly how the models were impacted? Is the main page of the R2G models automatically updated to the new stats or is it still reflecting the info from prior to this fix?

Nov 22, 2014 8:25:07 AM       
geov
Re: IMPORTANT: SP estimate series were re-built due to recent fix

I think the impact on R2G models from changes to P123 data and fixes should be investigated. In particular older models could be affected a lot. Perhaps all R2G models should be converted into sims and re-run at periodic intervals to provide updated performance figures in line with most recent P123 data. Currently historic R2G performance is frozen and not updated due to "fixes".

Nov 22, 2014 8:42:56 AM       
RJJ
Re: IMPORTANT: SP estimate series were re-built due to recent fix

Thank you for posting that George. I'm surprised more folks don't have a raised eye brow to this. If the historic data is indeed locked down on the main R2G model screens shouldn't there be a big red disclaimer that states "THIS DATA MAY NO LONGER BE ACCURATE DUE TO FIXES MADE ON X-X-XX DATE"? How would subscribers feel about seeing that? It would make me think "what is the relevance of inaccurate data?" Perhaps the data is still very close to being accurate but how do we know? What about all the folks that don't see notices of "fixes" in the community? I would assume that once errors are fixed the data of the models would automatically be reflecting the new stats. I have no idea how many fixes have been made to P123 data that may have affected the stats of the models.

Very concerned with reliability of accurate data and model performance stats.

Nov 22, 2014 11:28:01 AM       
primus
Re: IMPORTANT: SP estimate series were re-built due to recent fix

I disagree with geov. P123 should embrace bug fixes and improvements to data quality. Making a big to do over a minor fix disincentives it from doing so. Besides, if a minor change in a single parameter causes a disproportionate change in performance, then the whole model was fragile in the first place.

"The world is. The world is. Love and life are deep maybe as his eyes are wide." - Rush, "Tom Sawyer"
http://the-world-is.com/blog

Nov 22, 2014 11:51:10 AM       
geov
Re: IMPORTANT: SP estimate series were re-built due to recent fix

Primus, there have been a long list of "fixes" over time, not all of them minor. I remember when the short interest ratio was "fixed" it caused havoc to sims. All those that had this in the ranking system were affected. Of course, bugs must be fixed, but it is not fair to R2G subs to provide outdated performance figures.

I have first hand experience of this. Middle of August 2014 I inadvertently converted the R2G portfolio of Best10 into a sim and ran the simulation. (This is now not possible anymore, because P123 fixed this.) After this the model included all the "fixes" to that date. Performance figures were adversely affected. So at least for Best10 the performance figures are up-to-date now.

Nov 22, 2014 12:41:13 PM       
primus
Re: IMPORTANT: SP estimate series were re-built due to recent fix

I understand that users should be able to tell the difference between hypothetical performance. But what about real performance? You can't just go changing a port's forward looking historical performance. Out of sample performance is perhaps the only unbiased indicator of skill we, as quant investors, can latch on to.

Why can't p123 implement a function which copies a port's rules over to a sim for direct comparison? The trick would be to keep the rules and ranking systems private (proprietary).

"The world is. The world is. Love and life are deep maybe as his eyes are wide." - Rush, "Tom Sawyer"
http://the-world-is.com/blog

Nov 22, 2014 7:30:51 PM       
iavanti
Re: IMPORTANT: SP estimate series were re-built due to recent fix

Perhaps it should be a mix,
- After every change or even simply periodically if possible just in case, Historical data (pre-launch of R2G) should be re-run (if it is not already)
- OOS data / data after the launch of the R2G should remain as is, because those were the signals sent to subscribers, no matter what the bug/changes are.

Nov 23, 2014 4:40:37 AM       
Posts: 13    Pages: 2    1 2 Next
 Last Post