I’m not the p123 person with whom you interacted and your post is the first time I’m seeing your concern.
As you accurately noted, changes in data, changes in metrics, etc. do happen and they have been discussed in the forums extensively in the past.
We have also made many changes to the platform over the years. Consistent with what you suggest about Engineering 101, we do know what we have done to our product over time, and in fact, you can easily trace it back in time through the “Recent Feature Releases” thread that’s always visible.
The changes about which you are really concerned now, however, are of a different sort. They have occurred because of decisions in the world outside Portfolio123, and often outside of Compustat that we do not and cannot influence or control.
The central issue here revolves around the differences between engineering (and the physical sciences in general- as I understand them) and finance. In finance, the past insofar as we can articulate and use it is not fixed. There is no equivalent of laws of physics or chemistry on which we can count.
How did the economy perform in 2016? I don’t know. Nobody does or nobody can. The best we can do is refer to the output of some models that were built to describe the economy as best we can (GDP, etc.). But we don’t know if these are really as firm as the what we know about mixing certain chemical compounds or properties of electricity or water flow. GDP is simply a model we agree to use unless and until somebody comes up with something we think mat be better.
And we don’t even really know what GDP was last year? We only know what the Commerce Department thought it was when they finally decided to stop revising based on new information. Ditto for every other economic indicator.
We apply the same thing to company financials. Here, there are certain things we can know with reasonable comfort. What was Apple’s dollar sales in 2016? But it takes time to get the number right, although at least technology is helping internal accountants go from the field to a HQ aggregate a lot more quickly over time. But items like sales, and debt are unusual in their certainty. Many others are a lot harder to pin down and many derive purely from corporate modeling. So when you get down to it, we really don’t know what Apple earned in 2016. Nobody does, not even Tim Cook. All we and he know is what the company’s internal and auditor-approved and auditor enhanced models say the company earned.
And then, of course, databases model the data they receive from the companies. As explained in a data white paper I posted in the Help area when we switched to Compustat, raw “accurate” data would be useless to us on Portfolio123 because it would be impossible for us to compare any two companies. The difference from one data provider to the next isn’t so much accuracy (if a company reports sales as 983.6 million, we can be pretty comfortable assuming all will pick up that number and if they don’t a collection error will get fixed incredibly rapidly) but in the quality of the models they build (data sanitization protocols).
Logging ever4y change in every “model” that can impact your results is a brutal, if not impossible, task and I’m not even sure who would do the logging and who would compile and log the compilation of all the logs. And if it could be done and you were to receive it, you’d probably discard it quickly due to information overload.
None of this, however, should discourage you. While there is plenty of bad news about the absence of precision regarding the past, there is also good news – better news. We don’t need precision and in fact, we have all the precision we need to build successful models of the future, and then some. Is it enough precision to fly an airplane at 35,000 feet or send humans into space? No. But we’re not trying to do that. We’re trying to use the past as best we can model it to increase our probabilities of success in the unknowable and potentially very different future and for this, portfolio123 is magnificently positioned.
The key is how we build that bridge connecting the modeled past to the unknowable future. That’s the role played by our models.
From having screened and modeled since the ‘80s when data-bearing floppy disks went into drive B as the program disk sat in drive A and the drive-door lights blinked on and off as the pc read back and forth between them. I will tell you that I have never ever encountered a situation where the success of my effort or lack thereof was traceable to anything regarding the kinds of issues described above re: how the past is modeled. Never. Not once in many thousands of models over the years.
So, putting all of this together and bringing it back to the changes in sim that caught your notice, and recognizing your desire and need to learn from the situation, I will tell you that even if the sort of log you seek could be produced, it would not help you. Economic, company and data provider models of the past are such that if the sort of changes that occur throw an already-created simulation out of whack, the problem is likely to bed that the model left itself too exposed to randomness. Minor changes are part of the game. Big changes should not be.
I recommend you look more closely at the model for areas of potential vulnerability. The principles under which successful models work are explained in the Strategy Design on-line course posted in the Tutorials section. Also, you are welcome to show me your model – off line in order to preserve the privacy of your work – and we can work together to diagnose the situation.