WTF!!! We're doomed.

I just got back from checking out the battleship docked near me (it’s Fleet Week in NYC). I was shocked by what I saw inside a military transport vehicle; little laptop computers (netbooks actually) mounted at each seat running on . . . Windows Vista. VISTA!!!

So much for fighting to win.

:frowning:


I hope the military is not adding random noise to the laptop hard drives. Maybe Microsoft already does this: it would explain a lot. Of course, a little random noise in the missile launch codes would be a laugh riot! Who wouldn’t want to see that? Just joking :slight_smile:

Marc, No one wants random noise added to their private sims. Litteraly no one. Are you still committed to this? Maybe Marco accidentally lumped in some of the ideas for R2G presentations to the public? Please clear up any possible confusion on this.

Thank you in advance for your response.

Jrinne, what does adding random noise mean?

SZS, details on noise in sims presented here with my strong believe this is all about public ones only (R2G presentation).

This probably mean SOME designers on the forum believe their models OOS is much like missile launch in terms of hitting the target accuracy. SOME designers stated their missiles while tested was hitting one targets, but actually will hit other intended ones. It is by design. Actually missiles do forced by a lot of noise: weather conditions, terrain surface, anti-missile shields and so on and still hit the targets. But SOME designers missiles will not actually hit intended target because of bad weather, wrong surface, prices for condoms in Honduras, but missiles designed OK. Will SOME designers missiles hit the targets and what targets we will see. I expect we will get prove to famous “expect 50% less AR and 50% more DD”. And we can expect those SOME designers models will double the expectations.

SZS, congratulation on the first post.

Not sure how the random noise topic worked its way into what I thought would be a fun thread, but as long as people remain so passionate about this . . .

Personally, I don’t want noise introduced into my sims. That said, the particular change being introduced is something i do support since I see it as offering valuable information without detracting from what’s important to sim developers.

The mockup Marco showed me simply took the default summary price chart and showed it with alternative execution price assumptions layered into the visual. The rest of the sim remains as is, and even in what changes, one can opt to show the traditional no-noise version. So . . .

  1. What’s the big deal. If you want the old presentation, you have the old presentation. Just opt for it.

  2. Why one earth would I be upset about seeing how my sim would have performed if, say, instead of assuming open prices, we use code prices or average of high-low. If my sim can’t withstand changes like that, that’s not a p123 problem; that’s a me problem. So i appreciate being able to get such useful info so effortlessly. And i especially appreciate retention of an aspect of sim presentation I told Marco was important; the ability, say, of a money manager to take screen shots from p123 and incorporate it directly into a client presentation.

  3. Now as far as I can discern (not easy since the passion actually obscures rather than clarifies concerns), the three-year rolling tests seem to have gotten lumped in with this. Not sure why it’s happening now. I thought we already showed that forthcoming feature. But for what it’s worth, it’s really nothing new; it’s simply the addition of a valuable screener feature (Rolling Backtest formerly known as Advanced Backtest) into sim and portfolio. Again, why on earth would anybody not want information like this? If my model is fun when I launch and set rebalancing intervals with reference to a 12/31/04 start date but fails if I start, say on 2/13/04, that’s an issue. With my own work, I already work with the screener rolling backtest in order to find this sort of thing out, and i often change start dates manually.

  4. Also as to the rolling tests, and perhaps this may be more important than the conduct of rolling tests per se, is the setting of expectations and interpretation of the results. No strategy can ever be expected to work over each and every slice of o-o-s time one may examine. That’s the challenge of extrapolating into the future, which is quite different from what is done in other research-oritned disciplines. So yes, on the surface, models will be shown to look “bad” in the sense that periods of underperformance will very likely be revealed in any series of rolling tests. but whether it really is bad or not bad depends on why the periods of weakness occurred. Sometimes, it’s one of those things where a particular approach used factors that went out of fashion for a while; it happens periodically to everything. Sometimes, the market changes in structural ways and once-good strategies lose effectiveness, perhaps permanently (this happens a heck of a lot, and vice versa). And other times, it happens because models were simply too dependent on factors unrelated to those that influence stock pricing. Either way, this is an invitation to critical examination and a search for answers, and revisions, if warranted. And also, it shows the historic magnitude of periods of non-performance; how bad the bad times got. Often, the result can be seen as quite encouraging. It’s analogous to the drawdown stat cherished and discussed by so many, but it’;s a much more meaningful version. Instead of pointing to the 2008 crisis and saying “you sucked, ha ha on you” which is pretty much all we get now from Max Drawdown, statistic, the rolling tests . . . particularly the relative-to-market presentation, will give us a bigger picture of how far out on the risk limb the strategy really goes.

  5. Finally, I need to go back to a topic I raised yesterday, the legal aspects of what we do. I’ve been refreshing my research into that area given our expectation of R2G expansion and actually, my level of worry has increased. I’m not going into details now pending further research and, hopefully input from additional legal sources, but for now, suffice it to say the sort of things being done with introduction of “noise” actually is conduct consistent with the various boilerplate disclosures that must accompany PIP (that’s what regulators call sim; “pre inception performance”) presentation, which is quite a difference from a lot else I’ve seen in the forums which is tantamount to saying (“we know we have to have the boilerplate but we think its BS and don’t really mean it”). When regulators get around to looking at p123 specifically (they haven’t reached us yet, but everything I’m seeing shows that they are traveling down the freeway and are heading straight in our direction), the things being introduced could go a long way toward casting us in a very favorable light, which is a good thing since the things they require are the things that should be required of anyone who offers advice, ideas, etc. to others.

Marc,
Thank you for your response.

It was just a simple question. Marco showed a graph with different prices but it was unclear how (or if) it related to his post about noise: I can’t read his mind. I thought I might ask. As to where I posted it: anywhere I could hope to get an answer (it did work) and I am not the only one who posted the same question/request as you can easily verify. As to the post itself: If there is anyone out there who does not at least entertain the possibility that Microsoft adds random noise to their operating system then I congratulate them on their purchase of a Mac. Sorry for my poor humor.

Also, it would be my opinion that framing what looks like a good idea as a solution to Steve’s comments about data may have caused-- for me at least-- difficulties understanding all of its purposes. I don’t see how your plan for sims will solve any data issues but that wasn’t my question anyway.

Thanks again for the clarification.

I would have another question or 2 but I would not want a question construed as a criticism. The other questions may not be important anyway and the final result will probably provide all the answers I desire.

It looks like it might work and I look forward to seeing the final result!

No matter what, I will always hope P123 succeeds. You and everyone at P123 have done much for me.

Best regards,

Jim

Hi Marc and Konstantin thank you very much for your answers. So what Marco meant by “random noise” is simply different execution timing? “random noise” sounds more like something is manipulating/disrupting the actual data somehow so something like that would be troubling. Glad to know this is not the case?.

Will R2G still show year by year hypothetical results and intended start dates? In some models the start date is chosen so as to revalance before other investors do especially due to tax sales so dates can be a meaningful difference in performance (happy to share academic research on this matter if anyone is interested). Thank you Marc for your efforts in making R2G even better!

I don’t have all the details, but I suggested some of my ports for Marco’s test run and hope to see something this coming week . . . I expect it to focus on the 3-year rolling tests. It’ll be a lot easier to explain benefits after there’s something tangible to show and discuss. I prefer to discuss with reference to specific examples, or case studies, rather than in the abstract; that’s just the way my mind works.

As to the execution price “noise,” I suspect you may find it surprisingly benign (unless your result flashes an alarm, which should concern you – if there’s a problem, its better to see it in the sim then with real money). And ditto anything else that may be introduced. (I came up with the idea of introducing a factor category called alternative, where you would substitute different but also reputable definitions/formulas for ratios and see if your model is sensitive to that sort of thing. Example: Gross margin. Many data providers exclude depreciation from COGS. Compustat includes it, which I like better (and is how I learned years ago). It might be useful for you to be able to flip to an alternative valid definition like that and see if your models is sensitive to this sort of thing. Not sure how do-able it is. But it’s not a behind the scenes manipulation of data; if it can be done, it would be presented as something you’d have to go out of your way to do if you choose to do on your own.

BYW, I have no idea if Microsoft introduces noise into its military systems, but I’ve finally waved the white flag oon Mister Softie and bought a MacBook Pro and and am thrilled with it. A new era. . .

I think the original intent was to automagically conduction some type of sensitivity analysis on the model. What could happen by;

  1. varying some of the input parameters; buy/sell prices, model start date, etc
  2. perturbing some of the rank weights by small amounts, say +/- 2% for example
  3. perturbing some of the fundamental data
  4. enabling/disabling the hedge module
  5. etc

I’m not saying that these were proposed. Just that if someone were to ask me what I would try to vary to test a model, I would start with these items.

The transaction model and environment would never change.

Walter

I am hoping that using Vista somehow saves taxpayer dollars…kudos for not upgrading to Windows 8!

Somehow I think like Walter here. The list I believe was proposed on the forum but I am not sure if this ever was confirmed by P123.

tkp - I explained my objections to what is proposed. I know that you are struggling with the concept of optimization so why not let it be.

As for the noise proposal, it should be an interesting addition and will shake out a lot of basic issues.

R2G - I can see there are two things going on with R2G, one is the existing R2G, which (I believe) was originally intended as a freedom of the press initiative but now has been complicated by Trade. So there may be some regulatory issues to deal with. Then there may be perhaps some sort of Motif like initiative that is also being referred to as “R2G” or an extension to it. Just wondering if the new initiative should be called something else to avoid confusion?

Rolling backtest - I expressed my feelings on it and sorry the whole thing turned into a free-for-all. In some ways that sort of thing can be beneficial as everyone gets their thoughts out in the open and then start afresh. Perhaps its like a turbulent marriage, the marriage often tends to last longer as the feelings aren’t kept inside. Anyways, I went to the Motif site, and superficially at least, it appears that most of the Motifs are either completely passive, or based on well-defined strategies such as the Ivy asset allocation. From this perspective, “proper” backtest makes sense to me. But our existing R2G models are the equivalent of “actively managed funds”. I don’t think that actively managed ETFs provide back-data even in a prospectus. Passively managed ETFs, on the other hand, generally follow an index that has been published over a period of time. From a regulator perspective, I believe that making a case for presenting backtest data for an actively managed fund (i.e. probability bands, etc) would be a difficult sell.

However, I’m glad Marc is the man sticking his neck out for R2G and not me. If he can make the regulators happy with rolling backtests then more power to him.

Steve

Hi Steve backtests are widely used in the financial industry as long as it is clearly stated that they are hypothetical and not real traded accounts and may not be representative of future performance etc…

Backtests/Historical studies are also widely used by academics. The main point is how the experiment itself was designed.

I do think that if the model designers know the strategy’s backtest was obtained via formal optimization they should have a checkbox to decide whether or not to show the hypothetical results if he or she feels it misrepresents the strategy.

SZ - I understand that backtests are used and the boilerplate is used for publishing “simulated performance” or “pre-inception” as P123 does. I’ve been torturing Marco for 10 years trying to change the word “inception” to something else. Its good to see Marc using the term pre-inception.

Now P123 proper is a s/w program and is not offering a financial product and can get away with the boilerplate. (I’m not sure about Trade).

But I don’t see any boiler plates in ETF prospectuses or on ETF provider websites. The only time pre-launch performance is listed is when there is an index for which the ETF follows and the NAV index is used. The index is not a backtest and finance companies pay a lot of money to have an index generated.

I’m having difficulties finding the right section in the securities act to see what they say for funds/financial products, but in Canada, Mutual Funds come with the statement:

“Securities regulations do not allow us to report performance for a mutual fund that has been available for less than one year. This fund was launched August 11, 2014.”

Canadian regulations are usually not far off US regulations. In any case, I would be surprised if the SEC opens the door for backtests to be used to substantiate performance for financial products. And I think this must be where Marc is concerned about, providing a financial product like Motif.

I’m not sure why Motif would have different rules than a mutual fund or ETF. It looks to me as if Motif is on shaky ground :slight_smile:

Where is the P123 boilerplate? It seems to have disappeared.

As far as declaring that a model is optimized… From what I can see, you (SZ) are the only designer who can semi-legitimately claim that your model is not optimized as it stops picking stocks when none are found that meet your absolute valuation rules (please correct me if I am wrong about this).

The rest of us will be clicking the check-box, or telling a little white lie :slight_smile:

Steve

[quote]
tkp - I explained my objections to what is proposed. I know that you are struggling with the concept of optimization so why not let it be.
[/quote]Steve, for sure you did explained. I know that you are struggling with your R2G business falling apart so why not let it be.

You don’t need to. Just get out of your engineer designer world a bit there all your subjective argumentation have born and struggle a bit to listen to other people. You will find my understanding of models optimization explained and could understand fallacy of your optimizer confession from investor perspective.

At the end of the day who are you selling your R2Gs to? So let investor decide to see your backtest performance and noise applied or not. Not only performance matter but its consistency as well.

Я уверен, что вы можете разговаривать на Русском языке, т.к Литва когда-то была частью СССР.
Поэтому мне бы хотелось отметить, что, в этом мире и приветствуется свобода слова, но никак не приветсвуется плохое отношение и/ или
оскорбления. Если у вас имеются какие-либо проблемы, свяжитесь со мной, и мы это обсудим.
Steve

To lighten the mood of this hijacked thread:
https://www.youtube.com/watch?v=B-H06H86XiM

Hey all-

Can we give all the pointed remarks and personal jabs a rest in this forum? After all, we’re here for the same reason - to learn from each other and to make money using the great service and tools that P123 provides. Having used a number of other stock screening services, I have to say that P123 is by far the best…and that includes this community. Let’s not let this forum digress into personal attacks. I respect differing points of view and have learned quite a bit from many of you on here. Let’s keep it non-personal and get back to what we’re here for - learning from each other and making money.

Best,
Brent

Clicking imaginary “like” button

+1

Let’s hope the people who this comment is for, will realize it is for them :slight_smile: