Value Line PIT issue

Report a problem or error
1

Even after the recent update I noticed different simulation results when the exact same sim was run on different dates. I decided to ‘debug’ it a bit before posting:

It appears that the Value Line universe as of 11/17/2008 changed from 1146 (rundate Monday October 20th 2014) to 1013 (rundate Wednesday October 22nd 2014).

Could you please look into this issue? It would be really nice if the inconsistent backtest results can be done with once and for all.

Screen - Value Line - PIT - 20081117 condensed - rundate20141020.png
Screen - Value Line - PIT - 20081117 condensed - rundate20141020.png1137×204 20.6 KB

Screen - Value Line - PIT - 20081117 condensed - rundate20141022.png
Screen - Value Line - PIT - 20081117 condensed - rundate20141022.png1229×204 20.7 KB

2

Do you have specific examples of stocks that disappeared in the second run ?

Thanks

3

No, I cannot be 100% for sure which stocks disappeared. Display and download limitations make ‘debugging’ a bit challenging. But I took some screenshots which are a bit big to post.

However, using the two screenshots, I noticed some differences in certain VL values for certain stocks:

From 20th to 22nd:

  • PRSC changed its VLSafety value from 5 to 4.
  • HL changed its VLTechnical value from 4 to 5.
  • KMG changed its VLSafety value from 5 to 4.
  • ACAD changed its VLPerf1W value from 4 to 5.
    Edit: Please note that this list is probably not exhaustive as I only have screenshots of a subsample.

    Thanks again for looking into this.
4

I think I know the problem. It only happens on Monday because VL data is only available Monday morning before market open. We don’t do a full server reload when we get the data , just an update to VL data only and it’s messing up the VL PIT (you are looking into the future one week). Once the servers reload on Monday night it fixes itself.

In short: simulations using VL data run on Mondays between 7AM-10:30PM are wrong. Re-running them fixes the problem.

We’ll have a fix soon.

ALSO… this reminds me that R2G’s should not be allowed to use VL data since it is available too close to market open. But we never added a check for that. Please let us know if any R2G’s are using VL data (they will have to be taken down).

5

There’s probably a way to delay rebalance of R2G’s that use VL data until the VL data comes in. It’s tricky though.

Let us know if this is something you need, otherwise we’ll simply disallow R2G’s with VL data

6

Thanks for the fast update Marco. I did not check, but I would not be surprised if its a Monday only issue and that you have found the cause of it to happen.

Regarding the R2G’s, I think you should specifically address that community as they have interests and concerns that many other users do not have.

Again, thank you! This platform is getting better and better…