| Index | Recent Threads | Who's Online | User List | Search |
|
|
 |
Portfolio123 » List all forums » Forum: Screener » Thread: Is the database Reliable |
|
Total posts in this thread: 22
|
[Request a Feature]
[Post new Thread] [Add To My Favorites] [Watch this Thread] |
| Author |
|
|
feldmail
Member
|
I have been working on a simple screen with a simple ranking system. The parameters in the screen and ranking system consist of MktCap, AvgRec, AvgDailyTot, #AnalystsCurQ, SI%Float and a few more items. No OTC stocks are included. Backtesting results show brilliantly; great results for 9 years (from 3/31/01 - 12/31/09). Then, for the last year and a half, they were awful. I wonder if the data beyond a year is reliable. Is the database really accurate for the whole time? Does anyone have knowledge of this? |
||
|
|
mgerstein
Advanced Member
|
As far as I can see, the data is fine. But the world does change, and has definitely done so since 2008 and many members have seen models that worked well before 2008 falter afterward, although many performed well again in 2009-10. (I've been finding 2011 to be a rough year.) I notice that two of your factors involve use of analyst-based data. This area, especially, has changed. Wall Street firms have downsized dramatically, reducing the number of analysts, replacing many with lower-paid less-experienced analysts, outsourcing a lot of estimate-generating work (i.e. to India), etc. Also in recent years, firms have felt pressure to be more systematic in the way they generate recommendations. I used to work a lot with analyst-based data and find i can still do so to some degree, but it's much harder nowadays. Recommendations have, in my experience, definitely become much less useful (analysts are trained to evaluate and explain companies, not pick stocks, so when they try to get serious about disciplined stock picking, it turns out they may actually be doing worse than they did back when they used seat-of-the-pants, finger-in-the-wind type approaches. Because of the way the sell side has been changing, you may have to consider the notion that what you're seeing with your models is likely to persist indefinitely if not permanently. If you want to continue to work with analyst-based data, I suggest you experiment with data points relating to estimate revision and surprise, which may still have more juice than recommendations. |
||
|
|
feldmail
Member
|
Thank you, mgerstein, for your response and advice about analysts. While I understand that markets have changed and analysis has changed, I still find it peculiar that the screen only stops working in the last year. Analysts are only one part of the screen. It worked well in the huge bull times of 2009, well in the bear markets of 2002 and 2008. It also worked during the bull times of 2003 - 2007. It was only in 2010/2011 that it had trouble. The consistency of the results is what is most peculiar. The backtest averaged large positive returns and there were virtually no negative periods until recently. The bulk of all negative periods comes only in the last year. I'm not a statistics expert but the odds of having such phenomenal gains for 10 years and so poor for the last year are probably over 10,000:1. This makes me suspicious of the database. |
||
|
|
feldmail
Member
|
I am finding major problems with this database and do not understand how members can trust the backtesting of any screen. For example, do a screen today for any stock in the S&P500 with a SI%Float greater than 20%. The screener gives MET (Metlife), which actually has a SI%Float of only 3.8%!! I do not understand and don’t see how one can make financial decisions about screens that can have such significant errors. |
||
|
|
marco
|
Those are the values we are getting from reuters. The Reuters values for METlife SI%Float jumped around the week of 8/20/2011-8/27/2011 from around 2.5 to over 25% and is currently is 34.6%. This 10-fold jump does not seem to be corroborated by the SIRatio, which remained in single digits. The ratios we offer are quite diverse and they are made available by several systems/databases at Reuters. It's possible that a new system went in place a few weeks ago at reuters and affects other companies. It's also possible that it's company specific. Finally, this may be a Vickers problem since reuters licenses their data (but were planning to do it themselves). We will ask them right away to check/explain. Thanks ---------------------------------------- Portfolio123 Staff |
||
|
|
feldmail
Member
|
As I sent you by email, the problem is the Float. Your system shows the float as 84m shares when, in fact, the actual float is 840m shares. Again I ask, how can one make backtesting decisions on the data which can be subject to errors. |
||
|
|
marco
|
Looks like the Float data is wrong. It went from 834 -> 84 around 8/13/11. The factor of ~10 is also fishy. This had a ripple affect to other ratios. We've reported it to them and should be corrected soon. Hopefully it's company specific ---------------------------------------- Portfolio123 Staff |
||
|
|
dwpeters
Advanced Member
|
Hi, You raise a number of points. That a screen stopped working only in the last year is not unusual at all. With the extensive backtesting capabilities on this site it is all too easy to fit a strategy to past data. The future is always different and so this site is littered with strategies that were phenomenal in past only to breakdown in real time. Here is a post that identified a number of them: http://www.portfolio123.com/mvnforum/viewthread?thread=2998#13715 ... in a thread that talks about how to avoid this very problem. And this problem is the fault of the users, not of the application. The application is not perfect and neither is the data sourced from Reuters or provided by the application. There are more errors than I would like to see but the support team is generally very quick to respond to errors and continually working to improve the functionality and reliability of the application. You ask the question:
All I have to do is look at my accounts and I can easily make decisions based on the data from this site. Don |
|||
|
|
marco
|
Don thanks for your comments. We got a reply from reuters regarding MetLIFE SI%Float : "I’ve confirmed that data correction is required. On investigation we found that the analyst had incorrectly captured the free float values. We have now done the required correction. Please find the details below." I've updated the historical values since it started going bad (about 5 weeks). I think they will fix it by tonight so it should be ok in the sim servers after the reload around 1:30 AM It appears to be an isolated case. Thanks ---------------------------------------- Portfolio123 Staff |
||
|
|
rjacob
Member
|
Don, You are to be congratulated on your portfolio performance. Unfortunately, I did not have the same results. I also started trading P123 ports in 2007, had amazing results, then lost all the gains very quickly. I spent many hours running simulations and tried some different strategies, but I was never able to make money again trading the portfolios. I continue to read the forum posts hoping to see evidence that there is something in the works that I would feel confident trading. Unfortunately, the recent performance of the model portfolios is certainly not compelling; and it does seem to me that there is an increasing problem with database reliability. R. Jacob |
||
|
|
[Show Thread Printable Version] [Post new Thread] |
