| Index | Recent Threads | Who's Online | User List | Search |
|
|
 |
Portfolio123 » List all forums » Forum: Ranking Systems and Factors/Functions » Thread: Ranking and Back Test problems |
|
Total posts in this thread: 8 |
[Request a Feature]
[Post new Thread] [Add To My Favorites] [Watch this Thread] |
| Author |
|
|
dougmm7261
Member  UNITED STATES Joined: Apr 27, 2005 Posts: 3 Status: Offline |
I'm new here and I find this sight very useful. However, in just running some very simple Rankings and Back Tests, I've encountered so many "strange results" (bugs) that I'm not sure that I'd have any certainty about any of the results. For example, in running rankings, I got results that showed all stocks in the entire universe of almost 7000 stocks ranked above 50. How can this be? Next, when I ran back tests on a simple ranking system using only price strength and close (and price) as the screening criteria, the resulting back test included both stocks that didn't meet the criteria (i.e., penny stocks under .50 were included even though I screened for stocks over $5) and not enough stocks twere bought to fill the ranking screen (i.e., returning only 8 stocks ranked over 98 one week when other weeks resulted in 60 stocks selected). Am I missing something? I thought that the rankings automatically distributed the stocks evenly and gave them rankings between 1-100. If there are 6000 stocks then each unit of ranking should cover 60 stocks, right? How can I have a ranking system that ranked all the stocks in the universe and yet listed all of them with a rank of over 50? I also ran a back test based entirely on 4-week price performance that produced a return of 12,934,434,433,959 dollars on an initial $100 investment. How can this be? Finally, the ranking performace charts showed that historically stocks ranked between 5-15 produced negative returns, but when I did the back test from the screening page for the same time period, the results for this same ranking group showed average weekly returns of plus 0.5%. Are there reasonable explanations for these problems, or are there bugs in the software? Or both? Any help would be appreciated! Thanks, Doug |
||
|
|
marco
|
Doug, in order for us to investigate please be more specific. Either post a link to the system, portfolio or sim or email us if don't wish to make it public. If the question regards a Rank Perf, then include all parameters used. Thank you for your help! ---------------------------------------- Portfolio123 Staff |
||
|
|
marco
|
I believe you are seeing this with the ranking system you created that only has one factor: PEG. More than half of the stocks have no PEG. The algorithm assigns these stocks the next lowest value from the worst valid PEG. That is why you are seeing many stocks with exactly the same rank of around 56. This was by design so that a stock is not over-penalized for having an NA value. In the future we'll make this behaviour user-configurable. For exampl you'll be able to specify a rank of 0 for NA's. ---------------------------------------- Portfolio123 Staff |
|||
|
|
dougmm7261
Member UNITED STATES Joined: Apr 27, 2005 Posts: 3 Status: Offline |
Marco, Thanks for replying so quickly. I will forward the exact parameters that are questionable next time the problems come up. (I didn't save any of the runs that seemed to be problematic). Thanks for providing such a great site! Also, what would be really REALLY great is if we could take the monthly returns from different simulations/portfolios and combine them into one composite portfolio. The purpose of this would be so that we could see how uncorrelated strategies might overlap to reduce drawdown while still producing substantial returns. Thanks again, Doug |
||
|
|
swheat
Advanced Member  UNITED STATES Joined: Jul 23, 2004 Posts: 32 Status: Offline |
I have a question that is similar to Doug’s. It seems that no matter what ranking system I use the stocks in all buckets outperform the market. For example today I created a silly ranking system just to see what would happen. I created a formula that always returned true (1=1) and ranked stocks by it. Surprisingly, stocks that returned true (all over $3) produced an amazing return of just over 15% whereas the market gained just 1.5%. I added more silliness by dividing the ten day average volume by the number of employees (the ranking system I used is called Junk). All ranks but one solidly beat the market and the top two earned 140% Why does this occur? It seems to me that returns for stocks ranked in a random fashion should appear roughly clustered around the market. Thanks, Sam |
||
|
|
swheat
Advanced Member UNITED STATES Joined: Jul 23, 2004 Posts: 32 Status: Offline |
bump I know there is probably an obvious answer to the above but I dont still dont get it.  |
||
|
|
marco
|
The ranking system perf tools allocates equal weights to all the stocks while the indices are market cap weighted. For example in the SP500 the biggest 50 stocks make up about 50% of the total market cap of the sp500. ---------------------------------------- Portfolio123 Staff |
||
|
|
Sterling
Advanced Member
|
I noticed something with the Ranking Performance test. When I analyzed a single factor I'm studying it shows a distribution where rankings above 50 are predominantly over 0 and rankings under 50 are predominantly less than 0. My question is why is it that when I edit the properties of the factor formula in the ranking system from higher is better to lower is better and nothing else that the distribution after performance testing doesn't change? After reversing the rankings shouldn't the distribution also flip? Where before rankings above 50 were predominantly above 0 and rankings below 50 were less than 0 shouldn't those above a rank of 50 now be below 0 and those with a rank less than 50 be above? Instead the distribution seems the same as before. Any idea as to why? |
||
|
| [Show Thread Printable Version] [Post new Thread] |
