| Index | Recent Threads | Who's Online | User List | Search |
|
|
 |
Portfolio123 » List all forums » Forum: General Comments » Thread: BLOG: Measuring Ranking System Performance (Chapter 13) |
|
Total posts in this thread: 4 |
[Request a Feature]
[Post new Thread] [Add To My Favorites] [Watch this Thread] |
| Author |
|
|
mgerstein
Advanced Member
|
EXCERPT We've already seen how easy it is to assess the merits of a screen through backtesting, turning the clock back to see how it would have performed had it been used in the past. Not surprisingly, we can also conduct similar tests on ranking systems. The most sophisticated assessment tools will come later, in the material on simulation. But the ranking system interface contains Performance testing capabilities that can quickly provide a strong sense of how good a job your system would have done in the past in separating market winners and losers. FULL ARTICLE. BY: Marc Gerstein CATEGORY: P123 Step-By-Step |
||
|
|
o806
Advanced Member
|
Marc: This is a great article. It shows the importance of using several different performance charts to evaluate at a ranking system. A new comer to P123 will really benefit from seeing how to identify the strengths and weaknesses of a ranking system by looking at a series of chart perspectives such as the ones you used. As you well realize, it takes a fair amount of user mouse clicks to generate all these different charts. And all but two of your charts are based on the same universe and date range. It would be a major improvement to P123, if the performance ranking module could produce a sets of charts all at once. What I have in mind is something like the way P123's simulation test generates several charts from a single run. In the past, I have suggested improvements to the ranking performance module but very little has changed (with the exception of custom universes, which is much appreciated). Any chance you might suggest to management that a single performance test would generate a set of charts? Your article clearly indicates you appreciate the value of have multiple charts showing the behavior of a ranking system. What charts could be included in a standard report set? Here is my list to get a discussion started: 1. a bar chart based on 20 buckets 2. a bar chart based on 100 or 200 buckets (user selectable) 3. a bar chart of the right most 5 buckets, with each bucket containing 10 stocks. (with a user option to change this to 5 buckets of 5 stocks or 5 buckets of 3 stocks) 4. a historical graph of the right most 5 buckets, with each bucket containing 10 stocks (or just 5/bucket or 3/bucket if the user so selects) 5. an addition to the bar chart showing the maximum drawdown of each bucket. I know that the P123 servers already calculate drawdown for each bucket (they have to be able to draw the historical graphs for each bucket). It would be great if that drawdown number could be included on the performance bar chart, or a separate bar chart. Thanks for listening Brian |
||
|
|
fxq
Advanced Member
|
I am a new member and find the information in this module very informative. I have seen a lot of "ranking systems" on this site with bar charts that look like they will take my portfolio to the moon! If only it were that easy. I now see that those ranking systems have buckets containing so many stocks that it would be a problem to buy then all. I did a test that I would like you to comment on. I thought what if I subjected "First Rank" to a run where I set the Universe to the S&P, allowed all prices of stocks, and had only 2 buckets. I expected that the results would show the S&P performance to lie between those of the 2 buckets, but both historical and annual returns for the buckets lagged the sp500. Any insights here? Thanks, Mark |
||
|
|
marco
|
The SP500 is market cap weighted, our buckets are equally weighted. ---------------------------------------- Portfolio123 Staff |
||
|
| [Show Thread Printable Version] [Post new Thread] |
