| Index | Recent Threads | Who's Online | User List | Search |
|
|
 |
Portfolio123 » List all forums » Forum: Simulations and Portfolios » Thread: Simulating Rank Performance |
|
Total posts in this thread: 6 |
[Request a Feature]
[Post new Thread] [Add To My Favorites] [Watch this Thread] |
| Author |
|
|
MattBro
Member
|
I seem to get a disconnect between the performance reported by the performance page for a given ranking and my attempts to simulate this performance via the simulation tool. I was doing this primarily to try to simulate a short portfolio by creating a portfolio using the lowest ranked stocks first. An example would be to take the balance4 ranking and try to emulate the 20% or below ranking. To simulate this I removed all buy and sell rules other than a price constraint (which can also be specified in the ranking performance tool.) Every time I try to run this I get a performance that is much higher than that reported by the ranking performance tool. The only difference should be that the simulation tool would work with a smaller number of stocks than the ranking tool. Is there something I am missing here? The results are very different, and I wonder which one is real? As an example, explain why my unbalanced4 simulation does not even come close to the (poor) performance predicted by the 10% rank performance for balance4? ---------------------------------------- [Edit 1 times, last edit by MattBro at Aug 16, 2004 12:06:05 PM] |
||
|
|
marco
|
I ran your simulation but instead of ~40 stocks I used ~200 with 0 commissions and 0 slippage, and I got a total return of -16.75% vs the SP return of -7% for an active return of -9.7%. The 0-20 bucket in the rank performance graph returns -28% for an active return of -21%. I don't think a difference of -11% is disconnected. The 0-20 bucket has about 1500 stocks which cannot be simulated since there's a limit of 200 stocks in a simultion. Probably the main reason why a smaller set of stocks does a little better is for contrarian reasons. A stock that ranks extremely low will have a lot of room to grow. You can see this by trying 12 buckets in the rank performance graph (you can enter more than 10 buckets although it says that you are limited to 2-10). In this case the lowest bucket (0-8) actually outperforms the higher one (8-16) by about 10%. This is exactly what's happening in your simulation  (I just love it when things correlate! )---------------------------------------- Portfolio123 Staff |
||
|
|
MattBro
Member
|
This result is a little strange and it may take a while for me to accept it. Let's say I want to emulate the performance of the 10% ranking for balance4. According to the performance results for this, it consistently performs below the S&P 500. (I believe I force a minimum price of $10 so it may differ a bit from your results) Now there may be 100s of stocks in that bucket, but I shouldn't have to buy (or short) all of them to achieve the same mean performance. If I statistically subsample this set, over a long time average I should get similar performance, albeit noisier. What you are saying, however, is that within the low rank bucket, the balanced4 ranking may actually be negatively correlated with performance. One may suppose then that there is a 'flaw' in the ranking system. However I am seeing very similar results with some other simple ranking systems, such as my Value&Growth for example, which only uses a few very simple factors. ---------------------------------------- [Edit 1 times, last edit by MattBro at Aug 16, 2004 12:37:37 PM] |
||
|
|
gviersen
Advanced Member
|
Just a helpful hint I have noticed in short modeling. I have noticed that within factors, the stocks with "Not Available" or "Not Meaningful" data do not seem to be excluded from the rankings (based on reviewing ranking reports), but are dropped to the bottom. Therefore your very bottom decile (quintile, whatever) set of stocks will include those with NA or NM data. So if you choose Worst Stocks First in the simulation, you will be including all the stocks with NA/NM data, which will throw some noise into the system. I have found a better way to test short strategies is to reverse the direction of each factor, and then test for the highest rated (now the worst) stocks. That seems to be more consistent in the tests I have done as it seems to eliminate the NA/NM set (they drop to the bottom again). For that reason, as well, be very careful using "lower values rated higher" in your components. Again, the component will include the NA/NM stocks at the bottom of the factors as part of the lower(better) ratings. Always use "higher values rated higher" in the components (seems to work okay for factors). Marco, if that is not right, let me know. I came across that issue while trying to manually understand the rankings based on downloaded ranking reports. (Still cannot replicate Normal rankings, but I get the drift from my other post.) Hope that helps those trying short models |
||
|
|
marco
|
First of all the option to pick form the "Worst stocks first" is not very useful until we implement the shorting calculations. It was a quick test we added just to see what would happen. If the results underperformed then we knew we were on the right track with the ranking system. NA values are always at the bottom regardless of whether the factor is "higher is better" or "lower is better". Therefore I think reversing all the arrows is equivalent to picking from the worst stocks first. The main issue with NA's that we are still experimenting depends on the method you are using: For simple ordering NA's go to the bottom, but how far? Should NA's get a 0, or get the next lower score from the lowest valid value, or get some midpoint between 0 and the lowest score? Right now we assign the next lower score, but we might just make it user selectable. Think of PE for example, about 50% of the stocks have NA. What percentile score should those get? I'm starting to think maybe a midpoint value is best. For ranks based on normal distribution where the factors are transformed to aproximate normal distrib, what should an NA get? ---------------------------------------- Portfolio123 Staff |
||
|
|
gviersen
Advanced Member
|
You're correct - no right answer to the problem. It depends on your goal for the model. Do you want a company with negative earnings slammed out of your model or still considered based on the other factors? I like the user selectable idea, for the reason stated above - it depends on the user's goal for the model! I would add the ability to EXCLUDE as one of the options, if that is possible. While we are on the subject  , please, please, please include Buy/Sell criteria that would allow selection based on ranking model sub-factors. I realize it may be horribly complex to program buy/sell criteria functions on any factor on anyone's model. However, it could be as simple as an ALLRANK function so that you could have a criteria e.g., RANK > 90 and ALLRANK > 50. That way a stock would have to be > 90 for the aggregate rank and > 50 on each individual factor. Or an ALLRANK(%,threshold) where it had to be greater/less than the threshold on a % of the factors. Why? I tend to favor the stocks that are more consistent across the rankings when choosing between two stocks above my main ranking threshold. I would rather have the one ranked 2 points lower that is consistent across all the factors than the higher ranked one with one bad factor ranking. I can only do that manually now with the current system. |
||
|
| [Show Thread Printable Version] [Post new Thread] |
