Soon we will be adding a new way to handle NAs for Ranking Systems. Currently NAs drop to the bottom and get the lowest score, thereby penalizing stocks with NAs. This works well for long systems since you are mostly picking from the top stocks.
For shorting it does not work very well. More specifically for a system were you go long the top ranked stocks, and short the lowest ranked stocks. Your shorts will be made up of stocks with many NAs (depending on your ranking system). Furthermore the lowest ranked stocks will probably be stocks with sparse data (ie penny stocks), which are also hard to short.
For this reason we want to introduce a new parameter in a Ranking System specification to change the way NAs are handled. We believe this will make better hedging systems that use only one Ranking System. This new feature will not be very useful until the shorting and hedging capabilities are introduced to P123. However it will provide a jump start to start designing Ranking Systems using the Rank Performance tool.
Attached please find the description of the current method of handling NAs (which will remain the same), and the new method which we are currently testing, and should be launched soon.
Will this help the problem I have with Sims automatically selling positions if the Rank goes to 0 (usually because of an N/A or stock dropping out of a Universe) - even if Rank is NOT used in any Sell rule ?
If not, can we have some other solution to this ? - such as say a sell rule NORANK=false would not sell for zero or no rank, default would be NORANK=true which is how it is now.
NORANK is different. There’s no overall rank, regardless of inputs. It can mean two things: stock dropped out of universe as you say, or it stopped trading (either merger or BK).
I’m assuming you are using a screen to define your universe. Wouldn’t it be easier to relax the screen a bit? Try looking into the NORANK transactions, record the date and run the screen in that date to see which rules are making it fall off.
An option to ignore NORAnK only when stock fall off universe, but obviously not when it stops trading, will require some tinkering. But it’s the kind of tinkering that goes against the concept of “universe”, no?
I like the idea of removing NAs from the analysis. However we need to consider these cases:
One concern on assigning a neutral score to the NAs is that typically when I assess the quality of a ranking system I look for a continuous stairstep up on the performance by bucket as the percentile moves towards 100. The proposed NA technique will cloud that observation.
I think it might be useful to have an NA_Flag where if set to ON then the NA’s are flagged and removed from the ranking thereby retaining the stairstep appearance. Likewise, if NA_Flag is OFF then they are not flagged and can appear in the middle ranks as suggested.
A second concern is that I recall an article where the writer suggested that some Ranks are most effective in the middle, versus at the ends. Again, putting NAs in the middle would cloud this analysis. And again, a flag for choosing what to do with NA’s would resolve the issue.
IMO, it would be interesting to be able to compare shorts and longs in terms of a backtested attractiveness of expected return within the same ranking system. But that’s quite different from using a single screen where the top ranks are long candidates and the bottom ranks are good short candidates.
IMO, there is more work and no benefit to that sort of “two end” ranking usage compared to just using two separate rankings for generating longs and shorts.
That said, I think the feature you describe will be useful for long only (or short only) portfolios as it will help to with things like naturally comparing stocks that have earnings estimates and those that don’t.
Thank you for your comments. We’re not sure what this will do, so this is all conjecture. Initially we were thinking about two ranking systems for long/short, but I think this is worth a shot. A two ranking system hedging system can be added later.
Carl, two comments:
What do you mean removing NAs? Does that mean that if any NAs appear for a company it is excluded? I think it might be too drastic, and with systems with many factors the universe will shrink considerably
As far as the middle ranks. Does it really matter if the middle is flat? Aren’t the tails where the information really resides?
I hadn’t thought about the affect of removing NAs on Ranks with a lot of factors - good point. I have serveral rankings that are one, two and three factor… and for these it would be worth it to see the clean ranking (NAs cleaned out).
Regarding whether the value of ranking is at the tails, my answer is a qualified yes. If a ranking shows stair stepping from lowest to highest rank I have a lot more confidence in the rank’s ability to differentiate performance across the full spectrum, and therefor especially at the tails. Conversely when a ranking shows a lot of bouncing around and then one tail out/under performing I tend to pass it up.
I still want to explore rankings that do well in the middle, as sometimes the bell curve shows up instead of the stairstep. So I want to be able to ensure that the bell curve is not caused by neutralized NAs.
So, given my low factor count rankings and desire to see clean stairstepping/bell curves I would lean to having the option to turn off the NAs.
That said, I still think your approach to neutralize the NAs is a good one. I would just want a way to remove the clouding to the ranking graph.
One sudden insight/question… are you neutralizing the NAs only for the short Ranking system (which I will assume is auto built from the long Ranking system) in a long/short sim/port? If that is the case, then I have no problem with neutralizing for the short side. My concerns above have been related to how I use the Ranking pages to manually develop my ranking system.
BTW, the way I currently deal with the NA issue for shorting is that I reverse the ranking on all the factors for a long ranking system. This pushes all the NAs to the lower ranks on the short ranking while my short candidates are on the upper ranks. Maybe an alternative would be a “reverse arrows & boolean” on the Ranking>Factors for the ranking.
Why not make it a little more flexible and the user specify the rank to give to NAs? Personally, I would like to ‘punish’ them a little for having an NA, so 50 is too high. So I might want to give all NAs a rank of 35. Obviously you would only want to change it from the default of 50 if the ranking system was only going to be used for long only systems.
Allowing the user to set this in their My P123 section would be okay, but setting within each RS would be better so that you could leave it set to 50 for long/short RSs and alter it for long only RSs.
I think what I wrote above about “two separate rankings” was unclear. What I meant is that I didn’t see the benefit of being able to use the one ranking system to automatically generate longs as high ranks while simultaneously manually generating shorts as low ranks. In that setup, the low ranked “shorts” wouldn’t participate in any sort of simulation that was trying to pick between longs and shorts. Meanwhile, some factors are more difficult to write if one requires them to provide a useful total ordering on all stocks vs. just pick out attractive stocks at the extremes of the range. So overall, I don’t see the use case for having better ability to find longs as high ranks and shorts as low ranks in the same ranking.
I would like to have the ability to compare shorts and longs within the same ranking system where attractive shorts and attractive longs both have the ability to yield high ranks, and the “historical performance” and simulations would correctly understand which was which when calculating returns.
Is this feature released yet? Pl. advise how to fix existing ranking systems to be NA-neutral, so we can use lowest ranked stocks for short-selling.
Also, how can i evaluate the impact NA has on existing ranking systems today? e.g. if i could tell that a ranking system is not “polluted” by NAs, could i start shorting the low ranked stocks today, even before this feature is released?
No, the neutral NAs option is not implemented yet.
I think that the best way to develop short systems is to create an inverse of your long system like Carl suggested above. Copy your long system and reverse the arrows on High or Low is better (except for the top composite). That way all the stocks that were at the bottom for each factor of your original system will now be at the top except for the NAs which will still be at the bottom.
Anyone else have some good approaches for short systems?
