| Index | Recent Threads | Who's Online | User List | Search |
|
|
 |
Portfolio123 » List all forums » Forum: Ranking Systems and Factors/Functions » Thread: Found problem with ranking |
|
Total posts in this thread: 19
|
[Request a Feature]
[Post new Thread] [Add To My Favorites] [Watch this Thread] |
| Author |
|
|
marco
|
Dear All, I found a problem related to the ranking a factor/formula vs the Industry when used with a universe other than ALL stocks. In its current implementation the rank within industry ignores the universe. I'll illustrate with an example using the SP500 universe within the Communicatios Equipment Industry (COMEQP). There are currently 10 companies in the SP500 for that industry, therefore the ranks for a factor within an Industry should be 90,80,70...0 (assuming there are no NAs). This is not what's happening because it ranks all the stocks first, then picks out the ones in the SP500. See below for images of the current and soon to be released algorithm. This will only affect systems that rely on universes. Ranking vs the industry using ALL the stocks is not affected. We're very sorry for this problem. Please contact us if you need more info on this. The fix will be released very soon. ---------------------------------------- 
CurrentRankvsIndustryExample.gif
(9582 bytes)
(Download Count: 244)
----------------------------------------
ProposedRankvsIndustryExample.gif
(10706 bytes)
(Download Count: 244)
---------------------------------------- Portfolio123 Staff ---------------------------------------- [Edit 1 times, last edit by marco at Feb 27, 2007 7:25:17 PM] |
||
|
|
dcnelson
Advanced Member
|
If i understand this correctly it should only effect ranks of factors that are industry dependent. I assume stock factors are being ranked against the chosen universe ie the SP 500. Am I understanding that correctly. |
||
|
|
marco
|
This has now been released. It affects ranks of factors/functions vs the Industry, or the nodes with the Yellow industry icon in {} ---------------------------------------- Portfolio123 Staff |
||
|
|
dcnelson
Advanced Member
|
Is there anything we need to do to our ranking systems, portfolios or simulations |
||
|
|
marco
|
You should re-test your systems that use these type of nodes. I haven't seen much of a difference in results in my tests, but the possibility of big differences exists. ---------------------------------------- Portfolio123 Staff |
||
|
|
Circolo
Advanced Member  CANADA Joined: Dec 10, 2006 Posts: 29 Status: Offline |
Well this is an significant if unexpected development! I have retested alot of my screens and sims and unfortunately, there are significant differences in the resutls (so far almost all for the worst, I'm afraid) Some how the old system used to boost my results. Now % return are cut by as much as 20% or more and drawdowns significantly increase. On the surface your adjustment certainly seems more logical, but for me after reviewing of or individual results it might mean going back to the drawing board. I would love to hear what other users retest results are and if they are experiencing a similar outcome. As an aside I would retest anything involving Pr4W%ChgInd ect. factor even if it invloves the full Universe. Regards, Circolo  |
||
|
|
dcnelson
Advanced Member
|
This fix cuts my active returns in half on a lot of my sim's Back to the drawing boards |
||
|
|
z8735
Advanced Member
|
Marco, I have to protest: This might have been a bug, or it could be called unintended feature. Regardless of how you label, it had certain characteristics which provided the ranking functionality and around which P123 users spent countless hours developing their systems. By “fixing” it, you are changing the underlying characteristics around which systems were built and are in fact making our work obsolete, why would you want to do so? Take for example the Large Cap system published by Steve (Stittsville123) couple of months ago in http://www.portfolio123.com/mvnforum/viewthread?thread=2133#8672 Now run the same system under the new “fix” and indeed you’ll see that it has been “fixed”. Original system return: 42.5% annually. “Fixed” return: only 31% annually… I believe you should respect all the work done by your users / customers and not obsolete our work without a good reason. In that case I don’t think there is a good reason – the behavior of the ranking might have been different than originally meant, but it was still a valid system that differentiated between lower quality and higher quality stocks according to well defined criteria, so it worked and has been used as a foundation of many trading systems. Its logic should remain available for the sake of the many users who have devoted their efforts to developing for it. If you still think that for the sake of developing new system the new characteristics should also be made available then by all means – just make it as an option in the ranking system and universe selection screen, but please allow everybody who has existing systems to keep using the previous logic as before if he/she so wishes. In general - changing fundamentals around which the system works should really be avoided as much as possible. It might be a reasonable thing to do for a product in beta, but not for a commercial product with many users. Please do not undermine our efforts. Respectfully, Z. ---------------------------------------- [Edit 4 times, last edit by ZviEintracht at Feb 28, 2007 11:05:10 PM] |
||
|
|
dimitri
Advanced Member UNITED STATES Joined: Oct 26, 2006 Posts: 38 Status: Offline |
ZviEintracht, I do sympathize and totally see your point. A lot of my sims have changed also. But I have to say that it makes more sense to correct the problem for the sake of all future development. Consider this: as I understand it if the problem remains uncorrected then trying to rank a stock among its peers in a Universe would simply rank the stock among every other stock, which can easily be accomplished already. Why pretend you're ranking against a Universe try when you're not? On the other hand if a sim relies on ranking against all stocks to get the desired performance it can be changed to do so explicitly. Also it may not all be the fault of that bug Another possibility for altered sim performances is price cleaning that might have taken place. see this thread for an example: http://www.portfolio123.com/mvnforum/viewthread?thread=2307 I just think it makes more sense to get things fixed. dimitri ---------------------------------------- [Edit 2 times, last edit by dimitri at Feb 28, 2007 11:59:17 PM] |
||
|
|
jtbaccarat
Advanced Member
|
1. I wondered about this a while back myself; I figured it was purposeful, even though I found it odd. 2. I am glad it has been fixed as I originally thought it was this way it has now be changed to; but found it was not. 3. I applaud Marco's integrity for identifying a 'problem', rectifying it and explaining it to his customers; it is rare in business, unfortunately. Overall, it makes me even more comfortable with the service: if there are other issues, I am confident they will be addressed 4. If an entire model has its returns reduced by 50% when one input is changed this begs a few questions of the stature of the model; just my opinion, although I am sure it will not be well received |
||
|
|
[Show Thread Printable Version] [Post new Thread] |
