| Index | Recent Threads | Who's Online | User List | Search |
|
|
 |
Portfolio123 » List all forums » Forum: Feature Suggestions » Thread: Real Portfolio |
|
Total posts in this thread: 10 |
[Request a Feature] [Add To My Favorites] [Watch this Thread] |
| Author |
|
|
DennyHalwes
Advanced Member UNITED STATES Joined: Apr 28, 2004 Posts: 1532 Status: Online |
Marco, A review of all of the Forum feedbacks show a need for the ability to keep a real portfolio with P123 that is separate from the current portfolios that P123 provides. There are a lot of members that try to make their ports match their real trading by entering their actual trades with a manual transaction into their ports, and not using the port recommendations. I think this is a mistake. I don't think that the ports that are based on P123 Sims should be used for this task. If we use manual transactions instead of leaving the ports purely computed recommendations in place, we no longer can follow how good the computed Port is actually performing, or know whether or not we should modify the Buy and Sell Rules to improve its performance. What I suggest is an ability to track a real port with only manual transactions. What I in vision is a new tool that looks a lot like the current Ports (and has the same name as a Sim based Port), but it wouldn't have the need for a Ranking System, and Buy or Sell Rules. A member would simply enter the real trades that were made, and the Real Port would track the performance daily like the current Ports do, but would not make any weekly rebalance calculations at all. This approach would allow accurate tracking of Real Ports while maintaining the computed Port unmodified, allowing direct comparison between the two. You can simulate this with current ports by having two ports with the same name, but add ‘Real’ to one of the names. In the Real one, remove all the Buy and Sell Rules, and make only manual transactions. This way it will make no recommendations. In the other one accept all rebalance recommendations. Now you will be able to compare the performance of the real one to the computed one, and make decisions on improvements to the computed one (or stop rejecting the recommendations because your decisions aren’t as good As the computers!). Denny  ---------------------------------------- "The significant problems we face cannot be solved at the same level of thinking that we were at when we created them". Albert Einstein |
||
|
|
lynneeb
Member
|
As a newbie to this site, I find the distinction between computed portfolios and sims confusing. To me, a fully automatic model portfolio IS a simulation. My expectation of a portfolio based on a simulation is that it should reflect real-world trades if the owner is actually using the portfolio that way. In my case, I set up two portfolios based on two simulations that have good results and fit my style. In the largest portfolio, I manually entered stocks I already held and let the portfolio's sell rules time the sales of these stocks. Obviously, I'm not using the portfolio to test the simulation; I'm using the simulation to improve an existing portfolio. Having all the features of P123 such as rankings and rebalancing in a real portfolio is a plus for me. When examing community portfolios, however, perhaps it would be simpler to just flag the portfolios that have manual transactions. Then the distinction between real and model portfolios would be more obvious. |
||
|
|
vladinvest
Advanced Member
|
---- vladzaff: after editing my numerous spelling errors, sorry: ---- lynneeb, hi! Just a comment on you input here. I find a very useful difference in the Sims and the Portfolios. I am planning to post a larger "blog-like" forum message on system optimization. I have some experience in that part of mathematics and artificial intelligence systems. Some theory of it is very strongly related to what we all try to do he on P123. But just if I can lay out something in short: If you do Sim-ing, which basically combined with purposeful modifications of the parameters of the Sim is a sort of an optimization you can fall in to what P123 already explains in the user guides called "data mining trap" or "optimization trap". This is you optimized your Sim (your ranking system + your trading rules) so much that it got very specifically fit to the data that we already know about. In many cases systems that are strongly fit to specific data that we know about may not perform as well over unknown data (future market). That usually happens to Sims that through the optimization minimized the stocks number and the trades it made money on so much that the Sim may be completely inapplicable to any future market events. One very easy way for one to test their Sim for not being in a "optimization trap" is to test the systems sensitivity to modifying the parameters. Say you had a rule to only buy stocks where Price > $3 and Price <$7. If your Sim changes it performance drastically by changing the rule to > $3.5 and Price <$8 for example that is a definite indication you over-optimized you Sim and fell in an optimization trap. That is valid for any of you trading parameters including number of stock you are trying to trade etc. I have seen some Sim on P123 that show great results with let's say exactly 5 stock but if you change them to trade with 4 or 6 stock they show very different results (let's say times different). What that has to do with Sims vs Ports is a method for controlling over optimization called cross validation. And just shortly, since my message is getting too long for anybody to be interested to read: Cross Validation is a method where you optimize the Sim on known data but you leave some of the data for "crash testing". That's a bit hard to be done when the optimization is done by a human being but basically with this method you are allowed to optimize the Sim as much as you want based on the results over the known data but you should not continue optimizing any further of the performance over the spare "unknown" data do not reduce. Well to me the Ports in P123 are excellent tool for this cross validation. I have a chance to run my Sims over data that is as unknown as it can be. It's the tomorrow’s markets. That's why Ialso think like Danny there should be very little manual control over what happens in the Ports apart from may be more discretionary rebalancing. Ideally, and I know this will be huge effort from P123, I would like to see an automated optimization feature where I can say optimize Parameter ABC from X to Y and in addition to that do cross validation (hide some of the data from the optimization algorithm and show me the performance over it). I hope this was ont too much and too confusing. Happy trading, Vlad ---------------------------------------- Mind is like a parachute - it only works when it is open. ---------------------------------------- [Edit 1 times, last edit by vladinvest at Nov 24, 2004 1:43:59 AM] |
||
|
|
rannaswamy
Advanced Member
|
Since Portfolio123 is a dream-come-true application, I am adding a bit more to Vladzaff's feature request  1. Automated Robustness ranking of parameters: Batch run each parameter with settings around the chosen values and rank which parameters are robust and which could be 'optima traps'. 2. Cross-validation testing or Automatic out of sample testing: There could be two ways to do this. For instance, Build sim over years 2000-2002 and test it with 2003-2004. This way you get into bull sims tested on bear market etc. Or The sim might skip alternate periods during development and run on the remaining periods for OOS testing. This way you can even validate a sim DURING its development. Anyways, Marco and team, what you have created is a true marvel. You should even consider publishing Portfolio123 recommendations newsletter for Mutual fund managers or individual investors who may not be tech savvy enough to use even this piece of cake application. And you need lot more exposure, outside of Domash's column, you are not very heard-of. Let us know what else we can do. Best wishes Ravi ---------------------------------------- [Edit 1 times, last edit by rannaswamy at Nov 24, 2004 12:58:02 AM] |
||
|
|
vladinvest
Advanced Member
|
I total y concur with Ravi about the state of excellence of P123 plus having in mind the owners seem to be continuously working on new desired features still to come. And again and again, I am realizing that a lot of what we talk here are “nice-to-have”’s that may not be put at high priority or at all in the improvements pipeline. There are so many important things that seem to be underway already like daily fundamental data feed, short selling etc. Yes, Ravi, optimization functionality can be achieved many different ways. What you mention totally makes sense. One thing that could be done for cross validation is the validation data period (s) to be defined/selected by the user. That way he/she can decide whether they want to “train” on bull market and cross validate on bear etc. Vlad ---------------------------------------- Mind is like a parachute - it only works when it is open. |
||
|
|
lynneeb
Member
|
Hi, vladzaff! Thanks for your explanation of optimzation traps. That may explain why I haven't yet consistently improved on any of the community Sims . But I've been thinking some more about Portfolios versus Sims. When a Portfolio rebalances automatically, it isn't using future data so much as "new old" data from the last closing date. Since P123 lets us easily re-run a Sim, Sims do not have to be frozen in time; they can be refreshed with the same data that is input to the portfolios. If you take a model Portfolio, such as "GARP Top Ranked" with current total return of 460.19% and create a Simulation, you get a different total return: 402.22%. If you compare transactions between the Portfolio and the Simulation, they were identical from the start on 4/2/2001. But something happened in May. The Simulation rebalanced on 5/29/2001 and sold 4 stocks, including OXY. The Portfolio actually rebalanced on 5/25/2001 and did not sell OXY. Both sets of transactions appear to use the same closing data, but by waiting a few days, the Simulation saw OXY as a Laggard where the Portfolio didn't. Strictly speaking, the "correct" rebalance date was 5/29. Undoubtedly, the accumulation of such differences would change the total returns between the two. So which is the real "GARP Top Ranked" -- the real time Portfolio or the Simulation of it? I'd go with the Simulation. Simulations have the advantage (I think) of reflecting the accumulated fixes to both P123 programs and data (and any introduced errors). The past behavior of P123 in the model Portfolios can never be reproduced. But future behavior (I hope) can be anticipated based on good Simulations that avoid the kind of optimization traps you mention. |
||
|
|
vladinvest
Advanced Member
|
lynneeb, I am glad what I said about optimization made some sense. Yes my simulations too are different form my portfolios because some times I take decisions to manually rebalance. Sims are always fully automatic. One other difference I is that a Sims can start from a different date that the one a portfolio was open on and than can trade slightly different set of stocks. Referring to robustness of a Ranking System and Trading System (in the Sim or Port), the performance of a good, stable system should not be affected much by shifting the re-balance dates or any other small change of parameters. A good strategy should have 1) a Ranking System that should be based on common sense and should produce results not influenced by small differences of conditions and 2) a set of Trading Rules that also should produce relatively consistent results regardless of small changes in conditions. I use all the performance indicators from the P123 only for what they are - “indicators”. A realistic portfolio will never have exact same results. A strategy that generalizes well some rules to select good investments will always show good results regardless of whether it is traded exactly like the Sim or the auto- Port. When I get different results from similar Sims ot Profolios, to be able to predict what is going to happen in reallity I simply assume the lowet denominator. This way if I have a verison of a Sim that retuns 40% on the firt 3 stocks, one that retunrs 45% on the firt 5 and 38% on the first 20, I simply assume 38% as a good general guidance. I do not find it wise to think that chosing to trade exactly 5 stocks will improve the quality of my strategy. Thanks for sharing your thoughts. ---------------------------------------- Mind is like a parachute - it only works when it is open. |
||
|
|
DennyHalwes
Advanced Member UNITED STATES Joined: Apr 28, 2004 Posts: 1532 Status: Online |
All, Well, I sure started a good discussion with my first post! With respect to optimizing Sims without data mining; After I have optimized the Sim over the total time from 03/31/2004 to the latest date for which data is available, I run two different tests for robustness. First, I list every 4th stock from the Transactions page in the Restrict Buy List, and rerun the Sim. This forces the Sim to buy 25% new stocks, but allows it to keep most of the stocks it bought first. I'll repeat this by removing every 5th stock, which forces it to buy even different stocks. I don't want to force the sim to buy too many different stocks, because, if I have done a good job of developing a Sim that only buys the stocks most likely to increase in value (and not data mining the stocks that just happened to increase in value), then I don't want to throw all those stocks away and only buy second rate stocks. I don't like to use different time frames like many members are suggesting, because if the results are significantly different, I don't know if it's because there was too much data mining in the first Sim or the market was just significantly different during the time frame of the second Sim. I need to compare apples to apples as much as possible. Denny ---------------------------------------- "The significant problems we face cannot be solved at the same level of thinking that we were at when we created them". Albert Einstein |
||
|
|
vladinvest
Advanced Member
|
Denny, interesting way to cross-validate. In any case my feeling is one has to try an oprimized system over data that was not used for the optimization. Your method certainly is very interesting. Vlad ---------------------------------------- Mind is like a parachute - it only works when it is open. |
||
|
|
marco
|
Thank you all for your comments. Right now P123 requires a portfolio to have a Ranking System associated with it. This will change in the future to allow for other types of trading systems, such as a portfolio managed by a screen, and to allow for manual portfolios. At the moment there is no way to distinguish portfolios that have had manual transactions. You could look into the transaction list and look for the string "Manual Transaction" in the notes, but that is far from ideal. What can be easily done is add a parameter in the Community search page to only return portfolios that are "Automatically" managed. As far as avoiding the data-mining trap with simulations, all your suggestions are valid. I'd just like to remind everyone of the feature in Step 5, the "Add From Previous Runs" button. The way it's used is that after a simulation is run and looks reasonable, re-run the sim again with all the same parameters, but click on the button which adds all the stocks that were purchased in the previous run to the restricted list. The ideal result would be a somewhat lower performance for every subsequent run. Another idea we're toying with is rolling 1 year sims. The way it works is that multiple 1 year sims are run at different starting periods. The results of each 1Y sim are tabulated with other relevant statistics, such as avg, stddev, excess return etc. A good system should show consistent results for each 1Y sim. Finally, I'll respond to Ravi's comments on a separate thread since it might interest other people. Marco ---------------------------------------- Portfolio123 Staff |
||
|
| [Show Thread Printable Version] |
