Index | Recent Threads | Who's Online | Search

Posts: 87    Pages: 9    Prev 1 2 3 4 5 6 7 8 9 Next
Last Post
New Thread
This topic has been viewed 3411 times and has 86 replies
mgerstein
Re: To Quant Or Not To Quant, That Is The Question

Interesting thoughts coming in.

Generally, I’m sensing that my view of the hi-quant opportunity is reasonable; that whatever its virtues may be feature for feature, it’s not adding up as a viable commercial path for us.

So the family office thing is about marketing but also, it seems from Steve’s latest post on the topic, a matter of site presentation. Kurtis also made some interesting site-presentation points and some others I need to digest more. I’m very on board with all of Doug’s views and want to think more about some details he puts forth. All in all, much to chew on here.

Jim, I don’t necessarily grasp all the points folks have been addressing to you, but I hope you find their perspectives helpful.

Marc Gerstein
Corporate Advisor, Portfolio123
Director of Research, Chaikin Analytics, featuring <i>Power Gauge</i> model, designed on Portfolio123
Blog: https://actiquant.com
I predict the future, as soon as it becomes the past

Dec 4, 2019 4:30:55 PM       
Jrinne
Re: To Quant Or Not To Quant, That Is The Question



Generally, I’m sensing that my view of the hi-quant opportunity is reasonable; that whatever its virtues may be feature for feature, it’s not adding up as a viable commercial path for us.

Jim, I don’t necessarily grasp all the points folks have been addressing to you, but I hope you find their perspectives helpful.

Marc,

Thank you for looking at this. Their perspectives, and yours, are very helpful.

-Jim

From time to time you will encounter Luddites, who are beyond redemption.
--de Prado, Marcos López on the topic of machine learning for financial applications

Dec 4, 2019 5:15:24 PM       
marco
Re: To Quant Or Not To Quant, That Is The Question

I just read this thread. Sorry I've been doing coding again, and time flies when you are having fun.

Judging from the response (zero) of what we just released "New Opinion & Watchlist functions!" it's safe to say that these are not the enhancements you, the active forum user, were hoping for. But this is inline with what we've been doing for over a year: working on the new (free) Invest section, linking with brokers to make the investing process easier, real time prices, planning an advisor biz, and adding tools for the "stock pickers". We didn't completely stop enhancing the Research tools , but all recent additions have been small. The last big effort if I recall was the variable position sizing. A great , complex tool, that very few use.

Yes, we've taken a breather with the Research tools . I'm surprised nobody yet screamed: why the f&^%*! you doing things nobody wants ?? Obviously we think that once all the pieces come together many will want it. But lets back up a little to what started this...

About a year ago S&P changed the rules, and wanted even more changes for the next renewal. I'd rather not get into specifics; it was BAD. I've made peace with it. Data providers are all similar, if not the same, and all have hungry sales people with hungry wives & babies. You just have to learn to play the game. It also doesn't help when you are developing a product that may be viewed as cannibalizing some of their own; which is what would happen if we become more quantish.

With that in mind , and the fact that the quant clientele didn't exactly grow exponentially , we decided to explore different directions that would not need expensive data, and would not piss off providers (with Factset we have a rev sharing agreement which has a better chance of a long term relationship)

That's why we've taken a break from making P123 more quant-ish. And yes, some of us do not have the quant know-how, me included. I'm more of a stock picker/programmer, and my statistics is long in the past (also it doesn't help that I've done really well stock picking biotechs using a simple screen and the fundamental chart). But we do have quant people that can help us like Marc & Riccardo, university professors, students, and WE DO want to get back to work on Research and making P123 more appealing to quants/programmers. That's fun to. Some quant additions are low hanging fruits and would please many. Other things I would like are APIs, automation, better long short, IC, parameters for custom factors, and some of Jrinne suggestions which I would love to discuss at some point.

So what are some low hanging fruits? What are the quant additions that would generate good ROI ( with ROI defined as quant guy happiness inversely related to development time) ?

Thanks

Portfolio123 Staff.

Dec 4, 2019 8:48:07 PM       
Edit 1 times, last edit by marco at Dec 4, 2019 8:49:24 PM
yuvaltaylor
Re: To Quant Or Not To Quant, That Is The Question

I’m not a P123 power user, but I was involved in the development of several software products for data processing (billions of transactions a year) that were used by a range of users between “analyst” and “C++ Coder”, and I think some of the frameworks we implemented may apply:

1: Custom Data Series: I’ve tried using this feature in P123 and was unable to do what I wanted. Custom Data Series was a basic feature needed at the “analyst” level on the products I managed. Even StockCharts.com has this feature. I believe this is an “analyst” level feature but would also be used by developers.

2: Extended Custom Formulas: P123 has this feature, but it is severely limited in my opinion because of field length restrictions (I had to break my single formula into 30 separate formulas). These should also have the ability to access the custom data above for reads and possibly writes. I believe this is an “analyst” level feature but would also be used by developers.

3: Simple Automation: I think sometimes it’s useful for “analysts” to automate the export of screen results.

4: Read Webservice API: Anything that the user sees on the screen should theoretically be accessible through a web service call. For example, it should be possible to access the results of a simulation, or a screen, by making a web service call and getting back JSON data. I believe this is a developer level feature, which would then allow developers to write custom code in the language of their choice on their own remote platform.

5: Write Webservice API: It should be possible to create a new simulation, or ranking system, etc. by making web service calls. I believe this is a developer level feature, which would then allow developers to write custom code in the language of their choice on their own remote platform.

6: Embedded Authoring and Runtime Engine: In one of my old products, we also integrated an open-source Java Scripting engine, so that analysts could write code within the product. As our runtime engine processed data, there were callout points to the custom code. One difference between this and the existing P123 “custom functions” is that users could define user variables within different scopes. So, for example, start of simulation, call function x, which might initialize some custom variables that could be accessed by other custom functions. Our software ran locally, so custom code could also make callouts to the local system, databases, etc. How much additional functionality is needed is not clear.

In the products I worked on, we also had a custom reporting framework, an audit framework, a validation framework, a mapping framework, and a private label framework, but these are I believe less relevant.

I believe #1 to #3 above are basic features P123 should have. A limited set of APIs (#4 and #5) seem like they wouldn’t be too hard. For example, if the custom data series above had a Write Webservice API, a developer could write some Java/VBA/... code on their own machine to extra data from source A and insert it into a P123 custom data series, which custom functions could then access when a simulation was started through a WebService Call, and the simulation results processed through yet another web service call. I believe #6 above could also be reasonably done but would have to be very well defined through use cases.

My expectation would be that anything that Jim or other Quants wanted to do would then be implementable through their own custom code using the standard frameworks above, but that the frameworks would have a lot of other uses as well. But I'm not a Quant, so perhaps I'm offbase.

Anyway, I think P123 is a great product.


Thanks a million for these suggestions. Allow me to go through them one by one.

1. What do you mean by custom data series? I'm intrigued. I can call up what I think of as custom data series using the Custom Series tool. For example, if I want IBM's operating margin as a series, I would create a custom series with two rules: UnivSubset("IBM") and UnivAvg("1=1","OpMgn%TTM"). I could then name that series and use it with the GetSeries command. The custom series feature is very powerful. But you probably mean something else--please do explain.
2. We have a 500-character limit. Would a higher limit do the trick? Or is something else called for? I find it easier to manage data by embedding custom formulas rather than putting everything into one formula. In order to calculate intrinsic value, for example, I use about thirty different custom formulae, but each one can then be used for other things too.
3 through 5. Much of this can be done--and has been done--by users using Python and Ruby, but I agree, in principle, that it would be good for P123 to make it easier, and I will discuss your suggestions soon.
6. This is quite ambitious, but again, I'll certainly bring it up for discussion. If, in the meantime, you could address my questions about 1 and 2 I'd appreciate it.

Thanks!

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Dec 4, 2019 9:05:01 PM       
yuvaltaylor
Re: To Quant Or Not To Quant, That Is The Question

I think a lot of people are confused as to how to best utilize those tools (arcane to the average investor off the street) in a sound methodology to make money. This requires knowledge sharing. I do agree with the approach P123 has made to do more general user hand-holding.

. . . if I were just some general mildly curious investor who decided to give P123 a trial run for a few months, I can see where it's just too dense to follow through on with a long term financial commitment. I think there needs to be better curating of the knowledge that has been shared by the very smart people who have generously shared their hard won wisdom on these forums over years and years. I think the knowledge shared on these forums is an asset for P123 that goes underutilized.


We are going to be starting a series of webinars to address these concerns, probably within the next few weeks. Stay tuned. If a user wants to conduct a webinar for other users, that might be worthwhile too.

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Dec 4, 2019 9:13:27 PM       
yuvaltaylor
Re: To Quant Or Not To Quant, That Is The Question

For what it's worth, here are my two cents on the subject. To increase interest in your service, Portfolio123 should do the following:

A) Focus less on promoting optimized ranking systems. The severe underperformance of the DM models should have set off alarm bells. Perhaps the overemphasis of super-optimized quant models--and not the changing dynamics of the market, i.e., growth vs. value--is the main culprit. The result of this poor performance has led to 1) the people designing these models dropping out of your service and 2) newcomers looking at the results and getting discouraged (That DM ranking screen is horrible advertisement for your service!).​ To believe that these DM builders would have outperformed their benchmarks if they only had more quant tools is hard to believe. (Note: I'm all for new tools! I just don't think that is the solution to the main problem of over-optimized models leading to poor performance, leading to fewer subscribers)


Great points. There's a very basic problem that's at the root of this.

Portfolio123 offers users a chance to design and backtest strategies. Because of this capability, users are going to want to optimize those strategies, and the most active users are the ones who will spend the most time doing so. I speak from experience because I was one of those users myself, and I'm still trying to figure out whether or not to optimize, and what alternatives there are to doing so, and what exactly constitutes optimization. So the urge to optimize is inherent in what P123 offers. It takes a tremendous amount of discipline to fight that urge. For us to fight optimization would be like a videogame maker telling its players not to try to win. People are always going to treat P123 like a game, no matter what we do. It's part of the fun of using this site: look, Ma, my backtest gets a 50% CAGR!

If P123 takes the stance of "optimization will harm your out-of-sample performance," we'd be basically telling our users not to use our tools. And we can't KNOW that it's the case. I've been optimizing systems for four years and have maintained a 30% CAGR during that time. Maybe I would have done even better with less optimized systems. I don't know.

P.S., I believe the "showing off" of optimized, simulated portfolios in threads (that show amazing stats, of course) is wrong and misleading, especially without proper disclosure. Why do people rarely, if ever, include out-of-sample performance? As a relative newcomer, I look at these beautiful charts and then look up the person's DM performance and say "what the hell?"


I agree, even though I have been guilty of this myself. If people want to crow about their out-of-sample performance, as Andreas and I do quite often, that's immodest but harmless. In a way, so is crowing about a 50%-per-annum CAGR in a simulation, except that it encourages overoptimization and data-mining. But the worst is to present a simulated model as if it were actually implemented. That is a no-no.

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Dec 4, 2019 10:08:25 PM       
sthorson
Re: To Quant Or Not To Quant, That Is The Question

2. We have a 500-character limit. Would a higher limit do the trick? Or is something else called for? I find it easier to manage data by embedding custom formulas rather than putting everything into one formula. In order to calculate intrinsic value, for example, I use about thirty different custom formulae, but each one can then be used for other things too.



I have north of a thousand custom formula because I have to embed them. Last I checked the character limit was 250. It would be helpful if the custom formula character limit was increased substantially.

MORE IMPORTANTLY, allow the formula input box to be expanded, so that one can see all that is being entered.

Dec 4, 2019 10:32:55 PM       
yuvaltaylor
Re: To Quant Or Not To Quant, That Is The Question

2. We have a 500-character limit. Would a higher limit do the trick? Or is something else called for? I find it easier to manage data by embedding custom formulas rather than putting everything into one formula. In order to calculate intrinsic value, for example, I use about thirty different custom formulae, but each one can then be used for other things too.



I have north of a thousand custom formula because I have to embed them. Last I checked the character limit was 250. It would be helpful if the custom formula character limit was increased substantially.

MORE IMPORTANTLY, allow the formula input box to be expanded, so that one can see all that is being entered.


It's 500 characters now. Thanks for the suggestion, and I'll see what can be done.

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Dec 4, 2019 10:37:30 PM       
sthorson
Re: To Quant Or Not To Quant, That Is The Question

Thank you Yuval

Dec 4, 2019 10:39:11 PM       
Jrinne
Re: To Quant Or Not To Quant, That Is The Question


So what are some low hanging fruits? What are the quant additions that would generate good ROI ( with ROI defined as quant guy happiness inversely related to development time) ?

Thanks

Marco,

Thank you.

P123 is a wonderful tool! It works well exactly as is. Machine learning does work better for certain situations. For example, the linear (and normalized) equations now used for the ranks are one reason model performance drops off (generally) with models that have more than 5 stocks (I would love to show you this some day). But it is a fact that P123 cannot do as well as other methods with more than 5 stocks when the data is not linear.

This is all to say P123 works well. And if there are any limitations one can work around most of them—at least to some extent. For example, run three 5 stock models instead of a 15 stock model.

Getting to the point. All of Python’s useful features could be unleashed if we could have access to the m x n matrix (also called an array or DataFrame) that you (must) create when you run a sim. The array would have a hierarchical ticker and date row index. The columns would be the returns over the rebalance period (e.g., next week) and all of the factors or functions in the sim. The data points would still be the rank.

Question: Isn’t this matrix sitting in memory or on the hard drive at some point during a sim run? If not can it be created easily?

If so, why not move this matrix over to a PC loaded with Python (all Open Source) and allow us to have the most advanced quant system that one could create? With no download to the user that a data provider could object to.

Outline of points:

1) Python is free. The extensive Libraries are also free and Open Source.

2) The Matrix already exists or may not be hard to create.

3) It could be done on a laptop (for one individual). I defer to you as to how much you want to scale this up. But I would pay for a PC to get this started. And what I really mean is you could probably buy the equipment with reasonable user fees. Equipment that can do a lot with a little: one server and not that big of a server, I think.

Summary: The overhead many not be big.

How useful really?

First: not like Quantopian. Quantopian is extremely limited FOR WHAT WE DO HERE. P123 is better even without the Python. Perhaps we can take this as a given for now.

There is a huge body of people with machine learning skill out there.

MARC IS A MACHINE LEARNER. He really is. The econometrics he learned when getting his degree is a type of machine learning. It is also called multiple regression.

RedShield and pvdb are two P123 members who have already explicitly expressed interest in doing multiple regression. RedShield ran a hedge fund that used multiple regression as its quantitative method.

Maybe not Marc but everyone with a degree in Finance would want to try an econometrics model.

But there is an entire world of machine learners out there who use other methods. Tom Yani another P123 user has used BigML to run Random Forests, which is a common and relatively easy machine learning method. RedShield has run some Random Forests.

There are a few million people who run (and believe in) Random Forests. If they invest in stocks they will want to check it out. Where else could they do it? Nowhere else.

There is a huge body of machine learners who have gone beyond multiple regression and Random Forests. They may be in the Silicon Valley, work at NASA, work in Universities, work for GOOGLE or FaceBook. If you count all of the technophiles in Europe and Asia then I believe you have a huge market. They will want to run Support Vector Machines, Kernel Regressions, LASSO Regression, Ridge Regression, LOESS, MARS, EARTH, Random Forests, non-linear regressions, polynomial regressions, splines, C4.5,…...even Deep Learning. There are a lot of people that do this. I see them on the internet.

All of the above algorithms can be run with Python and the libraries. I have done all of them (except deep learning) on a laptop (some with R). We do not need to understand them to make them available. Only Python and the data we already have would need to be managed.

I can show you that some of the methods above work well.

You probably already know that PANDAS in Python was created by AQR Capital Management for their use. De Prado from AQR Capital Management has a book about how this can be used by retail investors. It is a serious tool now available as Open Source.

Bottom line: one matrix (per customer), one PC (possibly scaled up) and you have a HUGE market. And it is state of the art quantitative investing.

Ease of use:

For me the hardest part of Python is the data wrangling. After that the programing is not hard. The scripts are extremely short. People could share the scripts. Maybe P123 could write a few scripts.

I had one Fortran Course in college. I did audit a DOS course after I got out of Medical School so I could use Windows. If I can do it anyone interested in quantitative investing can do it: even if it is a hobby—as is my situation.

So that is it. If that m x n matrix is sitting in memory after running a sim you could reap a big profit from it, I think.

If that m x n matrix already exists you should use it.

What is a pitch deck? Should I get one;-)

Thank you Marco. I would love to answer any questions. Maybe I could even show a few examples through email, over the internet or even in person.

-Jim

From time to time you will encounter Luddites, who are beyond redemption.
--de Prado, Marcos López on the topic of machine learning for financial applications

Dec 5, 2019 4:48:16 AM       
Edit 26 times, last edit by Jrinne at Dec 5, 2019 7:56:53 AM
Posts: 87    Pages: 9    Prev 1 2 3 4 5 6 7 8 9 Next
 Last Post