Index | Recent Threads | Who's Online | Search

Posts: 373    Pages: 38    Prev 3 4 5 6 7 8 9 10 11 12 Next
Last Post
This topic has been viewed 18551 times and has 372 replies
tissa123
Re: FactSet beta site v1.0, NOW LIVE

Yuval,

Please verify NetPlant, Inventory, Recvbl and Payables.

Apr 22, 2020 3:23:22 PM       
yuvaltaylor
Re: FactSet beta site v1.0, NOW LIVE

When running one of my sims, I get

Ranking failed on 01/02/1999: Invalid criteria in Rule 6. Error near 'FINANC': Invalid command 'FINANC'

rule 6 of my Universe contains:
Sector!= FINANC


Those names were deprecated a while back, though they still work. We'll get them to work on the beta site, but we're not planning for them to work once we make the switch to FactSet. Please use the names listed here: https://www.portfolio123.com/doc/doc_detail.j...ector&popUpFullDesc=1

It should be Sector != FINANCIAL

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Apr 22, 2020 3:46:57 PM       
atw
Re: FactSet beta site v1.0, NOW LIVE

Hi Aaron and Yuval,

I have a confusing update...

The Trello Bug log suggests that issues with WeeksIntoQ have been resolved.

But when I remove WeeksIntoQ as a variable, my results in the Beta CompuStat w/preliminary data are improved and much closer - not the same but also not showing the heart stopping differences I have otherwise been seeing - to what I get on the existing website with or without WeeksIntoQ.

There is no reason I can think of that removing WeeksIntoQ should improve my results. It's not a major contributor when I include it but excluding it should only make results worse.

Therefore I would guess that WeeksIntoQ OR some aspect of the Beta w/prelim database upon which WeeksIntoQ is acting must not be right.

Separately, could there be platform issues going back and forth between the existing site and the Beta site? I notice when I run something on one or the other that settings are retained on the other. In other words, if I remove a rule using WeeksIntoQ on the Beta site the rule remains removed on the existing site. I suppose this is as things should be but can we be sure the cache is cleared? Maybe it would be better to make copies and run each copy exclusively on the existing or Beta site?

By the way, is it possible to run simulations simultaneously on the existing and Beta site? That would sure speed things up.

Finally, at the risk of repeating myself, if you could provide some concrete “before and after” examples related to several companies and variables affected by the annual preliminary data-related fix, it would sure be appreciated. This has been discussed in a number of ways. Speaking for myself, I’m really not clear about how the change plays out, what it means, and/or what I might do to adjust, if that’s even a possibility.

Thank you.

Hugh

Apr 22, 2020 6:41:09 PM       
rodbarcv2
Re: FactSet beta site v1.0, NOW LIVE

I tested several portfolios, and performance / drawdown / Sharpe Ratio are either same or better on FactSet. Only a few portfolios had a similar performance with slightly lower Share Ratio, which is still ok. But I have 2 sims (and respective Live Portfolios) with significant worse performance with FactSet. Could you please take a look to determine what is driving the worse performance in FactSet?

Both simulations exclude prelim. Both 10 year simulation.

https://beta.portfolio123.com/port_summary.jsp?portid=1526326
Compustat: 376% total return, -16% max drawdown (1.20 Sharpe Ratio);
FactSet: 173% total return, -38% max drawdown (0.76 Sharpe Ratio);


https://beta.portfolio123.com/port_summary.jsp?portid=1527527
Compustat: 312% total return, -18% max drawdown (1.31 Sharpe Ratio);
Factset: 246% total return, -21% max drawdown (1.12 Sharpe Ratio);


Thank you,


Rod

Apr 22, 2020 6:44:40 PM       
atw
Re: FactSet beta site v1.0, NOW LIVE

Hi Rod,

Is it your intention to restrict access to these links or is that something the Forum is doing?

Thank you.

Hugh

Apr 22, 2020 7:11:20 PM       
yuvaltaylor
Re: FactSet beta site v1.0, NOW LIVE

I tested several portfolios, and performance / drawdown / Sharpe Ratio are either same or better on FactSet. Only a few portfolios had a similar performance with slightly lower Share Ratio, which is still ok. But I have 2 sims (and respective Live Portfolios) with significant worse performance with FactSet. Could you please take a look to determine what is driving the worse performance in FactSet?

Both simulations exclude prelim. Both 10 year simulation.

https://beta.portfolio123.com/port_summary.jsp?portid=1526326
Compustat: 376% total return, -16% max drawdown (1.20 Sharpe Ratio);
FactSet: 173% total return, -38% max drawdown (0.76 Sharpe Ratio);


https://beta.portfolio123.com/port_summary.jsp?portid=1527527
Compustat: 312% total return, -18% max drawdown (1.31 Sharpe Ratio);
Factset: 246% total return, -21% max drawdown (1.12 Sharpe Ratio);


Thank you,


Rod

I'm afraid not. What we're trying to do is to isolate data points that are different and try to figure out why they're different. It would take us many hours to look into simulations with lots of different rules and ranking systems to try to figure out what's causing differences when so many data points are different to begin with.

What you should do is analyze specific instances. Try to figure out why your results are so different by looking at the particular stocks your systems are picking. Dig deep to find out what the differences are. If they're NOT related to fundamentals, then you may have found a bug, and you should let us know immediately. If they ARE related to fundamentals, is there a way you can make the two different numbers make sense? Or are the two different numbers so far apart that one of them HAS to be wrong? Can you compare the numbers with the actual financial statements to see what FactSet and Compustat are doing differently? Or do you think we're using the wrong datapoint? These are the kinds of investigations that can really help us.

Thanks, and I'm sorry I can't do what you ask.

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Apr 22, 2020 9:14:08 PM       
yuvaltaylor
Re: FactSet beta site v1.0, NOW LIVE

Hi Aaron and Yuval,

I have a confusing update...

The Trello Bug log suggests that issues with WeeksIntoQ have been resolved.

But when I remove WeeksIntoQ as a variable, my results in the Beta CompuStat w/preliminary data are improved and much closer - not the same but also not showing the heart stopping differences I have otherwise been seeing - to what I get on the existing website with or without WeeksIntoQ.

There is no reason I can think of that removing WeeksIntoQ should improve my results. It's not a major contributor when I include it but excluding it should only make results worse.

Therefore I would guess that WeeksIntoQ OR some aspect of the Beta w/prelim database upon which WeeksIntoQ is acting must not be right.

Separately, could there be platform issues going back and forth between the existing site and the Beta site? I notice when I run something on one or the other that settings are retained on the other. In other words, if I remove a rule using WeeksIntoQ on the Beta site the rule remains removed on the existing site. I suppose this is as things should be but can we be sure the cache is cleared? Maybe it would be better to make copies and run each copy exclusively on the existing or Beta site?

By the way, is it possible to run simulations simultaneously on the existing and Beta site? That would sure speed things up.

Finally, at the risk of repeating myself, if you could provide some concrete “before and after” examples related to several companies and variables affected by the annual preliminary data-related fix, it would sure be appreciated. This has been discussed in a number of ways. Speaking for myself, I’m really not clear about how the change plays out, what it means, and/or what I might do to adjust, if that’s even a possibility.

Thank you.

Hugh

The Trello site noticed issues with WeeksToQ and WeeksToY. Issues with WeeksIntoQ have not yet been resolved. WeeksIntoQ is basically completely broken on the beta site right now. Thank you for calling this to our attention.

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Apr 22, 2020 9:20:45 PM       
yuvaltaylor
Re: FactSet beta site v1.0, NOW LIVE

Just to update you--a few more bugs have been fixed, though there are still more. You can see most of them on the Trello board. In addition, KEEPNA on the beta site was sometimes returning 0 instead of NA. You shouldn't see that happening any more.

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Apr 22, 2020 9:29:44 PM       
dnevin123
Re: FactSet beta site v1.0, NOW LIVE

Differences in ROI% Calculation example:

For Ticker("DHT") if I look at the calculated ROI% on the Production Server and on the Beta Server using Prelim I see fairly big differences in the calculated values.


Production Server
ROI%5YAvg = 3.88
ROI%TTM = 6.07
ROI%PTM = -0.66

Beta Server
ROI%5YAvg = 5.10
ROI%TTM = 6.61
ROI%PTM = -0.32

Is this due to the way preliminary data is being handled? Seems odd that it would have a larger effect on the 5YAvg then on the TTM and PTM calculations.

Thanks,

Daniel


I have looked into this a little deeper to try and understand the differences. I don't know the exact details of the ROI calculation, but assume it is similar to the following

ROI%TTM = IncAftTaxTTM / InvCapTTM

where

InvCapTTM = Avg(EqTotQ+DbtLTQ,EqTotPYQ+DbtLTPYQ)

If I compare this estimated ROI calculation to the actual ROI calculation on the Production Server I get fairly similar results. For example, for ticker "DHT" on the date 10/15/2013 I get the following

IncAftTaxTTM = -114.47
InvCapTTM = 414.90
my Estimated ROI%TTM = -27.59
P123 ROI%TTM = -26.50

However, on the beta test server I get the following for "DHT"

IncAftTaxTTM = -114.47
InvCapTTM = 414.90
my Estimated ROI%TTM = -27.59
P123 ROI%TTM = -9.8974

So something is being handled quite a bit differently in the ROI%TTM calculation underneath the surface on the beta test server. Curious if you guys have an explanation. Here is the link to the screen I am using to test the differences (https://beta.portfolio123.com/app/screen/summary/240746?st=1&mt=1), not sure if that link will work or not.

Apr 23, 2020 8:45:48 AM       
rodbarcv2
Re: FactSet beta site v1.0, NOW LIVE

Hi Rod,

Is it your intention to restrict access to these links or is that something the Forum is doing?

Thank you.

Hugh



Hi Hugh,

These ports are private, I posted the link case it helped p123 staff. Yuval posted the approach that I should be using about this, so I'll try to do that, first to find out if it's fundamental related or not, and then why the discrepancy on the data for the stock.


Rod

Apr 23, 2020 11:41:14 AM       
Posts: 373    Pages: 38    Prev 3 4 5 6 7 8 9 10 11 12 Next
 Last Post