Index | Recent Threads | Who's Online | Search

Posts: 373    Pages: 38    Prev 4 5 6 7 8 9 10 11 12 13 Next
Last Post
This topic has been viewed 18552 times and has 372 replies
rodbarcv2
Re: FactSet beta site v1.0, NOW LIVE

I tested several portfolios, and performance / drawdown / Sharpe Ratio are either same or better on FactSet. Only a few portfolios had a similar performance with slightly lower Share Ratio, which is still ok. But I have 2 sims (and respective Live Portfolios) with significant worse performance with FactSet. Could you please take a look to determine what is driving the worse performance in FactSet?

Both simulations exclude prelim. Both 10 year simulation.

https://beta.portfolio123.com/port_summary.jsp?portid=1526326
Compustat: 376% total return, -16% max drawdown (1.20 Sharpe Ratio);
FactSet: 173% total return, -38% max drawdown (0.76 Sharpe Ratio);


https://beta.portfolio123.com/port_summary.jsp?portid=1527527
Compustat: 312% total return, -18% max drawdown (1.31 Sharpe Ratio);
Factset: 246% total return, -21% max drawdown (1.12 Sharpe Ratio);


Thank you,


Rod

I'm afraid not. What we're trying to do is to isolate data points that are different and try to figure out why they're different. It would take us many hours to look into simulations with lots of different rules and ranking systems to try to figure out what's causing differences when so many data points are different to begin with.

What you should do is analyze specific instances. Try to figure out why your results are so different by looking at the particular stocks your systems are picking. Dig deep to find out what the differences are. If they're NOT related to fundamentals, then you may have found a bug, and you should let us know immediately. If they ARE related to fundamentals, is there a way you can make the two different numbers make sense? Or are the two different numbers so far apart that one of them HAS to be wrong? Can you compare the numbers with the actual financial statements to see what FactSet and Compustat are doing differently? Or do you think we're using the wrong datapoint? These are the kinds of investigations that can really help us.

Thanks, and I'm sorry I can't do what you ask.



Thank you, I will try to do the suggested approach to determine if it's related to fundamentals or not first, and then try to pinpoint specific factors or data in the stock driving these divergences.


Rod

Apr 23, 2020 11:42:16 AM       
rodbarcv2
Re: FactSet beta site v1.0, NOW LIVE

Hi Yuval,

How is PEG calculated, or how should we be calculating PEG?

One of the factors on one of the sims that are driving the most discrepancies is PEG. Take for example, TAP (Molson Coors Beverage Co). In Compustat, PEG is 0.43. In FactSet, PEG is 4.81.


Rod

Apr 23, 2020 12:44:41 PM       
Edit 1 times, last edit by rodbarcv2 at Apr 23, 2020 12:52:14 PM
Chipper6
Re: FactSet beta site v1.0, NOW LIVE

Daniel it's working for me now with a minor discrepancy in IncAftTaxTTM between Compustat and Factset. See my screen (ID: 240780). Your screen is set to private so I couldn't look at it.

Apr 23, 2020 12:50:43 PM       
yuvaltaylor
Re: FactSet beta site v1.0, NOW LIVE

Differences in ROI% Calculation example:

For Ticker("DHT") if I look at the calculated ROI% on the Production Server and on the Beta Server using Prelim I see fairly big differences in the calculated values.


Production Server
ROI%5YAvg = 3.88
ROI%TTM = 6.07
ROI%PTM = -0.66

Beta Server
ROI%5YAvg = 5.10
ROI%TTM = 6.61
ROI%PTM = -0.32

Is this due to the way preliminary data is being handled? Seems odd that it would have a larger effect on the 5YAvg then on the TTM and PTM calculations.

Thanks,

Daniel


I have looked into this a little deeper to try and understand the differences. I don't know the exact details of the ROI calculation, but assume it is similar to the following

ROI%TTM = IncAftTaxTTM / InvCapTTM

where

InvCapTTM = Avg(EqTotQ+DbtLTQ,EqTotPYQ+DbtLTPYQ)

If I compare this estimated ROI calculation to the actual ROI calculation on the Production Server I get fairly similar results. For example, for ticker "DHT" on the date 10/15/2013 I get the following

IncAftTaxTTM = -114.47
InvCapTTM = 414.90
my Estimated ROI%TTM = -27.59
P123 ROI%TTM = -26.50

However, on the beta test server I get the following for "DHT"

IncAftTaxTTM = -114.47
InvCapTTM = 414.90
my Estimated ROI%TTM = -27.59
P123 ROI%TTM = -9.8974

So something is being handled quite a bit differently in the ROI%TTM calculation underneath the surface on the beta test server. Curious if you guys have an explanation. Here is the link to the screen I am using to test the differences (https://beta.portfolio123.com/app/screen/summary/240746?st=1&mt=1), not sure if that link will work or not.

Yesterday there was a difference in how KEEPNA was handled by the beta site and the production site. For certain factors it was incorrectly going to 0 on the beta site, which would lead to different numbers for ROI. This has now been fixed. The actual formula for ROI is 100 * (NetIncBXor(0, TTM, KEEPNA) + IntExp(0, TTM, KEEPNA) * .65) / ((EqTot(0, QTR, KEEPNA) + DbtTot(0, QTR, KEEPNA) + EqTot(4, QTR) + DbtTot(4, QTR)) / 2) Let me know if you're still seeing differences. I checked all the stocks in the S&P 500 and didn't see any. If you do see differences, then try to isolate which of these variables is causing them, please. Thank you for being so diligent about this.

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Apr 23, 2020 1:18:20 PM       
Quantonomics
Re: FactSet beta site v1.0, NOW LIVE


COUNTRY is not that important a function in and of itself...

Thank you.
Hugh


Talk for yourself. It's important if you are managing an international equity strategy with ADRs and you want for example to exclude countries with too much inflation, corruption, etc.

Apr 23, 2020 1:24:40 PM       
dnevin123
Re: FactSet beta site v1.0, NOW LIVE

Yesterday there was a difference in how KEEPNA was handled by the beta site and the production site. For certain factors it was incorrectly going to 0 on the beta site, which would lead to different numbers for ROI. This has now been fixed. The actual formula for ROI is 100 * (NetIncBXor(0, TTM, KEEPNA) + IntExp(0, TTM, KEEPNA) * .65) / ((EqTot(0, QTR, KEEPNA) + DbtTot(0, QTR, KEEPNA) + EqTot(4, QTR) + DbtTot(4, QTR)) / 2) Let me know if you're still seeing differences. I checked all the stocks in the S&P 500 and didn't see any. If you do see differences, then try to isolate which of these variables is causing them, please. Thank you for being so diligent about this.


Thanks Yuval, I think I have traced it down. The difference appears to be in the data (which isn't surprising). What is a little interesting to me is that there is a big difference between IncAftTax and NetIncBXor, and by looking at the Line-Item reference you would expect them to be similar for most securities.

Indeed, on the production server if you pull up the Fundamental Chart and compare NetIncBXor(0,TTM,KEEPNA) with IncAftTax for "DHT" you will see that they are basically identical over the last 10 years.

That is not the case for the beta test server. If you look closely at the same Fundamental Chart comparison over the last 10 years for "DHT" you will see some pretty significant differences between NetIncBXor and IncAftTax that are not present on the production server. Indeed, for the data in question (10/15/2013) the NetIncBXor = -45.39 while the IncAftTaxTTM = -114.47. (On the production server NetIncBXor = IncAftTaxTTM = -114.47.)

Do you guys have any thoughts as to the reason for the differences? Are NetIncBXor and IncAftTax being mapped in a different fashion on the beta server than on the production server, or does FactSet include something in NetIncBXor that Compustat does not?

Thanks,

Daniel

Apr 23, 2020 2:09:29 PM       
yuvaltaylor
Re: FactSet beta site v1.0, NOW LIVE

Hi Yuval,

How is PEG calculated, or how should we be calculating PEG?

One of the factors on one of the sims that are driving the most discrepancies is PEG. Take for example, TAP (Molson Coors Beverage Co). In Compustat, PEG is 0.43. In FactSet, PEG is 4.81.


Rod


The formula for PEG is very complicated, Rod. But it essentially depends on four numbers: price, EPSExclXORTTM, NextFYEPSMean, and LTGrthMean. You'll notice that only EPSExclXORTTM is going to be different between FactSet and Compustat at the moment because we haven't yet loaded FactSet's estimates on our servers. So that's the number you want to look at when you notice differences in PEG. I'll pretty much guarantee you that every time you see a significant difference in PEG between FactSet and Compustat you're seeing a significant difference in EPSExclXorTTM.

EPSExclXorTTM is calculated very differently by Compustat and FactSet in many cases. I don't know all the reasons, but there are going to be significant differences in almost every quarter in NetIncBXorQ between FactSet and Compustat.

Yuval Taylor
Product Manager, Portfolio123
invest(igations)
Any opinions or recommendations in this message are not opinions or recommendations of Portfolio123 Securities LLC.

Apr 23, 2020 2:57:00 PM       
Edit 1 times, last edit by yuvaltaylor at Apr 23, 2020 2:57:45 PM
dnevin123
Re: FactSet beta site v1.0, NOW LIVE

Found another security (BTI) with some interesting data differences. In particular, the Income Statement numbers on the Beta server are roughly 2x the Income Statment Numbers on the Production Server, while the Balance Sheet numbers appear to be the same.

BTI Most Recent Data - Production Server
NetIncBXorTTM = 7554.38
IncAftTaxTTM = 7746.42
SalesTTM = 34271.4
EqTotQ = 84631.81
DbtLTQ = 50067.62

BTI Most Recent Data - Beta Server
NetIncBXorTTM = 17444.27
IncAftTaxTTM = 14530.13
SalesTTM = 65644.62
EqTotQ = 84654.17
DbtLTQ = 50080.85

If you look at the datapoints over time on the Fundamental Chart it is clear that the differences on the Income Statement aren't simply a scalar factor. The Balance Sheet curves are basically identical.

Apr 23, 2020 3:21:44 PM       
Edit 3 times, last edit by dnevin123 at Apr 23, 2020 3:23:33 PM
Chipper6
Re: FactSet beta site v1.0, NOW LIVE

I think that the FactSet TTM numbers for BTI are counting the prior 4 "quarters", but each "quarter" is really six months. I am doing more research on this.

EDIT: Some other ADRs are also showing Sales TTM numbers double the Sales Annual for FactSet. But many are not.

Apr 23, 2020 4:37:11 PM       
Edit 2 times, last edit by Chipper6 at Apr 23, 2020 4:42:50 PM
rodbarcv2
Re: FactSet beta site v1.0, NOW LIVE

Hi Yuval,

How is PEG calculated, or how should we be calculating PEG?

One of the factors on one of the sims that are driving the most discrepancies is PEG. Take for example, TAP (Molson Coors Beverage Co). In Compustat, PEG is 0.43. In FactSet, PEG is 4.81.


Rod


The formula for PEG is very complicated, Rod. But it essentially depends on four numbers: price, EPSExclXORTTM, NextFYEPSMean, and LTGrthMean. You'll notice that only EPSExclXORTTM is going to be different between FactSet and Compustat at the moment because we haven't yet loaded FactSet's estimates on our servers. So that's the number you want to look at when you notice differences in PEG. I'll pretty much guarantee you that every time you see a significant difference in PEG between FactSet and Compustat you're seeing a significant difference in EPSExclXorTTM.

EPSExclXorTTM is calculated very differently by Compustat and FactSet in many cases. I don't know all the reasons, but there are going to be significant differences in almost every quarter in NetIncBXorQ between FactSet and Compustat.



Thanks Yuval. Is the plan to load FactSet's estimates during the beta tests? If so, any ballpark timeframe that we can expect that?


Rod

Apr 23, 2020 8:21:37 PM       
Posts: 373    Pages: 38    Prev 4 5 6 7 8 9 10 11 12 13 Next
 Last Post