Data minor & PIT Method

I get this PIT error in data miner.
Am I using it incorrectly?

Thanks
Tony


Hi Tony. You have the PIT Method line twice in the script. My script still runs with 2, but try deleting one of those lines.
Also, do you have a Compustat license? If not, you need to change the Vendor line to Vendor: Factset or just remove it since Factset is the default.

If you still get the error, please try moving the PIT Method line from the first line to another place in the script. I want to see if you get an error related to the new first line in the script - maybe the script is failing and the error message will mention whatever the first line is.

Also, verify that you have the latest DataMiner. You should see v1.1.3 in the top left corner.

If none of those things resolve your error, please let me know any I will recreate your script exactly to find the issue.

Thanks Dan. It works better if I follow the examples accurately. I am using v1.1.3

I do not have Compustat license but I was probing to see what would happen.

One more question. What is the difference between these two universe sections?
I noticed I cannot use my custom universe in the second, larger universe section.

Tony


Starting Universe works like the Starting Universe on the Universes page on the web site. You select the Starting Universe and then whatever rules you add are used to filter the Starting Universe to create a custom universe. There is a limited number of universes available as the Starting Universe and you cannot use your custom universes as you mentioned.

Here is a is a DataMiner example of how to create a custom universe in the script:

Universe:
        Rules:
            - mktcap < 700
            - AvgDailyTot(20) > 200000
            - CompleteStmt = 1
        Starting Universe: Prussell3000

The other option is to create a custom universe on the web site and use that directly in your script like Universe: myUniverse

Thanks Dan.
Is it possible to use the ‘Rank Performance’ operation in data miner but build the RS on the fly using the InLine RS building feature?

I see examples to do one or the other but not both in the same script.

Thanks
Tony

Yes it is possible Tony. I’ll make a cleaner version of the example below and add it to the other examples, but this should be good to answer your question for now.

Main:
    Operation: RankPerformance
    On Error:  Stop
Default Settings:
    Vendor: Factset
    PIT Method: Prelim
    Buckets: 10
    Start Date: 2010-11-01
    End Date: 2010-12-01
    Rebalance Frequency: 1Week
    Benchmark: SPY
    Minimum Price: 3
    Method: Long
    Slippage: 0

#Can use an existing  universe or create a custom universe inline
#    Universe: #SP500
    Universe:           
        Rules:
            - mktcap > 50000
            - AvgDailyTot(20) > 200000
            - CompleteStmt = 1
        Starting Universe: Prussell3000

#Could use an existing Ranking system:
#    Ranking: "Core: Growth"

#Can use a single factor ranking system defined inline. This is called 'quick rank'.
#    Ranking: 
#         Formula: "PEG"

#Or define a multifactor ranking system inline. This is a short example. The ranking system can have multiple composites or nested composites.
    Ranking: 
        Rank: Higher # ( [Higher] | Lower | Summation )
        Method: NAsNegative #( [NAsNegative] | NAsNeutral )
        Nodes:
        -
            Type: Composite
            Name: Value Composite
            Weight: 100 # ( [0] - 100 where 0 indicates equal weight)
            Rank: Higher # ( [Higher] | Lower | Summation )
            Nodes:
            -
                Type: StockFormula
                Name: EarnYield
                Weight: 50  # ( [0] - 100 where 0 indicates equal weight)
                Rank: Higher # ( [Higher] | Lower | Boolean )
                Formula: EarnYield
                Scope: Universe # ( [Universe] | Industry | Sector )
            -
                Type: StockFormula
                Name: Pr2SalesTTMvsInd
                Weight: 50
                Rank: Lower
                Formula: Pr2SalesTTM
                Scope: Industry

#Each iteration will use the settings specified in that iteration instead of the default settings above.
Iterations:
    -  
        Name: Baseline  #uses the settings above with no changes.
    -  
        Name: Single factor rank
        Ranking:
             Formula: EarnYield
    -  
        Name: 2 week rebalance  #Test multiple rebalance periods.
        Rebalance Frequency: 2Weeks
    -  
        Name: 4 week rebalance
        Rebalance Frequency: 4Weeks
    -  
        Name: Univ P3000  #use an existing universe instead of the custom universe defined in the top section.
        Universe: Prussell3000
    -  
        Name: Univ mktcap 100-500  #Use a different universe defined inline here.
        Universe:
            Rules:
                - Between(mktcap,100,500)
                - AvgDailyTot(20) > 200000
            Starting Universe: Prussell3000
    -  
        Name: Univ mktcap 500-1000
        Universe:
            Rules:
                - Between(mktcap,500,1000)
                - AvgDailyTot(20) > 200000
            Starting Universe: Prussell3000
    -  
        Name: MomentiumValue ranking  #Can create a ranking system on the fly here also.
        Ranking: 
            Rank: Higher # ( [Higher] | Lower | Summation )
            Method: NAsNegative #( [NAsNegative] | NAsNeutral )
            Nodes:
            -
                Type: Composite
                Name: Value Composite
                Weight: 100 # ( [0] - 100 where 0 indicates equal weight)
                Rank: Higher # ( [Higher] | Lower | Summation )
                Nodes:
                -
                    Type: StockFormula
                    Name: EarnYield
                    Weight: 50  # ( [0] - 100 where 0 indicates equal weight)
                    Rank: Higher # ( [Higher] | Lower | Boolean )
                    Formula: EarnYield
                    Scope: Universe # ( [Universe] | Industry | Sector )
                -
                    Type: StockFormula
                    Name: Pr26W%Chg
                    Weight: 50
                    Rank: Higher
                    Formula: Pr26W%Chg
                    Scope: Universe

    -  
        Name: XML ranking example  #Can copy/paste this from the Ranking System page on the site using the text editor.
        Ranking: 
            Nodes: #xml format copied from the text editor on the Ranking System, Factors page of the web site.
                <RankingSystem RankType="Higher">
                    <StockFormula Weight="0%" RankType="Higher" Name="test" Description="" Scope="Universe">
                        <Formula>eval(InsNetShrPurch>0,InsNetShrPurch/InstitutionalShsNet(0),NA)</Formula>
                    </StockFormula>
                    <StockFactor Weight="50%" RankType="Lower" Scope="Universe">
                        <Factor>Pr26W%Chg</Factor>
                    </StockFactor>
                    <StockFactor Weight="50%" RankType="Higher" Scope="Universe">
                        <Factor>TotRevisions4W</Factor>
                    </StockFactor>
                </RankingSystem>

Dan very nice.
Can data miner be called to execute a specific yaml file from the command line?

And. How large of a yaml file can this handle?
If I programmatically built a giant yaml file to tests thousands of iterations, could it handle that?

Tony

I tried one of my large RS as raw XML. It choked on something.


Hi Tony,
I am not aware of any capability to run DataMiner from the command line, but I will ask Dev.

If you send me the id for the large ranking system that is giving you the problem, I will figure out what the issue is. The id is the number found in the URL while you are in that ranking system.

Here ya go dan: 387697

Thanks
Tony

Dan, is there a way to pinpoint which XML tag is causing this in RankUpdate.py

I can email the file is that is helpful.

API request failed: Open quote is expected for attribute “Name” associated with an element type “StockFormula”. (on line 1)

EDIT: Dan, it appears that having an “&” in a factor name causes this error.

Thanks
Tony

Dan, I am not sure how many of these bugs you would like to see but…

In the example script: ScreenRun.py

This seems to fail:

from p123api import Client, ClientException
try:
client = p123api.Client(api_id=‘100’, api_key=‘xxxxxxxxxxxxx’) #Insert your API Id and Key

With this error:
except p123api.ClientException as e:
NameError: name ‘p123api’ is not defined

While this succeeds:

import p123api
try:
client = p123api.Client(api_id=‘100’, api_key=‘xxxxxxxxxxxxx’) #Insert your API Id and Key

Regarding the large ranking system and XML - The bug that was fixed today resolved the issues. Below are the steps I used to create a DataMiner script:
Go to the ranking system page and then to the text editor tab.
Copy all the text from the editor into Notepad++ or another editor.
Highlight the text you just pasted and press tab as many times as needed to indent the text so that it will be in the correct column when pasted into the DataMiner script.
Copy the text from Notepad++ to your DataMiner script. Note: this text goes under the “Nodes:” line in the Iterations section.

Some of the formula names in your ranking system have colons in them. That is a special character in Yaml, so we need to either remove those or do this:
Copy the XLM portion of the script into Notepad++ or some other editor.
Escape each double quote (") by replacing each double quote with "
Wrap the entire XML that comes after “Nodes:” in double quotes ie one " at the start and one at the very end.

Handling special characters is explained in the README.TXT file that is in the DataMiner dropbox:
YAML uses a few special characters to format the data: #, :, -, [, ], {, }, ", |, >, + and -. In most cases, the YAML parser is smart enough to handle them, but there could be
rare occasions where these special characters cause a formula to fail. If you should receive an error, you can fix the formula by wrapping the whole property value in double quotes thus explicitly telling the parser that it is dealing with a string. Examples:

  • If a formula contains the special character ‘#’, then you may need to wrap the formula in double quotes.
    For example, the formula #AnalystsCurQ would be “#AnalystsCurQ”.
  • Double quotes may need to be escaped if used inside property.
    For example, the rule Rating(“Core: Growth”) > 25 would be “Rating("Core: Growth") > 25”.

For the question regarding running DataMiner from the command line: That is not currently possible.

If you want to only import the Client and ClientException classes, then you need to remove the ‘p123api.’ from the 2 lines below. Like this:

client = Client(api_id=myid, api_key=mykey)
except ClientException as e:

“In the example script: ScreenRun.py” - Is there an API example file somewhere that is using ‘from p123api import Client, ClientException’? If so, could you please let me know where you found it so I can fix it?