API and DataMiner enhancements

We have released some enhancements to the API and DataMiner. The enhancements in this release will require users to modify some existing DataMiner and API scripts.

  1. The DataUniverse and Ranks operations in DataMiner now accept a date range and a frequency instead of a single As Of Date.
    The ‘As Of Date’ parameter was replaced with ‘Start Date’ to be consistent with the Data operation. Please replace ‘As Of Date’ in your scripts with ‘Start Date’. Also keep in mind that if a Start Date is specified, but no End Date is given, then the result is that the script will run for all dates between the Start Date and todays date.
    New parameters:
    Start Date (Required)
    End Date (Optional. Default is todays date.)
    Frequency (Optional. Default is 1week.)

  2. Users will need to change any existing DataMiner scripts that use the Data operation.
    The Iterations section was removed and the formulas now go under the Default Settings section:

Old syntax:

Iterations:
     - 
        Name: PE
        Formula: PEExclXorTTM
     -
        Name: Pr2Sales
        Formula: PR2SalesTTM

New syntax:

Formulas:
       - PE: PEExclXorTTM
       - Pr2Sales: PR2SalesTTM
  1. The includeNodeDetails parameter in the Ranks endpoint of the API was deprecated and replaced with a new parameter called nodeDetails.
    IncludeNodeDetails will still work for now, but users should change any API scripts that use the Ranks endpoint to use nodeDetails.
    nodeDetails accepts the following values:
    Returns the primary rank.
    composite Returns the primary rank and also the composite ranks.
    factor Returns the primary rank, composite ranks and factor level ranks.

Please use the new DataMiner build from this point forward.
Location: https://www.dropbox.com/home/p123/P123%20Data%20Miner/latest_release

The formulas used to calculate the API request costs have also changed for some of the operations. For the Data, DataUniverse and Ranks operations, every 25,000 data points will count as 1 API request. Multiplying the number of rows times the number of columns of the data returned gives you the number of data points.

Dan - the dropbox link doesn’t work. Dropbox claims the files don’t exist.
Steve

Hi Steve,
Thanks for letting me know.
Here is a new link: [url=https://www.dropbox.com/sh/okb02vvkn3epclg/AAD0Y0jAx6xVMQZikHlE9nFYa?dl=0]https://www.dropbox.com/sh/okb02vvkn3epclg/AAD0Y0jAx6xVMQZikHlE9nFYa?dl=0[/url]

Dan - I am now installing p123api via !pip and it isn’t recognizing NodeDetails. (See image). It also gives an error on IncludeNodeDetails.

it isn’t clear to me what has changed in the Rank API other than IncludeNodeDetails changing to NodeDetails. Are there some changes from the DataMiner that I need to be aware of when calling the API? i.e. the date specifications, frequency, etc?


OK - it turns out I was using NodeDetails instead of nodeDetails. I have gotten past that error message. I am still curious about the date specifications. The DataMiner was changed, but the API was not? Just to confirm… I cannot concatenate multiple dates into one API call?

I suspect my data miner crashed this afternoon after your implemented those changes and I lost 20,000 requests. Is there a way I can get my file or requests back plz? Also will the old syntax still work? I will convert to new syntax but it will take me some time. Thanks

factorbased - I am not doing anything with DataMiner. The last time DataMiner was changed (by P123) was back in December. P123 should investigate however as you probably have a legitimate case if it crashed on you.

Steve - You are correct. The change was made to DataMiner so that it now makes an API call for each date in the start/end date range. The API was not changed and still takes a single date.

I’m using API.
I want to call api multiple times but I got an error API request failed: Cannot deserialize asOfDt parameter value .
It looks like now it is impossible ? So this API has little value for me.
Other issue is that I can not shift dates automatically to get the next date that is X trading days ahead.

The solution that I’m looking for is to implement start_date and end_date to the data_universe API .

dates = ["2021-02-30", "2021-03-06"]

try:
    client = p123api.Client(api_id='x', api_key='x')

    df_initial = client.data_universe(
        {
            "universe": "test1",
            "asOfDt": "2021-02-23", 
            'formulas': [
                'Open(0)',
                'Close(-5)',
                'AsOfDate'
            ]
        }, True)  #True to output to dataframe
    
    for date in dates:
        df = client.data_universe(
            {
                "universe": "test1",
                "asOfDt": date, 
                'formulas': [
                    'Open(0)',
                    'Close(-5)',
                    'AsOfDate'
                ]
            }, True)  #True to output to dataframe

        df_initial.append(df, ignore_index=True)
        
except p123api.ClientException as e:
    print(e)

The deserialize error is because the 2021-02-30 date is invalid. 2021-02-28 would work. I also changed the append line. I dont have an answer for the trading days part of your question.

import p123api

try:
    client = p123api.Client(api_id='x', api_key='x')
    dates = ["2021-02-27","2021-03-06"]

    df_initial = client.data_universe(
        {
            "universe": "5StockUniv",
            "asOfDt": "2021-02-20",
            'formulas': [
                'Open(0)',
                'Close(-5)',
                'AsOfDate'
            ]
        }, True)  # True to output to dataframe

    for date in dates:
        df = client.data_universe(
            {
                "universe": "5StockUniv",
                "asOfDt": date,
                'formulas': [
                    'Open(0)',
                    'Close(-5)',
                    'AsOfDate'
                ]
            }, True)  # True to output to dataframe
        df_initial = df_initial.append(df, ignore_index=True)

    print(df_initial)

except p123api.ClientException as e:
    print(e)