Join our Community in its new Home - The Datorama Trailblazer Community Group!

It's been an amazing 3 years coming together in this forum to collaborate, innovate, support, and inspire each other about our shared usage of Datorama. While this is not quite a goodbye, we are excited to announce that we are getting a fresh start in our new home within the Salesforce Trailblazer Community. We have a ton of fun new content planned and you may even see the revival of some of our most popular posts from the past few years.

We’ll be keeping this group around for a bit for you to peruse, but as of November 15, we will no longer be allowing new posts or comments. Be sure to join our new group at https://www.salesforce.com/success-datorama to keep the conversation going.

We can’t wait to see you there!

Python Connector - MOAT Custom Reports GET API Request

VishalPatelVishalPatel LondonSYS_ADMIN IMG42
edited August 2020 in Developer Zone
Hey all,

I wanted to share some python code that could help others setup either MOAT custom reports or python based API GET retrievals which have a json bytes response.

Use case: MOAT is an ad verification service and their API allows you to export more data than what's currently available through the standard MOAT Datorama connectors.

The code shared below is an example of how data can be retrieved and then transformed into a format that Datorama can ingest. The code would be pasted in a TotalConnect Data Stream which uses a Python retrieval function.
Further info: https://support.datorama.com/en/support/solutions/articles/4000121991-python-data-stream-retrievals




Key points I want to share in regards to the setup :-

1) For the python terminal in Datorama, I found that the the code for importing packages should be set at the top of the script otherwise errors can occur unexpectedly.

2) The packages used below are friendly with Datorama's python library.

3) The query below is specific to MOAT documentation however conceptually this could be based on your desired connection's relevant query.

4) The code below contains script which converts the bytes api response into a string and it uses then regex to get the elements needed to convert the response into a Dataframe.

5) Once data is in a Dataframe, it should be converted to csv which is the format required for Datorama to ingest.

6) It's always better to build and test the code in a separate python terminal (e.g. Jupyter) before utilising it in Datorama


#http://api.moat.com/docs/

# play here with the token to see

import datorama

import urllib3

import datetime

import pandas

import json

import re

token = ‘--------TOKEN GOES HERE ---------'

auth_header = 'Bearer {}'.format(token)

# yesterday = (datetime.date.today() - datetime.timedelta(days=1)).strftime('%Y-%m-%d')

metrics = ('date','level1','level2','level4', 'impressions_analyzed')

query = {

    'metrics': ','.join(metrics),

    'start': '2020-07-20',

    'end': '2020-07-20',

    'brandId': 11376  #put your brand id here

}

http = urllib3.PoolManager()

resp = http.request('GET', 'https://api.moat.com/1/stats.json',

                    fields=query,

                    headers={'Authorization': auth_header})

#print(resp.status)

#print(resp.data)

#from io import StringIO

# resp.data is a bytes file, we make it a string first

s=str(resp.data,'utf-8')

# we then make it a nice string using regex to grab the data (which is inside the 'details' list)

ns = re.search('details":(.+])',s).group(1)

# we use the json library to make the json string into a json object (aka list of dictionaries)

data = json.loads(ns)

df=pandas.DataFrame(data)

y=df.to_csv()

#Saving the csv to Datorama

datorama.save_csv(y)

Comments

Sign In or Register to comment.