Join our Community in its new Home - The Datorama Trailblazer Community Group!

It's been an amazing 3 years coming together in this forum to collaborate, innovate, support, and inspire each other about our shared usage of Datorama. While this is not quite a goodbye, we are excited to announce that we are getting a fresh start in our new home within the Salesforce Trailblazer Community. We have a ton of fun new content planned and you may even see the revival of some of our most popular posts from the past few years.

We’ll be keeping this group around for a bit for you to peruse, but as of November 15, we will no longer be allowing new posts or comments. Be sure to join our new group at https://www.salesforce.com/success-datorama to keep the conversation going.

We can’t wait to see you there!

Datorama Platform API (Part 1): Getting Started

In this series of posts, we are going to be going on a tour of the Datorama Platform API and how it can be valuable to you. We will be delving into four different use cases to introduce the different elements that will help you get started and make the most of this Datorama feature:
  1. Data Processing & Reliability
  2. Managing Multiple Accounts
  3. Triggering Processes Externally to Datorama
  4. Dynamic Datorama Documentation
The series is based off our recent webinar looking at the Developer offering - you can catch up with it on demand here!

Let’s start by going over some of the basics....

An API, Application Programming Interface, allows applications to communicate with one another - basically, two computers talking to each other. If you want to learn more - Zapier have produced a really comprehensive introduction to APIs and is a great place to get started here 

Datorama Platform API lets you save time and get more value by managing your Datorama implementation with automatic actions and bulk updates. You can automated almost everything you do via the Datorama UI!

Datorama platform allows you to automate a lot of your previously manual tasks like pulling data and compiling Excel-based reports. The Platform API is the next level of automation, automation squared say, by automating Datorama specific tasks and workflows, removing even more of your manual tasks and providing efficiency through automation.


Now lets jump into our first example...

Use Case 1: Data Processing and Reliability

One of the most useful and common use cases of the Platform API is to ensure your data processing is up to date and reliably running. It enables the ability to set up, reprocess and manage your API data streams in an automated manner; as well as making changes to the data stream settings in bulk.

By creating automated process to manage you data processing, you can:
  • Improve data quality & reduce data discrepancies
  • Reduce manual intervention & improve efficiency
  • Update all DSs of a specific data source type easily
Now, let’s look at this example in action. We want to select all failed data streams of a specific data stream and reprocess them. First, think about how would we do that in the platform UI?
  1. Navigate to the data stream list under Connect & Mix
  2. Filter the data streams by data stream source, run status and status
  3. Select all the data streams
  4. Click ‘Process’, enter the data range and go!


Second, think about what questions we are ask to Datorama to replicate this with the Platform API and how we would format them using the Platform API. In human readable form, the questions we want to ask are:
  1. Datorama, can you give me a list of all data streams that failed?
  2. Datorama, please to reprocess each data stream on this list?
Like with human communication has etiquette to guide their interactions, computers have something similar called protocol. This is the rules that govern the communication between two computers and are clearly defined. The primary protocol used on the web, and therefore for the Datorama Platform API, is HTTP (hyper-text transfer protocol) which is centered around requests and responses. The request is what you (the client) send to Datorama (the sever), i.e. your question, and the response is what you receive back.

There are a few key ingredients to a HTTP request:
  • Endpoint - the address where the request is sent (required)
  • Method - the type of request (required)
  • Header parameters - specific instructions to enable communication (e.g. authentication & authorization) (required)
  • Other parameters - specifications for the response type
  • Payload/Data - the data to be created, updated, deleted, etc.


In our examples, we are going to use Python and specifically the requests package to create our requests. This is an easy to use and well documented package, but you can generate your HTTP requests however you find easiest.

Now, let’s make convert these questions to the language Datorama will understand. For the first question, what are our ingredients for the HTTP request:
  • Endpoint: https://app.datorama.com/v1/workspaces/{workspace_id}/data-streams This can be found on the Developer Portal - dependent on the platform entity you want to access, what information you require and your instance of Datorama
  • Method: GET This is used to retrieve data, but should have no other effect on the data
  • Header: {'token': token, 'Content-Type': 'application/json; charset=UTF-8'} You should use your Datorama API token and other parameters specified in the Developer Portal. Find how to access your Datorama API token here

Putting this altogether, we can create the attached Python script, reprocessScript.py, for reprocessing failed data streams. This script consists of the following elements:
  • Import statements: import the required python packages
  • Global Variables: define the global variables to be used throughout the script
  • filter_data_streams Function: Produces a list of data stream IDs by a specific data stream type, run status and status. This performs the HTTP request we discussed earlier in the article. The required parameters are: workspace id, header parameters and data stream type for filtering
  • Utility Functions today_string & back_string: Return todays date in required string format and return previous data in required string format respectively.
  • reprocess Function: Reprocess data streams, given a list of data stream IDs, start and end date. This performs a secondary HTTP request.
Once you have mastered these basic concepts, you can use the same method to create HTTP requests for all your automation needs in Datorama. Now you have this script, you’re probably think ... Okay, how do I run this script? In the next post, we are going to be covering various execution methods to get you started. Subscribe to the Platform API tag here on the community to get notified of the next part of this series.

Whenever you are working with the Platform API, everything you need to know can be found in the Developer Portal which is accessible for all Datorama users. This is a great place to continue with the Platform API in the meantime. 

Comments

Sign In or Register to comment.