There is no doubt that working with the Adobe Experience Cloud Solution UI’s is comprehensive and fun. But there are times when you need to extract or process data externally or you just need to automate processes as doing repetitive stuff over and over again is undoubtfully no fun at all. Take Adobe Analytics. Workspace is by far one of the most powerful inbuild data visualization and reporting tools on the market. Just drop some dimension and metrics, hit the right mouse button and click on visualize, done. Even intermediate statistical calculation are included, take Frederiks posts about advanced time series analysis in Workspace as an example. Although you can do a lot of stuff within the tool itself, there are times where you want to leave the box and do some more advanced stuff or you just need to extract data somewhere else. Adobe wouldn’t be Adobe if there wasn’t any solution for this. And thanks to the experience cloud APIs we can basically do whatever you can imagine. While there are numerous Python implementations for various solutions, I never really found a straighforward nor comprehensive implementation of all experience cloud APIs. The landscape is rather fragmented and each use case will take you on a wild ride on google, to find the right implementation. This is why I decided to create something that will solve all my experience cloud tasks once and for all.
As of today this project is not complete. The latest version includes mostly an Analytics implementation. Contributers are more then welcome !
How to get?
You can get the source on Github:
Or simply get it via pip:
pip install experiencecloudapis
How to get started?
No API is an API without authentication. And the same applies for all experience cloud APIs. There’s no question that this is the most annoying part when starting to work with the APIs. But luckily Adobe uses the same method across all solutions and by now Adobe I/O has improved at lot. Gone are the days where you needed to dig down into forums in order to understand how the hell you get the authentication running. This process was nerve wracking, fullstop. I consider myself a good person, so I try to save you from suffering.
Adobe provides three types of authentication:
- API Key Integration (Authorization Only)
- OAuth Integration (OAuth 2.0 authentication flow)
- Service Account Integration (JWT authentication flow)
As we focus on automation processes (and only JWT is implemeted as of today), I will guide you through the Service Account Integration which is based on JSON Web Tokens (JWT). In order to get the integration running, you need to have access to Adobe I/O console, as this will be the place to create your projects. From high level the process is easy:
- Create a project in the I/O console
- Assign the JWT Authentication to your project
- Add the required APIs to your project
- Export the credentials
But let’s start from scratch. We start in the Adobe I/O projects tab. Click on Add to Project to start.
Clicking on the button will open a dropdown. A blank project won’t have an authentication method assigned initially. While adding your first API to your project, you will be asked to create an authentication automatically. But let’s do it step by step. Click on API in order to add the first API to our new project.
As the focus on this article is about experience cloud APIs, we obviously click on Experience Cloud and add the Adobe Analytics API (mainly because the current version of the implementation does not support much more). So hit Adobe Analytics and proceed.
The process asks you to assign an authentication method to your project. You can basically add both methods if you like. But as mentioned earlier, the whole point of this implementation is to automate processes. Therefore we create a service account (JWT) authentication. Let me say a few words to both methods. The big plus (but also risk) of the JWT method is, that you do not need to grant your application access manually. The Service Account method will basically allow you to authenticate reading a JSON file and encrypting the payload with a private key. There is no more need to grant access whatsoever. Therefore you will need to make sure to protect these files. If you create an application on a remote server, you absolutely need to make sure that no other user can read these files. Otherwise you risk to compromise your application. When using the JWT method, your application will be able to retrieve access and refresh tokens without any interaction of your side. Therefore be very careful and know what you do if you expose these files outside of your local machine. On the other hand the OAuth methods adds a layer inbetween. You, as an resource owner will need to grant your application access before data can be retrieved. As long as you haven’t granted access, your application won’t be able to retrieve an access token and data. Also, when the token expires, this process needs to be repeated. The downside of this approach is, that you will need to add a manual interaction step between your application and the Adobe API. If you wish to retrieve data on a regular basis, let’s say a CRON, then this method won’t serve you much. Nevertheless, let’s continue!
After you selected the JWT method, the process will ask you for credentials. You can either ask Adobe to create a private/public key pair for your, which is very convenient, or you can provide your own key pair. There are tons of tutorials how to create a key pair. Personally, I find ssh-keygen a very neat way to quickly create a pair. What ever way you chose, create your key pair and proceed. When you pick the Adobe path, credentials will automatically be downloaded after clicking on continue.
Here we are. In the overview you can see your public key. Also your browser should have downloaded the key pair in the meantime.
Lets have a glimpse in the archive. We see two files. The first file is called certificate_pub.crt and serves as your public key. To be more precisely: your certificate. A certificate is basically a signed public key, but let’s not go into this topic. If you want to know more about public key cryptography, as Wiki! You will need the private.key file later as an input for the implementation. Make sure to keep this file.
Each API you add to your project will have it’s own configuration steps. For example, Adobe Analytics API will ask you to select a product profile to your project in order to scope the access to your data. This is a nice security layer, as even if your credentials get compromised, the attacker won’t be able to go beyond the scope. It’s best to make sure to grant your application only the minimal scope in order to get the job done. Select the necessary profiles and go on.
After selecting the profiles, you will be taken back to the overview page. Notice that the sidebar have changed and you see an entry for both Credentials and APIS. If you want to add either a new authentication method or API, hit the Add to Project button and select what you desire. Click on Service Account (JWT) as we need one more file for the implementation.
This will take you to the JWT overview page. This will show you credential details and also allows you to generate a JWT token. You will find a Generate JWT tab. Click on it, select your private.key from a earlier step and hit the generate button. This will create a Token for you, which you need for authentication token exchange. The authentication token is what you will need eventually in order to retrieve data from Adobe APIs. If you want to see how the process works, try it out. Otherwise, the implementation will to the job for you. But in order to do so, you will need to hit the Download JSON button in the top navigation. This will download the JWT payload as a JSON file. Eventually you will need both, the private.key file and this JSON file for the python implementation. Make sure to save both (and keep them safe!).
Let’s get started!
Okay, time to recap. We installed the experiencecloudapis packe, created a Adobe I/O console project, added a Service Account JWT authentication method, added Adobe Analytics API into it and saved the required files. Next we will write some python code to retrieve data from the Analytics Reports API. When requesting report data, I find it easiest to use the Workspace debugger. Since a couple of releases it’s super easy to activate the debugger. In Workspace click on Help and select activate debugger. This will add a little cockroach above your table. Clicking on it, you will be directed to a API request overview page. Select the JSON format and copy the request payload. The implementation will be able to read the format and request the data. This is by far the most convenient way to set up your jobs. But don’t worry, a request payload builder will be added soon to the implementation. Before we start, we also need to know the company id for the Analytics Reports API. This is by far the most nerv wrecking part and initially almost led me to write a complaint letter to Adobe. Let’s see the “straightforward” (and also recommended) way to retrieve it.
You will need to access the Adobe Analytics Swagger page. Hit the login button and authenticate. Swagger uses an OAuth authentication behind the curtains, so this gives you a glimpse of what you can do for cool stuff with these methods.
After authentication, hit reports and try out a request. Hit Execute and see the response below. In the little curl window, you will see the full API request URL. In the URL path, just after /api, you will find your company id. Easy, isn’t it?
But now, let’s get our hand dirty!
One final note. The AnalyticsReport class will try to retrieve segment/metric/dimension translation while processing the response. Otherwise you would be left with abstract IDs.
This project is not final and will be continuously maintained. Contributers welcome !