NOTE: The Data Push API is not recommended for new implementations. We recommend that you use the Data Ingestion API to pull your data into the Celonis Platform.
\nThis page describes how to get started with the Data Push API in a simple way using Postman.
\nThe Data Push API provides an easy way to transfer data into Celonis. For pushing data, Celonis currently uses micro-batching. Instead of either inserting huge chunks all at once or inserting record by record, data meant to be pushed into the platform is structured into these so-called \"micro-batches\". As an exchange format, the commonly used Parquet or CSV file formata are used. The API provides an easy way of creating, executing, and monitoring micro-batch jobs/Data Push jobs. Once the data is available in the system, you can transform, analyze and work with it as you would with a Celonis-provided extractor.
\nThe API to submit a push job is rate limited on a team basis. These limits are fixed to guarantee platform stability for all of our customers:
\nFirst you need to determine the URL for your team and cluster.
\nhttps://<team>.<cluster>.celonis.cloud/data-ingestion/api\nTo find the team and the cluster, please check the URL you use to access the Celonis EMS and retrieve the team and\ncluster from it.
\nFor example, the base URL for dev team in US-2 cluster would be:
\nhttps://dev.us-2.celonis.cloud/data-ingestion/api
You'll also need the Data Pool ID, for any further operations related to Data Push API. The Data Pool ID is displayed in the URL when you open a Data Pool in your Celonis team.
\nSince the Data Push API is a REST API, you will need to have a way to interact with it. There are multiple ways to\ndo this, but to make things simple, we will use Postman, which is a very popular and free REST client.
\nIf you already have a REST client, you can ignore this section.
\nOtherwise, go to the Postman Download page and download it for your Operating System. Once\nit's downloaded, install it.
\nOnce installed, it should look like something like this:
\nTo start using it, create an account or click on \"Skip and go to the app\".
\nEach request to the API must be authenticated with a Celonis Application or API key. There are multiple ways of doing this:
\nWhile it is possible to use API keys and Application keys with Celonis,\nit's important to note that they are not the industry standard. Both keys have an unlimited lifetime and cannot be “rotated”. This means that users need to delete and recreate them to achieve the equivalent of key rotation, which can be a cumbersome process.
\nOne issue (particularly relevant for API Keys) is that they give any\napplication that uses them complete access to almost all the Celonis APIs.\nThere is no way to restrict their access to only certain resources.
\nOAuth 2.0 is a well-established industry standard that makes it easier to\nintegrate customer-managed applications with Celonis. OAuth uses scopes to manage access to resources, which means that the OAuth clients can only access APIs allowed by the scopes they are configured to use.
\nFor example, we can have the scope studio that gives access to Studio or the scope audit.log:read that gives read-only access to audit logs. For an OAuth client to be able to access a certain resource (API), the client must have the OAuth scope and the Celonis permission to do so. This follows the security principle of least privilege so that an OAuth client gets only the required privilege to perform a certain task and not more.
You can find out how to create an API key by following\nour User API Keys\nguide.
\nThe Celonis API uses Bearer Token Authentication for verifying consumer access. The credentials must be sent in an\nAuthorization header in the HTTP request. Credentials sent in the URL or body of the request will be ignored.
\nTo authenticate using Bearer Token Authentication:
\nMDg5MGVkNDktNjMwZC00ODdiLTkyNGItMjNmMzMxNjRmM2IwOkhNUVRMUis4SGh6NHhBY21Vck9GaWdkem5rYzBrb3p0N056WUM0bGlqczMM\n\nAuthorization: \n\nBearer\n\n\nMDg5MGVkNDktNjMwZC00ODdiLTkyNGItMjNmMzMxNjRmM2IwOkhNUVRMUis4SGh6NHhBY21Vck9GaWdkem5rYzBrb3p0N056WUM0bGlqczMM\n\nYou can find out how to create an Application by following\nour Application API Keys guide.
\nTo authenticate using AppKey Authentication:
\nYjNiZDI4MWUtYzAyNS00NTJjLTk1MjAtZjE4ZGEzMjM5MzIzOlVTRzVhNis2djMzaUtBbFNoNVlFMDBsSFA5OEJ2MDA2c0REK3N2OWIzVHdm\n\nAuthorization: \n\nAppKey\n\n\nYjNiZDI4MWUtYzAyNS00NTJjLTk1MjAtZjE4ZGEzMjM5MzIzOlVTRzVhNis2djMzaUtBbFNoNVlFMDBsSFA5OEJ2MDA2c0REK3N2OWIzVHdm\n\nThe newly created Application Key is required to have necessary permission to be used for any Data Push API related operations.\nThere are two methods to provide these permissions:
\nOnce you have completed the previous steps, you can start making requests to the API. This example will use Postman for that.
\nSample URL: https://dev.us-2.celonis.cloud/data-ingestion/api/v1/data-push/bea4fb85-5d27-4336-b4db-b83207e8a03a/jobs/
\nA POST request should be made to create a Data Push Job, which will create a new Data Push Job and provide a unique ID for this newly created job as a result.\nThis ID should be used for the remaining operations performed with regards to this particular Data Push Job.\nThe request body must provide values for the mandatory fields.
\nOnce the Data Push Job has been created, use its ID to upsert chunks/files. Multiple files can be uploaded in the context of a single Data Push job.\nWe support PARQUET and CSV file as chunks to be uploaded.
\n\nGET request (with URL : https://dev.us-2.celonis.cloud/data-ingestion/api/v1/data-push/bea4fb85-5d27-4336-b4db-b83207e8a03a/jobs/00e7706a-e2f7-49be-a7be-6098628774d9)\ncan be used to get the details of a particular Data Push job.
\n