Current release: 1.1.0
\nSpecifies the action to be taken if an error occurs while inserting the data. There are three available options: CONTINUE - the error is swallowed, THROW - the error is allowed to propagate, RETRY - The exception causes the Connect framework to retry the message. The number of retries is set by connect.ems.max.retries. All errors will be logged automatically, even if the code swallows them.
\n| Type: | string |
|---|---|
| Default: | RETRY |
| Valid Values: | [continue, throw, retry] (case insensitive) |
| Importance: | high |
The maximum number of times to re-attempt to write the records before the task is marked as failed.
\n| Type: | int |
|---|---|
| Default: | 10 |
| Valid Values: | [1,...] |
| Importance: | medium |
The time in milliseconds between retries.
\n| Type: | long |
|---|---|
| Default: | 15000 |
| Valid Values: | [1000,...] |
| Importance: | medium |
If set to 'true', connector will continue when invalid input errors occur. Invalid records will be sent to the DLQ, if configured. Please note that this does not cover errors that happen in Converters during de-serialization.
\n| Type: | boolean |
|---|---|
| Default: | false |
| Valid Values: | |
| Importance: | medium |
activate/deactivate remote EMS logging
\n| Type: | boolean |
|---|---|
| Default: | true |
| Valid Values: | |
| Importance: | low |
The URL of the EMS extractor events endpoint the appender will submit log entries to. Only needed when `connect.ems.uploader=CBP`.
\n| Type: | string |
|---|---|
| Default: | null |
| Valid Values: | (Optional) Value must be a valid URL |
| Importance: | low |
The frequency at which the appender will try to flush the buffered log entries
\n| Type: | long |
|---|---|
| Default: | 60000 (1 minute) |
| Valid Values: | [1,...,9223372036854775807] |
| Importance: | low |
The minimum amount of events that need to be in the buffer before this can be flushed.
\n| Type: | int |
|---|---|
| Default: | 50 |
| Valid Values: | [1,...,2147483647] |
| Importance: | low |
The maximum amount of buffer entries before the appender will start to drop log entries.
\n| Type: | int |
|---|---|
| Default: | 1000 |
| Valid Values: | [1,...,2147483647] |
| Importance: | low |
The maximum time (in milliseconds) the appender should wait in between two consecutive flushes of log entries.
\n| Type: | long |
|---|---|
| Default: | 60000 (1 minute) |
| Valid Values: | [1,...,9223372036854775807] |
| Importance: | low |
The approximate frequency at which the EMS heartbeat log message should be produced
\n| Type: | long |
|---|---|
| Default: | 180000 (3 minutes) |
| Valid Values: | [1,...,9223372036854775807] |
| Importance: | low |
The number of time the connector underlying consumer is called with a zero records before the telemetry gets disabled
\n| Type: | int |
|---|---|
| Default: | 10 |
| Valid Values: | [1,...,2147483647] |
| Importance: | low |
The access key to be used to authenticate to the telemetry endpoint. Only needed when `connect.ems.uploader=CBP`
\n| Type: | password |
|---|---|
| Default: | null |
| Valid Values: | |
| Importance: | low |
The AWS access key
\n| Type: | password |
|---|---|
| Default: | null |
| Valid Values: | |
| Importance: | high |
The AWS access secret
\n| Type: | password |
|---|---|
| Default: | null |
| Valid Values: | |
| Importance: | high |
The AWS S3 endpoint
\n| Type: | string |
|---|---|
| Default: | null |
| Valid Values: | (Optional) Value must be a valid URL |
| Importance: | high |
The AWS region. Ignored if you are using https://**.celonis.cloud/api/data-ingestion/continuous as endpoint
\n| Type: | string |
|---|---|
| Default: | eu-central-1 |
| Valid Values: | |
| Importance: | high |
The name of the S3 Bucket
\n| Type: | string |
|---|---|
| Default: | null |
| Valid Values: | |
| Importance: | high |
Prefix of the S3 object keys
\n| Type: | string |
|---|---|
| Default: | null |
| Valid Values: | |
| Importance: | high |
Target table. This can be a string containing source topic and partition, that will be replaced with actual values. Example: \"table_{topic}_{partition}\"
\n| Type: | string |
|---|---|
| Default: | {topic} |
| Valid Values: | |
| Importance: | high |
How to upload the files. Available values: cbp, s3, noop.
\n| Type: | string |
|---|---|
| Default: | s3 |
| Valid Values: | [cbp, s3, noop] (case insensitive) |
| Importance: | high |
The number of bytes of the row groups in the Parquet file. Default is 16777216.
\n| Type: | int |
|---|---|
| Default: | 16777216 (16 mebibytes) |
| Valid Values: | |
| Importance: | medium |
For debug purpose, set the setting to true for the connector to keep the local files after an upload. Default is false.
\n| Type: | boolean |
|---|---|
| Default: | false |
| Valid Values: | |
| Importance: | low |
Rather than writing to the host file system, buffer parquet data files in memory
\n| Type: | boolean |
|---|---|
| Default: | false |
| Valid Values: | |
| Importance: | medium |
The folder to store the temporary files as it accumulates data. If not specified then [/var/folders/_9/59m1tvp57pl816byf01cc71w0000gp/T//ems] is being used.
\n| Type: | string |
|---|---|
| Default: | /var/folders/_9/59m1tvp57pl816byf01cc71w0000gp/T//ems |
| Valid Values: | |
| Importance: | low |
The type of obfuscation to apply. Available methods: NONE,FIX,SHA1,SHA512,RANDOM
\n| Type: | string |
|---|---|
| Default: | NONE |
| Valid Values: | [sha512, fix, random, none] (case insensitive) |
| Importance: | low |
A comma-separated list of paths to apply obfuscation to. The format is: 'topicName:field1.field2...fieldN'. topicName may be omitted to match path of all topics. An additional array selector may be used to obfuscate fields nested in arrays, like in 'users[].password,users[].creditCard'
\n| Type: | list |
|---|---|
| Default: | \"\" |
| Valid Values: | |
| Importance: | low |
An optional salt string to use during obfuscation. Only applied for SHA1 and SHA512 methods.
\n| Type: | string |
|---|---|
| Default: | \"\" |
| Valid Values: | |
| Importance: | low |
Optional length of obfuscated output. Only used for FIX and RANDOM obfuscation methods
\n| Type: | int |
|---|---|
| Default: | 5 |
| Valid Values: | [0,...,2147483647] |
| Importance: | low |
Proxy host including scheme (required) and port (optional), eg https://my-proxy.com or http://my-proxy.com:8080
\n| Type: | string |
|---|---|
| Default: | null |
| Valid Values: | |
| Importance: | medium |
Proxy BASIC auth username
\n| Type: | string |
|---|---|
| Default: | null |
| Valid Values: | |
| Importance: | medium |
Proxy BASIC auth password
\n| Type: | password |
|---|---|
| Default: | null |
| Valid Values: | |
| Importance: | medium |
A comma separated list of non-proxied hosts
\n| Type: | list |
|---|---|
| Default: | \"\" |
| Valid Values: | |
| Importance: | medium |
The conversion strategy to use for persisting decimal values into EMS. Available values (plainString, decimal)
\n| Type: | string |
|---|---|
| Default: | plainString |
| Valid Values: | [plainstring, double] (case insensitive) |
| Importance: | low |
The accumulated file maximum size before it is uploaded to EMS. It cannot be less than 1MB.
\n| Type: | long |
|---|---|
| Default: | 25000000 |
| Valid Values: | [100000,...] |
| Importance: | high |
The maximum number of records in the accumulated file before it is uploaded to EMS.
\n| Type: | long |
|---|---|
| Default: | 10000 |
| Valid Values: | [1,...] |
| Importance: | high |
The time interval in milliseconds to upload the data to EMS if the other two commit policies are not yet applicable.
\n| Type: | long |
|---|---|
| Default: | 180000 (3 minutes) |
| Valid Values: | [1000,...] |
| Importance: | high |
Embed Kafka metadata such as partition, offset and timestamp as additional record fields.
\n| Type: | boolean |
|---|---|
| Default: | true |
| Valid Values: | |
| Importance: | medium |