Kafka Topic
This page describes how to set up a Kafka Topic as a target for a data sync
1. Select "Kafka Topic" as your data sync Destination (Image 1).
Image 1: Kafka Topic Destination
- 1.Add in the following information about your Kafka Topic Target (Image 2):
Parameter | Description |
|---|---|
Boostrap Servers | Bootstrap Servers are a list of host/port pairs to use for establishing the initial connection to the Kafka cluster.
This parameter should a CSV list of "broker host" or "host:port"
Ex: localhost:9092,another.host:9092 |
Topic Name | This is the name of the Kafka Topic that messages will be produced to. |
Use SSL | Check this if you want to connect to Kafka over SSL |
SASL Mechanism | Select the SASL (Simple Authentication and Security Layer) Mechanism to use for authentication:
- None
- PLAIN
- SCRAM-SHA-256
- SCRAM-SHA-512
- OATHBEARER (default)
- OATHBEARER (oidc) |
Image 2: Basic Parameters
- 1.Click "Add Column Mapping" (Image 3).
Image 3: Column Mappings
2. For each of your source columns, map it to a target column in Kafka (Image 4). The names you specify in your "Target Column" value will turn into attributes in a JSON payload that will be constructed and pushed to Kafka. The name of this target column can be whatever you choose, but we recommend maintaining your naming convention across columns for simplicity.
Image 4: Column Mappings
When using a Kafka Topic as a sync target, you must select Delta as your synchronization pattern (Image 5).
Image 5: Sync Behaviour
A Kafka Topic sync target can be set up either as a batch or a real time sync.
If batch: it will push a message per record.
If real time: It will push a message per record with the triggering event being a single record. For real time event syncs, you must set up your configuration in the Listener Config table.
Last modified 9mo ago