Kafka Topic

This page describes how to set up a Kafka Topic as a target for a data sync

1. Destination

1. Select "Kafka Topic" as your data sync Destination (Image 1).

Image 1: Kafka Topic Destination

2. Basic Parameters

1. Add in the following information about your Kafka Topic Target (Image 2):

Parameter	Description
Boostrap Servers	Bootstrap Servers are a list of host/port pairs to use for establishing the initial connection to the Kafka cluster. This parameter should a CSV list of "broker host" or "host:port" Ex: localhost:9092,another.host:9092
Topic Name	This is the name of the Kafka Topic that messages will be produced to.
Use SSL	Check this if you want to connect to Kafka over SSL
SASL Mechanism	Select the SASL (Simple Authentication and Security Layer) Mechanism to use for authentication: - None - PLAIN - SCRAM-SHA-256 - SCRAM-SHA-512 - OATHBEARER (default) - OATHBEARER (oidc)

Image 2: Basic Parameters

3. Column Mappings

1. Click "Add Column Mapping" (Image 3).

Image 3: Column Mappings

2. For each of your source columns, map it to a target column in Kafka (Image 4). The names you specify in your "Target Column" value will turn into attributes in a JSON payload that will be constructed and pushed to Kafka. The name of this target column can be whatever you choose, but we recommend maintaining your naming convention across columns for simplicity.

Image 4: Column Mappings

4. Sync Behaviour

When using a Kafka Topic as a sync target, you must select Delta as your synchronization pattern (Image 5).

Image 5: Sync Behaviour

5. Sync Type

A Kafka Topic sync target can be set up either as a batch or a real time sync.

If batch: it will push a message per record.

If real time: It will push a message per record with the triggering event being a single record. For real time event syncs, you must set up your configuration in the Listener Config table.