Kafka Topic Source Example
This page provides an example data sync using Kafka Topic as a source
Last updated
This page provides an example data sync using Kafka Topic as a source
Last updated
In this example, we are syncing from a Kafka Topic source to a Cinchy Table target.
We want to sync the following data from Kafka and map it to the appropriate column in the "Sync Target 2" table in the "Kafka Sync" domain.
Kafka Source | Cinchy Column |
---|---|
This is what the Connections UI will look like with the aforementioned example parameters and data.
Your source tab should be set to "Kafka Topic" and have the following information (Image 1):
Tip: Click on an image in this document to enlarge it.
Column 1 (Standard Column) Parameters | Example Data |
---|---|
Column 2 ( Standard Column) Parameters | Example Data |
---|---|
Your destination tab should be set to "Cinchy Table", and have the following information (Image 2):
Domain: The domain where your destination table resides. In our example we are using the "Kafka Sync" domain.
Table: The name of your destination table. In our example we are using the "Sync Target 2" table.
Degree of Parallelism: This is the number of parallel batch inserts and updates that can be run. Set this to 1 for our example.
Under the Sync Behaviour tab, we want to use the following parameters (Image 3):
Synchronization Pattern: Full File
Sync Key Column Reference Name: Employee Id
New Record Behaviour: Insert
Dropped Record Behaviour: Delete
Change Record Behavior: Update
The following code is what the XML for our example connection would look like:
Column 1 (Standard Column) Parameters | Example Data |
---|---|
Column 2 (Standard Column) Parameters | Example Data |
---|---|
$.employeeId
Employee Id
$.name
Name
Name
$.employeeid
Alias
Employee Id
Data Type
Number
Name
$.name
Alias
Name
Data Type
Text
Trim Whitespace
True
Source Column
Employee Id
Target Column
Employee Id
Source Column
Name
Target Column
Name