Overview
Cinchy provides two methods of synchronizing data: batch sync and real-time sync. You can run a batch sync as a one time data load operation, or you can schedule it to run periodically. The real-time sync pushes individual events in real-time to the target system through the Cinchy listener and worker process.
High Level Architecture Diagram
At a high level, running a syncdata operation performs these steps:
- 1.Connects to Cinchy and creates a log entry in the Execution Log table with a status of running.
- 2.Streams the source and target into the CLI. Any malformed records or duplicate sync keys are written to source and target errors csvs (based on the temp directory)
- 3.Compares the sync keys to match up source and target records
- 4.Checks if there are changes between the matched records
- 5.For the records where there are changes, groups them into insert, update, and delete batches.
- 6.Sends the batches to the target, records failures in sync errors csv and Execution Errors table.
- 7.Once complete, updates Execution Log entry with final status and execution output.
At a high level, this is the process a real-time message goes through. We assume there is a listener and worker already set up.
- 1.Listener is successfully subscribed and waiting for events from streaming source
- 2.Listener receives a message from a streaming source and pushes it to SQL Server Broker.
- 3.Worker picks up message from SQL Server Broker
- 4.Worker fetches the matching record from the target based on the sync key
- 5.If there are changes detected, the worker pushes them to the target system. Logs successes and failures in the worker's log file.
Last modified 1yr ago