Data Integration

A new year brings new capabilities! We’re excited to kick off 2026 with one of our largest CData Sync releases yet, this Q1 roll out is packed with enhancements designed to give you more flexibility, control, and power across your data pipelines. Here’s what’s new in Sync Q1 2026: ❄️ Apache Iceberg Destination Support CData Sync now supports the Iceberg file format and catalog, enabling modern lakehouse architectures with open, reliable table formats. Iceberg destinations are now available for:  Amazon S3  Azure Blob Storage  Azure Data Lake Storage  Google Cloud Storage  🚦 Visual Pipelines in the UI We’re excited to preview Pipelines, an upcoming point-and-click UI experience for managing job and event dependencies in Sync. Pipelines will make it easier than ever to orchestrate and visualize Sync workflows.🔄 CDC Catalog Enhancements: SAP HAN

Hi,I’m trying to use Sync to replicate data from Salesforce into a Postgres database.  I have a full load working, and every night it truncates then reloads the tables.I’m having a lot of problems setting up another job that then runs an incremental update periodically.  It seems straightforward to set up.  My jobs incremental settings are:Start Date:  2026-01-06 Replication Interval : 1 dayWhen I run the job, I consistently get the same error, regardless of which Salesforce object I try to incrementally update.  In this case my source is Building__c, and the destination is the Postgres table salesforce.building. The error I get is:  [0] Updating Row is failed. ERROR: Server error [SQL state: 42P01]: relation "salesforce_building__r_1329269084" does not exist (Character 13) , AffectedCount: 0I can’t find any mention of this type problem in the documentation, and have followed all the steps in the docs to set this up.

How can I get this Sql Server dbQuery to work? The INSERT is completing fine, but I need to get the inserted identity PK and Arc is not picking it up.<arc:set attr="db.query"> insert into table (...) values (...); select SCOPE_IDENTITY();</arc:set><arc:call op="dbQuery" in="db" out="results"> ...Is there a better way to get the identity? (I have to use ArcScript, the upsert connector is not an option)

When configuring a replication Job after creating a Connection to your SQL Server database, you may initially find that you are unable to utilize this Connection as a Destination.The Connection appears as intended when selecting a Source for the Job:However, the same Connection does not appear when selecting a Destination:What can be done to resolve this?

The CData Sync clustering feature allows multiple Sync installations to operate together as a single, scalable, and highly available environment. By distributing workloads across multiple nodes, clustering ensures continuous operation, automatic failover, and consistent performance even as data movement needs grow.In this article, you’ll learn how to: Set up Sync nodes to share a unified application directory Configure a shared application database for consistent metadata and job management Enable and activate Cluster Mode across all Sync installations Ensure reliable, distributed job execution with automatic node balancing To learn more, read: Configuring Clustering for CData Sync

Working with PostgreSQL data that includes “Infinity” values? Learn how to manage and sync these special values effortlessly using CData Sync. Our newest guide walks you through the necessary steps and best practices to keep your data clean and reliable.Full guide: https://www.cdata.com/kb/articles/sync-postgresql-infinity-values.rst

In the input there is a value “<N201>Germany GmbH & Co. KG</N201>”. After it is mapped it is “<Address>Germany GmbH & Co. KG Hahnstraße 70</Address>”. How can I set it up that special characters are still escaped after the mapping. Because now the next Json connector is failing because of the & character.

OverviewSometimes APIs return data to the connecting client in a page based format. This can happen when the requested dataset to be returned is too large for the API to return in one response, or it could be the way that the API administrator has built the API to return responses.For example, if an API request is sent to GET the total number of items in an order, and the total number of items is 500, the API might return the results in a series of 10 pages where there are 50 items per page. The client requesting the information would then have to issue 10 separate API calls to get each page. This behavior is often coined “pagination”.The REST Connector and the FlowPagination isn’t something that is natively supported within the REST connector however, it is typically possible for Arc users to develop a solution to accomplish this by use of some custom script within a script connector and the utilization of custom headers to hold the page numbers (or

Greetings, I sent a detailed example of this to support and I’ll put it here in case it’s something simple. I’m running into a lot of issue with dbamp and ordering by the account name.  (both directly and also in the joins).For example the 2 queries should produce the same results both return the same values.  They do not because the order returned is different even when the values are the same.    select top(5) account.id from [salesforce].cdata.salesforce.account order by account.id COLLATE        SQL_Latin1_General_CP1_CI_AS asc;    select top(5) cast(account.id as nchar(18)) from [salesforce].cdata.salesforce.account order by cast(account.id as nchar(18)) COLLATE SQL_Latin1_General_CP1_CI_AS asc; If I change the query to do a case sensitive ordering like so, then the results match.  select top(5) account.id from [salesforce].cdata.salesforce.account order by account.id COLLATE      SQL_La

How do I delete unused template files?

Let me explain the overall flow i am trying to achieve. The flow is exposed as rest service to client. Client will call the rest service to send a file to a specific location. Client will trigger the flow, there is a requirement where i need to call a rest service inside the flow . Client is passing the required data in headers and the same header values i need to set while calling the rest service. I tried adding rest connector but body is disabled. I am not able to send custom request body to a service through rest connector. can you please guide me how to do it. 

version: 4.1.6.0Description: All of my sql jobs pulling variations of sf_refresh are failing as of 10/11/2025. Currently not bring any data down in our production environment ERROR:     10/11/2025 5:02:02 AMLog        Job History (SF_Refresh Salesforce - ChargentBase__Gateway__c)Step ID        1Server        AZSQL02Job Name        SF_Refresh Salesforce - ChargentBase__Gateway__cStep Name        1Duration        00:00:00Sql Severity    16Sql Message ID    0Operator Emailed    Operator Net sent    Operator Paged    Retries Attempted    1MessageExecuted as user: NT SERVICE\SQLSERVERAGENT. --- Ending SF_Refresh. Operation FAILED. [SQLSTATE 42000] (Error 50000)  --- Starting SF_Refresh for ChargentBase__Gateway__c V4.1.4 [SQLSTATE 01000] (Error 0)  05:02:02: Using Schema Error Action of yes [SQLSTATE 01000] (Error 0)  OLE DB provider "DBAmp.DBAmp" for linked server "Salesforc

i get error when trying to pull down application data that changed in last 4 days. I’m querying the linked server directly like this . it seems like i’m hitting some threshold and when it’s over a certain amount it gives me this error. this didnt happen in my old dbamp version. ideas onf ix?                select *                --  select count(*)FROM [MATCHFORCE].[CData].[Salesforce].[Application__c_QueryAll]WHERE [LastModifiedDate] <   dateadd(second,-1,(dateadd(day, datediff(day, 2, getdate()),0)))                AND [LastModifiedDate] > datead

Hey CData Community! 👋Looking to strengthen data protection and compliance in your integration environment?Our latest guide, “How to Secure CData Arc (.NET / Windows Edition)”, walks through key security configurations to help safeguard your data and automate workflows confidently.Check out the full article here: How to Secure CData Arc (.NET / Windows Edition)

Dear Support ArcESB,We are using Arc 2023 Professional on Windows Server 2022 Standard. Our customer exchanges data with us via AS2.The customer has recently migrated their AS2 system to a new platform which uses HTTP Authentication requiring a username and password. The username length is 78 characters, and the password length is 81 characters, as shown in the attached image.Could you please verify if your program supports usernames and passwords of this length? We would appreciate your confirmation and any recommendations you may have.Thank you for your assistance.Best regards,Parinya  P.

Hey CData Community!Microsoft’s SQL Server 2025 brings AI into the heart of the database with new native vector support, enabling you to store, search, and analyze embeddings directly inside SQL Server. When paired with CData Sync, you can easily connect and replicate data from 250+ sources to power AI workflows like semantic search, RAG pipelines, and anomaly detection — all within SQL Server.Check out the full guide here: https://www.cdata.com/kb/articles/sync-sqlserver25-ai-workflows.rst

Getting the below error in Prod OLE DB provider "DBAmp.DBAmp" for linked server "SALESFORCE_LS" returned message "Error 5103 : Error: Zipped response from salesforce was missing valid zip header.". [SQLSTATE 01000]

 We are on DBAmp version  4.1.8.We have a SQL Server Agent job running daily on our test org. After Salesforce Winter ‘26 was applied to the org, we started getting the following failure message. How can we resolve this? OLE DB provider "DBAmp.DBAmp" for linked server "SALESFORCE" returned message "Error 5103 : Error: Zipped response from salesforce was missing valid zip header.".Msg 7303, Level 16, State 1, Line 1Cannot initialize the data source object of OLE DB provider "DBAmp.DBAmp" for linked server "SALESFORCE". 

We are upgrading from DBAMP v.4.1.8 to v.22.  Because we are restricted from using xp_cmdshell, most of our SQL Server Agent jobs run sf_replicate.  If I’m understanding the documentation correctly, we won’t be able to call sf_replicate directly on v.22.  The CMDEXEC looks like this on 4.1.8:  '"C:\Program Files\DBAmp\DBAmpNet2.exe" replicate pkchunk,batchsize(25000) Account MyServerName  MyDatabase SALESFORCE'Will the call have to look something like this on v.22? "C:\Program Files\CData\CData DBAmp\bin\DBAmpAZ.exe" MirrorCopy Account MyServerName MyDataBase SALESFORCE pkchunk,batchsize(25000)  

Is it possible to create my own Operations that I can call via <arc:call>? Or simple reusable methods/functions using <arc:include> perhaps that can return data? Or?

I have a trading partner who’s sending EDI with a file extension (.xml) and a Content-Type: text/plain and they’re also sending EPCIS xml files which are also of Content-Type: text/plain.  I need to determine which files are legit xml/edi and route them accordingly.I use a script on my EDI files to add additional headers, but I don’t think this will work on the xml EPCIS files because they are not EDI.<!-- code I use to add additional headers to edi files --><arc:set attr="edi.File" value="[FilePath]" /><arc:call op="x12Scan" in="edi" out="edi"> <arc:enum item="edi"> <arc:set attr="output.header:[_attr]" value="[_value]" /> </arc:enum></arc:call><arc:set attr="output.filepath" value="[filepath]" /><arc:push item="output" />Since the EPCIS files are true xml, I feel like this will bomb on the op=”x12Scan”.  A

Fall is here and so is CData Sync’s Q3 2025 release! Our latest updates are designed to help you unlock more value from your data, faster and more efficiently. ⚡ Enhanced CDC Catalog Expansion: You asked, we delivered Sync now supports DB2 protocol AS400/iSeries and MySQL for even broader data replication capabilities! 🔄 Reverse ETL Catalog Expansion: New destinations now supported, including Salesforce Pardot, Salesforce Marketing Cloud, Sage Intacct, and Veeva Vault to enhance your reverse ETL capabilities ✨ Open Delta Table Support: Now available for all your favorite data lake destinations such as: Amazon S3, Azure Blob Storage, Google Cloud Storage, and Azure Data Lake Storage powering your delta integrations across Databricks, Fabric, and more! ⚡ Faster File Replication: CSV, Avro, Parquet, and One Lake Mirroring are now 90% faster for smoother large-workload handling. 🛠 Generic JDBC Connector: Unlock source-only connectivity and schema disco

I am trying to use httpGet to retrieve binary files, then push them through via a Script connector: <arc:set item="httpGetInput" attr="url" value="[params.pdfUrl]" /><arc:call op="httpGet" in="httpGetInput" out="response"> <arc:set attr="output.data" value="[response.http:content]" /> <arc:set attr="output.filename" value="test.pdf" /> <arc:push item="output" /></arc:call>It downloads the file, and sends it through, however the file is garbled, I assuming this is a encoding problem somewhere.Downloading the files directly from the source URL, the files render fine.Said connector is inside an API Request, with Response content type set to “application/pdf”:There must be something I am missing.

I am new to arcESB and we have a basic requirement where we need to call a rest service, get the data in json format and same data i need to use to later while using smtp connector (filename need to be modified). I am unable to map the data, i tried using arcScript in event section of rest connector but it seems to be not working. Any suggestion will be really appreciated. thanks in advance.