Connector Improvement: Ability to do initial load through flat files UNLOADed from DB2 z/OS
Not plannedOrganizations migrating large IBM DB2 z/OS tables face a challenge: some tables are too large or too constrained by mainframe resource limits to be ingested through Fivetran's standard DB2 z/OS connector (CDC or full table scan).
We would like Fivetran to support an alternative ingestion path using flat files produced by the DB2 UNLOAD utility. The flow would work like this:
- A DB2 UNLOAD job exports a table to a flat file (fixed-length or delimited) on the mainframe
- The file is transferred to a cloud storage bucket
- Fivetran picks up the file, parses the schema, and loads it into the destination
This is the standard mainframe practice for bulk data movement and would avoid long-running queries on production z/OS systems — critical for teams with strict MIPS/capacity constraints. It would also unlock initial load scenarios where the DB2 z/OS connector is used for ongoing CDC but cannot handle the first full load of large tables.
Expected behavior:
- Support for common UNLOAD output formats (fixed-length binary, delimited text, or both)
- Schema mapping via a provided DDL or column definition file
- Compatible with any existing Fivetran-supported destination
-
Official comment
Hi Michel,
Thanks for being a Fivetran customer! This is a very interesting use case. I'll reach out to discuss it in more detail with you.
Dan
Please sign in to leave a comment.
Comments
1 comment