Connector Improvement: Fixing the Broken Schema: Avoiding instance_# Field Names
Answered
The schema becomes messy due to dynamically named fields like instance_1, instance_2, etc. This creates a problem every time new fields are added or the field order changes, as it shifts instance numbers and breaks downstream mappings or logic.
What we need is a stable schema with actual field names mapped consistently, regardless of their position or how many fields are present.
What we need is a stable schema with actual field names mapped consistently, regardless of their position or how many fields are present.
Ideally:
• Each field should map directly to a named column e.g., gl_account rather than a numbered instance_x,
• The connector should reference fields by a unique identifier or label, not by position.
• Each field should map directly to a named column e.g., gl_account rather than a numbered instance_x,
• The connector should reference fields by a unique identifier or label, not by position.
Could we explore fixing this at the connector or schema level so that field mappings remain consistent even as new fields are added?
-
Official comment
Hi Sanja,
Thanks for sending this request. Can you share what the format of your report looks like? I see in Workday there are options to define your explicit column names in the "Column Heading Override XML Alias". Have you tried that?
Frank
-
Hi Sanja - Thanks for submitting this request. Can you please clarify which connector you are having this issue with?
-
We’re using the Fivetran Workday Financial Management connector. It pulls Workday custom report which contains Journal Lines, Ledgers etc from Workday into BigQuery, which supports our reporting and analytics layer.
Please sign in to leave a comment.
Comments
3 comments