Fivetran Veeva Vault Connector: Long Text / Rich Text fields truncated at 250 characters — request to implement LONGTEXT() function in VQL SELECT queries
Connector name: Veeva Vault
Table name: quality_event__qdm (and all Veeva objects containing Long Text / Rich Text fields)
Column to add:LONGTEXT(description__c), LONGTEXT(risk_analysis__c), LONGTEXT(comments_long_text__c) .. and few others
(Note: these columns already exist but are truncated at 250 chars — requesting full-length values via LONGTEXT() function)
API documentation link: https://general.veevavault.dev/vql/functions-options/longtext/
Additional details:
ISSUE: Long Text and Rich Text field values synced by the Fivetran Veeva Vault connector are truncated at 250 characters. Veeva VQL returns only 250 characters by default unless the LONGTEXT() function is explicitly used in the SELECT clause. The connector is currently not using LONGTEXT(), causing permanent data loss for any field value exceeding 250 characters.
FIVETRAN SUPPORT CONFIRMATION: This has been investigated by Fivetran Support (agent: Anthony, ticket ref: # 409149). Support has confirmed the connector does not use LONGTEXT() and has gathered evidence. They have asked us to file this feature request so they can escalate to the product/engineering team with their findings attached.
EVIDENCE: We verified the same record (id: OP400000014R038) via a parallel IICS pipeline using LONGTEXT() — it returns the full text. The Fivetran-synced table returns the same fields truncated at 250 characters.
AFFECTED FIELDS CONFIRMED:
- description__c
- risk_analysis__c
- comments_long_text__c
LIKELY BROADER IMPACT: All Long Text and Rich Text fields across all Veeva Vault objects synced by this connector are potentially affected.
BUSINESS IMPACT:
- Silent data loss in Snowflake tables
- Analytics pipelines produce degraded results on truncated input
- Cannot decommission parallel extraction pipeline until Fivetran provides complete data
REQUEST:
1. Implement LONGTEXT() wrapping for all Long Text and Rich Text field types in the connector's VQL SELECT queries
2. Provide a re-sync / backfill mechanism to recover historically truncated data after the fix is deployed.
-
Official comment
Hi Shreenandan, we’ve rolled out support for retrieving the full contents of Long Text and Rich Text fields in the Veeva Vault connector, so these fields are no longer truncated at 250 characters. Thanks for raising this request and sharing the detailed examples!
-
Below are the sample tables and columns - which are getting truncated by this issue.
TABLE_NAME COLUMN_NAME MAX_DATA_LENGTH SCHEMA_MAX_LENGTH ROWS_AT_250_CAP AUDIT_QDM ADDITIONAL_DOCUMENTS_REVIEWED_C 250 256 1 AUDIT_QDM AUDIT_STANDARDS_C 250 512 348 AUDIT_QDM EXECUTIVE_SUMMARY_C 250 512 1125 AUDIT_QDM NARRATIVE_C 250 512 631 BATCH_DISPOSITION_V JUSTIFICATION_V 250 256 115 CAPA_ACTION_QDM COMMENTS_C 250 512 349 CAPA_ACTION_QDM DESCRIPTION_C 250 512 2109 CAPA_ACTION_QDM TITLE_C 250 256 10 CHANGE_ACTION_QDM CHANGE_CONTROL_TITLE_C 250 256 129 CONNECTION_SYS DESCRIPTION_SYS 252 256 1 DATASET_ITEM_SYS EXCLUDED_COLUMNS_SYS 250 256 18 DATASET_ITEM_SYS FILTER_SYS 250 256 1 EFFECTIVENESS_CHECK_QDM COMMENTS_C 250 256 61 EFFECTIVENESS_CHECK_QDM DESCRIPTION_C 250 256 332 EFFECTIVENESS_CHECK_QDM RESULT_1_C 250 512 444 EFFECTIVENESS_CHECK_QDM TITLE_C 249 256 1 EXCEPTION_ITEM_SYS ITEM_DATA_SYS 250 256 131 EXCEPTION_MESSAGE_SYS MESSAGE_BODY_JSON_SYS 250 256 35 INVESTIGATION_QDM COMMENTS_C 250 256 136 INVESTIGATION_QDM DESCRIPTION_C 250 512 864 INVESTIGATION_QDM RESULTS_1_C 250 512 2079 INVESTIGATION_QDM TITLE_C 250 256 8 QMS_ORGANIZATION_QDM CANCELLATION_REASON_C 249 256 1 QMS_ORGANIZATION_QDM PRIMARY_CONTACT_INFORMATION_C 250 256 4 QMS_ORGANIZATION_QDM SERVICE_ADDRESS_C 248 256 1 QMS_ORG_QUALIFICATION_V QUALIFICATION_ANALYSIS_C 250 512 95 QMS_SCAR_V COMMENTS_LONG_TEXT_C 250 256 24 QMS_SCAR_V DESCRIPTION_V 250 512 156 QUALITY_EVENT_QDM COMMENTS_LONG_TEXT_C 250 512 4067 QUALITY_EVENT_QDM COMPLAINT_CORRESPONDENCE_C 250 512 1129 QUALITY_EVENT_QDM DESCRIPTION_C 250 512 22453 QUALITY_EVENT_QDM DISTRIBUTION_RESTRICTIONS_C 248 256 1 QUALITY_EVENT_QDM RESPONSE_SUMMARY_1_C 250 512 3798 QUALITY_EVENT_QDM RISK_ANALYSIS_C 250 1024 11947 QUALITY_EVENT_QDM SUPPLIER_RESPONSE_AND_CAPA_SUMMARY_C 250 512 146 QUALITY_EVENT_QDM TITLE_C 250 512 104 QUALITY_INCIDENT_V DESCRIPTION_C 250 512 518 RECORD_CHECK_MATCH_RECORD_V JUSTIFICATION_V 250 512 97 RECORD_CHECK_RESULT_V APPLIED_FILTERS_V 250 256 1548 RECORD_CHECK_RESULT_V DEFAULT_FILTERS_V 250 256 829 RECORD_CHECK_RESULT_V MATCHING_SEARCH_TERMS_V 250 256 21 RELATED_CAPA_C CAPA_TITLE_C 250 512 21 RELATED_CAPA_C QUALITY_EVENT_TITLE_C 250 512 20 RELATED_QUALITY_EVENT_QDM INBOUND_QE_TITLE_C 250 512 41 RELATED_QUALITY_EVENT_QDM RELATED_QUALITY_EVENT_TITLE_C 250 512 38
Please sign in to leave a comment.
Comments
2 comments