Support ServiceNow sysparm_fields Parameter to Retrieve Only Selected Columns
AnsweredWhich connector?
ServiceNow Connector (QA & Production)
Business Justification / Additional Details
We have been working with Fivetran Support (Ticket #387592) to investigate long-running syncs and API timeout issues affecting our ServiceNow connector. During the investigation, Fivetran support team confirmed that when columns are deselected in the connector schema, the connector still retrieves the complete record from the ServiceNow REST API and performs column filtering internally before loading data into the destination. The connector does not currently utilize the ServiceNow sysparm_fields parameter to request only the selected fields.
Our ServiceNow investigation identified that a significant portion of the API execution time is spent performing ACL evaluations across all fields in large tables such as cmdb_ci, dscy_net_base, dscy_router_interface, dscy_switchport, and dscy_route_next_hop. Because the connector requests all fields, ServiceNow must evaluate ACLs for every column, resulting in API execution times exceeding 60 seconds and causing request timeouts, retries, and extended synchronization durations.
Support implemented a temporary workaround by configuring internal startLimits settings to begin synchronization with smaller page sizes for affected tables. While this reduces retry overhead, it does not address the root cause because the connector still retrieves all fields from ServiceNow.
Requested Enhancement
Add support for passing the ServiceNow sysparm_fields parameter based on the fields selected in the connector schema so that only required columns are retrieved from the ServiceNow API.
Expected Benefits
- Reduce API payload size.
- Reduce ServiceNow ACL evaluation overhead.
- Decrease API execution times and timeout occurrences.
- Improve synchronization performance for large ServiceNow tables.
- Reduce connector retries and overall sync duration.
- Lower load on both ServiceNow and Fivetran infrastructure.
Fivetran Support explicitly confirmed that this functionality is not currently available and recommended submitting a Feature Request for Product team evaluation.
-
Official comment
Hi Devanshu,
This is a valuable idea and has been added to our feature improvements backlog for the ServiceNow connector for further exploration.
We'll keep this thread updated with any progress or changes regarding this request.
Thanks,
Frank
Please sign in to leave a comment.
Comments
1 comment