Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

ServiceNow connector — translate row filters into sysparm_query instead of filtering after extraction

Answered
Santiago Chapa User

Which connector?:  ServiceNow

 

Additional details:

Request: For tables with a row filter configured, append the filter to the sysparm_query parameter of the Table API call so non-matching records are never extracted. Today the filter is applied after extraction, so the connector pays the full API and transfer cost of every row it discards.

Proposed behavior — the connector already sends an encoded query for the incremental cursor, so this is an append:

Current:   /api/now/table/em_alert?sysparm_query=sys_updated_on>=<cursor>
Proposed:  /api/now/table/em_alert?sysparm_query=sys_updated_on>=<cursor>^u_suppress_escl=Normal

Must apply to both historical and incremental syncs, and fall back to post-extraction filtering if a predicate can't be expressed.

Why this matters:

  • Filters give no performance benefit today — sync time scales with total table size, not the requested subset.
  • ServiceNow tables often have Table Cleanup rules that delete records after N days. When extraction is slower than the source's growth, records are purged before the connector reads them — permanent, silent data loss.
  • Historical syncs can fail to converge. If extraction throughput is below the table's row-generation rate, the sync never reaches the present and never switches to incremental, with no error raised.
  • Sync frequency and rollback window cannot compensate, because the constraint is extraction volume, not scheduling.

Measured on our connector: 371,366 rows extracted to retain 1,249 (0.34% yield), at a throughput below the table's daily growth — so the sync could never converge.

Please sign in to leave a comment.

Comments

2 comments

Date Votes
  • Official comment
    Frank Fivetranner

    Hi Santiago,

    This is a valuable idea, and the request to translate row filters into the sysparm_query parameter to limit the extraction in the ServiceNow connector has been added to our feature improvements backlog.

    We will keep this thread updated with any progress or changes related to this feature.

    Thanks,
    Frank

  • Azhar Mohammed User

    +1 — blocking issue for us.

    ServiceNow sys_audit historical sync times out because row filters are applied after extraction. We need source-side sysparm_query pushdown for filters like specific fieldname values and sys_created_on >= 2025-01-01 (~1.7M rows / ~2.5% of the table). Without that, large system tables aren’t practically syncable.