Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Enable HVR Compare Utility Support for Type JSON/parquet Files Stored in AWS S3 Targets

Naresh Kumar Grandi User

Which connector?: AWS S3 as target

Additional details:

As part of the TMNA migration from Qlik Replicate to Fivetran HVR, we extensively use AWS S3 JSON file targets for downstream data integration and application consumption. HVR currently supports generating JSON files to S3 through FileFormat=Json; however, the Compare utility does not support JSON file locations as a compare source or target.
When attempting to execute Compare against a JSON/S3 location, HVR returns:
F_JR0F08:File capture using action FileFormat with parameter Json is currently not supported.  
This prevents users from validating data consistency between source databases and JSON file-based targets.   

Business Need

Many organizations use S3 JSON outputs as the final delivery layer for:

  • Data lakes
  • Analytics platforms
  • Downstream application feeds
  • Cloud-native integrations
  • Kafka/S3 landing zones

Currently, there is no native mechanism within HVR to validate that JSON files generated in S3 match the source data.

Requested Capabilities

  • Compare Source Database ↔ JSON Files in S3.
  • Compare Database Target ↔ JSON Files in S3.
  • Compare JSON Files ↔ JSON Files.
  • Support:
    • Row count validation
    • Primary key comparison
    • Non-key column comparison
    • Difference reports
    • Scheduled compare jobs
    • Native HVR compare reporting

Business Benefits

  • Faster migration testing and cutover validation.
  • Reduced operational effort.
  • Improved CDC data quality verification.
  • Better support for modern data lake architectures using S3 JSON targets.
  • Feature parity with enterprise validation requirements commonly used in Qlik Replicate environments.

Please sign in to leave a comment.

Comments

0 comments