Learn how consumption-based pricing works with HVR, how HVR stores and sends usage data to Fivetran, and how you can integrate HVR with Fivetran.
HVR customers using version 6.1.0/3 and above can use Fivetran’s consumption-based pricing model. We calculate your usage based on the number of monthly active rows (MAR) across all connectors (sources) and destinations (targets) in your account. Once you register your hub system with a Fivetran account, the HVR application securely sends back metadata to Fivetran regarding your usage.
Fivetran calculates MAR for your HVR hub systems based on this metadata. You can register multiple hub systems to your Fivetran account. If you are syncing data from one source to multiple destinations, we count MAR for each destination.
We calculate MAR the same way across all cloud and on-premise connectors. You can share a single budget for both Fivetran and HVR.
Understand HVR data storagelink
HVR uses a special data structure called HyperLogLog (HLL) to track MAR. These data structures are stored in disk, in the hub system, in hour granularity buckets, per table. HVR uses a hash value from the primary keys to update the data structure.
Each data structure is around 2 KB. HVR stores an hour of data per table in a single 2 KB data structure. So for each active table, HVR stores 24 data structures every day. Each active table needs 48 KB of disk space per day. HVR recommends a week of storage allocation on the hub system to store the data that HVR uses to track MAR. HVR stores these data structures for a maximum of 1 month.
The original primary keys can’t be retrieved from the HLL data structure. In addition, HVR doesn’t store the primary keys directly but uses their hashes to update this structure. The hashes are irreversible and can’t be retrieved.
Understand HVR usage datalink
The HVR application establishes a connection with Fivetran servers and transfers the MAR usage data over the network hourly. HVR sends two data files to Fivetran:
- MAR data
- HVR hub snapshot
MAR datalink
The MAR data that HVR sends to Fivetran comprises the following:
- Hub name, hub ID, description, and hub server URL
- Channel name
- Location names, description, and type
- Table names and base names
- Refresh events, including the timestamps and metadata
- HLL records for Integrate and Refresh jobs
The MAR and usage data that HVR sends to Fivetran is stored on AWS S3, encrypted at rest. The data does not contain any personally identifiable information of the customer. HVR uploads this file hourly, and Fivetran sends back a 7-day license key daily as part of this data exchange. HVR purges the MAR data after 24 hours.
To view the MAR data sent from HVR to Fivetran:
- Execute the following command on the HVR CLI:
hvrlicense -m <filename>. - Use a text editor to open the file and view the data.
HVR automatically uploads hourly files. To view the hourly data:
- Execute the following command on the HVR CLI:
hvrlicense -m<filename> -B now-1h. - Use a text editor to open the file and view the data.
Snapshotlink
On receiving a request from the Fivetran server, HVR uploads a usage snapshot per hub every 30 days. The snapshot contains the following details:
- a week of events
- a week of statistics
- channel definitions
- job state
NOTE: The snapshot doesn’t contain log files, history, or transaction files.
To save the snapshot data into a zip file, execute the following command on the HVR CLI:
hvrsnapshotsave -r "metering-usage-snapshot account_id={account_id}" -e now-1w -l none -s now-1w -t none -V redact {hub} {filename}.zip
Integrate HVR with Fivetranlink
Follow these instructions to integrate your HVR hub server with Fivetran so that you can use consumption-based pricing.
Prerequisiteslink
To integrate each of your HVR hub system with Fivetran, you must:
-
Register your HVR hub system. You must create a Fivetran account or log in to Fivetran.com.
NOTE: You must register each hub system individually.
-
Enable outbound access from the HVR hub system running the hub server to the following IP addresses (on port 443):
- 34.110.137.135
- 34.120.211.189
Configure Consumption-Based Pricing for HVRlink
The configuration process is different for existing and new HVR customers.
Existing HVR customers
To start using consumption-based pricing on your HVR hub system, do the following:
-
Log in to the HVR web UI.
-
On the left navigation pane, click System.
-
Click the ellipsis button and select Licensing.
-
Click Register with Fivetran account.
-
On the Fivetran account setup form, enter your details and click Sign up.
NOTE: If you already have a Fivetran account, click Sign in here.
-
Once you log in to your Fivetran account, click Continue to HVR.
The registration process is complete, and you are redirected to the HVR UI.
NOTE: Once you complete the registration, the HVR deployment’s Licensing section displays the new Fivetran account with a VALID status. To view the HVR registration logs, click Registration ID.
New HVR customers
If you are a new HVR customer or a Fivetran customer on the Business Critical Plan, do the following:
-
Download HVR from either of the following:
- HVR account
- Fivetran account management page > Settings tab. Navigate to the HVR Settings tab and click Download HVR 6.1.
-
To get started with HVR and learn more about installation and configuration, see HVR’s Getting Started documentation.
Manage Fivetran licenselink
Licenses are provided through the registration process, or a license key is provided directly through email. Manage your license from the Licensing page of your HVR deployment. You can delete the license or remove the Fivetran registration.
The Fivetran registration process provides you access to the HVR software in the form of a refreshing license key. If you’re actively using HVR on our consumption model and if you remove your Fivetran registration before the contract ends, you won’t have access to the HVR software after 7 days. Your access to the HVR software will end at the end of your subscription term.
For more information about licenses, see HVR Licensing.