Other: Other: Refresh option for multiple tables at the same time
In the case of other cdc solutions, two main settings are possible when using a function similar to refresh.
1. Number of tables refreshed simultaneously
2. Number of parallels used when refreshing one table
In the case of HVR, only the second case appears to be possible.
It would be nice if feature 1 was also added.
-
HVR supports both options which can be used together, or independently.
1. Refresh parallelism: how many processes are working to refresh all selected tables. The default for this is 4 (option -P), and can be changed for every refresh. For example, if you refresh 100 tables at parallel 4 then at any time 4 tables will be refreshed. As soon as one is done the process will move on to the next table.
2. Slicing is the term HVR uses to break up a large table into smaller increments. Every increment can be served by a separate process. E.g. you may break down a table into 10 slices, then up to 10 parallel processes - depending on the previously-mentioned setting selected - may be running concurrently to load a single table.
For more information, see:
- https://fivetran.com/docs/hvr6/getting-started/concepts/refresh
- https://fivetran.com/docs/hvr6/user-interface/channels/refreshing-data
- https://fivetran.com/docs/hvr6/command-line-interface/command-reference/hvrrefresh
Hope this clarifies.
Thank you,
Mark.
Please sign in to leave a comment.
Comments
1 comment