Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Dashboard Improvement: Managing HVR 6 slicing

Iain Tait User

One if the advantages of slicing in HVR5 was the ability to run a few slices, then once complete, pause the slicing, run the integrate to catch up the current changes, and then start another batch of slices.

This would mean we were able to keep current changes more up to date.

With HVR6 we no longer have any control over when the slices are run, to allow us this control.  Can we get some functionality to allow us to do this. as the only way currently is to create multiple jobs with a different selection of slices within each job. 

Please sign in to leave a comment.

Comments

1 comment

Date Votes
  • Mark Van de Wiel User

    Hi Iain,

    Slicing in HVR is designed to speed up the initial load of a single large table. The aspects of slicing that allow us to do this include:

    • Parallelism - deliver into the target in parallel. Please note that only some target technologies support parallel append loads; others require an exclusive table lock hence parallelism is not possible.
    • Incremental recovery - should the load fail then we can recover the sliced load by resuming just the slices that were not yet completed.

    Note that a bulk load by default starts with a truncate of the target table.

    Also note that HVR allows you to run capture independent of refresh. Integrate cannot currently run at the same time (which we are addressing).

    The implementation is HVR 5 was arguably a hack, resulting in a job per slice. This is not how we ever wanted to release the capability, but we did so due to the existing architecture. In HVR 5 we had to play a trick with Restrict /RefreshCondition on the target to avoid every slice truncating the table. This slows down the load. Also, the use of controls that align initial and incremental changes cannot be guaranteed in HVR 5's architecture, and you need to configure Resilient processing manually (depending on data).

    Is your request regarding current changes relative to the table you are refreshing, or to the other tables in the channel?

    Of course you highlight a benefit you see from the technique used in HVR 5. Please note that you can perform a similar approach if you plan for this with HVR 6. Limit the refresh to a subset of the slices (e.g. with a separate name). Please do note that such approach introduces extra overhead since you must introduce configuration settings to avoid truncation for subsequent sliced jobs.

    Thanks,
    Mark.