How can I store my connectors data in a preferred location in BigQuery?
Example use case: I want to store my data in the location US_EAST_4 ( Northern Virginia). Our AWS infrastructure is in
US_EAST_4, so we need our data to be stored at the same location.
In the BigQuery setup form, you will notice 2 fields; Data Location and Data Processing Location.
- Set the Data Location as
US_EAST_4from the dropdown list.
- Set Data Processing Location as per the instructions in the article mentioned here.
Once you set the Data Location as
US_EAST_4, all of your connectors’ schemas will get created in this location. Please note that Dataset Location can't be changed once the warehouse is created.
If you have already created a warehouse with a different Dataset Location and want to use the same BigQuery project ID, please follow the below steps:
- Create another warehouse with the same project and select Data Location as
US_EAST_4from BigQuery warehouse setup form.
- Pause the connector for which you want to change the data set location and remove the particular connector data set from BigQuery.
In the new warehouse, create connectors having a different schema name to the previous warehouse. The same schema name can't exist in a BigQuery dataset.