Connector Improvement: Capture changes both from online redo logs & archive log files for Oracle version 19c
We recently ran into an issue with a nightly job that truncates tables, then re-inserts rows in our Oracle 19c source database. The truncate portion of the job was written to an archive log, but the insert portion was not. Those transactions were still in the online redo log. Following the creation of the archive log that contained the truncate, the tables were synced to the Snowflake destination. As a result, the value of the _fivetran_deleted fields in those tables in Snowflake were set to TRUE. Had the changes in the online redo logs been used, the value would have been accurately set to FALSE.
We recently were made aware that Fivetran does not use the online redo logs if the Oracle database is 19c. Prior to 19c, the online redo log changes were captured with syncs. This seems like the connector functionality took a step backward with 19c. I would like to request that the online redo logs are used when syncing.
From Fivetran documentation - https://fivetran.com/docs/databases/oracle#logminer
For Oracle versions below 19c, we capture the changes both from active log files and archived log files. For Oracle 19c, we only capture the changes from archived log files.
Thank you.
Ken
-
Official comment
Hi Ken,
We do read from the online and archive logs for our Oracle connector for version 19c. I believe we have supported this since about early 2021 if I recall correctly!
It seems that the docs you linked are inaccurate. I will ensure these docs get fixed. Perhaps in this scenario the connector did not begin its sync before the truncate job. I am not certain what may have caused this issue but I recommend a support case if it is still an issue, as we do read from both online and archived logs.
Please let me know if you have any other questions!
Please sign in to leave a comment.
Comments
1 comment