Support explicit DECIMAL/BigDecimal type mapping for numeric columns in file connectors
Request: Allow Custom Data Type Mapping for file connectors (including SFTP CSV) to map a numeric column to an exact fixed-point type, such as BigDecimal, landing in Snowflake as NUMBER(p,s). Ideally users could also set precision and scale per column, e.g. AMOUNT → DECIMAL(38,2).
Problem: CSV files carry no type metadata, so Fivetran infers types from the values. Any column containing a decimal point is inferred as DOUBLE and created as FLOAT in Snowflake. Monetary values shouldn't pass through binary floating point, and casting the FLOAT to NUMBER afterward isn't equivalent to ingesting the original value exactly.
Real-world data: Source exports often drop trailing zeros, so a single money column can contain -54, 8011.5, and 336.71. The mapping needs to accept values with fewer decimals than the configured scale. Values with more decimals than the scale should fail or be flagged, not be silently rounded.
Current workaround: Mapping the column to String and converting downstream. That adds transformation logic, and it depends on Fivetran landing the raw CSV text without parsing it to DOUBLE first.
Impact: This affects anyone ingesting financial, accounting, billing, or transaction data from file sources. Exact decimal mapping would remove the need for string-based workarounds.
Please sign in to leave a comment.
Comments
0 comments