Which connector?: SFTP

Additional details:

We’re evaluating the SFTP connector to ingest files from our Sterling File Gateway (SFG) instance and have encountered some limitations that are pushing us to create multiple connectors instead of managing everything under a single connection. Specifically:

File format configuration scope
Current behavior: File Format options are defined at the connector level.
Impact: We handle multiple file types (CSV, TSV, fixed-width, JSON/parquet, etc.) in SFG. Because format is connector-scoped, we must create a separate connector per file/type, which increases operational overhead.
Request: Support file-level or table-level format configurations within a single SFTP connector (e.g., allow per-path or pattern-based format settings).

Primary key configuration scope
Current behavior: The primary key used for file processing and load is also defined at the connector level.
Impact: Different files require different approaches, forcing separate connectors for each file/table.
Request: Allow primary key definitions at the file/table level, ideally via rules based on directory path or filename pattern.

Compression and PGP encryption
Current behavior: Limited flexibility for handling compressed and PGP-encrypted files within one connector.
Impact: Our SFG delivers files with varying compression (e.g., .gz) and PGP encryption settings. We currently need distinct connectors to accommodate these variations.
Request: Support per-file or per-path configuration for compression and PGP (including key management), so that one connector can process a mix of compressed and encrypted files with different encryption keys.

We would like to manage multiple file types through a single SFTP connector, driven by directory/filename patterns, each with its own format, primary key, and compression/encryption settings. This would significantly reduce connector sprawl and maintenance overhead.