Connector Improvement: Elasticsearch: Support AWS hosted Opensearch
PlannedWe currently use AWS hosted Elasticsearch, and the connector as-is will not work with our setup as it requires a username and password or API key for access. We currently use networking controls to allow access to these clusters, without username/password logins, and the connector does not allow it to be setup without those included. Further, our strategy for access going forward for these is actually to lean on IAM roles to increase security, not use HTTP basic authentication. As noted below from their documentation, this is an either-or approach, both cannot be used.
"If a resource-based access policy contains IAM users or roles, clients must send signed requests using AWS Signature Version 4. As such, access policies can conflict with fine-grained access control, especially if you use the internal user database and HTTP basic authentication. You can't sign a request with a user name and password and IAM credentials. In general, if you enable fine-grained access control, we recommend using a domain access policy that doesn't require signed requests."
Source: https://docs.aws.amazon.com/opensearch-service/latest/developerguide/fgac.html
We would love to see this connector updated to better support AWS hosted Elasticsearch.
-
Official comment
Hello Jason,
Thanks for your post here! Our first launch of Elasticsearch only included Elastic Cloud and self-hosted Elasticsearch, as you mentioned.
I have good news that we are committed to launching Opensearch support in Q2! So this new connector should be available before the end of July.
Could you share more details on your Opensearch version? Are you using the legacy Elasticsearch versions (aka OpenDistro) or the new Opensearch versions (https://opensearch.org/docs/latest/version-history/)?
-
I believe I have an email from someone on your team that says you are using OpenDistro 7.7, but it doesn't hurt to confirm.
-
Kevin Kim Thanks for the response! That's great to hear!
We are currently using legacy Elasticsearch 7.x versions (OpenDistro). I'm not sure on the timeline to move to OpenSearch for us currently, so if this could also support OpenDistro that would be fantastic. In either case we'd love to see an approach that allows for the use of IAM roles to facilitate access as an option, either via a bastion similar to other connectors or OIDC.
Thank you!
Please sign in to leave a comment.
Comments
3 comments