This is a sample CDK application to illustrate a cross account setup of OpenSearch Serverless with private access through a VPC endpoint. It allows you to ingest data from one AWS account to an OpenSearch Serverless collection in a different AWS account.
The setup is split in two accounts:
- ingestion account: a Lambda Function that ingests data from certain sources (DynamoDB, S3, ...) to an OpenSearch Serverless collection in a different account. The Lambda Function is placed in a VPC and assumes an IAM role in the search account to write data to the OpenSearch Serverless collection by leveraging a VPC endpoint.
- search account: an OpenSearch Serverless collection that allows network access from a VPC endpoint and data access for an IAM role.
💡 You can use the same setup to read from an OpenSearch Serverless collection, i.e. query data. Instead of having an ingestion account, just replace it with a read account and let your Lambda Function perform queries against OpenSearch (but consider the supported API operations).
The setup unfortunately requires step-wise deployments because the resources have cyclic dependencies. Look at bin/main.ts which contains comments explaining which fields you need to set. Here's a general overview:
- Deploy the
IngestionStack
to create the Lambda Function and a VPC endpoint - Note the VPC endpoint id of the stack output because it's required for the
SearchStack
. - Deploy the
SearchStack
to create the OpenSearch Serverless collection -> it requires a VPC endpoint id. - Note the collection id and IAM role ARN of the stack outputs because they are required for the Lambda Function of the ingestion stack.
- Deploy the
IngestionStack
again with the updated parameters.
Now you can run the Lambda Function to test the connection and ingestion of data.
⚠️ This is a sample application. You might want to further restrict permissions of the VPC endpoints, Lambda Function or OpenSearch Serverless collection.
npm run build
compile typescript to jsnpm run watch
watch for changes and compilenpm run test
perform the jest unit testsnpx cdk deploy
deploy this stack to your default AWS account/regionnpx cdk diff
compare deployed stack with current statenpx cdk synth
emits the synthesized CloudFormation template
I'm a cloud consultant focusing on AWS and happy to help & learn. Just create an issue or send me a message.