-
Notifications
You must be signed in to change notification settings - Fork 1k
Interludic2000 feature cdk apigw sqs lambda sns #2798
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
*.js | ||
!jest.config.js | ||
*.d.ts | ||
node_modules | ||
|
||
# Python | ||
__pycache__/ | ||
*.py[cod] | ||
*$py.class | ||
*.so | ||
.Python | ||
env/ | ||
venv/ | ||
.venv/ | ||
pip-log.txt | ||
pip-delete-this-directory.txt | ||
|
||
# CDK asset staging directory | ||
.cdk.staging | ||
cdk.out | ||
|
||
# Parcel default cache directory | ||
.parcel-cache | ||
|
||
# npm | ||
npm-debug.log* | ||
.npm | ||
|
||
# Yarn | ||
yarn-error.log | ||
|
||
# dotenv environment variables file | ||
.env | ||
|
||
# IDE | ||
.vscode/ | ||
.idea/ | ||
|
||
# OS | ||
.DS_Store | ||
Thumbs.db |
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,141 @@ | ||||||
# Webhook SNS Pattern - CDK Version | ||||||
|
||||||
This is a simplified AWS CDK implementation of a webhook integration pattern that receives webhook events via API Gateway, queues them in SQS, and processes them with Lambda to send SMS notifications via SNS. | ||||||
|
||||||
## Architecture | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The architecture explanation is brief and lacks details on how the components interact. Adding an architecture diagram might be even better. |
||||||
|
||||||
``` | ||||||
API Gateway (POST) → SQS Queue → Lambda Function → SNS (SMS) | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
``` | ||||||
|
||||||
## Components | ||||||
|
||||||
- **API Gateway**: REST API endpoint to receive webhook events | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
- **SQS Queue**: Decouples the API from processing and provides reliability | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
- **Lambda Function**: Processes messages and sends SMS via SNS | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
- **SNS**: Sends SMS messages to phone numbers | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
## Prerequisites | ||||||
|
||||||
- AWS CLI configured with appropriate permissions | ||||||
- Python 3.9+ | ||||||
- AWS CDK v2 installed (`npm install -g aws-cdk`) | ||||||
|
||||||
## Deployment | ||||||
|
||||||
1. Install Python dependencies: | ||||||
```bash | ||||||
pip install -r requirements.txt | ||||||
``` | ||||||
|
||||||
2. Bootstrap CDK (if not done before): | ||||||
```bash | ||||||
cdk bootstrap | ||||||
``` | ||||||
|
||||||
3. Deploy the stack: | ||||||
```bash | ||||||
cdk deploy | ||||||
``` | ||||||
|
||||||
4. Note the API Gateway endpoint URL from the output. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
## Testing | ||||||
|
||||||
### SMS Sandbox Verification (Required for New AWS Accounts) | ||||||
|
||||||
If your AWS account is in SMS sandbox mode (default for new accounts), you'll need to verify your phone number before receiving SMS messages: | ||||||
|
||||||
1. **Check if your account is in sandbox mode:** | ||||||
```bash | ||||||
aws sns get-sms-sandbox-account-status --region your-region | ||||||
``` | ||||||
|
||||||
2. **If in sandbox mode, verify your phone number:** | ||||||
```bash | ||||||
# Add your phone number to sandbox | ||||||
aws sns create-sms-sandbox-phone-number --phone-number "+your-phone-number" --region your-region | ||||||
|
||||||
# Check your phone for verification code, then verify it | ||||||
aws sns verify-sms-sandbox-phone-number --phone-number "+your-phone-number" --one-time-password "YOUR_CODE" --region your-region | ||||||
``` | ||||||
|
||||||
3. **Alternative: Request production access** through the AWS Console (SNS → Text messaging → Sandbox) for unrestricted SMS sending. | ||||||
|
||||||
### API Testing | ||||||
|
||||||
The API is protected with an API key. After deployment, you'll need to retrieve the API key value before testing. | ||||||
|
||||||
#### Get the API Key Value | ||||||
|
||||||
1. **Note the API Key ID from the deployment output**, then get the actual key value: | ||||||
```bash | ||||||
aws apigateway get-api-key --api-key YOUR_API_KEY_ID --include-value | ||||||
``` | ||||||
|
||||||
2. **Copy the `value` field from the response** - this is your actual API key. | ||||||
|
||||||
#### Send Test Request | ||||||
|
||||||
Send a POST request to the API Gateway endpoint with the API key header: | ||||||
|
||||||
Example using curl (update URL with your own API domain, API key, and phoneNumber with your phone number including country code e.g. +1234567890): | ||||||
```bash | ||||||
curl -X POST https://your-api-id.execute-api.region.amazonaws.com/prod/ \ | ||||||
-H "Content-Type: application/json" \ | ||||||
-H "x-api-key: YOUR_ACTUAL_API_KEY_VALUE" \ | ||||||
-d '{"phoneNumber": "+your-phone-number", "message": "Hello from webhook!"}' | ||||||
``` | ||||||
|
||||||
Expected response: | ||||||
```json | ||||||
{ | ||||||
"SendMessageResponse": { | ||||||
"ResponseMetadata": { | ||||||
"RequestId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" | ||||||
}, | ||||||
"SendMessageResult": { | ||||||
"MD5OfMessageAttributes": null, | ||||||
"MD5OfMessageBody": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", | ||||||
"MD5OfMessageSystemAttributes": null, | ||||||
"MessageId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", | ||||||
"SequenceNumber": null | ||||||
} | ||||||
} | ||||||
} | ||||||
``` | ||||||
|
||||||
You should also receive an SMS on your mobile with the following message: | ||||||
``` | ||||||
Hello from webhook! | ||||||
``` | ||||||
|
||||||
## Important Notes | ||||||
|
||||||
- **Phone Number Format**: Use E.164 format (e.g., +1234567890) | ||||||
- **SMS Costs**: SNS SMS messages incur charges - be mindful of costs | ||||||
- **Permissions**: Ensure your AWS account has SNS SMS permissions enabled | ||||||
- **PoC Only**: This is a simplified pattern for proof of concept. For production usage, consider implementing authentication/authorization, Dead Letter Queues (DLQ) for failed message processing, proper error handling and retry logic, adding monitoring and alerting (CloudWatch alarms), rate limiting, and further input validation and sanitization. | ||||||
|
||||||
## Use Cases | ||||||
|
||||||
This pattern works well for various webhook integrations including: | ||||||
- Marketing automation platforms | ||||||
- CRM systems | ||||||
- E-commerce platforms | ||||||
- Monitoring and alerting systems | ||||||
- Any system that needs to send SMS notifications based on webhook events | ||||||
|
||||||
## Cleanup | ||||||
|
||||||
To remove all resources: | ||||||
```bash | ||||||
cdk destroy | ||||||
``` | ||||||
|
||||||
## Customization | ||||||
|
||||||
- Modify `lambda/app.py` to change message processing logic | ||||||
- Update the CDK stack to add additional features like DLQ, encryption, etc. | ||||||
- Add API authentication/authorization as needed | ||||||
- Integrate with different notification channels (email, Slack, etc.) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,93 @@ | ||
{ | ||
"title": "Webhook integration with SMS notifications", | ||
"description": "API Gateway webhook integration that queues events in SQS and processes them with Lambda to send SMS notifications via SNS", | ||
"language": "Python", | ||
"level": "200", | ||
"framework": "AWS CDK", | ||
"introBox": { | ||
"headline": "How it works", | ||
"text": [ | ||
"This pattern demonstrates a reliable webhook integration that receives HTTP POST requests via API Gateway, queues them in SQS for decoupling and reliability, then processes the messages with Lambda to send SMS notifications through SNS.", | ||
"The pattern uses SQS to decouple the API from message processing, providing resilience and the ability to handle traffic spikes. Lambda processes messages from the queue and sends SMS messages to phone numbers in E.164 format." | ||
] | ||
}, | ||
"gitHub": { | ||
"template": { | ||
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/apigw-sqs-lambda-sns", | ||
"templateURL": "serverless-patterns/apigw-sqs-lambda-sns", | ||
"projectFolder": "apigw-sqs-lambda-sns", | ||
"templateFile": "webhook_sns/webhook_sns_stack.py" | ||
} | ||
}, | ||
"deploy": { | ||
"text": [ | ||
"pip install -r requirements.txt", | ||
"cdk bootstrap", | ||
"cdk deploy" | ||
] | ||
}, | ||
"testing": { | ||
"text": [ | ||
"Send a POST request to the API Gateway endpoint:", | ||
"curl -X POST https://your-api-id.execute-api.region.amazonaws.com/prod/ -H \"Content-Type: application/json\" -d '{\"phoneNumber\": \"+1234567890\", \"message\": \"Test message\"}'", | ||
"Replace the phone number with a valid number in E.164 format for actual SMS delivery." | ||
] | ||
}, | ||
"cleanup": { | ||
"text": [ | ||
"Delete the stack: <code>cdk destroy</code>." | ||
] | ||
}, | ||
"authors": [ | ||
{ | ||
"name": "Shaun Guo", | ||
"image": "https://media.licdn.com/dms/image/C5103AQG3KMyMdEIKpA/profile-displayphoto-shrink_800_800/0/1517283953925?e=1692835200&v=beta&t=AxJ9ST_8K_bw8nqTPDaJB2F5dnQspES9FuJ64DBScC8", | ||
"bio": "Shaun is a Senior Technical Account Manager at Amazon Web Services based in Australia", | ||
"linkedin": "shaun-guo" | ||
}, | ||
{ | ||
"name": "Robbie Cooray", | ||
"image": "https://media.licdn.com/dms/image/v2/C5603AQFK28pyKxfNiQ/profile-displayphoto-shrink_200_200/profile-displayphoto-shrink_200_200/0/1603248682998?e=1729728000&v=beta&t=YcFUfepq9JCKMvcqzaHMhTJFks6nrsfxS6v8JpolvEc", | ||
"bio": "Robbie is a Senior Solutions Architect at Amazon Web Services based in Australia", | ||
"linkedin": "robbiecooray" | ||
} | ||
], | ||
"patternArch": { | ||
"icon1": { | ||
"x": 10, | ||
"y": 50, | ||
"service": "apigw", | ||
"label": "API Gateway" | ||
}, | ||
"icon2": { | ||
"x": 35, | ||
"y": 50, | ||
"service": "sqs", | ||
"label": "Amazon SQS" | ||
}, | ||
"icon3": { | ||
"x": 60, | ||
"y": 50, | ||
"service": "lambda", | ||
"label": "AWS Lambda" | ||
}, | ||
"icon4": { | ||
"x": 85, | ||
"y": 50, | ||
"service": "sns", | ||
"label": "Amazon SNS" | ||
}, | ||
"line1": { | ||
"from": "icon1", | ||
"to": "icon2" | ||
}, | ||
"line2": { | ||
"from": "icon2", | ||
"to": "icon3" | ||
}, | ||
"line3": { | ||
"from": "icon3", | ||
"to": "icon4" | ||
} | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
#!/usr/bin/env python3 | ||
import aws_cdk as cdk | ||
from webhook_sns.webhook_sns_stack import WebhookSnsStack | ||
|
||
app = cdk.App() | ||
WebhookSnsStack(app, "WebhookSnsStack") | ||
|
||
app.synth() |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
{ | ||
"app": "python3 app.py", | ||
"watch": { | ||
"include": [ | ||
"**" | ||
], | ||
"exclude": [ | ||
"README.md", | ||
"cdk*.json", | ||
"requirements*.txt", | ||
"source.bat", | ||
"**/__pycache__", | ||
"**/*.pyc" | ||
] | ||
}, | ||
"context": { | ||
"@aws-cdk/aws-lambda:recognizeLayerVersion": true, | ||
"@aws-cdk/core:checkSecretUsage": true, | ||
"@aws-cdk/core:target-partitions": [ | ||
"aws", | ||
"aws-cn" | ||
], | ||
"@aws-cdk-containers/ecs-service-extensions:enableLogging": true, | ||
"@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true, | ||
"@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, | ||
"@aws-cdk/aws-iam:minimizePolicies": true, | ||
"@aws-cdk/core:validateSnapshotRemovalPolicy": true, | ||
"@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true, | ||
"@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, | ||
"@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true, | ||
"@aws-cdk/aws-apigateway:disableCloudWatchRole": false, | ||
"@aws-cdk/core:enablePartitionLiterals": true, | ||
"@aws-cdk/aws-events:eventsTargetQueueSameAccount": true, | ||
"@aws-cdk/aws-iam:standardizedServicePrincipals": true, | ||
"@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true, | ||
"@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true, | ||
"@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true, | ||
"@aws-cdk/aws-route53-patters:useCertificate": true, | ||
"@aws-cdk/customresources:installLatestAwsSdkDefault": false, | ||
"@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true, | ||
"@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true, | ||
"@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true, | ||
"@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true, | ||
"@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true, | ||
"@aws-cdk/aws-redshift:columnId": true, | ||
"@aws-cdk/aws-stepfunctions-tasks:enableLogging": true, | ||
"@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true, | ||
"@aws-cdk/aws-apigateway:requestValidatorUniqueId": true, | ||
"@aws-cdk/aws-kms:aliasNameRef": true, | ||
"@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true, | ||
"@aws-cdk/core:includePrefixInUniqueNameGeneration": true, | ||
"@aws-cdk/aws-efs:denyAnonymousAccess": true, | ||
"@aws-cdk/aws-opensearchservice:enableLogging": true, | ||
"@aws-cdk/aws-normlizedkeys:props": true, | ||
"@aws-cdk/aws-kms:reduceCrossAccountRegionPolicyScope": true, | ||
"@aws-cdk/aws-opensearchservice:usingLatestEngineVersion": true | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.