Skip to content

Commit bf8ca11

Browse files
FalltradesFalltrades
committed
feat: ✨ aws-s3+lambda+eventBridge+dynamoDB Event-Driven Pub/Sub website
1 parent f8b5933 commit bf8ca11

15 files changed

Lines changed: 677 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,3 +20,5 @@ This is an example repository to showcase some IaC usage with different cloud pr
2020
- [aws-ecs+rds](html-db-website/aws-ecs+rds)
2121
- [aws-eks](html-db-website/aws-eks)
2222
- [aws-s3+lambda+rds](html-db-website/aws-s3+lambda+rds)
23+
- event-driven-website
24+
- [aws-s3+lambda+eventBridge+dynamoDB](event-driven-website/aws-s3+lambda+eventBridge+dynamoDB)

‎event-driven-website/aws-s3+lambda+eventBridge+dynamoDB/README.md‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
# AWS-S3+Lambda+eventBridge+RDS
2+
3+
This is an example repository containing Terraform code. It contains the code to deploy a demo Event-Driven Pub/Sub application with S3, Lambda, eventBridge and DynamoDB.
4+
We are using Api gateway and Cloudfront to expose the application.
5+
6+
## Tree
7+
```
8+
.
9+
├── app
10+
│   ├── lambda.zip
11+
│   ├── on_connect.py
12+
│   ├── requirements.txt
13+
│   ├── trigger_event.py
14+
│   └── zip.sh # Helper script to run to generate lambda.zip
15+
├── misc
16+
│   └── architecture.dot.png # Generated with https://github.com/patrickchugh/terravision.
17+
├── README.md
18+
└── terraform
19+
├── iam.tf
20+
├── main.tf
21+
├── outputs.tf
22+
├── provider.tf
23+
├── s3.tf
24+
├── templates
25+
│   └── index.html.tftpl
26+
└── variables.tf
27+
```
28+
29+
## Architecture diagram
30+
31+
<img src="./misc/architecture.dot.png">
32+
33+
## Infracost
34+
35+
```shell
36+
Name Monthly Qty Unit Monthly Cost
37+
38+
aws_apigatewayv2_api.event_ws
39+
├─ Messages (first 1B) Monthly cost depends on usage: $1.00 per 1M messages
40+
└─ Connection duration Monthly cost depends on usage: $0.25 per 1M minutes
41+
42+
aws_cloudfront_distribution.s3_distribution
43+
├─ Invalidation requests (first 1k) Monthly cost depends on usage: $0.00 per paths
44+
└─ US, Mexico, Canada
45+
├─ Data transfer out to internet (first 10TB) Monthly cost depends on usage: $0.085 per GB
46+
├─ Data transfer out to origin Monthly cost depends on usage: $0.02 per GB
47+
├─ HTTP requests Monthly cost depends on usage: $0.0075 per 10k requests
48+
└─ HTTPS requests Monthly cost depends on usage: $0.01 per 10k requests
49+
50+
aws_dynamodb_table.connections
51+
├─ Write request unit (WRU) Monthly cost depends on usage: $0.000000625 per WRUs
52+
├─ Read request unit (RRU) Monthly cost depends on usage: $0.000000125 per RRUs
53+
├─ Data storage Monthly cost depends on usage: $0.25 per GB
54+
├─ On-demand backup storage Monthly cost depends on usage: $0.10 per GB
55+
├─ Table data restored Monthly cost depends on usage: $0.15 per GB
56+
└─ Streams read request unit (sRRU) Monthly cost depends on usage: $0.0000002 per sRRUs
57+
58+
aws_dynamodb_table.event_history
59+
├─ Write request unit (WRU) Monthly cost depends on usage: $0.000000625 per WRUs
60+
├─ Read request unit (RRU) Monthly cost depends on usage: $0.000000125 per RRUs
61+
├─ Data storage Monthly cost depends on usage: $0.25 per GB
62+
├─ On-demand backup storage Monthly cost depends on usage: $0.10 per GB
63+
├─ Table data restored Monthly cost depends on usage: $0.15 per GB
64+
└─ Streams read request unit (sRRU) Monthly cost depends on usage: $0.0000002 per sRRUs
65+
66+
aws_lambda_function.on_connect
67+
├─ Requests Monthly cost depends on usage: $0.20 per 1M requests
68+
├─ Ephemeral storage Monthly cost depends on usage: $0.0000000309 per GB-seconds
69+
└─ Duration (first 6B) Monthly cost depends on usage: $0.0000166667 per GB-seconds
70+
71+
aws_lambda_function.trigger
72+
├─ Requests Monthly cost depends on usage: $0.20 per 1M requests
73+
├─ Ephemeral storage Monthly cost depends on usage: $0.0000000309 per GB-seconds
74+
└─ Duration (first 6B) Monthly cost depends on usage: $0.0000166667 per GB-seconds
75+
76+
aws_s3_bucket.frontend
77+
└─ Standard
78+
├─ Storage Monthly cost depends on usage: $0.023 per GB
79+
├─ PUT, COPY, POST, LIST requests Monthly cost depends on usage: $0.005 per 1k requests
80+
├─ GET, SELECT, and all other requests Monthly cost depends on usage: $0.0004 per 1k requests
81+
├─ Select data scanned Monthly cost depends on usage: $0.002 per GB
82+
└─ Select data returned Monthly cost depends on usage: $0.0007 per GB
83+
84+
OVERALL TOTAL $0.00
85+
86+
*Usage costs can be estimated by updating Infracost Cloud settings, see docs for other options.
87+
88+
──────────────────────────────────
89+
23 cloud resources were detected:
90+
∙ 7 were estimated
91+
∙ 16 were free
92+
93+
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
94+
┃ Project ┃ Baseline cost ┃ Usage cost* ┃ Total cost ┃
95+
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━╋━━━━━━━━━━━━┫
96+
┃ main ┃ $0.00 ┃ - ┃ $0.00 ┃
97+
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━┻━━━━━━━━━━━━┛
98+
```
99+
100+
## Helpful informations

‎event-driven-website/aws-s3+lambda+eventBridge+dynamoDB/app/lambda.zip‎

12.9 MB
Binary file not shown.

‎event-driven-website/aws-s3+lambda+eventBridge+dynamoDB/app/on_connect.py‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
import json
2+
import boto3
3+
import os
4+
from decimal import Decimal
5+
6+
class DecimalEncoder(json.JSONEncoder):
7+
def default(self, obj):
8+
if isinstance(obj, Decimal): return int(obj)
9+
return super(DecimalEncoder, self).default(obj)
10+
11+
db = boto3.resource('dynamodb')
12+
hist_table = db.Table(os.environ['HIST_TABLE'])
13+
conn_table = db.Table(os.environ['CONN_TABLE'])
14+
15+
def handler(event, context):
16+
route_key = event['requestContext']['routeKey']
17+
connection_id = event['requestContext']['connectionId']
18+
19+
# 1. If it's just the initial connection, just save and exit
20+
if route_key == "$connect":
21+
conn_table.put_item(Item={'connectionId': connection_id})
22+
return {'statusCode': 200}
23+
24+
# 2. If it's the history request, fetch and send
25+
domain = event['requestContext']['domainName']
26+
stage = event['requestContext']['stage']
27+
gatewayapi = boto3.client('apigatewaymanagementapi',
28+
endpoint_url=f"https://{domain}/{stage}")
29+
30+
response = hist_table.scan()
31+
items = response.get('Items', [])
32+
items.sort(key=lambda x: x.get('unix_time', 0))
33+
history_to_send = items[-20:]
34+
35+
try:
36+
gatewayapi.post_to_connection(
37+
ConnectionId=connection_id,
38+
Data=json.dumps({"action": "history", "data": history_to_send}, cls=DecimalEncoder)
39+
)
40+
except Exception as e:
41+
print(f"Error: {e}")
42+
43+
return {'statusCode': 200}

‎event-driven-website/aws-s3+lambda+eventBridge+dynamoDB/app/requirements.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
boto3==1.28.0

‎event-driven-website/aws-s3+lambda+eventBridge+dynamoDB/app/trigger_event.py‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
import json
2+
import boto3
3+
import os
4+
import time
5+
import uuid
6+
from decimal import Decimal
7+
8+
# Helper to convert DynamoDB Decimals to standard Integers
9+
class DecimalEncoder(json.JSONEncoder):
10+
def default(self, obj):
11+
if isinstance(obj, Decimal):
12+
return int(obj)
13+
return super(DecimalEncoder, self).default(obj)
14+
15+
db = boto3.resource('dynamodb')
16+
conn_table = db.Table(os.environ['CONN_TABLE'])
17+
hist_table = db.Table(os.environ['HIST_TABLE'])
18+
19+
def handler(event, context):
20+
try:
21+
domain = event['requestContext']['domainName']
22+
stage = event['requestContext']['stage']
23+
gatewayapi = boto3.client('apigatewaymanagementapi',
24+
endpoint_url=f"https://{domain}/{stage}")
25+
26+
body = json.loads(event.get('body', '{}'))
27+
28+
event_payload = {
29+
"id": str(uuid.uuid4())[:4],
30+
"type": body.get('event_type', 'System Update'),
31+
"timestamp": time.strftime('%H:%M:%S'),
32+
"unix_time": int(time.time())
33+
}
34+
35+
# 1. Save to history
36+
hist_table.put_item(Item=event_payload)
37+
38+
# 2. Get all connections
39+
connections = conn_table.scan(ProjectionExpression="connectionId")['Items']
40+
41+
# 3. Broadcast - Use the encoder to prevent "Internal Server Error"
42+
message = json.dumps(event_payload, cls=DecimalEncoder)
43+
44+
for conn in connections:
45+
cid = conn['connectionId']
46+
try:
47+
gatewayapi.post_to_connection(ConnectionId=cid, Data=message)
48+
except gatewayapi.exceptions.GoneException:
49+
conn_table.delete_item(Key={'connectionId': cid})
50+
51+
return {'statusCode': 200}
52+
53+
except Exception as e:
54+
print(f"CRASH: {e}") # This will show up in CloudWatch Logs
55+
return {'statusCode': 500, 'body': str(e)}

‎event-driven-website/aws-s3+lambda+eventBridge+dynamoDB/app/zip.sh‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
#!/bin/bash
2+
3+
set -euo pipefail
4+
5+
rm -rf build lambda.zip
6+
mkdir build
7+
8+
cp *.py build/
9+
10+
# Build dependencies for Lambda using Docker
11+
docker run --rm \
12+
-v "$(pwd)/build":/var/task \
13+
-v "$(pwd)/requirements.txt":/var/requirements.txt \
14+
--entrypoint "" \
15+
public.ecr.aws/lambda/python:3.11 \
16+
python3.11 -m pip install -r /var/requirements.txt -t /var/task
17+
18+
cd build
19+
zip -r ../lambda.zip .
20+
cd ..

‎event-driven-website/aws-s3+lambda+eventBridge+dynamoDB/misc/architecture.dot.png‎

320 KB
Loading

‎event-driven-website/aws-s3+lambda+eventBridge+dynamoDB/terraform/iam.tf‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
resource "aws_iam_role" "lambda_exec" {
2+
name = "event_dashboard_lambda_role"
3+
4+
assume_role_policy = jsonencode({
5+
Version = "2012-10-17"
6+
Statement = [{
7+
Action = "sts:AssumeRole"
8+
Effect = "Allow"
9+
Principal = { Service = "lambda.amazonaws.com" }
10+
}]
11+
})
12+
}
13+
14+
# Attach policy to allow Lambda to access DynamoDB and API Gateway
15+
resource "aws_iam_role_policy" "lambda_policy" {
16+
role = aws_iam_role.lambda_exec.id
17+
policy = jsonencode({
18+
Version = "2012-10-17"
19+
Statement = [
20+
{
21+
Action = [
22+
"dynamodb:PutItem",
23+
"dynamodb:DeleteItem",
24+
"dynamodb:Scan",
25+
"dynamodb:GetItem"
26+
],
27+
Effect = "Allow",
28+
Resource = "*"
29+
},
30+
{
31+
Action = ["execute-api:ManageConnections"],
32+
Effect = "Allow",
33+
Resource = "arn:aws:execute-api:*:*:*/*"
34+
}
35+
]
36+
})
37+
}
38+
39+
resource "aws_s3_bucket_policy" "allow_access_from_cloudfront" {
40+
bucket = aws_s3_bucket.frontend.id
41+
42+
policy = jsonencode({
43+
Version = "2012-10-17"
44+
Statement = [
45+
{
46+
Sid = "AllowCloudFrontServicePrincipalReadOnly"
47+
Effect = "Allow"
48+
Principal = {
49+
Service = "cloudfront.amazonaws.com"
50+
}
51+
Action = "s3:GetObject"
52+
Resource = "${aws_s3_bucket.frontend.arn}/*"
53+
Condition = {
54+
StringEquals = {
55+
"AWS:SourceArn" = aws_cloudfront_distribution.s3_distribution.arn
56+
}
57+
}
58+
}
59+
]
60+
})
61+
}
62+
63+
resource "aws_s3_bucket_public_access_block" "frontend_secure" {
64+
bucket = aws_s3_bucket.frontend.id
65+
66+
block_public_acls = true
67+
block_public_policy = true
68+
ignore_public_acls = true
69+
restrict_public_buckets = true
70+
}

0 commit comments

Comments
 (0)