Production-grade image processing platform built with Java and Spring Boot. Architected an event-driven microservices system using AWS Lambda for async image transformations, SNS/SQS for decoupled messaging, and DynamoDB single-table design for metadata storage. Implemented end-to-end distributed tracing with OpenTelemetry across service boundaries, enabling cross-service visibility from API through message queues to Lambda. Features include async image upload with status polling, automatic resize and watermarking, presigned URL downloads, and conditional DynamoDB updates for optimistic concurrency control. Infrastructure defined with Terraform modules and local development environment using LocalStack and Docker Compose.
- Sourcehttps://github.com/dtszwai/media-processing-service
- StackJavaSpring BootAWS LambdaS3DynamoDBSNSSQSOpenTelemetryTerraformLocalStack
System Architecture
Technical Highlights
Event-Driven Architecture
Decoupled API from processing using SNS/SQS, enabling independent scaling of ingestion and CPU-intensive transformations.
Distributed Tracing
End-to-end OpenTelemetry instrumentation with trace context propagation through message headers across service boundaries.
Single-Table DynamoDB
Composite key design (PK=MEDIA#{id}, SK=METADATA) with conditional updates for optimistic concurrency control.
Infrastructure as Code
12+ modular Terraform components including VPC, ECS, Lambda, and observability stack for reproducible deployments.
Processing Flow
Upload
Client uploads image → API stores in S3, metadata in DynamoDB
Publish
API publishes media.v1.process event to SNS topic
Queue
SNS forwards message to SQS queue for reliable delivery
Process
Lambda consumes event, resizes image, applies watermark
Complete
Lambda stores result in S3, updates status to COMPLETE
Download
Client polls status, downloads via presigned URL
REST API
| Method | Endpoint | Description |
|---|---|---|
POST | /v1/media/upload | Upload an image for processing |
GET | /v1/media | List all media with metadata |
GET | /v1/media/{id}/status | Get processing status |
GET | /v1/media/{id}/download | Download processed image |
PUT | /v1/media/{id}/resize | Resize to new width |
DELETE | /v1/media/{id} | Delete media |
Observability Stack
Distributed Tracing
OpenTelemetry with trace context propagation through SNS/SQS message headers
Metrics Collection
Custom counters for upload, process, resize, and delete operations with success/failure tracking
Grafana Dashboards
P95 latency, throughput, error rates, and trace waterfall views for debugging