- Overview
- Project Philosophy
- Infrastructure Visualization
- Infrastructure Deployment Flow
- Tools and Practices
- Project Structure
- Getting Started
- Useful Links
- Next Steps
- Security and Monitoring
- Personal Note
This repository contains the infrastructure configuration for the K'óoben project using Terraform. It is designed to deploy a complete AWS-based architecture for a carpentry business application, consisting of:
- Backend: Built with NestJS, containerized with Docker
- Frontend: Built with NextJS, containerized with Docker
- Complete Infrastructure: VPC, subnets, security groups, load balancers, auto-scaling groups, RDS, and more
Related Repositories (Work in Progress):
- Frontend Repository: kooben-fe - NextJS e-commerce application (WIP)
- Backend Repository: kooben-be - NestJS API for the application (WIP)
The key philosophy behind this project is Infrastructure as Code (IaC) with approximately 98% of the infrastructure created and managed through Terraform and Terraform Cloud, minimizing the need for AWS Console interaction. Docker images are pulled directly from Docker Hub when EC2 instances are provisioned, using a bash script embedded in the Launch Template.
The core principles guiding this project:
- Complete Terraform Automation: Minimize AWS Console interaction, ensuring reproducible deployments
- Modular Design: Well-structured code divided into logical modules for maintainability
- Security First: Utilizing tools like tfsec to identify and address security concerns
- Cost Optimization: Implementing scheduling for non-production resources to reduce costs
- Documentation: Comprehensive documentation for future reference and learning
- Continuous Improvement: A platform for learning and evolving better design patterns
The only manual interactions required with AWS were creating secret keys in AWS Secrets Manager and copying Route53 DNS records to configure in Namecheap domain settings.
The project includes visual representations of the infrastructure deployed with Terraform:
This detailed diagram shows the AWS architecture with a clear representation of the different availability zones, subnets, and services used in the project. It provides a comprehensive view of how the components interact within the AWS Cloud.
This automatically generated diagram shows the relationships between Terraform resources and the overall architecture of the system. It represents all the AWS resources and their connections as defined in the Terraform code. 😬 The graph might look a bit chaotic due to the complex relationships between resources, but it provides valuable insights into the infrastructure dependencies.
- Locations:
- Cloud Architecture:
documentation/Cloud-Architecture.png - Terraform Graph:
documentation/graph.svg
- Cloud Architecture:
- How to update the graph: Run
terraform graph | dot -Tsvg > documentation/graph.svgfrom theinfradirectory - Requirements: GraphViz must be installed (
brew install graphvizon macOS)
The deployment of the Kooben infrastructure follows a logical sequence that builds components in the correct dependency order. Below is an overview of the deployment process:
- VPC created with CIDR block 10.10.0.0/24
- Public and private subnets deployed across two availability zones
- Internet Gateway, NAT Gateway, and Route Tables configured
- Security Groups established for controlled traffic
- S3 bucket created for configurations and Docker Compose files
- RDS PostgreSQL instance deployed in private subnet
- KMS keys created for encryption of various resources
- Secrets Manager configured for sensitive environment variables
- Application Load Balancers created for frontend and backend
- SSL/TLS certificates provisioned via ACM
- Target groups configured for health checks
- Route53 DNS records created for domain routing
- Launch Templates configured with user data scripts
- Auto Scaling Groups established for frontend and backend
- Docker containers pulled and deployed automatically
- SNS notification topic created with email subscription prompt
- Key Interaction Point: User receives email to confirm SNS subscription
- CloudWatch Events/EventBridge rules created for:
- EC2 instances: Start at 21:45, Stop at 05:45 (Berlin time)
- RDS database: Start at 21:30, Stop at 05:15 (Berlin time)
- Auto Scaling Groups: Scale in/out based on schedule
- CloudTrail enabled for comprehensive audit logging
- CloudWatch Logs for application monitoring
- VPC Flow Logs for network traffic analysis
- KMS encryption for data at rest and in transit
- IAM roles with least privilege principle
This project leverages several tools and best practices:
- terraform fmt: For consistent code formatting
- tflint: To catch errors and enforce best practices before applying changes
- tfsec: To identify potential security issues in Terraform configurations
- infracost: To estimate the monthly cost of AWS resources
- External modules:
- Custom fork of EC2/RDS scheduler module for Auto Scaling Groups, EC2 and RDS instances (terraform-aws-ec2-rds-scheduler) (fixed security issues)
- Random module for generating unique S3 bucket names
- Security: KMS for encryption, IAM for access management, VPC for network isolation
- Secrets Management: AWS Secrets Manager for sensitive application configuration
.
├── infra/ # Terraform infrastructure files
│ ├── modules/ # Terraform modules
│ │ ├── alb/ # Application Load Balancer module
│ │ │ ├── main.tf
│ │ │ ├── outputs.tf
│ │ │ ├── variables.tf
│ │ ├── backend/ # Backend application modules
│ │ │ ├── asg/ # Auto Scaling Group for backend
│ │ │ ├── launch_template/# Launch Template for backend
│ │ ├── frontend/ # Frontend application modules
│ │ │ ├── asg/ # Auto Scaling Group for frontend
│ │ │ ├── launch_template/# Launch Template for frontend
│ │ ├── iam/ # IAM configurations
│ │ ├── networking/ # Networking-related modules
│ │ │ ├── main.tf
│ │ │ ├── outputs.tf
│ │ │ ├── variables.tf
│ │ ├── rds/ # RDS database module
│ │ ├── route53/ # Route53 DNS configurations
│ │ ├── S3/ # S3 Bucket module
│ │ ├── security_groups/ # Security Groups module
│ │ ├── sns/ # SNS notifications module
│ │ ├── kms/ # KMS module for encryption
│ │ │ ├── main.tf
│ │ ├── launch_template/ # EC2 Launch Template module
│ │ │ ├── main.tf
│ │ │ ├── user_data.sh # Script for EC2 user data
│ │ │ ├── variables.tf
│ │ ├── networking/ # Networking-related modules
│ │ │ ├── modules/ # Submodules inside networking
│ │ │ │ ├── flow_logs/ # VPC Flow Logs module
│ │ │ │ │ ├── main.tf
│ │ │ │ │ ├── outputs.tf
│ │ │ │ │ ├── variables.tf
│ │ │ │ ├── routing/ # Routing-related configurations
│ │ │ │ │ ├── main.tf
│ │ │ │ │ ├── variables.tf
│ │ │ │ ├── vpc/ # VPC module
│ │ │ │ │ ├── main.tf
│ │ │ │ │ ├── outputs.tf
│ │ │ │ │ ├── variables.tf
│ ├── locals.tf # Local variables for Terraform
│ ├── main.tf # Main entry point for Terraform
│ ├── outputs.tf # Terraform output definitions
│ ├── providers.tf # Terraform provider configurations
│ ├── terraform.tfvars # Environment-specific variables (not committed)
│ ├── variables.tf # Variable definitions
├── .gitignore # Files ignored by Git
├── README.md # Project documentation
├── documentation/ # Project documentation folder
│ ├── s3.md # Documentation for S3 configuration
│ ├── COMMANDS.md # List of useful Terraform commands
│ ├── graph.svg # Infrastructure visualization
│ ├── tools-i-used/ # Documentation for tools used
│ │ ├── infracost.md # Infracost documentation
│ │ ├── tfenv.md # tfenv documentation
│ │ ├── tfsec.md # tfsec documentation
│ │ ├── tflint.md # tflint documentation
Ensure you have the following installed:
- Terraform
- AWS CLI
- tfenv (for managing Terraform versions) (Documentation)
- TFLint (for linting) (Documentation)
- Infracost (for cost estimation) (Documentation)
- tfsec (for security analysis) (Documentation)
# Install and use the correct Terraform version
tfenv install 1.10.5
tfenv use 1.10.5
terraform init # Initialize Terraform
terraform validate # Validate Terraform configuration
terraform plan # Preview changes
terraform apply # Apply changes to AWSinfracost breakdown --path ./infra # Cost estimation
tflint # Run Terraform Linter to check for best practices
tfsec # Perform security analysis on Terraform configuration- Terraform Documentation
- AWS CLI Documentation
- Infracost Documentation
- TFLint Documentation
- tfenv Documentation
- tfsec Documentation
- S3 Configuration Documentation
- Project Presentation Slides
- Configure Launch Templates for backend instance ✅
- Implement cost optimization through scheduling ✅
- Deploy new services (EC2-backend, EC2-frontend, RDS) ✅
- Integrate Route 53 for domain management ✅
- Create Auto Scaling Group for dynamic scaling ✅
- Configure Launch Templates for frontend instance ✅
- Deploy an Application Load Balancer (ALB) ✅
- Implement HTTPS with AWS Certificate Manager ✅
- Implement monitoring (CloudWatch, CloudTrail) ✅
- Implement security scanning with AWS CodeGuru ✅
- Migrate from Docker Hub to Amazon ECR for container registry
- Implement AWS native automation tools (CodePipeline, CodeBuild, CodeDeploy)
- Explore AWS container management services (ECS/EKS) for improved orchestration
CloudTrail has been implemented to provide comprehensive logging of all API calls made within the AWS account. This service:
- Records all API calls made to AWS services
- Enables security analysis, resource change tracking, and compliance auditing
- Stores logs securely in an S3 bucket with appropriate retention policies
- Helps identify unusual activity and potential security threats
CodeGuru Security has been integrated into the CI/CD pipeline to automatically scan code for security vulnerabilities:
- Identifies critical security issues in application code
- Provides detailed recommendations for remediation
- Integrated with GitHub Actions for automated scanning on each push to main
- Generates SARIF reports that are uploaded to GitHub Security tab for easy tracking
I'm incredibly proud of what has been accomplished with this project. It represents countless hours of learning, troubleshooting, and refining. Working with Terraform and Terraform Cloud has been a revelation - these tools have fundamentally changed how I approach infrastructure development.
The ability to create and destroy complex infrastructure within seconds, without having to manually click through the AWS Management Console, has accelerated my learning process immensely. It's empowering to describe infrastructure as code and watch it come to life through automation.
This project is just the beginning. As I continue to gain experience, I plan to refine this codebase, implement more AWS-native services, and incorporate more sophisticated patterns and practices. The modular structure I've established provides a solid foundation for future improvements.
Happy coding!