Today, we are embarking on a brand new series: K8s with Pravesh 🚀 where well break down Kubernetes, understand what it really is, and more importantly, how you can actually use it in a practical, no-BS way.

In todays blog, well dive into the fundamentals Deployments, Services, and ConfigMaps and use them to deploy a three-tier application on Minikube.

Now you might be thinking Whats new here? There are already thousands of blogs doing the same thing.

And honestly, youre not wrong.

But hold your horses for a second 🐎

This isnt just another apply this YAML and it works kind of tutorial. Were going to:

• Understand whats really happening under the hood

• Debug real issues (yes, the ones that actually happen)

• And build intuition so you dont just run Kubernetes you get it

So lets dive in. 🔥

🛠 Pre-Requisites

Before we dive deep, there are a couple of things you need to have set up. Nothing fancy just the essentials to get your Kubernetes playground up and running.

🔹 Docker / Docker Desktop

Well be running Minikube using Docker, so make sure you have Docker installed on your system.

👉 Install it from here: https://docs.docker.com/get-started/get-docker/

🔹 Minikube

Think of Minikube as your personal Kubernetes cluster lightweight, local, and perfect for experimenting and learning all the cool stuff without needing a cloud setup.

👉 Download it from here: https://minikube.sigs.k8s.io/docs/start/?arch=%2Fmacos%2Farm64%2Fstable%2Fbinary+download

🎥 Practical Demonstration

https://youtu.be/ZYlRwMf4lYA

🤔 What is Kubernetes (K8s)?

At its core, Kubernetes is a container orchestration tool.

Now that sounds fancy, but lets simplify it a bit.

Think of Kubernetes as a Head Chef in a restaurant 👨🍳 It makes sure:

• Everyone is doing their job properly

• Work is flowing smoothly

• And if something breaks it steps in and fixes it

Thats the layman definition.

The Real Meaning

In technical terms, Kubernetes is responsible for:

• Managing containers

• Scaling them

• Ensuring they are always running

• Handling communication between them

You can think of it as an advanced version of Docker Compose but built for production-grade systems.

The Smallest Unit: Pod

In Kubernetes, the smallest deployable unit is a Pod.

👉 A Pod is basically a wrapper around your container(s)

• It can run one or more containers

• These containers share:

  • Network

  • Storage

But heres the thing

Managing Pods manually? 😵💫 Not a great idea.

Enter Deployments

To solve that, we have Deployments.

A Deployment is like a blueprint for your Pods.

You define:

• Container image

• Number of replicas

• Ports

• Volumes

• Other configurations

And Kubernetes takes care of:

• Creating Pods

• Scaling them

• Replacing them if they crash

💥 Much easier to manage.

How Do Pods Talk to Each Other?

Back to our restaurant analogy 🍽

The waiter needs to communicate with the chef, right?

But in Kubernetes 👉 Pods dont automatically talk to each other

We need something in between.

Services: The Communication Bridge

Services act as a bridge between Pods.

They provide:

• Stable networking

• Internal DNS

• Load balancing

There are 3 main types:

🔹 ClusterIP

• Default type

• Used for internal communication only

• Not accessible from outside the cluster

🔹 NodePort

• Exposes the service on a specific port on the node

• Accessible from outside using:

  <Node-IP>:<Port>
  

🔹 LoadBalancer

• Exposes the app to the outside world

• Commonly used in cloud environments (AWS, GCP, etc.)

ConfigMaps: Handling Custom Configurations

Back to the restaurant

Imagine a customer walks in and says:

I want a Caff macchiato, with a little bit of soy, enough to make me go OH BOY! Kevin Hart fans, you know 😄

Handling custom requests manually can get messy

But in Kubernetes, we have ConfigMaps for this.

👉 ConfigMaps allow you to:

• Store non-confidential data

• Use it inside your applications

• Keep configs separate from your code

For sensitive data? 👉 Use Secrets

YAML: The Language of Kubernetes

All resources in Kubernetes are defined using YAML files.

You describe:

• What you want

• And Kubernetes makes it happen

If you want to explore more, check out the official docs: 👉 https://kubernetes.io/docs/setup/

Practical Demonstration

Enough with the theory now lets get our hands dirty 🔥

So far, weve covered:

• Deployments

• Services

• ConfigMaps

And to bring all of this together, well deploy a three-tier application (NginxNodeRedis).

Ive actually used this same app in one of my earlier projects to demonstrate CI/CD workflows with GitHub Actions and Terraform. If youre curious, check it out here: 👉 https://blog.praveshsudha.com/cicd-for-terraform-with-github-actions-deploying-a-nodejs-redis-app-on-aws

Step 1: Clone the Project

git clone https://github.com/Pravesh-Sudha/nginx-node-redis.git

Open the project in your favorite editor (VS Code works great).

Understanding the App

This is a simple Node.js application that:

• Displays a request counter

• Increments the count on every refresh

• Stores data in Redis

• Uses Nginx as a reverse proxy (serving on port 80 instead of 5000)

Step 2: Run with Docker Compose

Before jumping into Kubernetes, lets run it locally:

docker-compose up --build

Make sure Docker Desktop is installed and running.

You should see logs in your terminal and the app running in your browser.

Once done:

Ctrl + C

Step 3: Move to Kubernetes

Now comes the interesting part.

Inside the project:

cd nginx-node-redis/kube-config/

Youll find three directories:

• nginx/

• node/

• redis/

Each contains:

• Deployment YAML

• Service YAML

📦 Nginx Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

        volumeMounts:
        - name: nginx-config-volume
          mountPath: /etc/nginx/nginx.conf
          subPath: nginx.conf
          
      volumes:
      - name: nginx-config-volume
        configMap:
          name: nginx-config

A Note on AI & YAML

The best thing about AI? 👉 You can generate YAML files instantly.

But what happens when things break?

Thats where fundamentals matter.

Lets break this down 👇

Understanding the Deployment

1. API Version & Kind

Defines what resource we are creating:

kind: Deployment

2. Labels (IMPORTANT)

Labels appear in three places and each has a role:

• metadata.labels tagging the Deployment

• spec.selector.matchLabels tells Deployment which Pods to manage

• template.metadata.labels applied to Pods (used by Services)

👉 This is how Kubernetes connects resources.

3. Container Spec

image: nginx:1.14.2
ports:
  - containerPort: 80

Defines:

• Image

• Port

4. ConfigMap Mount

volumeMounts:
- name: nginx-config-volume
  mountPath: /etc/nginx/nginx.conf
  subPath: nginx.conf

👉 This mounts your custom Nginx config into the container.

🌐 Nginx Service

apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  type: ClusterIP
  selector:
    app: nginx
  ports:
    - port: 80
      targetPort: 80

Here:

• We use ClusterIP

• Selector matches:

  app: nginx
  

👉 This connects the Service to Pods.

Nginx ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-config
data:
  nginx.conf: |
    events {}

    http {
      upstream loadbalancer {
        server node-service:5000;
      }

      server {
        listen 80;

        location / {
          proxy_pass http://loadbalancer;
          proxy_set_header Host $host;
          proxy_set_header X-Real-IP $remote_addr;
          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }

        location = /favicon.ico {
          log_not_found off;
          access_log off;
        }
      }
    }

👉 Here we:

• Override default Nginx config

• Route traffic to:

  node-service:5000
  

Step 4: Deploy to Minikube

# Start Minikube
minikube start

# Go to config directory
cd nginx-node-redis/kube-config/

# Deploy Redis
cd redis/ && kubectl apply -f deploy.yml -f svc.yml
cd ..

# Deploy Node
cd node/ && kubectl apply -f deploy.yaml -f svc.yml
cd ..

# Deploy Nginx
cd nginx/ && kubectl apply -f deploy.yml -f svc.yml -f configmap.yaml

Wait for Pods

kubectl get pods -w

Wait until all pods are:

Running

Access the App

minikube service nginx-service

👉 This opens your app in the browser now running on Kubernetes 🎉

Self-Healing in Action

Heres where Kubernetes shines.

Lets break something 😈

kubectl delete pod <pod-name>

Now check:

kubectl get pods

👉 Youll see:

• A new pod automatically created

🧠 What just happened?

Kubernetes ensures:

Actual state = Desired state

Even if you:

• Delete a pod

• Crash a container

👉 Kubernetes will bring it back

🔍 Whats Happening Under the Hood?

Now that everything is up and running, lets take a step back and understand how things are actually working behind the scenes 👇

1. Accessing the Application

When you run:

minikube service nginx-service

👉 Minikube exposes your service and gives you a URL with a port.

2. Request Hits Nginx Service

Once you hit that URL:

• The Nginx Service receives the request

• It looks at its selector:

  app: nginx
  

• And forwards the request to all matching Nginx Pods

3. Inside the Nginx Pod

Inside the pod:

• Nginx uses the custom config (via ConfigMap)

• The request is proxied to:

  node-service:5000
  

4. Node Service Load Balancing

Now the interesting part 👀

• node-service is a ClusterIP Service

• It has multiple pods (replicas = 3)

👉 Kubernetes automatically distributes traffic:

node-service
   
 
  node-pod1 node-pod2 node-pod3
 

5. Node App Talks to Redis

Inside your Node app:

• It connects to:

  redis-service
  

• Stores:

  • Request count

  • Cache data

6. Response Flow

Finally, the response travels back:

Redis  Node  Nginx  Browser

🎉 And you see the updated request count

🧠 Key Insight

Notice something important here

👉 We never used a single IP address.

Everything works using:

• Service names

• Internal DNS

• Labels & selectors

This is called Service Discovery one of the most powerful features of Kubernetes.

Scaling Made Easy

Want more traffic handling capacity?

Just update:

replicas: 3

👉 Increase or decrease as needed

👉 No changes required anywhere else

Kubernetes handles the rest

Cleanup

Once youre done experimenting, you can delete the cluster:

minikube delete

🎯 Conclusion

And thats a wrap for this one! 🚀

In this blog, we didnt just deploy an application on Kubernetes we actually understood whats happening behind the scenes. From Deployments and Services to ConfigMaps and internal service discovery, you now have a solid foundation to start building real-world K8s projects.

More importantly, you saw how:

• Kubernetes replaces static setups like Docker Compose with dynamic, scalable systems

• Services enable seamless communication without worrying about IPs

• And how the system self-heals to match the desired state

This is just the beginning of the K8s with Pravesh series. In the upcoming blogs, well go deeper into more advanced concepts and build even more powerful systems 💥

🔗 Lets Connect

If you found this helpful, feel free to connect with me and follow along for more DevOps and Kubernetes content:

• 💼 LinkedIn: https://www.linkedin.com/in/pravesh-sudha

• 📝 Blog: https://blog.praveshsudha.com

• 💻 GitHub: https://github.com/Pravesh-Sudha

If you have any questions, got stuck somewhere, or just want to discuss ideas my DMs are always open 🙌

Until next time Keep building, keep learning, and keep shipping 🚀

Hey folks, welcome to the world of Agentic Tools and DevOps.

Today, were diving into CI/CD pipelines and exploring how we can debug them efficiently and almost instantly using AI. In this project, well build an AI-powered CI/CD Copilot whereAWS Lambdaserves as the core logic layer. This Lambda function will interact with the Google Gemini API to analyze pipeline failures and help us debug them intelligently.

The goal of this project is not just to integrate AI into a CI/CD workflow, but to help you understand how to build your own AI agent from scratch one that can assist in real-world DevOps scenarios.

So, without further ado, lets get started.

💡 Prerequisites

Before we begin, make sure you have the following requirements in place:

• Docker & Docker Hub account
  We will run parts of this project inside Docker containers. Later, well push our custom image to Docker Hub, so make sure you have both Docker installed and a Docker Hub account ready.

• Jenkins (Our CI/CD Tool)
  Well use Jenkins for demonstration purposes. You can either:

  • Run Jenkins as a Docker container, or

  • Install it directly from the official website.

• Terraform
  We will provision our infrastructure including the Gemini API key (stored securely) and the AWS Lambda function using Terraform.

  Make sure:

  • Terraform CLI is installed

  • Your AWS credentials are configured

  • The IAM user has permissions forAWS LambdaandAWS Secrets Manager

  If youre new to Terraform setup, you can follow this guide:
  👉https://blog.praveshsudha.com/getting-started-with-terraform-a-beginners-guide#heading-step-1-install-the-aws-cli

🎥 Youtube Demonstration

💡 How It Works

The complete source code for this project is available in this GitHub repository:
👉https://github.com/Pravesh-Sudha/ai-devops-agent

Navigate to thecicd-copilotdirectory to follow along.

If youve been following my work, you might recognize this project. I originally used this sameNode.js Book Reader applicationto demonstrate how Docker works with Node.js. For this AI-powered CI/CD Copilot, Ive made specific modifications particularly in theJenkinsfileand theterra-configdirectory.

Inside theterra-configdirectory, youll find:

• main.tf Provisions:

  • AWS Lambda function

  • AWS Secrets Manager secret (to securely store the Gemini API key)

• lambda.zip The packaged Lambda deployment artifact (zippedlambda_function.py)

• lambda_function.py The core of this project.
  This file contains the AI agent logic and the structured prompt sent to the Gemini API.

• iam.tf Defines the IAM roles and permissions required for:

  • AWS Lambda

  • AWS Secrets Manager

Architecture Overview

The core idea behind this project is simple:

1. Jenkins detects a pipeline failure.

2. It collects contextual information (stage name, build ID, logs).

3. It sends that data to AWS Lambda.

4. Lambda calls the Gemini API.

5. Gemini analyzes the logs and returns structured debugging insights.

Payload Sent to Lambda

The Lambda function expects a JSON payload in the following format:

{
   stage: $stage,        # Name of the stage where the pipeline failed
   job: $job,            # Job name (e.g., cicd-copilot)
   build_id: $build_id,  # Build ID number (e.g., 1, 2, 3)
   logs: $logs           # Last 200 lines of failure logs
}

This structured input allows the AI agent to understand the pipeline context before analyzing the logs.

Prompt Sent to Gemini API

Inside the Lambda function, we make a POST request to the Gemini API with the following structured prompt:

You are a senior CI/CD Copilot specialized in Jenkins pipelines.

Pipeline context:
- Stage name: {stage}
- Expected outcome: Build an artifact usable by later stages

Your tasks:
1. Identify the failure category (build / runtime / config / infra / dependency / auth / unknown)
2. Identify the most likely root cause
3. Provide actionable fixes
4. Suggest a patch ONLY if clearly inferable

Respond ONLY in valid JSON with this schema:
{{
  "failure_category": "",
  "root_cause": "",
  "actionable_fixes": [],
  "suggested_patch": {{
    "file": "",
    "line": "",
    "fix": ""
  }}
}}

Logs:
{logs}

The prompt dynamically injects two key variables:

• {stage} The pipeline stage name

• {logs} The failure logs

If youd like to explore the full Lambda implementation, you can view it here:
👉https://github.com/Pravesh-Sudha/ai-devops-agent/blob/main/cicd-copilot/terra-config/lambda_function.py

How It Integrates with Jenkins

You might be wondering how exactly does this connect with Jenkins?

Inside theJenkinsfile, each stage:

• Sets an environment variable for the stage name.

• Redirects command output (in case of failure) into aLOG_FILE.

If any stage fails:

• Thepost { failure { ... } }block is triggered.

• Jenkins constructs the JSON payload.

• It invokes the AWS Lambda function.

• The AI-generated failure analysis is printed directly into the Jenkins console output.

This gives you instant, structured debugging assistance right inside your CI/CD pipeline.

How to Integrate This in Your Own Workspace

To replicate this approach in your own pipeline:

1. Append log redirection to each command:

   ${LOG_FILE} 2>&1
   

2. Define an environment variable for the stage name.

3. Provision:

   • AWS Lambda

   • IAM roles

   • Secrets Manager (for the Gemini API key)
     using Terraform.

4. Add apost failureblock in your Jenkinsfile to invoke the Lambda function with the structured JSON payload.

Once configured, your CI/CD pipeline becomes AI-assisted capable of analyzing its own failures and suggesting actionable fixes.

💡 Practical Demonstration

Enough with the theory lets see this in action.

Step 1: Fork and Clone the Repository

First, head over to the GitHub repository andfork it under your own username.
Youll be intentionally modifying the code later to trigger pipeline failures, so forking is important.

After forking:

git clone https://github.com/your-username/ai-devops-agent.git
cd ai-devops-agent/cicd-copilot/terra-config

Step 2: Initialize Terraform

Inside theterra-configdirectory, initialize Terraform:

terraform init

Step 3: Generate Your Gemini API Key

To provision the infrastructure, youll need aGEMINI_API_KEY.

1. Go toGoogle AI Studio

2. Log in with your Google account

3. Navigate to theAPIsection

4. ClickCreate API Key

5. Give it a name and generate the key

6. Store it securely

Now, apply the Terraform configuration:

terraform apply -var="gemini_api_key=<Paste-your-key-here>" --auto-approve

Make sure the configured AWS IAM user has the required permissions (Lambda and Secrets Manager access), as mentioned in the prerequisites section.

Once completed, your infrastructure (Lambda function + IAM roles + Secret) will be up and running.

Step 4: Configure Jenkins Pipeline

Open your Jenkins dashboard (usually running onhttp://localhost:8080).

1. ClickCreate New Item

2. SelectPipeline

3. Name it:cicd-copilot

4. ChoosePipeline script from SCM

Configure the following:

• SCM:Git

• Repository URL:
  https://github.com/your-username/ai-devops-agent

• Branch Specifier:main

• Script Path:
  cicd-copilot/Jenkinsfile

ClickSave.

Step 5: Install Required Jenkins Plugins

Navigate to:

Manage Jenkins Plugins

Install the following plugins:

• Docker

• Docker Pipeline

• Docker Commons

Step 6: Add Docker to Jenkins PATH

Ensure Docker is accessible inside Jenkins.

In your terminal, run:

which docker

Copy the output path.

Now go to:

Manage Jenkins System Global Properties

Append the copied path to the existing PATH variable using:as a separator. Save the configuration.

Step 7: Add Docker Hub Credentials

Navigate to:

Manage Jenkins Credentials

1. Add a new credential:

   • Kind:Username with password

   • Username: Your Docker Hub username

   • Password: Your Docker Hub password

   • ID:docker-cred

Save it.

Step 8: Trigger the Pipeline

Now go back to yourcicd-copilotproject and clickBuild Now.

OpenConsole Output.

You will notice that the pipeline fails this is intentional.

The logs are automatically captured and sent to the AI Agent, which returns structured debugging analysis inside the Jenkins console.

In the first failure, the AI identifies a typo in theDockerfile.
For example:

apine

It should be:

alpine

Fix the typo in your forked repository and commit the changes.

Step 9: Second Failure (Version Mismatch)

Rebuild the pipeline.

This time, the pipeline fails again but for a different reason. There is a Docker image version mismatch.

The AI analysis might suggest that the image is private or unavailable. However, the real issue is in theJenkinsfile.

Inside theRun Containerstage, change the image version from:

v2

to:

v1

Commit the change and rebuild the pipeline.

Step 10: Successful Pipeline Run

Now, when you trigger the pipeline again:

• The build succeeds

• The Docker image is pushed to your Docker Hub account

• The container starts successfully

Visit:

http://localhost:3000

You should see the Book Reader application running.

Stop the Application

To stop the running container:

docker kill cicd-copilot

Clean Up Infrastructure

To avoid unnecessary AWS charges, destroy the infrastructure:

terraform destroy -var="gemini_api_key=<Paste-your-key-here>" --auto-approve

What We Achieved

In this project, we built an AI-powered CI/CD Copilot using:

• Jenkins for pipeline orchestration

• AWS Lambda for AI agent logic

• AWS Secrets Manager for secure API storage

• Google Gemini API for log analysis

The agent receives contextual pipeline information and failure logs, analyzes them intelligently, and provides structured debugging insights directly inside the CI/CD workflow.

Instead of manually scanning logs, you now have an AI assistant that understands context, categorizes failures, identifies root causes, and suggests actionable fixes making debugging faster, smarter, and more efficient.

💡 Conclusion

Modern CI/CD pipelines are powerful but when they fail, debugging can quickly become time-consuming and frustrating. In this project, we went a step further by integrating AI directly into the pipeline workflow.

By combining:

• Jenkinsfor orchestration

• AWS Lambdafor serverless execution

• AWS Secrets Managerfor secure API handling

• Google Gemini APIfor intelligent log analysis

we built an AI-powered CI/CD Copilot capable of understanding pipeline context, analyzing failure logs, identifying root causes, and suggesting actionable fixes all automatically.

This isnt just about log analysis. Its about shifting from reactive debugging to intelligent, context-aware automation.

As AI continues to evolve, integrating agentic systems into DevOps workflows will become increasingly common. Building projects like this not only strengthens your cloud and automation skills but also prepares you for the next wave of AI-driven infrastructure.

If you found this project helpful, feel free to connect with me and follow my work:

• 🌐Website:https://praveshsudha.com

• 📝Blog:https://blog.praveshsudha.com

• 💼LinkedIn:https://www.linkedin.com/in/pravesh-sudha

• 🐙GitHub:https://github.com/Pravesh-Sudha

• 🐦Twitter/X:https://x.com/praveshstwt

• 🎥 Youtube: https://youtube.com/@pravesh-sudha

I regularly share content on DevOps, AWS, Terraform, CI/CD, and building real-world cloud projects from scratch.

If you build your own version of this AI CI/CD Copilot, tag me Id love to see what you create.

Happy Building 🚀

Hola amigos 👋
Welcome to the world of AI and DevOps.

In this blog, I want to share my experience building SkillDebt.ai as part of the UI Strikes Back Challenge, hosted by the WEMakeDevs community in collaboration with Tambo AI.

The idea behind SkillDebt.ai is simple:
as developers, we often talk about technical debt in our code but we rarely think about the technical debt in our careers.

SkillDebt.ai takes your resume or tech stack, analyzes it using Tambo AI and Gemini, and turns that data into beautiful, interactive visual insights about your skills. Instead of long paragraphs or generic advice, you get a clear picture of where you stand in your field.

Beyond skill visualization, it also highlights:

• Skill decay tools and technologies you havent touched in a while

• Risk audits warning signs when core skills are becoming outdated

• Upgrade suggestions practical recommendations on what skills to add next to boost your career growth

This project isnt just about AI or UI its about giving developers clarity, direction, and a better way to plan their learning journey.

%[https://youtu.be/ItTKixXJF2I\]

🌟 Practical Demo

Lets see SkillDebt.ai in action.

Theres no heavy setup or complex prerequisites. All you need is your resume in PDF format.

Head over to the live demo here:
👉 https://my-repo-8k7lhiaxb-pravesh-sudhas-projects.vercel.app/

The Project has been taken down by 25 Feb 2026, you can follow the GitHub Guide to illustrate in your own local system

Once youre on the site, click on Upload Resume, select your PDF, and hit Analyze. Thats it.

From there, SkillDebt.ai walks you through a complete breakdown of your profile:

First, youll see a visual skill analysis chart that gives a quick overview of your strengths and gaps across different areas in your field.

Next comes the Skill Decay graph, which highlights technologies you havent actively used in a while and flags them based on risk. This part is especially useful because it surfaces skills you might be unknowingly neglecting.

After that, the Risk Audit section kicks in. It acts like a warning system, pointing out areas in your resume that could become problematic if left unaddressed.

Finally, you get career-focused upgrade suggestions specific skills you should consider adding or improving to stay relevant and boost long-term growth.

If youre curious about how everything works under the hood, the complete source code is open-source and available here:
🔗 https://github.com/Pravesh-Sudha/ui-strikes-back

🌟 How I Built It

Going into the hackathon, I had one clear goal:
I didnt want to build something flashy but forgettable. I wanted to build something novel and actually useful.

The AI agent space is already crowded. Everywhere you look, theres another code debugger, another productivity hack, another AI assistant doing roughly the same thing. At the same time, with the rapid rise of AI especially tools like Claude Code and autonomous agents AI engineering has gone through the roof.

Thats when I paused and thought:
instead of building yet another tool to replace engineers, why not build something that helps engineers upskill and stay ahead of the curve?

That idea became SkillDebt.ai a system focused on helping developers understand where they stand today, what theyre falling behind on, and how they can adapt to this AI-driven future instead of getting left behind.

From an implementation perspective, the most challenging part for me was configuring the Tambo Generative UI components. Getting the components to behave correctly, respond to the data, and render meaningful insights wasnt straightforward at first. I ran into plenty of invalid input errors along the way.

But once I understood how the pieces fit together, things started clicking. The documentation played a huge role here it turned what initially felt overwhelming into a structured learning process. After a lot of trial and error (mostly Invalid config for components) , I finally got all four core components working together smoothly.

The main heart of the Project is the tambo.config.ts file inside the src/tambo directory, it handles the prompt for the generative UI components. Have a look at it:

import { z } from 'zod';
import { SkillRadarChart } from '../components/adaptive/SkillRadarChart';
import { SkillDecayTimeline } from '../components/adaptive/SkillDecayTimeline';
import { RiskWarningCard } from '../components/adaptive/RiskWarningCard';
import { UpgradeSuggestionCard } from '../components/adaptive/UpgradeSuggestionCard';
import { ExplanationToggle } from '../components/adaptive/ExplanationToggle';

export const tamboConfig = {
    components: [
        {
            name: 'skill_radar_chart',
            description: 'Visualizes the balance between depth and breadth of skills, or compares multiple skill categories.',
            component: SkillRadarChart,
            propsSchema: z.object({
                title: z.string().describe("Title of the chart, e.g., 'Frontend Skill Balance'").default("Skill Analysis"),
                data: z.array(z.object({
                    skill: z.string().describe("Name of the skill, e.g., 'React'").default("Unknown Skill"),
                    value: z.number().min(0).max(100).describe("Skill level from 0 to 100").default(50),
                    fullMark: z.number().default(100).optional(),
                })).describe("Array of 3-6 skills to visualize.").default([]),
            }),
        },
        {
            name: 'skill_decay_timeline',
            description: 'Shows a timeline of skills and their freshness/decay status based on last usage.',
            component: SkillDecayTimeline,
            propsSchema: z.object({
                data: z.array(z.object({
                    name: z.string().describe("Name of the skill, e.g. 'jQuery'").default("Unknown Skill"),
                    lastUsed: z.string().describe("Year or timeframe like '2023', 'Current'").default("Unknown"),
                    decayLevel: z.string().describe("Decay level: 'low', 'medium', 'high', 'critical'").default("medium"),
                })).describe("List of data points regarding skill usage and decay for the timeline.").default([]),
            }),
        },
        {
            name: 'risk_warning_card',
            description: 'Displays a warning about a specific career risk or skill obsolescence.',
            component: RiskWarningCard,
            propsSchema: z.object({
                title: z.string().describe("Short warning title, e.g. 'Legacy Stack Risk'").default("Risk Warning"),
                message: z.string().describe("Detailed explanation of the risk").default("Potential risk detected."),
                riskLevel: z.string().describe("Risk level: 'moderate', 'high', 'critical'").default('moderate'),
            }),
        },
        {
            name: 'upgrade_suggestion_card',
            description: 'Suggests a specific skill upgrade or learning path with potential impact.',
            component: UpgradeSuggestionCard,
            propsSchema: z.object({
                skill: z.string().describe("The recommended skill to learn").default("New Skill"),
                recommendation: z.string().describe("Why this skill is recommended").default("Recommended for career growth."),
                impact: z.string().describe("Impact: 'career_pivot', 'salary_bump', 'stability'").default("stability"),
            }),
        },
        {
            name: 'explanation_toggle',
            description: 'Can be used to provide deeper context or reasoning for a specific insight, hidden by default behind a toggle.',
            component: ExplanationToggle,
            propsSchema: z.object({
                reasoning: z.string().describe("The detailed reasoning or explanation to be hidden.").default("No additional details provided."),
                context: z.string().describe("Optional context or source data reference.").optional(),
            }),
        },
    ],
};

It wasnt easy at the start, but that struggle is exactly what made the project so rewarding.

After deploying the project, I posted about it on LinkedIn and dozens of Developers got their profile review using the system, and seeing real people interact with Generative UI components using Tambo made my DAY!

🌟 Conclusion

Building SkillDebt.ai was a genuinely fun and exciting journey. From shaping the idea, struggling through early implementation issues, to finally seeing the generative UI come together every step pushed me to think differently about how AI can be used to empower developers, not replace them.

Huge thanks to the WEMakeDevs community and Tambo AI for organizing the UI Strikes Back Challenge and creating a space that encourages experimentation, learning, and building in public. Challenges like these are what make the developer ecosystem so motivating.

If you found this project interesting or have ideas on how it can be improved, Id love to hear from you. You can find the code on GitHub, and feel free to connect with me on my socials:

• GitHub: https://github.com/Pravesh-Sudha

• LinkedIn: https://www.linkedin.com/in/pravesh-sudha/

• Twitter / X: https://x.com/praveshstwt

• YouTube: https://www.youtube.com/@pravesh-sudha

Thanks for reading and as always, keep building, keep learning, and stay curious 🚀

Adios 👋

Welcome, Devs 👋
Today, were stepping into the exciting intersection of AI, automation, and cloud infrastructure.

In this project, well explore how an AI-powered agent can actively participate in a real DevOps workflow, just like a senior reviewer on your team. This isnt a toy demo it closely resembles how real-world infrastructure changes are reviewed, validated, and approved in production environments.

Well use Terraform to provision cloud resources and GitHub Actions to automatically validate every pull request that modifies our HCL code. But heres the twist 👀
Instead of relying only on static checks, we introduce an AI agent into the pipeline.

Every infrastructure change is:

• Scanned using Terrascan

• Reviewed by an AI agent powered by Gemini

• Automatically approved, approved with changes, or rejected based on risk severity

If a pull request introduces dangerous or insecure infrastructure changes, the AI agent blocks the PR just like an automated infrastructure security reviewer.

Think of it as:

🧠 An AI-powered Infra Guardian that never gets tired of reviewing Terraform code.

So without further ado, lets dive in and see how we built an AI-driven, serverless DevOps workflow that brings intelligence directly into your CI/CD pipeline.

📽 Youtube Demonstration

🌟 Pre-requisites

Before we dive deep into the implementation, lets make sure your environment is ready. This project touches multiple tools across cloud, IaC, security, and CI/CD, so having these set up beforehand will save you a lot of time.

Make sure you have the following in place:

• AWS CLI installed and configured with an IAM user

  The IAM user should have permissions to create resources like ALB, ECS, Lambda, IAM, ACM, etc.

• Terraform CLI installed on your system

• GitHub account (pretty easy 😉)

• Terrascan installed locally
  👉 Follow the official guide here:
  https://runterrascan.io/docs/getting-started/

If youre completely new to AWS CLI or Terraform, dont worry. Ive already written a beginner-friendly guide that walks you through everything step by step:

📘 Getting Started with Terraform (Beginners Guide)
https://blog.praveshsudha.com/getting-started-with-terraform-a-beginners-guide#heading-step-1-install-the-aws-cli

Once these prerequisites are fulfilled, youre all set 🚀

🌟 Why AI Agents in Modern DevOps?

The current DevOps landscape is heavily influenced by AI-driven automation. What we now call AIOps has quietly become the de-facto standard for deploying, monitoring, and delivering software at scale.

AI agents are everywhere today but lets address the elephant in the room.

An AI agent is essentially a program that automates work which previously required human intervention. In many cases, it still follows a human-in-the-loop approach, but the heavy lifting analysis, validation, and decision-making is handled by the agent itself.

In this project, well bring that concept to life.

Well deploy a Super Mario Bros game (containerized using Docker) on a serverless AWS architecture, leveraging services like:

• Amazon ECS

• AWS Lambda

• Application Load Balancer (ALB)

• ACM for HTTPS

• GitHub Actions for CI/CD

This setup closely resembles a real-world production environment.

Now comes the interesting part 👀

Every time a Pull Request is raised against our Terraform codebase:

• GitHub Actions kicks in

• Terrascan scans our IaC for security and best-practice violations

• The scan report is sent to an AI agent powered by Gemini

• The AI analyzes the findings and decides whether to:

  • Approve

  • Approve with Changes

  • Reject the PR

In a real-world DevOps workflow, this kind of system can save hours of manual review, reduce human error, and provide actionable remediation suggestions along with architectural risk insights.

Think of it as an automated Infrastructure Reviewer one that never gets tired and scales with your team.

🌟 Practical Demonstration: Building the AI-Powered DevOps Workflow

Enough theory lets get our hands dirty and see this system in action.

To get started, head over to the following GitHub repository, fork it under your own GitHub username, and then clone it locally:

👉 Repository: https://github.com/Pravesh-Sudha/ai-devops-agent

git clone https://github.com/<your-username>/ai-devops-agent.git
cd ai-devops-agent

Now navigate into the main project directory:

cd terraform-review-agent

Open the project in VS Code (or your favorite editor). Youll notice two main subdirectories:

terraform-review-agent/
 lambda/
 terraform/

• lambda/ Contains the AI review Lambda function

• terraform/ Contains all infrastructure provisioning code

Lets walk through the Terraform configuration piece by piece.

🧩 Terraform Code Breakdown

🔹 provider.tf

Defines AWS as the cloud provider:

• AWS provider version: 6.26.0

• Region: us-east-1

This ensures consistent provider behavior across environments.

🔹 backend.tf

We store Terraform state remotely using Amazon S3 a production best practice.

use_lockfile = true

This enables state locking without DynamoDB, preventing concurrent state corruption using Terraforms native lockfile mechanism.

🔹 variables.tf

Only two variables are required:

• project_name fixed as mario-game

• gemini_api_key passed dynamically (never hardcoded)

This ensures our API key remains secure and out of version control.

🔹 outputs.tf

Provides useful runtime information after provisioning:

• ALB DNS name (where the game runs)

• ACM certificate ARN (used later for HTTPS)

🔹 networking.tf

Instead of using the default VPC, we create our own VPC using the official AWS VPC module:

• Two public subnets

• Clean network isolation

• Better control and scalability

🔹 security.tf

Security is handled via two separate security groups:

• ALB Security Group

  • Allows inbound traffic from anywhere (port 80 initially)

• ECS Task Security Group

  • Only allows traffic from the ALB

This follows the least privilege principle.
(We later extend this to support HTTPS on port 443.)

🔹 secrets.tf

The Gemini API key is securely stored using AWS Secrets Manager.

No plaintext secrets. No leaks. Production-safe by default.

🧠 The AI Brain: Lambda Function

🔹 lambda.tf

This file defines a Python-based AWS Lambda function responsible for reviewing Terrascan findings and acting as a CI/CD security gate.

At the heart of this Lambda is a carefully crafted prompt:

def build_prompt(findings: dict) -> str:
    return f"""
You are a senior DevOps and Terraform security reviewer acting as a CI/CD security gate.

Your task is to analyze Terrascan findings and decide whether the infrastructure
can be deployed based on **risk thresholds**, not perfection.

Decision Policy (STRICT)
- REJECT if:
  - Any HIGH or CRITICAL severity issue exists
  - OR MEDIUM severity issues  4
  - OR Application Load Balancer has **no HTTPS listener at all**
- APPROVE_WITH_CHANGES if:
  - MEDIUM severity issues are 13
- APPROVE if:
  - Only LOW or INFO issues exist

Output Format
Provide:
1. 🚨 Security issues ordered by severity (summary only)
2. 🛠 Required remediation (only actionable items)
3.  Risk justification (12 lines)
4. 📌 Final verdict: APPROVE | APPROVE_WITH_CHANGES | REJECT

Rules:
- Be concise
- Use bullet points
- Focus on AWS (ALB, ECS, VPC, IAM)
- Ignore Terrascan scan_errors
- Do NOT repeat raw JSON
- Verdict must strictly follow the Decision Policy

Findings:
{json.dumps(findings, indent=2)}
"""

This logic ensures:

• Security is enforced pragmatically

• No false rejections for minor issues

• HTTPS is mandatory for approval

• Clear, actionable feedback for developers

🔹 iam.tf

IAM roles and policies are defined here:

• Lambda is granted access to Secrets Manager

• ECS task role attaches:

  • AmazonECSTaskExecutionRolePolicy

This allows ECS to pull images, write logs, and function correctly.

🔹 ecs.tf

This is where the Mario game comes to life:

• ECS task definition using Fargate

• Docker image for Super Mario Bros

• ECS service to keep the task running

Fully serverless. No EC2 management required.

🔹 alb.tf

To expose the application publicly:

• Application Load Balancer

• Listener on port 80 (initially)

• Target group pointing to ECS tasks

Later, we enhance this with HTTPS + ACM, making the setup production-ready.

🚀 Provisioning the Infrastructure

Before running Terraform, we need to create the S3 bucket for state storage:

aws s3 mb s3://pravesh-terraform-mario-state

If you see BucketAlreadyExists, simply:

• Update the bucket name in backend.tf

• Re-run the command with a unique name

Now initialize Terraform:

cd terraform
terraform init

Gemini API Key Setup

Head over to Google AI Studio and generate a free Gemini API key.

Once you have it, keep it safe well pass it dynamically to Terraform.

Plan & Apply

Preview the infrastructure:

terraform plan -var="gemini_api_key=<YOUR_GEMINI_API_KEY>"

Review the plan and then deploy:

terraform apply -var="gemini_api_key=<YOUR_GEMINI_API_KEY>" -auto-approve

Provisioning takes around 57 minutes, mainly due to ALB setup.

🎮 Final Result

Once Terraform finishes:

• Copy the ALB DNS name from the outputs

• Open it in your browser

🎉 You should now see the Super Mario Bros game running on ECS, backed by a serverless AWS architecture and guarded by an AI-powered DevOps review system.

🌟 Terraform AI Review Agent in Action

Now comes the most exciting part seeing the Terraform AI review agent in action.

Lets simulate a real-world scenario by making a small change to our infrastructure code and opening a Pull Request. As soon as we do this, our GitHub Actions workflow will automatically kick in and run the AI-based review.

Before that, you need to add your AWS Access key and Secret Access key in your secrets of the repo. If you dont know how to do that, follow this guide and do the step 1 only, make sure you select the ai-devops-projects repo, not the nginx-redis-node.

Triggering the AI Review

Make a minor change in the Terraform code and raise a Pull Request. Once the pipeline runs, youll notice that the workflow fails .

Why did this happen?

If you check the Violation report, youll see that the AI agent rejected the changes. The reason is simple and important:

• Three MEDIUM-severity issues are related to the Application Load Balancer

• Our application is currently running only on HTTP

• Running production workloads over HTTP is not secure

Because our AI agent follows a strict policy (defined in the Lambda prompt), the absence of an HTTPS listener on the ALB results in a PR rejection.

This is exactly how a real-world AI-powered infrastructure gate should behave.

Fixing the Issue: Enabling HTTPS 🔒

To resolve this, well enable HTTPS by creating an ACM certificate and updating our ALB configuration.

Step 1: Update Security Group Rules

Inside security.tf, uncomment the ingress rule for port 443 so that HTTPS traffic is allowed.

Step 2: Enable HTTPS Listener on ALB

Open alb.tf and do the following:

• Uncomment the aws_lb_listener "https" block

• Uncomment the ACM certificate resource

• Remove the the existing app_listener (HTTP listener)

This ensures HTTP is no longer used for forwarding traffic directly.

Step 3: Update Domain Name in ACM Certificate

Inside the ACM certificate resource:

• Replace praveshsudha.com with your own domain name

• This is required because youll be adding CAA and CNAME records for certificate validation

Step 4: Add CAA Record (IMPORTANT )

Before creating the ACM certificate, make sure to add the following CAA record in your DNS provider:

• Type: CAA

• Name: @

• Flag: 0

• Tag: issue

• CA Domain: amazonaws.com

• TTL: Default

Important: Add this CAA record before applying Terraform, otherwise ACM certificate creation may fail.

Step 5: Enable ACM Output

In outputs.tf, uncomment the output block for acm_certificate_arn.
This will help us fetch validation details later.

Step 6: Apply the Changes

Run the following command:

terraform apply --var="gemini_api_key=<YOUR_GEMINI_KEY>" --auto-approve

This will:

• Create the ACM certificate

• Add an HTTPS listener to the ALB

Once completed, Terraform will output the ACM certificate ARN.

Step 7: Validate the ACM Certificate

Use the ARN and run:

aws acm describe-certificate \
  --certificate-arn arn:aws:acm:us-east-1:<ACCOUNT_ID>:certificate/<CERT_ID>

From the output:

• Copy the CNAME name (only up to mario, not the full domain)

• Copy the CNAME value

Add this CNAME record to your DNS provider.

Within a few minutes, the certificate status will change to ISSUED .

Step 8: Point Your Domain to the ALB

Now create a DNS record:

• Type: CNAME

• Name: mario

• Target: <YOUR_ALB_DNS_NAME>

• TTL: Default

After a few minutes, your application will be live at:

👉 https://mario.your-domain.com

Re-running the AI Review

Now that HTTPS is enabled, lets test the AI agent again.

Run the following commands:

git checkout -b test
git add outputs.tf security.tf alb.tf
git commit -m "testing ai-agent-workflow"
git push origin test

Go to your GitHub repository and open a Pull Request.

This time:

• GitHub Actions runs successfully

• Terrascan reports are generated

• Gemini analyzes the findings

• AI agent APPROVES the PR

🌟 Cleaning Up Resources

Once youre done experimenting with the project, its very important to clean up all the resources to avoid any unnecessary AWS charges.

Follow the steps below in order to safely delete everything we created.

Step 1: Destroy Terraform Resources

First, navigate to the terraform directory and run:

terraform destroy --auto-approve --var="gemini_api_key=<YOUR_GEMINI_KEY>"

This command will:

• Terminate ECS services and tasks

• Delete the Application Load Balancer

• Remove Lambda functions and IAM roles

• Clean up networking components like VPCs, subnets, and security groups

Step 2: Delete the Terraform State Files from S3

Once Terraform has destroyed all the resources, delete the remote state files stored in S3.

aws s3 rm s3://pravesh-terraform-mario-state --recursive

This removes all objects inside the bucket, including the Terraform state file.

Step 3: Remove the S3 Bucket

Finally, delete the empty S3 bucket:

aws s3 rb s3://pravesh-terraform-mario-state

🌟 Conclusion

This project goes far beyond deploying a Super Mario game on AWS it represents how modern DevOps is evolving with AI and serverless architectures.

By integrating Terraform, GitHub Actions, Terrascan, and Gemini, we built an AI-powered Terraform review agent that acts as a real CI/CD security gate. Every infrastructure change is evaluated based on risk, not guesswork. The AI summarizes security findings, suggests concrete remediations, and makes approval decisions that closely resemble how a senior DevOps engineer would review production infrastructure.

On the infrastructure side, we embraced a serverless-first approach using AWS ECS Fargate, Lambda, ALB, and managed cloud services. This setup reflects real-world architectures used in production today scalable, cost-efficient, and operationally simple, without managing servers manually.

The key takeaway from this project is clear:
AI in DevOps is not about replacing engineers its about empowering them.
By automating repetitive infrastructure reviews, we save valuable engineering hours, reduce human errors, and ship changes with higher confidence and security.

I highly encourage you to fork the repository, experiment with breaking changes, tune the AI decision thresholds, and extend this project further. This is just the beginning of what AI-assisted DevOps can achieve.

Happy building 🚀

🔗 Connect with me

• LinkedIn: https://www.linkedin.com/in/pravesh-sudha/

• Twitter / X: https://x.com/praveshstwt

• YouTube: https://www.youtube.com/@pravesh-sudha

• Blog: https://blog.praveshsudha.com

If this project helped you learn something new, feel free to share it with your network it truly helps a lot

Welcome Devs to another interesting blog from my side!
Its been a while since I first connected with Cognee, and exactly a month ago I actually built a Cognee Starter application from scratch using Flask and deployed it on AWS ECS using Terraform. If you havent checked it out yet, heres the link to that build youll enjoy it:

Since then, the Cognee team has been on fire. Their GitHub repo recently crossed 10K+ stars (absolutely deserved 🎉). And staying true to the momentum, they came up with a fun little community event the Secret Santa Mini Challenge.

So for this challenge, I decided to build something a bit unique
An Emotion-Aware Secret Santa powered by Gemini 2.5 Flash, with Cognee acting as the memory layer holding everything together.

How the Idea Hit Me 🤯 And Why Emotions Matter in Secret Santa

After going through the rules and criteria of the challenge, I started brainstorming ideas and suddenly something clicked on a very personal level.

In my friend group, Im the delightful one
Happy for no absolute reason, just vibing, giggling, randomly remembering something from Kevin Hart Special 😂

But my friends?
Total opposite personalities:

• One is stressed 24/7 because of career pressure

• Another is moody, unpredictable like Mumbai weather

• And the last one is the chill guy, relaxed in literally every situation

Reflecting on that, I thought:
Why not create a Secret Santa that understands emotions the same way we understand each other?

A Secret Santa that:

• Reads how each friend is feeling

• Understands their energy, mood, and stress

• Pairs them up based on emotional compatibility

• And even helps choose a meaningful gift

Thats how Emotion-Aware Secret Santa was born.

How It Works 🧠🎁 Turning Feelings Into Smart Gift Matches

Each friend gives:

• Their name, and

• A short description of their mood, week, stress level, or personality

For example:

• Alice is overwhelmed with work and feeling stressed.

• Bob had a great week and is feeling positive and energetic.

These tiny descriptions become the foundation for the AIs reasoning.

🧩 Step 1 Storing the emotional descriptions with Cognee

Each user description is added into Cognees memory layer using:

cognify.add(...)

Then using:

cognify()

Cognee processes all the data with Gemini, building:

• Semantic links

• Entities

• Relationships

• A mini knowledge graph

• Embeddings

(Ive shown this visually in my previous video its super cool to watch.)

🧠 Step 2 Cognee asks the right question

Cognee then asks:

What is the emotional state or mood of Alice?

Using RAG_COMPLETION, Gemini returns refined emotional states like:

• stressed

• excited

• lonely

• happy

• tired

🎅 Step 3 AI-Powered Secret Santa Pairing

Now the fun logic:

• Cognee assigns Secret Santa pairs

• Makes sure no one gets themselves

• And suggests a gift based on emotion

Gift suggestions are generated using a local gift dictionary (0 extra AI cost because while testing I hit the Gemini daily quota twice 💀😂).

🎉 Step 4 The Big Reveal

Finally, the program prints a beautiful Secret Santa reveal:

• Who is gifting whom

• Why they were paired

• And what gift matches their emotional state

Simple, wholesome, and powered by Cognees memory + Geminis reasoning.

Try It Yourself 🎄 Run the Emotion-Aware Secret Santa on Your Machine

Ive open-sourced the entire project so you can explore, modify, and have fun with it.
The code is available here:

👉 GitHub Repo: https://github.com/Pravesh-Sudha/secret-santa-cognee

Clone it to your system and youre ready to get started.

🔑 Step 1 Get Your Gemini API Key

To run this project, youll need a Gemini API key.
The good news? Google AI Studio gives you one for free.

Once you have your key:

1. Inside the project directory, create a .env file

2. Copy everything from .env.example

3. Replace the values of:

   • LLM_API_KEY

   • EMBEDDING_API_KEY
     with your Gemini key

And boom the setup is done.

🔧 Step 2 Install Dependencies

Inside your project directory, run:

uv sync

This will install all required dependencies cleanly.

📝 Step 3 Customise Your Friends & Gifts

You can now explore the code and make the project your own:

👥 Add your own friends

Open:

data/friends.json

Add your friends and their mood descriptions.
(Tip: try to keep it max 4 friends, otherwise you may hit the Gemini daily quota like I did 😭😂)

🎁 Customise the gifts

Inside:

gift_gen.py

You can update gifts for each emotion to make them more fun, personal, or chaotic your call.

Step 4 Run the Project

Once everything is set up, run:

uv run main.py

The program takes around 23 minutes, and then

🎉 You get a full Secret Santa reveal right in your terminal!

• Who got whom

• Their emotional reasoning

• And the perfect gift suggestion

All powered by Cognee + Gemini.

NOTE:
Initially, I planned to generate gifts using Gemini too
but Geminis Requests per Minute limit looked at me and said:
Not today, brother.

So I switched to a local gift list zero extra AI cost, much more reliable.

📽 Video Demonstration

🎄 Conclusion Building with Cognee Is Just Too Much Fun

This Secret Santa Mini Challenge by Cognee was the perfect excuse to experiment, break things, fix things, hit API limits twice 😭, and eventually build something that felt genuinely personal.

Using Cognee as the memory layer + Gemini 2.5 Flash for reasoning turned a simple holiday tradition into a small emotionally aware AI system and honestly, thats the kind of playful innovation that makes me love building these projects.

If you try it out, tweak it, or turn it into something wild and creative, Id genuinely love to see it.
And big shoutout to the Cognee team for organizing such a wholesome challenge and continuing to ship amazing updates to the ecosystem.

More AI projects, more experiments, and more community fun coming soon.
Till then keep building, keep learning, and keep vibing.

🌐 Connect With Me

If you enjoyed this project or want to follow my DevOps + AI journey, find me here:

• LinkedIn: https://www.linkedin.com/in/pravesh-sudha/

• Twitter/X: https://x.com/praveshstwt

• YouTube: https://www.youtube.com/@pravesh-sudha

See you in the next build! 🚀

Today, were not spinning up infrastructure, writing HCL, or fixing a broken state file (for once 😅).
Instead, were looking back at the last 8 monthsa journey filled with learning, building, breaking things, fixing them again, and slowly becoming that Terraform guy in my circle.

It all started in April 2025, when one of my LinkedIn buddies shared that he got selected as a HashiCorp Ambassador.
That was the first time I genuinely thought:
Damn, this looks exciting. Why dont I aim for it too?

Just a month before thatin March 2025I had been selected as an AWS Community Builder, and around the same time, I launched my YouTube channel to share DevOps and Infra-as-Code tutorials with the world.
Slowly, people started watching, sharing, and sending messages saying they actually deployed things using my guides.

So officially, in May 2025, I decided:

🔥 Im going to prepare for HashiCorp Ambassador 2026. Lets do this seriously.

Fast-forward 8 months
Ive built multiple Terraform projects, contributed to open-source repos, and crossed:

• 20,000+ views on my blogs

• 16,000+ views on YouTube

• 400+ awesome subscribers

More importantly, I went from Terraform looks complicated to Terraform is my comfort zone.

And today, Im sharing the exact resources, projects, repos, and lessons that helped me go from Zero Hero in Terraform and will help you too.

1. Building a Strong Terraform Foundation

Even though I already had some familiarity with Terraform (thanks to scattered YouTube videos and random experiments), I decided to start from absolute zerobecause if Im going to teach something, I need to understand it deeply myself.

So the first thing I created was a Getting Started with Terraform guide.
This wasnt just another intro blog. My goal was to help beginners understand:

• What Terraform really is

• Why DevOps engineers rely on IaC

• How providers work

• Structuring a project with main.tf, variables.tf, outputs.tf

• What configuration means in practice

Basically, the fundamentals you must know before touching any cloud resource.
If you are new, you can read the same guide here:
👉 Getting Started with Terraform A Beginners Guide

Once the basics were sorted, I went deeper into the most important part of Terraform
the thing that causes 90% of people stress when it breaks:

The Terraform State File

Where to store it?
How to keep it safe?
How to ensure teams dont overwrite each others state?

I wrote a complete blog explaining how to use AWS S3 + DynamoDB as a rock-solid remote backend for production-grade Terraform.
I even created a YouTube demo for those who prefer watching over reading.

📘 Blog: Where Should You Store Terraform State Files for Maximum Efficiency?
🎥 Video: Terraform Remote Backend on AWS (S3 + DynamoDB)

These two resources became the backbone of my Terraform foundationnot just for me, but for everyone following along.
Before I moved to advanced projects, pipelines, and multi-environment infra this foundation is what made everything else 10 easier.

2. Core Concepts & Best Practices (The Part Everyone Skips But Shouldnt)

Once the foundations were clear, I moved into the phase where most beginners either get overwhelmed or fall in love with Terraform.
For me, this was the moment things clicked.

Because understanding Terraform is not just learning commands its learning structure, security, and scalability.

a) Terraform Modules The Secret Sauce

One of the first Aha! moments for me was understanding modules.
If theres one thing that separates beginners from pros, its this.

Modules teach you:

• How to avoid repetitive code

• How to scale infra easily

• How to structure projects cleanly

• How teams collaborate efficiently

I wrote an in-depth blog breaking this down and also created a complete YouTube walkthrough.

📘 Blog: Terraform Modules The Secret Sauce to Scalable Infrastructure
🎥 Video: Terraform Module Explained with Demo

Learning modules improved how I thought about every project afterward.

b) Terraform Security Practices (Because IaC Without Security is Just a Script)

Infrastructure automation is amazingbut it also means if you make a mistake, you can take down everythinginstantly.

So I created a dedicated guide covering the Top 5 DevSecOps-focused Terraform security practices, including:

• Provider credential validation

• Role-based access

• Scanning Terraform code for vulnerabilities

• Remote backend security

• Avoiding manual state file edits (my favourite rule 😅)

📘 Blog: Terraform Meets DevSecOps 5 Security Practices You Cant Ignore
🎥 Video: Terraform Security Best Practices

This was the point where both my blog and channel started gaining real tractionpeople were searching for practical, real-world Terraform security advice.

c) Terraform Workspaces The Most Underrated Feature Ever

Workspaces are like that quiet student in class who actually knows everything.

Nobody talks about them
until they see how powerful multi-environment deployment becomes with a single command:

terraform workspace select dev
terraform workspace select prod

Workspaces changed how I approached multi-environment infra forever.

📘 Blog: Terraform Workspaces & Multi-Environment Deployments
🎥 Video: Complete Workspace Tutorial on AWS

d) Terraform Meets Ansible (IaC + Configuration Management = 🔥)

The next logical step was learning how Terraform works with configuration management tools.

Terraform builds the infra.
Ansible configures what lives inside the infra.

I created a hands-on guide where I showed exactly how Terraform provisions servers and Ansible configures them a production-ready combo.

📘 Blog: Terraform Meets Ansible Automating Multi-Environment Infra on AWS
🎥 Video: Terraform + Ansible Full Demo

This section was a turning point in my portfolio because it bridged two worldsInfra-as-Code and Configuration Management.

3. CI/CD, Automation & Modern DevOps (Where Everything Comes Together)

After covering modules, security, workspaces, and multi-environment provisioning, it was time to bring Terraform into the real world
the world of automation, pipelines, and DevOps workflows.

I had already used Terraform with Jenkins and even GitLab CI/CD in my previous projects (outside this series), so this time I wanted to do something fresh.

And whats more modern DevOps than using GitHub Actions?

So I decided to build a real project that connects:

• Terraform

• GitHub Actions

• AWS

• And a production-style multi-component application (Node.js + Redis + Nginx)

The Project: Request Counter App Deployment on AWS

This wasnt just a hello world project.
It involved:

• A Node.js API handling increment-count logic

• Redis to store the counter

• Nginx as a reverse proxy

• Terraform to provision AWS infra

• GitHub Actions to automatically deploy everything on push

In short, a full Infrastructure + Application + CI/CD pipeline the kind of thing you actually do in real companies.

I documented the entire workflow so anyone can recreate it step-by-step.

📘 Blog: CI/CD for Terraform with GitHub Actions Deploying a Node.js + Redis App on AWS
🎥 Video: GitHub Actions + Terraform Full Pipeline Demo

This project helped me understand how Terraform behaves inside a pipeline: the checks, the backend locks, the state consistency, the secret management all of it.
And more importantly, it leveled up my DevOps portfolio significantly.

Real-world + automated + cloud-native = a perfect trifecta.

4. Cloud-Native & Multi-Tier Application Deployments (Intermediate Projects)

Once I became comfortable with Terraform fundamentals, DevSecOps practices, and CI/CD automation, it was time to step into the cloud-native world where real production systems live and breathe.

This phase of my journey pushed me out of my comfort zone, because now I wasnt just creating small demos.
I was building multi-tier, scalable, mission-critical infrastructures the kind youd actually find in modern companies.

a) Deploying a Three-Tier Application on AWS EKS (with Best Practices)

Kubernetes + Terraform is a whole universe on its own.
So to challenge myself, I decided to deploy a complete three-tier application on AWS EKS, fully automated using Terraform following all real-world best practices.

This included:

• VPC with subnets

• Managed node groups

• Ingress controllers

• Load balancers

• Namespace separation

• Secrets + configs

• And a proper service-to-service communication workflow

It was one of the most complex setups Id built at that point and the most rewarding.

📘 Blog: Deploy a Three-Tier Application on AWS EKS using Terraform (Best Practices)
🎥 Video: EKS + Terraform Deployment Tutorial

(This video crossed 1,000+ viewsfirst big milestone!)

b) Deploying a Highly Scalable & Available Django Application (AWS + Terraform)

After EKS, I wanted to explore another real-world architecture something more traditional, but equally production-grade.

So I built a highly scalable Django application hosted on AWS using Terraform.
This project included all the standard AWS building blocks youd expect in a real enterprise setup:

• RDS for relational database

• Secrets Manager for secure credentials

• Application Load Balancer

• Auto Scaling Group

• EC2 instances for compute

• Private/public subnets

• Proper network isolation and high availability

This architecture reflected how an actual company would deploy a Python backend in production.

📘 Blog: Deploying a Highly Scalable Django Application on AWS with Terraform
🎥 Video: Django + AWS + Terraform Full Architecture Demo

This stage of my Terraform portfolio strengthened my confidence in handling full-stack cloud-native deployments not just isolated resources.

5. Game Deployments & Creative Infra Projects (The Standout Pieces)

Now, lets talk about the most fun part of my Terraform journey the projects that truly made my portfolio stand out.

Ive been a gamer since childhood, and fun fact:
I actually bought my PS5 using the prize money I won as the Dev.to Runner-H Challenge Winner (Scolded by my Dad for this unjust purchase, but HELL YEAH!!, it was worth it.)
So, naturally, I decided to merge my love for gaming with my DevOps career.

The result?
Some of the most unique, creative, and highly engaging Terraform projects Ive ever built.

a) Deploying Super Mario Bros on AWS EKS (Award-Winning Project)

This one will always be special.

I deployed the classic Super Mario Bros game on AWS EKS using Terraform complete with pods, services, ingress, and Kubernetes best practices.

This project wasnt just a hit among developers
it actually helped me win the AWS Containers 4x4 Challenge, and I received some amazing premium swags from the community.

📘 Blog: Deploy Super Mario on AWS EKS using Terraform (Step-by-Step)
🎥 Video: Super Mario on Kubernetes Demo

b) Deploying Tetris on AWS ECS with Terraform

After nailing Mario, I didnt want to stop.

Next, I deployed Tetris this time using Amazon ECS with Terraform.
This project explores how containerized applications run on ECS Fargate, how services scale, and how ALBs route traffic.

📘 Blog: How to Deploy a Tetris Game on AWS ECS with Terraform
🎥 Video: Tetris on ECS Full Walkthrough

c) Deploying Cognee AI Starter App on ECS (Collaboration Project)

One of the most exciting collaborations of my journey was with Cognee AI the memory layer of AI Agents.

I built a Flask-based Cognee Starter Application from scratch, containerized it, and deployed it on AWS ECS using Terraform as the IaC backbone.

This project taught me a lot about real product deployment workflows, container orchestration, and DevOps collaboration.

📘 Blog: Deploying Cognee AI Starter App on ECS with Terraform
🎥 Video: Cognee AI Deployment Demo

d) Deploying an Amazon Clone on AWS (Amazon on AWS Meta Enough?)

To wrap up this creative phase, I decided to do something hilarious and ambitious:

Deploying an Amazon Clone on AWS itself.
Yes Amazon on AWS. A true full-circle moment. 😂

This project used:

• Jenkins for CI/CD

• Terraform for infra

• AWS services like EC2, ALB, ASG, RDS, VPC

• And a full clone app architecture setup

📘 Blog: Deploy an Amazon Clone on AWS (Complete Guide with Jenkins + Terraform) 🎥 Video: Amazon Clone Deployment Tutorial

These projects became the highlights of my Terraform portfolio the kind that make recruiters pause, smile, and think:
Okay, this person actually enjoys building stuff.

6. Reflection & Growth (The End of This Portfolio But Not the Journey)

And now here we are.
Eight months later.
Countless blogs, videos, deployments, wins, failures, swags, and late-night debugging sessions later I finally paused and asked myself a very simple question:

If a complete beginner asked me for guidance today what would I say?

Because when youre learning something as powerful as Terraform, the hardest part isnt understanding .tf files
its navigating the early confusion, the overwhelming docs, the trial-and-error, and the mistakes we all make.

So to answer that question, I created a dedicated piece:
the top 5 mistakes beginners make while learning Terraform and how to avoid them.

📘 Blog: Dont Touch Terraform Before Avoiding These 5 Rookie Mistakes
🎥 Video: Top 5 Beginner Mistakes in Terraform

This video and blog were my way of giving back to anyone starting the same journey I took in April 2025.
If I could go back in time, this is exactly what I would hand to myself.

The Terraform Guide (Everything in One Place)

To make things easier for learners, I bundled every single blog foundations, modules, state files, workspaces, EKS, ECS, CI/CD, everything into a clean, structured series:

📚 Terraform Guide Series: All blogs in one place

This series now stands as a complete Zero-to-Hero path for anyone wanting to master Terraform using real projects.

Terraform Playlist (All Video Demonstrations)

And for visual learners, I created a dedicated playlist on YouTube containing every demo, from foundational projects to Kubernetes deployments and game infra:

🎥 Terraform Project Playlist

Conclusion

As I wrap up this 8-month Terraform journey, one thing is clear: this portfolio isnt just a collection of .tf files its a reflection of growth, discipline, creativity, and identity.

When I started back in April 2025, I didnt know Terraform beyond the basics. I simply made a decision in May 2025:
I will learn this tool properly and build something meaningful.

Eight months later, heres what that decision turned into:

• I learned Terraform from scratch

• Built real-world, production-style cloud infrastructures

• Won community challenges, including AWS Containers 4x4

• Collaborated on AI projects, like the Cognee AI deployment

• Published blogs that crossed 20,000+ views

• Grew my YouTube channel to 16,000+ views and 400+ subscribers

• And most importantly, built projects that genuinely reflect who I am as a DevOps engineer

This journey taught me that you dont need a perfect starting point you just need a consistent one.
It showed me that sharing your learning publicly builds connection, credibility, and confidence.
And it proved that passion projects (like deploying Super Mario and Tetris using Terraform) can teach you more than any textbook ever will.

If youre just beginning your Terraform journey, Ill leave you with this:

Start small. Stay consistent. Build publicly.

Because the internet remembers builders not perfectionists.

Thank you for reading my story.
I hope this motivates you to start building your own.

📌 Connect With Me

🔗 LinkedIn: https://www.linkedin.com/in/pravesh-sudha/
🐦 Twitter/X: https://x.com/praveshstwt
📺 YouTube: https://www.youtube.com/@pravesh-sudha
🌐 Website/Blogs: https://blog.praveshsudha.com

Lets keep learning and building together.

Welcome back, Devs!
A few weeks ago, I shared 5 Best Security Practices for Terraform. That guide was more for folks who already work with Terraform day in and day out the ones managing infra at scale, reviewing modules, and pushing changes through CI/CD pipelines.

But what about the beginners?

The ones who just wrapped up the basics of DevOps Linux, Networking, Docker, Git and are now stepping into the cloud world. For them, Infrastructure as Code can feel well, intimidating at first. Terraform looks simple from the outside, but when you actually start writing configurations, the learning curve hits hard. And its totally normal IaC is a tough nut to crack initially.

So to make your journey smoother and confusion-free, Im back with todays blog, where well break down the Top 5 Mistakes Beginners Make While Learning Terraform and how you can avoid them.

Without further ado lets get started!

🌟 Before We Dive In Do This First

Before we dive deep into the mistakes, lets get one thing out of the way make sure youve got Terraform ready on your system. No matter how good the guide is, nothing makes sense unless youve actually installed the CLI and can run those sweet terraform init and terraform apply commands.

Since Im an AWS Community Builder, I usually stick to AWS as my cloud provider for demos and explanations. If youre following along, youll need to connect Terraform to your AWS account. You can do that in two ways:

1. Export AWS credentials directly in your terminal
   (AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY)
   Works fine, but not the best option for long-term use.

2. Install the AWS CLI (recommended)
   This is cleaner, more secure, and helps you manage multiple profiles easily.
   Just create an IAM user with the right permissions and run aws configure.

If you dont know how to set that up dont worry. Ive already covered the entire process in my Beginners Guide to Terraform. You can check it out here:
👉 https://blog.praveshsudha.com/getting-started-with-terraform-a-beginners-guide#heading-step-1-install-the-aws-cli

Once your CLI and AWS credentials are set, youre all ready to explore the mistakes beginners make and how you can avoid them.

Mistake 1: Treating Terraform Like a Scripting Tool

This is hands-down the most common beginner mistake.

Most folks who get introduced to Terraform have already touched at least one programming language Python, Go, Rust, maybe even Bash scripting. And because of that prior experience, they naturally assume Terraform will behave the same way:

I wrote line 1 first, so Terraform will execute that first right?
Nope. Not at all.

Terraform is not a scripting tool.
It doesnt run your code line-by-line.
It doesnt care about the order in which you wrote your resources.

Terraform is declarative, not imperative.

Instead of following your code from top to bottom, Terraform reads all the resources in your configuration and builds something called a dependency graph. This graph tells Terraform which resources depend on which other resources and that determines the execution order.

So beginners often get confused:

Why is Terraform not creating things in the order I wrote them?

Because Terraform is smarter than that. It looks at dependencies, not line numbers.

Moral of the story:
With Terraform, you dont explain how to create each step. You only declare what you want like an EC2 instance, security groups, and a VPC and Terraform figures out the how automatically.

Once you understand this mindset shift, Terraform becomes much easier (and honestly, more fun) to work with.

Mistake 2: Hardcoding Everything Instead of Using Variables

If youre anything like me, you love going through official docs whenever you start learning a new tech. And if you visit the Terraform documentation, youll notice something instantly:

Most of their example configurations hardcode values.

Region? Hardcoded.
AMI ID? (Got from data type).
App name, DB name, instance type? All hardcoded.

And honestly that's fine when you're just starting out. Hardcoding helps beginners understand the structure of a resource without worrying about variables, files, or module structures.

But once you get a basic understanding of how Terraform works
hardcoding becomes your worst enemy.

Let me explain with a simple example.

Say youve deployed a two-tier application on AWS your VPC, ALB, EC2, RDS, security groups, everything. Now theres a new feature request, and you dont want to deploy the changes in the same environment.

So you decide to replicate the environment.

If everything is hardcoded, youre now stuck manually changing names and identifiers for every single resource.
Painful.
Time-consuming.
Highly prone to breaking things.

The Fix: Use Variables Like a Pro

Store all your important attributes inside a variables.tf file.
This gives you a clean, centralized location for every configuration value your entire project depends on.

In the above scenario, if your resources use variables, all you need to do is change a value in one place and Terraform will automatically reflect it everywhere.

Heres how you define a variable in Terraform:

variable "instance_type" {
  default = "t2.micro"  # EC2 Instance Type
}

Clean, reusable, scalable and exactly how real-world Terraform is written.

Mistake 3: Mixing Manual Changes Through the AWS Console

This one is a classic beginner trap.

When you provision infrastructure using Terraform, it stores a full blueprint of your resources inside the terraform.tfstate file. This file represents the current state of your infrastructure basically Terraforms memory of what exists.

Now imagine you want to update something small, like renaming an EC2 instance.
You open the AWS Console, click on the instance, change the name, hit save, and boom done.

Simple, right?

Yes. But also very wrong.

Making manual console changes creates something called drift a mismatch between:

• what actually exists in AWS

• what Terraform thinks exists (according to the state file)

This drift becomes a huge headache because Terraform will now get confused about what's changed, what needs to be replaced, or what should not exist at all. For beginners, handling drift is even more overwhelming because it's not always obvious where things went wrong.

Heres the golden rule:

If you provision resources using Terraform,
update or delete them using Terraform only.

Avoid the temptation of "quick fixes" in the AWS Console. A few clicks today can cause hours of debugging tomorrow.

Moral of the story:

Treat Terraform as your single source of truth.
No ClickOps. No shortcuts.

Mistake 4: Ignoring Terraform Resource Dependencies

Terraform is pretty smart when it comes to understanding relationships between resources. It automatically builds a dependency graph to figure out what needs to be created first and what depends on what.
Most of the time, this works beautifully.

But sometimes Terraform needs a little help.

There are scenarios where Terraform cannot infer dependencies on its own especially when two resources dont have an obvious reference to each other. Thats where beginners get stuck.

For example:

• Applying an S3 Bucket Policy before the bucket is actually created

• Applying IAM role attachments before the IAM role itself exists

• Creating a Lambda permission before the Lambda function is ready

In such cases, Terraform might try to create things in the wrong order, leading to errors.

The Fix: Use depends_on Smartly

Terraform gives us an escape hatch the depends_on meta-argument.

With depends_on, you can explicitly tell Terraform:

Hey, this resource should only be created after this other resource is fully ready.

It ensures the correct and predictable order of execution.

Example use case:

resource "aws_s3_bucket_policy" "bucket_policy" {
  bucket = aws_s3_bucket.my_bucket.id

  policy = data.aws_iam_policy_document.example.json

  depends_on = [
    aws_s3_bucket.my_bucket
  ]
}

Now Terraform knows for sure: Bucket first Policy second.

Moral of the story:

Understand how Terraform creates its dependency graph, and use depends_on wisely when Terraform cant figure things out on its own.

Mistake 5: Thinking Terraform Is Only About init, plan, and apply

I still remember when I started learning Terraform.
I watched a bunch of YouTube tutorials, and almost every single one followed the exact same pattern:

• Write config in main.tf, provider in provider.tf, vars in variables.tf, outputs in outputs.tf

• Run terraform init

• Run terraform plan (and lets be honest, most beginners skip this 😅)

• Run terraform apply --auto-approve

• And at the end, terraform destroy

And thats it.
End of tutorial.
Done.
You now know Terraform. 🙃

Except thats not the full story.

Terraform is not just about those three commands. There are many other commands that help you:

• write cleaner, more readable code

• avoid silly mistakes

• prevent accidental deployments

• stick to best practices from day one

But beginners (including me back in the day) completely ignore them.

Here are a few essential ones:

🔹 terraform plan (Seriously, use it)

Before applying, always check the plan.
It shows you what Terraform will create, modify, or delete.
Skipping this step is how disasters happen and trust me, Ive been guilty of this too. 😅

🔹 terraform fmt (Fix your code automatically)

Your code may work, but if it looks like a messy bowl of noodles, nobody wants to maintain it.
terraform fmt formats your HCL beautifully and keeps your files consistent.

🔹 terraform validate (Catch misconfigurations early)

Sometimes your code looks right but contains subtle mistakes wrong types, bad arguments, misplaced blocks, etc.
terraform validate helps detect these before you even think about applying them.

Moral of the story:

Make fmt, validate, and plan a daily habit.
They will save you from so many unnecessary headaches.

🌟 Hands-On: Deploying a Static Portfolio Website on an S3 Bucket

Alright, thats enough theory now lets actually get our hands dirty.
To make everything we discussed more practical, well deploy a static portfolio website on an S3 bucket using Terraform.

The complete code is available here:
👉 https://github.com/Pravesh-Sudha/terra-projects

Once you open the repo, navigate to the terra-mistakes directory. Inside it, youll find multiple Terraform files and a static/ folder. The static directory contains two HTML files:

• index.html your main portfolio page

• error.html fallback page for unexpected errors

Inside the main.tf, were:

• creating an S3 bucket

• enabling static website hosting

• uploading both HTML files

• attaching a bucket policy to allow public access

Youll also notice something important:

• We didnt hardcode values; even in this small project, the bucket name is stored in variables.tf

• We used depends_on so Terraform knows to create the Public Access Block before applying the bucket policy

These are the exact practices that prevent the mistakes we discussed earlier.

Run These Commands to See Everything in Action

Open your terminal inside the terra-mistakes directory and run:

terraform init          # Initialize provider plugins
terraform fmt           # Fix indentation, syntax, and formatting
terraform validate      # Detect any surface-level misconfigurations
terraform plan          # Preview what Terraform will provision
terraform apply --auto-approve   # Deploy the S3 website

Once the resources are created, Terraform will output the Website URL from the output.tf file.

Note:
This project is deployed in us-east-1, and the S3 website endpoint URL is hardcoded for that region.
If you want to deploy in another region, update both the provider and output.tf accordingly.

Now, open the URL in your browser and boom! 🎉
Your portfolio website is live on S3.

Updating the Policy the Right Way

Lets say you want to add delete object permission to the bucket policy.
The beginner instinct?
Head straight to the AWS Console IAM update the policy manually.

But we dont do ClickOps here.

Remember: Terraform is your single source of truth.

So instead, update your HCL code inside the bucket policy:

Statement = [
  {
    Sid       = "PublicReadGetObject"
    Effect    = "Allow"
    Principal = "*"
    Action    = ["s3:GetObject", "s3:DeleteObject"]  # Added delete permission
    Resource  = "${aws_s3_bucket.site_bucket.arn}/*"
  }
]

Then apply the changes:

terraform apply --auto-approve

Terraform will detect the change, update only the policy, and keep everything consistent no drift, no confusion.

To cross check the changes, go to S3 Console Navigate to the bucket and click on Permissions tab and See the Bucket Policy.

Cleaning Up

When you're done experimenting, dont forget to delete everything:

terraform destroy --auto-approve

This removes all AWS resources cleanly so you dont get billed unnecessarily.

🌟 Conclusion

Learning Terraform as a beginner can feel overwhelming not because the tool is hard, but because Infrastructure as Code requires a different mindset. The mistakes we covered today are extremely common, and almost every Terraform practitioner (including me 😅) has made them at some point.

But the good news?

Once you start writing declarative code, using variables wisely, avoiding ClickOps, understanding dependencies, and making fmt, validate, and plan part of your workflow Terraform becomes a powerful, predictable, and enjoyable tool to work with.

Take your time, practice often, and break things safely.
Every small experiment makes you a better DevOps engineer.

If you followed along with the hands-on demo, you now have a static website deployed on AWS using clean, beginner-friendly Terraform. Thats a solid milestone great job, Dev! 🚀

Feel free to explore more projects, improve your configurations, or even contribute to the repo. And if you ever get stuck, you know where to find me.

Connect With Me

If you enjoyed this blog or learned something new, lets connect:

• 🌐 Website: https://praveshsudha.com

• 🐦 Twitter/X: https://x.com/praveshstwt

• 💼 LinkedIn: https://www.linkedin.com/in/pravesh-sudha

• 📺 YouTube: https://www.youtube.com/@pravesh-sudha

Lets keep building, learning, and automating together! 🚀

Welcome, Devs, to the exciting world of Infrastructure and Cloud computing!

If you have been navigating the cloud landscape recently, you have undoubtedly encountered the industry buzzword: Infrastructure as Code (IaC). Simply put, IaC is the practice of specifying your infrastructure requirements in a code format rather than manually configuring servers and networks. This approach is the bedrock of modern DevOps, ensuring reliability and consistency across all your environments.

While there is a robust ecosystem of tools offering IaC capabilitiesincluding Pulumi, Chef, Puppet, and AWS CloudFormationtoday we are zeroing in on the undisputed heavyweight of this segment: Terraform.

If you are just starting out or want to dive deeper into the core concepts, I have covered a lot of ground regarding Terraform in my previous posts. I highly recommend checking out the full series here to get up to speed:

👉 Terraform Series

In today's guide, we are shifting our focus to Optimisation. As your infrastructure grows, so does the complexity and the time required for deployments. We will dive into the best practices to improve the performance of your Terraform configurations, ensuring your infrastructure remains efficient, fast, and secure.

So, without further ado, lets get started!

📽 Youtube Demonstration

🌟 Pre-Requisites

Before we jump into the optimisation techniques, let's ensure you have the necessary tools ready to roll.

As usual, for this guide, you will need the following set up on your machine:

• AWS CLI (Configured with an appropriate IAM user)

• Terraform CLI

• Docker and Docker-compose

If you don't have these installed yet or aren't sure how to configure them, dont worry! I have walked through the entire process step-by-step in my previous guide. Just follow the instructions there, and you will be good to go:

👉 How to Install AWS CLI and Terraform

Once you are all set up, proceed to the next section where we start optimizing!

🌟 How to Optimise Terraform

Now that we are set up, let's get into the meat of the matter. Here are 5 Best Practices to turbocharge your Terraform performance.

1. Use Remote Backends for State Management

By default, Terraform stores its "state" (the file that maps your code to real-world resources) locally on your machine (terraform.tfstate). While this works for solo side projects, it kills performance and collaboration in real teams.

The Fix: Store your Terraform State file in a remote location, preferably AWS S3.

• Why? It prevents conflicts when multiple team members (Dev, Stage, Prod) try to apply changes at the same time.

• The Performance Boost: Moving to a remote backend can improve I/O performance by 10-30% for large state files.

• State Locking: When configured correctly with State Lock enabled, you ensure that no two people can write to the state simultaneously. (Note: While traditionally done with DynamoDB, ensuring your backend supports locking is key to preventing corruption).

How to do it:

You simply need to create an S3 bucket and reference it in your configuration:

terraform {
  backend "s3" {
    bucket       = "my-terraform-state-bucket"
    key          = "prod/terraform.tfstate"
    region       = "us-east-1"
    use_lockfile = true
  }
}

Its that simple!

2. Modularise Your Code

Writing one giant main.tf file is a common rookie mistake. If you are defining a secure EC2 instance, you likely also need a VPC, Security Groups, Subnets, etc. Lumping this all together makes Terraform work harder to calculate dependencies.

The Fix: Break your code down into Modules.

Instead of rewriting the same sub-components for every service, use the DRY (Don't Repeat Yourself) principle. Create a module for your network stack, a module for your compute stack, etc.

• Benefit: This reduces complex dependency graphs.

• Speed: It enables better parallel execution because Terraform can process distinct modules concurrently.

3. Unlock True Parallelism

Terraform is capable of walking through the dependency graph and creating non-dependent resources at the same time. However, the default setting is often too conservative.

The Fix: Adjust the -parallelism flag.

By default, Terraform limits concurrent operations to 10. For modern systems, this is quite low.

• Guideline: You can safely increase this to 30-100 for normal development changes.

• Constraint: Don't go too high, or you might hit AWS API rate limits (throttling). Aim for roughly 512MB of system memory per 1,000 resources.

The Result:

Using this flag can cut down plan and apply times significantlypotentially dropping a 3-5 minute operation down to just 30-60 seconds.

terraform apply -parallelism=30

4. Optimise Provider Configuration

Sometimes, Terraform performs checks that aren't strictly necessary for every single run, especially in non-production environments.

The Fix: Tune your AWS Provider.

In your Testing, Staging, or Dev environments, you can tell the provider to skip certain validations to save time on API calls. You can also configure retry behaviors to handle network blips gracefully without failing the whole run.

Example Configuration:

provider "aws" {
  region                      = "us-east-1"
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  max_retries                 = 5
}

Note: While this is a great time-saver for Dev/Test environments, check your organization's policy before using strict validation skipping in Production.

5. Use Resource Targeting

Imagine you have a massive Terraform setup managing an S3 bucket, its Website Config, ACL rules, and specific objects (like index.html). If you just want to rename the bucket, Terraform might try to refresh the state of every single component related to it.

The Fix: Use the -target flag.

This allows you to apply changes only to specific resources, ignoring the rest of the infrastructure. It acts like a surgical strike rather than a carpet bombing.

• The Impact: This limits the operation's scope and prevents the recreation of dependent components that haven't changed.

• Efficiency: For targeted fixes, this can reduce execution time by 85-95%.

# Only apply changes to the specific bucket, ignoring the rest
terraform apply -target=aws_s3_bucket.my_website_bucket

🌟 Practical Demonstration

Enough with the theory! Let's get our hands dirty with some real Terraform code and the AWS Console.

To demonstrate these optimizations, we are going to deploy a Request Counter Application on AWS Elastic Beanstalk. This isn't just a "Hello World"; it is a multi-container setup running on an ECS Cluster involving:

1. Nginx: As a reverse proxy and load balancer.

2. Node.js: The application server (running two instances).

3. Redis: To store the request count data persistently.

Step 1: Create Your ECR Repositories

First, we need a place to store our Docker images. We will create two Public Repositories in AWS Elastic Container Registry (ECR)one for our web app and one for our custom Nginx image.

Run the following commands in your terminal:

aws ecr-public create-repository --repository-name nginx-node-redis-web
aws ecr-public create-repository --repository-name nginx-node-redis-nginx

Step 2: Get the Code

The complete source code for this project is available on my GitHub.

1. Fork the repo.

2. Clone it to your local system.

3. Open it in your code editor (like VS Code).

👉 GitHub Repo: Nginx-Node-Redis Project

Step 3: Test Locally (Get the Vibe)

Before we deploy to the cloud, lets make sure it works on your machine. Inside the project directory, run:

docker-compose up --build

This will spin up the Nginx, Redis, and two Web containers. Open your browser and go to http://localhost:8080.

You should see a nice Request Counter app. Every time you refresh, the counter increments, and you will see the server ID toggle between web1 and web2 as Nginx load-balances the traffic.

Once you are satisfied, press CTRL+C in your terminal to stop the application.

Step 4: Build and Push to Cloud

Now, let's move these images to AWS.

1. Navigate to the web directory in your terminal.

2. Go to AWS Console ECR Public Repositories.

3. Select nginx-node-redis-web and click the "View Push Commands" button.

4. Execute those commands one by one to build and push your web image.

Repeat for Nginx:

1. Navigate to the nginx directory.

2. In the AWS Console, select nginx-node-redis-nginx.

3. Click "View Push Commands" and execute them to push your Nginx image.

Step 5: The Terraform Configuration

Navigate to the terra-performance-config directory. This is where we have applied all the optimizations we discussed earlier:

• provider.tf: We are skipping unnecessary validation checks (skip_metadata_api_check, etc.) to speed up the provider.

• backend.tf: We are using an S3 bucket to host our state file remotely and enabling lock_file to prevent write conflicts.

• main.tf: We are using a Terraform Module for Elastic Beanstalk. It utilizes a Dockerrun.aws.json file (similar to docker-compose but for ECS) which is uploaded to S3 and referenced in the module.

• dockerrun.aws.json: Skelton of your Docker containers, make sure to replace the image URI of web1, web2 and Nginx, you can find it in your ECR public Repo.

• variables.tf: Contains our standard configuration using the default VPC and subnets in us-east-1.

CRITICAL: Architecture Check (ARM vs AMD)

I built the default Docker images on a MacBook Air M3 (ARM64 architecture).

If you are using Windows or Linux (Intel/AMD), your architecture is likely AMD64. You must make these two small changes in the code before proceeding, or the deployment will fail:

1. In variables.tf: Change the instance type from t4g.micro (ARM-based) to t3.micro or t3.small.

   

2. In Dockerrun.aws.json: Locate the Redis container definition. Change the image from the specific ARM tag to generic: "image": "redis:alpine".

   

Step 6: Testing Parallelism

Now for the moment of truth. Let's see how fast we can provision this stack using the Parallelism optimization.

Run the following commands:

cd terra-performance-config

# Create the S3 bucket for our artifacts
aws s3 mb s3://pravesh-ebs-terra-performance-bucket

# Initialize Terraform
terraform init

# Apply with high parallelism
time terraform apply --auto-approve -parallelism=30

The Result: Usually, a multi-container Elastic Beanstalk environment takes 10-30 minutes to provision. With -parallelism=30, you will likely see this drop to 3-8 minutes.

Once finished, Terraform will output the Elastic Beanstalk URL. Click it to see your live app!

Step 7: Testing Resource Targeting

Finally, let's look at the power of Resource Targeting.

We have made a small change to the Dockerrun.aws.json file (e.g., adding an Environment Variable).

If we ran a normal apply, Terraform might check every single resource. Instead, we will target only the specific components we are updating:

time terraform apply -parallelism=30 \
  -target=aws_s3_object.dockerrun \
  -target=aws_elastic_beanstalk_application_version.v1 \
  -target=module.elastic-beanstalk-environment \
  -auto-approve

The Result: Without targeting, Terraform might attempt to refresh the state of the entire VPC and Security Group structure. With targeting, this update happens in under a minute.

Cleanup

Don't forget to tear down your infrastructure to avoid unexpected AWS bills!

# Destroy Terraform resources
terraform destroy --auto-approve

# Clean up S3
aws s3 rm s3://pravesh-ebs-terra-performance-bucket --recursive
aws s3 rb s3://pravesh-ebs-terra-performance-bucket

# Delete ECR Repositories
aws ecr-public delete-repository --repository-name nginx-node-redis-web --force
aws ecr-public delete-repository --repository-name nginx-node-redis-nginx --force

🌟 Conclusion

And thats a wrap, folks!

We have successfully journeyed through the landscape of Terraform Optimization. We didn't just learn how to write Infrastructure as Code; we learned how to make it efficient, scalable, and blazing fast.

From moving our state to a Remote Backend to unlocking the speed of Parallelism, and from Modularising our logic to performing surgical strikes with Resource Targetingyou now possess the toolkit to take your DevOps game to the next level.

Remember, optimization isn't just about saving a few minutes here and there. It's about creating a developer experience where feedback loops are short, deployments are reliable, and your infrastructure can scale without becoming a bottleneck.

I hope you enjoyed this deep dive and the hands-on project. Go ahead and apply these techniques to your own infrastructure, and let me know how much time you saved on your next deployment!

If you found this guide helpful or have any questions, feel free to connect with me. I talk about Cloud, DevOps, and everything in between.

🚀 Let's Connect:

💼 LinkedIn: linkedin.com/in/pravesh-sudha

🐦 Twitter/X: x.com/praveshstwt

📹 YouTube: youtube.com/@pravesh-sudha

🌐 Website: blog.praveshsudha.com

Happy Coding and Happy Clouding! 🚀

Welcome to the world of Cloud and Automation, Devs!
Today, were going to explore one of the most powerful and widely used Infrastructure-as-Code (IaC) tools Terraform. In this guide, well learn how to use Terraform workspaces to manage multiple environments seamlessly from Development to Staging and finally Production.

By the end of this blog, youll not only understand how Terraform organizes infrastructure across environments but also see it in action through a hands-on demonstration deploying a static website on Amazon S3 with three isolated environments.

So, without further ado, lets dive in and uncover how Terraform simplifies multi-environment deployments in the cloud.

📽 Youtube Demonstration

💡 Prerequisites

Before we jump into the implementation, lets make sure your setup is ready. Youll need the following tools installed and configured on your system:

• AWS CLI Installed and configured with an IAM user that has full access to Amazon S3. If you dont know how to do that, Follow these steps 1-3 from my Terraform Starter blog. (Just change the Permissions from EC2 to S3FullAccess)

• Terraform CLI Installed and ready to execute Terraform commands.

Once these are in place, youll have everything you need to follow along with the tutorial and deploy your multi-environment infrastructure.

💡 What is Terraform Workspace?

When managing infrastructure with Terraform, its common to work with multiple environments such as Development, Staging, and Production. Each of these environments often requires its own set of resources and configurations. To keep things organized and maintain a clean infrastructure codebase, Terraform provides a powerful feature called Workspaces.

A Terraform Workspace allows you to create and manage separate environments within a single Terraform configuration. Each workspace is associated with its own state file, which means the resources for one environment are isolated from another, even though they share the same configuration files. This makes it much easier to manage multiple deployments all from a single codebase.

When you initialize Terraform for the first time, a default workspace named default is automatically created. Any infrastructure you create without explicitly switching workspaces will live in this default environment. You can then create new workspaces for different environments as needed.

Here are the available Terraform workspace commands:

terraform workspace --help

Usage: terraform [global options] workspace
  new, list, show, select and delete Terraform workspaces.

Subcommands:
    delete    Delete a workspace
    list      List workspaces
    new       Create a new workspace
    select    Select a workspace
    show      Show the name of the current workspace

Each workspace must have a unique name. When you switch between them, Terraform automatically updates the state file to match the selected workspace ensuring your deployments remain consistent and isolated per environment.

💡 Pros and Cons of Using Terraform Workspaces

Before deciding to use Terraform Workspaces for managing your environments, its important to understand both their advantages and limitations. While they provide a convenient way to organize infrastructure, they may not always be the best fit for every use case especially in large-scale or complex deployments.

Pros

• Single Configuration for Multiple Environments
  You can manage multiple environments like Dev, Stage, and Prod using a single Terraform configuration. This reduces code duplication and keeps your setup clean and maintainable.

• Easy Environment Switching
  Workspaces come with a simple built-in command to switch between environments:

    terraform workspace select <name>
  

  This makes moving between environments effortless and consistent.

• Simplifies Non-Production Environment Creation
  You can easily spin up non-production environments such as Development, QA, Beta, or UAT that mirror your production setup often as smaller, scaled-down versions.

• Resource and Variable Isolation
  Each workspace maintains its own state file and can have environment-specific variables, reducing the chance of misconfigurations or accidental resource overlap.

• Ideal for Small to Medium Projects
  For small teams or projects where environments share similar configurations, workspaces provide just the right balance of simplicity and control.

Cons

• Added Complexity for Large-Scale Infrastructure
  As projects grow, managing multiple environments and configurations through workspaces can become cumbersome and harder to maintain.

• Not Fully Isolated
  While each workspace has its own state file, they still share the same backend configuration. Without proper management, this can lead to state conflicts or accidental cross-environment changes.

• Limited for Advanced Use Cases
  Workspaces arent ideal for multi-provider setups or situations where resources need to be shared across environments. In those cases, using separate directories or repositories for each environment is often a better approach.

💡 Practical Demonstration

Now that weve covered the theory, its time to get our hands dirty with a real-world example.
The project well be working on demonstrates how to use Terraform Workspaces to deploy a static website on AWS S3for multiple environments Dev, Stage, and Prod.

You can find the complete code for this project on my GitHub repository:
👉 GitHub terraform-workspace-s3

Navigate to the terraform-workspace-s3 directory, and youll see the following files and folders:

• provider.tf Defines AWS as the cloud provider and specifies the region (us-east-1).

• output.tf Prints the website URL once deployment is complete. The output changes based on the environment.

• main.tf The core of the project. It:

  • Creates an S3 bucket named pravesh-{env}-terraform-workspace-site.

  • Configures it as a static website.

  • Uploads two HTML objects: index.html and error.html.

  • Includes configurations for ownership controls, public access, and bucket policies.

• index/ Contains three subdirectories: dev, stage, and prod.
  Each folder has its own index.html file with environment-specific content.

• create.sh A shell script that automates workspace creation and resource deployment.

• delete.sh A cleanup script that destroys all created resources to prevent unnecessary AWS charges.

Inside the main.tf, weve set locals.env equal to the current workspace name. This dynamic link ensures Terraform automatically detects the environment (Dev, Stage, or Prod) and applies the corresponding configuration.

Once youve cloned the repository locally, open your terminal and run the following commands:

cd terra-projects/terraform-workspace-s3
chmod u+x create.sh
./create.sh

This script will:

1. Create all the required Terraform workspaces.

2. Apply the configuration and deploy the static website for each environment.

If you prefer, you can also run the Terraform commands manually instead of using the script.

(I ran it manually while testing, Here are some Screenshots)

After the deployment completes successfully, Terraform will output a website URL for each environment.
Open the URL in your browser and youll see your static website live, hosted directly from your S3 bucket!

When youre done testing, make sure to delete the resources to avoid unwanted charges.
You can do this easily by running the cleanup script:

cd terra-projects/terraform-workspace-s3
chmod u+x delete.sh
./delete.sh

This will remove all S3 buckets, associated objects, and delete the Terraform workspaces you created.

🧩 Conclusion

As we wrap up this project, weve seen how Terraform Workspaces simplify managing multiple environments like Dev, Stage, and Prod all from a single configuration.
Instead of maintaining separate folders or repos, workspaces help isolate state files while keeping the infrastructure code consistent and scalable.

We also explored how to host a static website on AWS S3 using Terraform, learned about configuring ownership, access policies, and automating deployment across environments.

While Workspaces are great for small to medium-sized projects, they do have some limitations for larger infrastructure setups. Still, theyre a great way to get started with multi-environment IaC and understand how Terraform handles isolation through state management.

If youve followed along till the end, congratulations youve just taken a big step toward mastering Infrastructure as Code with Terraform!

🌐 Connect with Me

If you enjoyed this guide, share it with your DevOps buddies and stay tuned for more such projects!
You can also find me sharing tech content, tutorials, and behind-the-scenes DevOps experiments here 👇

• 💼 LinkedIn: linkedin.com/in/pravesh-sudha

• 🐦 Twitter/X: x.com/praveshstwt

• 📹 YouTube: youtube.com/@pravesh-sudha

• 🌐 Website: blog.praveshsudha.com

Welcome Devs, to the world of Cloud and Code 💻

Today, Ive got something really exciting for you. Were going to integrate Terraform with Ansible to showcase the true power of Infrastructure as Code (IaC) and Configuration Management all fully automated with a multi-environment setup.

This setup will give you a real-world glimpse of how modern DevOps projects operate with environments like Dev, Stage, and Prod, and how tools like Terraform and Ansible work together one handling infrastructure provisioning, and the other managing configuration.

So without further ado, lets dive in and start building 🚀

📽 Youtube Demonstration

Pre-Requisites

Before we jump into the setup, make sure you have the following requirements ready on your system 👇

• AWS CLI installed and configured with an IAM user that has full access to EC2 and VPC.

  🧭 If youre new to this part dont worry! Ive already covered it in one of my earlier blogs:
  👉 Getting Started with Terraform A Beginners Guide
  (Follow Step 1 to 3 itll help you install AWS CLI, configure your IAM user, and set up Terraform CLI as well.)

• Python and Ansible installed on your system.

  📘 You can check out Ansibles official installation guide here:
  👉 Install Ansible with pip

Once thats all set up, were good to go and start building our project 🚀

🚀 Getting Started

Alright Devs, now that weve set up the basics, lets dive into the real deal.

This project is all about provisioning infrastructure for a real-world multi-environment setup and then configuring it using Ansible just like its done in production systems.

You can find the complete project code here:
👉 Terra-Projects Repository
The directory for this particular setup is terra-ansible-starter.

Now, lets break down how the project actually works 👇

🧱 1. terra-config Directory

This directory contains the following files:

• provider.tf

• variable.tf

• output.tf

• main.tf

Heres what happens inside:
Were creating a key pair named tester-key, and a security group with three rules allowing:

• Outbound traffic on ports 80 and 22

• Inbound traffic from everywhere

Then, using a for loop, we spin up 6 EC2 instances two for each environment:

• dev

• stage

• prod

The output.tf file gives us the public IPs of all these instances, neatly categorized per environment.

🧮 2. scripts Directory

Inside this folder, we have a Python script named generate_inv.py.
Heres what it does:

• It reads the output.json file (generated after running terraform output command).

• Then, it dynamically creates a hosts.ini file inside the Ansible inventory directory.

This makes the integration between Terraform and Ansible completely automated no manual IP editing required!

3. ansible Directory

Here lies our configuration magic

• Inside this folder, theres a playbook.yml file which defines a role called webserver.

• The roles/webserver/tasks/main.yml file includes all configuration steps for the servers:

  • Install NGINX

  • Copy the index-{env}.html file (specific to each environment)

  • Restart the NGINX server

Theres also a files directory that contains separate index.html files for each environment (dev, stage, prod).

The inventory folder holds the hosts.ini file, which specifies the public IPs of instances and yes, its dynamically created using our Python script.

🔗 Connecting It All Together

That was an eagle-eye view of the project. Now lets zoom in and see how everything connects together.

At the heart of this automation lies the deploy.sh script the glue that ties Terraform and Ansible into one seamless workflow.

Heres what happens step by step 👇

🧩 Step 1: Provision Infrastructure with Terraform

First, the script navigates inside the terra-config directory and runs:

terraform init
terraform apply -auto-approve

This initializes Terraform and provisions the infrastructure across all three environments Dev, Stage, and Prod.

Once the resources are created, it generates an output.json file that stores the public IPs of every instance in JSON format categorized neatly by environment.

🐍 Step 2: Generate Dynamic Inventory with Python

Next, the script moves back to the main project directory and executes the generate_inv.py script.

This Python script:

• Reads the output.json file created by Terraform

• Formats it properly to generate an Ansible hosts.ini file

• Appends it with other essential details such as the SSH key path for authentication

A quick sleep 15 command ensures the EC2 instances finish their health checks before Ansible jumps in for configuration.

Step 3: Configure Instances with Ansible

Finally, once our hosts.ini file is ready, the script triggers the Ansible playbook command:

ansible-playbook playbook.yml

This command configures all six EC2 instances automatically each according to its environment.
Every environment (Dev, Stage, Prod) gets its own custom index.html file served through NGINX.

🧠 Enough Talk Lets Get Our Hands Dirty!

Alright, enough with the theory its time to get practical and actually see this automation in action 🔥

Before we launch the setup, we need an SSH key that Ansible will use to authenticate into our EC2 instances.

Run the following command to generate one:

ssh-keygen -t rsa -b 4096 -f ~/.ssh/appKey

This command creates a secure SSH key pair named appKey inside your ~/.ssh directory.
Well use this private key for all our six EC2 instances two for each environment (Dev, Stage, and Prod).

🚀 Lets Deploy Everything

Now, since were DevOps engineers (and we hate doing things manually 😎), well use the deploy.sh script to automate everything from provisioning infrastructure to configuring servers.

Make sure the script has execution permissions, then run it:

chmod u+x deploy.sh
./deploy.sh

Sit back, grab a coffee , and watch the logs as the magic unfolds.
Terraform will create your instances, generate the output file, the Python script will prepare your inventory, and Ansible will jump in to configure your NGINX servers all in one go.

🌐 Test the Setup

Once the process completes, visit the public IPs of your EC2 instances in your browser.
Youll see different index.html pages for each environment Dev, Stage, and Prod each showcasing the environment name and design differences.

🧹 Clean Up

Once youre done testing, dont forget to destroy your infrastructure AWS isnt running on free coffee beans 😅

Run the following command to safely tear everything down:

cd terra-ansible-starter/terra-config
terraform destroy --auto-approve

This will clean up all EC2 instances, security groups, and key pairs saving you from unwanted AWS charges 💸

🎯 Conclusion

And thats a wrap, folks! 🥳

We just walked through a complete automation pipeline where Terraform handled infrastructure provisioning, and Ansible took care of server configuration all in a multi-environment setup.
This hands-on project gives you a solid understanding of how real-world DevOps workflows look when code, automation, and infrastructure come together.

By combining these two tools Terraform and Ansible youve essentially built a foundation for scalable, environment-aware deployments.
Whether its deploying static sites, configuring app servers, or scaling microservices, the same workflow logic applies automate, version-control, and manage everything as code.

If you followed along till here, youve not just learned two tools
youve built the mindset of a DevOps engineer who thinks in systems and automates for efficiency

Keep exploring, keep experimenting and as always, build, break, and learn 💪

🌐 Connect with Me

If you enjoyed this guide, share it with your DevOps buddies and stay tuned for more such projects!
You can also find me sharing tech content, tutorials, and behind-the-scenes DevOps experiments here 👇

• 💼 LinkedIn: linkedin.com/in/pravesh-sudha

• 🐦 Twitter/X: x.com/praveshstwt

• 📹 YouTube: youtube.com/@pravesh-sudha

• 🌐 Website: blog.praveshsudha.com

Welcome Devs to the world of AI and Automation 👋

Today, were diving into an exciting hands-on project where infrastructure meets intelligence. Well explore Cognee AI a memory layer for LLMs that lets applications remember, retrieve, and build on prior context and see how it works in action through the Cognee Starter App, built with Flask.

But were not stopping there. Once the app is ready, well deploy it to AWS ECS (Fargate) using Terraform, bringing the power of Infrastructure as Code (IaC) to streamline and automate the entire deployment process.

By the end of this guide, youll:

• 🧠 Get familiar with Cognee AI and its role as a memory layer for LLMs.

• 🐳 Containerise and prepare a Flask application for production.

• Provision AWS infrastructure using Terraform.

• 🚀 Deploy the app seamlessly on AWS ECS with Fargate.

So without further ado, lets get started and build something awesome!

📽 Youtube Demonstation

🧰 Pre-Requisites

Before we roll up our sleeves and dive into the deployment, lets make sure your local environment is ready. Having the right tools installed will make the process smooth and error-free.

Heres what youll need on your system:

• 🪝 AWS CLI configured with an IAM user that has ECS, VPC, and IAM Full Access permissions.
  👉 If youre not familiar with this setup, check out my detailed step-by-step guide here: Learn How to Deploy a Three-Tier Application on AWS EKS Using Terraform.

• 🐍 Python since Cognee Starter runs on Flask.

• 🧱 Terraform CLI the star of this blog, which well use to provision and manage our AWS infrastructure.

Once youve checked these boxes , youre all set to move to the fun part building and deploying our application!

🧠 Understanding Cognee AI

Before jumping into infrastructure, lets take a moment to understand what Cognee AI actually does and why its important.

In simple terms: Cognee organizes your data into AI memory.

When you make a call to a Large Language Model (LLM), the interaction is stateless meaning it doesnt remember what happened during previous calls or have access to your broader document context. This makes it difficult to build real-world applications that require context retention, document linking, or knowledge continuity.

Thats where Cognee AI comes in. It acts as a memory layer for LLMs, allowing you to:

• Link documents and data sources together.

• Maintain context across multiple LLM calls.

• Create richer, more intelligent applications that can reason over previous interactions.

In this project, well be using the Cognee Starter App, which gives a hands-on introduction to this powerful memory layer and then well deploy it on AWS ECS (Fargate) to make it production-ready.

🧭 How Cognee Works

Cognee isnt just about storing data its about understanding and structuring it so LLMs can use it intelligently. When it comes to your data, Cognee knows what matters.

There are four key operations that power the Cognee memory layer:

• .add Prepare for Cognification
  This is the starting point. You send your data asynchronously, and Cognee cleans, processes, and prepares it for the memory layer.

• .cognify Build a Knowledge Graph with Embeddings
  Cognee splits your documents into chunks, extracts entities and relations, and links everything into a queryable knowledge graph the core of its memory layer.

• .search Query with Context
  When you search, Cognee combines vector similarity with graph traversal. Depending on the mode, it can fetch raw nodes, explore relationships, or even generate natural language answers using RAG (Retrieval-Augmented Generation). It ensures the right context is always delivered to the LLM.

• .memify Semantic Enrichment of the Graph (coming soon)
  This will enhance the knowledge graph with deeper semantic understanding, adding richer contextual relationships.

In our hands-on demonstration, well use the first three methods .add, .cognify, and .search to see how Cognee works in action before deploying the app on AWS ECS.

🧪 A Small Demo to Understand Cognee Functions

Before we jump into deploying our Flask application on AWS, lets take a few minutes to understand how Cognee AI works practically.

Ive already hosted the project code on GitHub 👇
👉 GitHub Repository terra-projects

Head inside the cognee-flask directory, and youll find the entire project structure there.

🧰 Step 1: Set up Environment Variables

Inside the project folder, create a .env file. You can refer to the .env.example file for the format.

Youll need a Gemini API Key, which is free to get. Paste it in your .env file like this:

LLM_PROVIDER="gemini"
LLM_MODEL="gemini/gemini-2.5-flash"
LLM_API_KEY="<your-gemini-key>"

# Embeddings
EMBEDDING_PROVIDER="gemini"
EMBEDDING_MODEL="gemini/text-embedding-004"
EMBEDDING_DIMENSIONS="768"
EMBEDDING_API_KEY="<your-gemini-key>"

👉 If youre using a different LLM provider, follow this guide to configure it properly: Cognee Docs Installation & Setup

Step 2: Install Dependencies

To install all the required dependencies, run:

uv sync

This will set up everything you need to run the demo locally.

🧠 Step 3: Understanding testing_cognee.py

Now, open the testing_cognee.py file. Heres what it looks like:

from cognee import SearchType, visualize_graph
import cognee
import asyncio
import os, pathlib

async def main():
    # Create a clean slate for Cognee -- reset data and system state
    await cognee.prune.prune_data()
    await cognee.prune.prune_system(metadata=True)

    # Add sample content
    text = "Cognee turns documents into AI memory."
    await cognee.add(text)

    # Process with LLMs to build the knowledge graph
    await cognee.cognify()

    graph_file_path = str(
        pathlib.Path(
            os.path.join(pathlib.Path(__file__).parent, ".artifacts/graph_visualization.html")
        ).resolve()
    )
    await visualize_graph(graph_file_path)

    # Search the knowledge graph
    graph_result = await cognee.search(
        query_text="What does Cognee do?", query_type=SearchType.GRAPH_COMPLETION
    )
    print("Graph Result: ")
    print(graph_result)

    rag_result = await cognee.search(
        query_text="What does Cognee do?", query_type=SearchType.RAG_COMPLETION
    )
    print("RAG Result: ")
    print(rag_result)

    basic_result = await cognee.search(
        query_text="What are the main themes in my data?"
    )
    print("Basic Result: ")
    print(basic_result)

if __name__ == '__main__':
    asyncio.run(main())

📝 Whats happening here:

• First, we purge any existing data using prune_data and prune_system.

• Then, we add new text data to Cognee using .add.

• We process and build a knowledge graph with .cognify.

• The graph is stored in .artifacts/graph_visualization.html.

• Finally, we run three types of searches:

  • GRAPH_COMPLETION explores relationships.

  • RAG_COMPLETION generates natural language answers.

  • Basic Search retrieves core themes.

Step 4: Run the Demo

Run the script with:

uv run testing_cognee.py

You should see outputs for Graph, RAG, and Basic results in your terminal. You can also open the .artifacts/graph_visualization.html file in a browser to view the knowledge graph Cognee has generated.

🧪 Step 5: Run the Flask App Locally

Before deploying to the cloud, lets test the Flask app locally:

uv run app.py

Once the server starts, open 👉 http://localhost:5000

Youll see the Cognee AI Starter App UI. Heres what you can do:

• Add your own data.

• Ask a query.

• Wait 12 minutes.

• Youll get:

  • 🧠 Graph Completion Result

  • 💬 RAG Completion Result

  • 📝 Basic Theme of the text

  • 🌐 A vector graph visualization generated by Cognee.

Click on View Knowledge Graph to explore the graph and see how Cognee structured your data.

With this, weve understood Cognees core functionality locally.
Next up well take this application to the cloud by deploying it on AWS ECS using Terraform 🚀

Deploying the Cognee App on AWS ECS Using Terraform

Weve successfully tested the Cognee AI Starter App locally now its time to take things to the cloud 🌥

Well use Terraform to provision all the required AWS infrastructure and deploy our Flask application on ECS (Fargate).

Inside your project directory, navigate to the terra-config folder. This is where all of our Terraform configuration files live.

🧱 Step 1: Understanding the Terraform Files

• provider.tf
  Defines AWS as our cloud provider and sets the region and provider configuration.

• default_config.tf

  • Fetches the default VPC and subnets.

  • Creates a security group for ECS tasks with port 5000 open to allow inbound traffic.

• main.tf (the heart of the project)

  • Creates an ECS cluster.

  • Defines the ECS task definition with:

    • Docker image

    • Container port

    • CPU & memory configurations

    • CPU architecture

  • Creates an IAM role for ECS.

  • Finally, provisions an ECS service using the task definition inside the cluster.

• get_ip.sh
  A simple shell script that uses AWS CLI to fetch the public URL where your application is running.

Step 2: Initialize and Deploy the Infrastructure

Run the following commands inside the terra-config directory:

terraform init
terraform plan
terraform apply --auto-approve

This will take around 2 minutes to provision the complete infrastructure on AWS including the ECS cluster, service, task definition, and networking setup.

🌍 Step 3: Get the Application URL

Once the deployment finishes, make the script executable and run it:

chmod u+x get_ip.sh
./get_ip.sh

This will output the URL where your application is hosted.
👉 Open the URL in your browser, and youll see your Flask Cognee application live on AWS ECS 🎉

From here, you can:

• Add your data.

• Ask queries.

• See the graph and AI-generated answers exactly like in the local demo.
  The only difference is its now running inside a scalable, production-grade ECS cluster.

🧹 Step 4: Clean Up Resources

Once youre done testing, its good practice to destroy the infrastructure to avoid unwanted AWS charges:

terraform destroy --auto-approve

This will tear down all ECS services, roles, and networking resources you created for this project.

And thats it!
Youve successfully deployed your Flask Cognee AI Starter App on AWS ECS using Terraform. With just a few commands, we automated the entire infrastructure provisioning and deployment pipeline.

🎯 Conclusion

Congratulations! 🎉 Youve successfully taken a Flask-based Cognee AI Starter App from your local machine all the way to the cloud using AWS ECS and Terraform.

In this blog, we learned how to:

• Understand Cognee AI and its memory layer for LLMs.

• Explore and test its key operations .add, .cognify, and .search.

• Run the app locally to see how Cognee organizes and queries your data.

• Provision AWS infrastructure using Terraform.

• Deploy the Flask application on ECS (Fargate) and access it via a public URL.

This hands-on project demonstrates the power of combining AI, containerization, and infrastructure automation to deploy intelligent applications in a scalable, production-ready environment.

🔗 Explore More

• Check out Cognee AI here: https://cognee.ai

• Follow me on socials for more DevOps, cloud, and AI tutorials:

  • GitHub: https://github.com/Pravesh-Sudha

  • Blog: https://blog.praveshsudha.com

  • LinkedIn: https://www.linkedin.com/in/pravesh-sudha

Keep exploring, keep building, and stay tuned for more hands-on projects combining AI and DevOps! 🚀

Welcome Devs to the world of code and automation 👨💻

Being around the DevOps landscape for more than two years, Ive often come across one of the biggest myths in the field DevOps Engineers dont code.
Well we do code, and quite a lot of it. From automating repetitive tasks, writing IAM policies, and managing infrastructure as code in HCL, to gluing together entire cloud workflows code is very much a part of a DevOps engineers daily toolkit.

Its been a while since I last got my hands dirty with Python, so I decided to change that. In todays blog, were kicking off another project in the Python for DevOps series.

Well be building a custom CLI tool using Python3 and Boto3 (AWS SDK for Python) to manage EC2 instances, S3 buckets, and Lambda functions all through the command line.

This project is a great way to:

• Strengthen your Python skills 🧠

• Get hands-on with Boto3 💻

• Automate AWS tasks like a pro

So without further ado, lets dive right in and start building something practical.

📽 Youtube Demonstration

🧰 Prerequisites

Before we start building our custom AWS CLI, lets make sure your system is ready to roll.
Heres what youll need:

• 🪄 AWS CLI Installed & Configured
  You should have the AWS CLI installed and configured with an IAM user that has full access to:

  • EC2

  • Lambda

  • IAM

  • S3

If you havent set this up yet, Ive got you covered check out the Step 1 section of my previous blog here:
👉 Learn how to deploy a Three-Tier Application on AWS EKS using Terraform

• 🐍 Python 3
  Make sure Python 3 is installed on your system.
  You can verify it by running:

    python3 --version
  

Once both are ready, we can move on to setting up our project structure and start coding.

🧠 Writing the Core Python Scripts EC2, S3 & Lambda Automation

The full source code for this project is available on my GitHub repo:
👉 Pravesh-Sudha/python-for-devops

Navigate inside the automate-ec2-s3-and-lambda directory, and youll find three main files:

• ec2_manager.py

• s3_manager.py

• lambda_manager.py

These files contain the logic to interact with AWS using Boto3, the official AWS SDK for Python.

🖥 1. EC2 Manager ec2_manager.py

This file handles everything related to EC2 instances: creating, listing, and terminating them.

import boto3
from botocore.exceptions import ClientError

class EC2Manager:
    def __init__(self, region_name="us-east-1"):
        self.ec2 = boto3.client("ec2", region_name=region_name)

    def create_instance(self, image_id="ami-0360c520857e3138f", instance_type="t3.micro", key_name=None):
        try:
            print("Creating a Ec2 Instance....")
            params = {
                "ImageId": image_id,
                "InstanceType": instance_type,
                "MinCount": 1,
                "MaxCount": 1
            }
            if key_name:
                params["KeyName"] = key_name

            response = self.ec2.run_instances(**params)
            instance_id = response["Instances"][0]["InstanceId"]
            print(f"EC2 instance Created successfully with id: {instance_id}")
            return instance_id
        except ClientError as e:
            print(f"Ec2 creation failed. Error occured: {e}")
            return None

    def list_instances(self):
        try:
            response = self.ec2.describe_instances()
            instances = []
            for reservation in response["Reservations"]:
                for instance in reservation["Instances"]:
                    instance_info = {
                        "InstanceId": instance["InstanceId"],
                        "State": instance["State"]["Name"],
                        "Type": instance["InstanceType"],
                        "PublicIP": instance.get("PublicIpAddress")
                    }
                    instances.append(instance_info)
            return instances
        except ClientError as e:
            print(f"Failed to get list of Ec2 instances. Error Occured: {e}")
            return []

    def terminate_instance(self, instance_id):
        try:
            self.ec2.terminate_instances(InstanceIds=[instance_id])
            print(f"Instance with id {instance_id} terminated successfully")
        except ClientError as e:
            print(f"Failed to terminate instance {instance_id}. Error Occured: {e}")

📝 Quick Breakdown:

• We initialize a Boto3 EC2 client inside the class constructor.

• create_instance() launches a new EC2 instance using a given AMI ID and instance type.

• list_instances() fetches and displays all existing instances with their state and public IP.

• terminate_instance() shuts down an instance by its ID.

• The try-except block ensures clean error handling and avoids breaking the script when AWS throws an exception.

🪣 2. S3 Manager s3_manager.py

This file deals with creating, listing, and deleting S3 buckets.

import boto3
from botocore.exceptions import ClientError

class S3Manager:
    def __init__(self, region_name="us-east-1"):
        self.s3 = boto3.client("s3", region_name=region_name)

    def create_bucket(self, bucket_name):
        try:
            print(f"Creating a new Bucket name: {bucket_name}")
            self.s3.create_bucket(Bucket=bucket_name)
            print(f"Bucket Created Successfully.")
        except ClientError as e:
            print(f"Failed to create Bucket. Error: {e}")

    def list_buckets(self):
        try:
            response = self.s3.list_buckets()
            buckets = [b["Name"] for b in response["Buckets"]]
            return buckets
        except ClientError as e:
            print(f"Failed to list the bucket. Error: {e}")
            return []

    def delete_bucket(self, bucket_name):
        try:
            print(f"Deleting Bucket: {bucket_name}")
            self.s3.delete_bucket(Bucket=bucket_name)
            print(f"Successfully deleted the bucket: {bucket_name}")
        except ClientError as e:
            print(f"Failed to delete the Bucket. Error: {e}")

📝 Quick Breakdown:

• Boto3s S3 client allows easy bucket operations.

• We can create new buckets, fetch all bucket names, and delete existing ones.

• Again, try-except helps us handle permission issues or invalid operations gracefully.

🧬 3. Lambda Manager lambda_manager.py

This one is a bit more advanced because Lambda also requires an IAM role to execute.

import boto3
from botocore.exceptions import ClientError
import zipfile
import os
import time, json

class LambdaManager:
    def __init__(self, region_name="us-east-1"):
        self.lambda_client = boto3.client("lambda", region_name = region_name)
        self.iam_client = boto3.client("iam", region_name = region_name)

    def _create_deployment_package(self, code_path, zip_name="lambda_function.zip"):
        with zipfile.ZipFile(zip_name, "w") as zf:
            zf.write(code_path, os.path.basename(code_path))
            print(f"Deployment package built Successfully")
        return zip_name

    def _get_or_create_role(self, role_name="LambdaBasicExecutionRole"):
        assume_role_policy = {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Principal": {"Service": "lambda.amazonaws.com"},
                    "Action": "sts:AssumeRole"
                }
            ]
        }
        try:
            role = self.iam_client.get_role(RoleName = role_name)
            print(f"Getting the IAM role for Lambda....")
            return role["Role"]["Arn"]
        except self.iam_client.exceptions.NoSuchEntityException as e:
            print(f"Creating the IAM role : {role_name}....")
            role = self.iam_client.create_role(
                RoleName = role_name,
                AssumeRolePolicyDocument = json.dumps(assume_role_policy) 
            )
            self.iam_client.attach_role_policy(
                RoleName = role_name,
                PolicyArn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
            )
            print(f"Waiting for the IAM role Propagation...")
            time.sleep(10)
            return role["Role"]["Arn"]

    def create_lambda(self, function_name, code_path, handler_name="lambda_function.lambda_handler", runtime = "python3.9"):        
        try:
            zip_file = self._create_deployment_package(code_path)
            role_arn = self._get_or_create_role()

            with open(zip_file, "rb") as f:
                zip_bytes = f.read()

            response = self.lambda_client.create_function(
                FunctionName = function_name,
                Code = {"ZipFile": zip_bytes},
                Runtime = runtime,
                Role = role_arn,
                Handler = handler_name,
                Timeout = 15,
                MemorySize = 128, 
            )
            print(f"Lambda function {function_name} created Successfully.")
            return response["FunctionArn"]
        except ClientError as e:
            print(f"Failed to create the lambda function. Error: {e}")

    def list_lambdas(self):
        try:
            response = self.lambda_client.list_functions()
            lambdas = response.get("Functions",[])
            return [
                {
                "Name": fn["FunctionName"],
                "Runtime": fn["Runtime"],
                "Arn": fn["FunctionArn"]
                }
                for fn in lambdas
            ]
        except ClientError as e:
            print(f"Failed to list Lambda Functions. Error: {e}")
            return []

    def delete_lambda(self, function_name):
        try:
            self.lambda_client.delete_function(FunctionName = function_name)
            print(f"Lambda function: {function_name} deleted Successfully.")
        except ClientError as e:
            print(f"Failed to delete Lambda Function. Error: {e}")

📝 Quick Breakdown:

• _create_deployment_package() zips your function code for Lambda deployment.

• _get_or_create_role() ensures an IAM role exists to let Lambda run. If not, it creates one and attaches the basic execution policy.

• create_lambda() uploads the zipped code and creates a new Lambda function.

• list_lambdas() and delete_lambda() handle listing and cleanup.

And heres our simple lambda_function.py file that acts as the Lambda code:

import json

def lambda_handler(event, context):
    print("Hello from Pravesh - Your AWS Community Builder!") 
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Pravesh - Your AWS Community Builder!')
    }

🧭 4. Connecting Everything With main.py (CLI)

To make this project actually feel like a real CLI tool, well use Pythons built-in argparse module.
This allows us to run commands like:

python main.py ec2 create
python main.py s3 list
python main.py lambda delete <function-name>

Heres the full CLI script 👇

import argparse
from ec2_manager import EC2Manager
from s3_manager import S3Manager
from lambda_manager import LambdaManager

def main():
    parser = argparse.ArgumentParser(
        description="AWS Automation CLI using Boto3 (EC2 & S3)"
    )

    subparsers = parser.add_subparsers(dest="service", help="Choose AWS service")

    # EC2 Commands
    ec2_parser = subparsers.add_parser("ec2", help="Manage EC2 instances")
    ec2_subparsers = ec2_parser.add_subparsers(dest="action", help="EC2 actions")

    ec2_create = ec2_subparsers.add_parser("create", help="Create EC2 instance")
    ec2_create.add_argument("--image-id", default="ami-0c94855ba95c71c99", help="AMI ID")
    ec2_create.add_argument("--instance-type", default="t2.micro", help="Instance type")
    ec2_create.add_argument("--key-name", help="Optional key pair name")

    ec2_subparsers.add_parser("list", help="List EC2 instances")
    ec2_terminate = ec2_subparsers.add_parser("terminate", help="Terminate EC2 instance")
    ec2_terminate.add_argument("instance_id", help="Instance ID to terminate")

    # S3 Commands
    s3_parser = subparsers.add_parser("s3", help="Manage S3 buckets")
    s3_subparsers = s3_parser.add_subparsers(dest="action", help="S3 actions")
    s3_subparsers.add_parser("list", help="List S3 buckets")
    s3_create = s3_subparsers.add_parser("create", help="Create S3 bucket")
    s3_create.add_argument("bucket_name", help="Bucket name to create")
    s3_delete = s3_subparsers.add_parser("delete", help="Delete S3 bucket")
    s3_delete.add_argument("bucket_name", help="Bucket name to delete")

    # Lambda Commands
    lambda_parser = subparsers.add_parser("lambda", help="Manage AWS Lambda functions")
    lambda_subparsers = lambda_parser.add_subparsers(dest="action", help="Lambda actions")
    lambda_create = lambda_subparsers.add_parser("create", help="Create Lambda function")
    lambda_create.add_argument("function_name", help="Lambda function name")
    lambda_create.add_argument("code_path", help="Path to Python file (e.g. lambda_function.py)")
    lambda_create.add_argument("--handler", default="lambda_function.lambda_handler", help="Handler name")
    lambda_create.add_argument("--runtime", default="python3.9", help="Runtime version")
    lambda_subparsers.add_parser("list", help="List Lambda functions")
    lambda_delete = lambda_subparsers.add_parser("delete", help="Delete Lambda function")
    lambda_delete.add_argument("function_name", help="Lambda function name to delete")

    args = parser.parse_args()

    # Service Handling
    if args.service == "ec2":
        ec2 = EC2Manager()
        if args.action == "create":
            ec2.create_instance(image_id=args.image_id, instance_type=args.instance_type, key_name=args.key_name)
        elif args.action == "list":
            instances = ec2.list_instances()
            if not instances:
                print("No EC2 instances found.")
            else:
                print("\nEC2 Instances:")
                for i in instances:
                    print(f"{i['InstanceId']} | {i['State']} | {i['Type']} | {i['PublicIP']}")
        elif args.action == "terminate":
            ec2.terminate_instance(args.instance_id)

    elif args.service == "s3":
        s3 = S3Manager()
        if args.action == "create":
            s3.create_bucket(args.bucket_name)
        elif args.action == "list":
            buckets = s3.list_buckets()
            if not buckets:
                print("No buckets found.")
            else:
                print("\nS3 Buckets:")
                for b in buckets:
                    print(f"- {b}")
        elif args.action == "delete":
            s3.delete_bucket(args.bucket_name)

    elif args.service == "lambda":
        lm = LambdaManager()
        if args.action == "create":
            lm.create_lambda(args.function_name, args.code_path, handler_name=args.handler, runtime=args.runtime)
        elif args.action == "list":
            functions = lm.list_lambdas()
            if not functions:
                print("No Lambda functions found.")
            else:
                print("\nLambda Functions:")
                for f in functions:
                    print(f"{f['Name']} | {f['Runtime']} | {f['Arn']}")
        elif args.action == "delete":
            lm.delete_lambda(args.function_name)
    else:
        parser.print_help()

if __name__ == "__main__":
    main()

This is where everything comes together. Each manager class is imported, and argparse routes the commands to the right function.

🧠 Pro Tip Write the Code Yourself

Now comes the most important part dont just copy the code.
Write the ec2_manager.py file line by line in your own workspace. Understand:

• Why we use try/except

• How boto3 clients are created

• How API calls are structured and handled

Then move on to S3 and Lambda.
If you get stuck look back at the repo for hints, but type it yourself. This will build muscle memory and confidence, which is crucial in the age of AI-assisted development.

Even though tools can generate code for you, knowing how to build it yourself gives you the real power

🛠 Setting Up the Virtual Environment

Once your files are ready, set up the virtual environment and install dependencies:

python3 -m venv venv
source venv/bin/activate
pip3 install boto3 botocore

🚀 Running the CLI Commands

EC2

python main.py ec2 create
python main.py ec2 list
python main.py ec2 terminate <instance-id>

S3

python main.py s3 create <bucket-name>
python main.py s3 list
python main.py s3 delete <bucket-name>

Lambda

python main.py lambda create <function-name> lambda_function.py
python main.py lambda list
python main.py lambda delete <function-name>

Before deleting your Lambda function, open the AWS Lambda Dashboard, find your function, and give it a test.

Youll see your lambda_function.py code running and the expected output returned in the console

🏁 Conclusion

And thats a wrap, folks! 🎉

We just built a fully functional Python CLI tool that can manage your AWS EC2, S3, and Lambda resources all from the command line. This might look simple at first, but projects like this help you truly understand how DevOps engineers use code to automate and simplify daily cloud operations.

This is just the beginning. You can extend this CLI to include:

• More AWS services like RDS, DynamoDB, and CloudWatch 📈

• Advanced features like error handling and logging 🧭

• Role-based actions with IAM policies 🔐

Ive pushed the entire project to GitHub.
👉 Make sure to fork and star the repo if you found it useful it really motivates me to create more such DevOps + Python projects.

If you build your own version, Id love to see it! Share it with me or tag me on socials:

• 🐦 Twitter: @praveshstwt

• 💼 LinkedIn: Pravesh Sudha

• 📝 Blog: blog.praveshsudha.com

• 🐙 GitHub: github.com/PraveshSudha

Until next time keep automating and keep building. 💪

Welcome to the world of Infrastructure and Automation! 🚀
In todays post, were going to explore one of the fundamental building blocks of Terraform the Module.

If youve been using Terraform for a while, you already know how quickly your configuration files can grow messy as your infrastructure expands. Thats where modules come to the rescue. They help you organize, reuse, and standardize your Terraform code making your infrastructure cleaner, more scalable, and much easier to maintain.

In this guide, well break down:

• What a Terraform module is

• How its structured

• Why using modules is beneficial

• And finally, how to create your own module for a real-world use case.

To wrap things up, well get hands-on and build a Terraform module for both VPC and EC2, deploy an instance, and serve a simple index.html page from it.

So, without further ado lets dive right in! 🌍

💡 Youtube Demonstration

💡 Pre-Requisites

Before we dive into building our Terraform module, lets make sure your workspace is all set up and ready to go. Heres what youll need:

• 🧑💻 An AWS Account with an IAM user that has Full Access to both VPC and EC2.
  The IAM user should have an Access Key generated and configured with the AWS CLI on your system.
  If youre new to this step, Ive covered it in detail here:
  👉 Set up AWS CLI and IAM User (Step 1 from my EKS blog)

• Basic knowledge of Terraform understanding what it is and how it works will make things much smoother.

Once you have these requirements in place, were all set to kick off our journey into Terraform Modules! 🎯

💡 What is a Terraform Module?

Terraform lets you define your infrastructure as code using HashiCorp Configuration Language (HCL). And like any programming language, it embraces one golden principle DRY (Dont Repeat Yourself).

Instead of repeatedly provisioning similar resources (like networking components or security groups) for every new environment, Terraform allows you to encapsulate those recurring configurations inside a module.

In simple terms, a Terraform module is a collection of Terraform configuration files (.tf or .tf.json) that live together in the same directory and work as a single logical unit.

Here are a few examples of what Terraform modules can represent:

• 🧱 An AWS VPC module containing subnets, route tables, and internet gateways

• 💾 A Microsoft SQL Always On cluster in Azure, including NSGs

• A GCP Project module that enables APIs and sets permissions

Why Modules Matter

Imagine youre setting up an EC2 instance the traditional way defining every component manually: the security group, IAM role, instance profile, and the instance itself. Thats a lot of repeated configuration!

Heres what that would look like 👇

resource "aws_security_group" "my_app_sg" {
  name        = "my-app-sg"
  vpc_id      = var.vpc_id
  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_iam_role" "my_app_role" {
  name = "my_app_role"
  assume_role_policy = <<EOF
{ ... }
EOF
}

resource "aws_iam_instance_profile" "my_app_profile" {
  name = "my_app_profile"
  role = aws_iam_role.my_app_role.name
}

resource "aws_instance" "my_app_server" {
  ami           = "ami-0c55b159fbd7718e9"
  instance_type = "t3.micro"
  vpc_security_group_ids = [aws_security_group.my_app_sg.id]
  iam_instance_profile   = aws_iam_instance_profile.my_app_profile.name
  subnet_id              = var.subnet_id
  user_data              = file("setup.sh")
  tags = {
    Name = "my-app-server"
  }
}

Now, lets see how modules make this effortless and elegant:

module "ec2_instance" {
  source  = "terraform-aws-modules/ec2-instance/aws"
  version = "5.6.0"

  name          = "my-app-server"
  ami           = "ami-0c55b159fbd7718e9"
  instance_type = "t3.micro"
  subnet_id     = var.subnet_id

  tags = {
    Name = "my-app-server"
  }

  # Configuration for managed resources
  create_iam_instance_profile = true
  iam_role_name               = "my_app_role"
  iam_role_policy_json        = "{ ... }"

  security_group_ingress = [
    {
      description = "HTTP access from anywhere"
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
    }
  ]
}

See the difference?
Instead of defining every resource from scratch, you just call a module and pass the required inputs. Terraform takes care of the rest!

Well create our own module in the practical demonstration later but this example perfectly shows why modules are so powerful.

Why Use Terraform Modules?

Heres how modules simplify your infrastructure management:

• 📦 Package related resources together into a reusable configuration

• 👨👩👧👦 Share standardized configurations across your team or projects

• 💡 Embrace DRY principles, reducing repetitive code

• Minimize human error when referencing complex resources manually

Quick Question: How Are Terraform Resources Different from Modules?

Thats a great distinction to understand early on 👇

• A resource in Terraform describes a single piece of infrastructure like a VPC, subnet, EC2 instance, or IAM role.

• A module, on the other hand, is a collection of resources grouped together to form a reusable and logical unit for example, a complete VPC setup or an EC2 deployment stack.

💡 Benefits of Using Terraform Modules

Now that we understand what modules are and how they work, lets explore why theyre such an essential part of Terraform best practices.

Using modules isnt just about cleaner code its about making your infrastructure reusable, scalable, and collaborative. Lets look at the key benefits 👇

🧩 1. Reusability

Just like every programming language has functions, classes, or libraries, Terraform has modules.
They allow you to abstract a set of resources into a single reusable component that can be used across multiple projects or environments.

For example, once youve built a VPC module, you can reuse it for your dev, staging, and production environments without rewriting a single line of networking code.

In short: build once, use anywhere.

🚀 2. Scalability

Modules make scaling your infrastructure seamless.

Lets say you have a security group defined for your development environment that allows traffic on port 27017 for MongoDB. Instead of manually updating that configuration across multiple environments, you can simply update the module and propagate the change everywhere its used.

This ensures consistency and saves tons of time especially when managing large-scale infrastructure across multiple environments.

🤝 3. Team Collaboration

As your Terraform usage grows across teams, modules help maintain standardization and best practices.

Platform teams can create a catalog of pre-approved modules (for example, VPC, EC2, or EKS modules) that application teams can directly use. This ensures that all deployed infrastructure meets security, compliance, and organizational standards while allowing developers to focus on their applications.

In short, modules enable smooth collaboration, reduce misconfigurations, and promote a shared IaC culture within your organization.

💡 A Practical Demonstration

Alright, now that weve covered the theory its time to get our hands dirty! 🧑💻

Earlier, we looked at an example using the official AWS EC2 module, but in this section, well create our own Terraform modules from scratch.
Our goal? To build a modular Terraform setup that provisions a VPC, Security Group, and EC2 instance, which in turn hosts a static portfolio page.

🧱 What Well Be Building

Well use some of the most common AWS infrastructure components:

• VPC (Virtual Private Cloud)

• Subnets

• Internet Gateway (IGW)

• Route Table (RT)

• Security Group (SG)

• EC2 Instance

These resources form the foundation of most web applications, whether its an e-commerce site, a personal portfolio, or a marketing page.

In production environments, infrastructure code is usually broken down into reusable modules to:

• Avoid duplication

• Improve maintainability

• Enable better team collaboration

For example, a DevOps team might maintain a VPC module thats reused across multiple environments dev, staging, and production ensuring consistent infrastructure everywhere.

Our demo follows the same principle. Well organize our project into separate modules for vpc, security-group, and ec2, each containing its own:

• main.tf core resource definitions

• variables.tf variable declarations

• outputs.tf exported outputs

Well keep variables flexible (no default values) so they can be customized per environment.

Automation with user_data

Inside our EC2 module, well use a user_data script to automatically install Apache and deploy a simple index.html page.

In real-world deployments, such automation scripts (or tools like Ansible or Chef) are used to configure servers automatically at startup eliminating the need for manual setup.
This approach is standard for bootstrapping web servers, application backends, or API services.

📂 Project Structure

The complete code is available in my GitHub repository:
👉 https://github.com/Pravesh-Sudha/terra-projects

Navigate to the terra-modules directory, and youll find the following structure:

terra-modules/
|
-- main.tf                 # Calls our custom modules
 variables.tf                 
-- outputs.tf
|
 modules/
    vpc/
       main.tf
       variables.tf
       outputs.tf
   
    security-group/
       main.tf
       variables.tf
       outputs.tf
   
    ec2/
        main.tf
        variables.tf
        outputs.tf

Here, each submodule (vpc, security-group, ec2) defines its own configuration logic, and the root main.tf file simply calls these modules and passes the required variables.

Once everything is set, our static website server configuration will be ready to deploy.

🚀 Running the Terraform Project

Now lets see it in action!

1. Initialize Terraform

    terraform init
   

   

   This initializes the project and fetches all three modules.

2. Review the execution plan

    terraform plan
   

   

   Youll see that Terraform plans to create 7 resources in total.

3. Deploy the infrastructure

    terraform apply --auto-approve
   

   

   Wait for a minute or two once the deployment completes, Terraform will output the public IP address of your EC2 instance.

4. View your static website
   Copy the public IP into your browser, and youll see your web server running serving the index.html page! 🎉

   

🧹 Cleanup

When youre done experimenting, make sure to tear down the infrastructure to avoid unnecessary AWS costs:

terraform destroy --auto-approve

With that, youve successfully created your own set of Terraform modules a foundational skill for structuring production-grade Infrastructure as Code (IaC).

In case you are wondering how to reuse the components, here is an example for your file to create another VPC using our custom module:

module "vpc" "another-vpc" {
    source = "./modules/vpc"
    vpc_cidr = "<Provide the CIDR>"
    subnet_cidr = "<Provide the CIDR>"
    app_name = var.app_name
}

variable "app_name" {
    default = "Another-App"
    type = string
    description = "Name of the Application"
}

💡 Conclusion

And thats a wrap! 🎉

In this guide, we explored the core concept of Terraform modules what they are, why theyre beneficial, and how they make your infrastructure reusable, scalable, and team-friendly.
We then put theory into practice by creating our own Terraform modules for VPC, Security Group, and EC2, and deployed a working static website on AWS all using clean, modular Terraform code.

By now, you should have a solid understanding of how to:

• Structure Terraform projects into reusable modules

• Simplify complex configurations

• Collaborate efficiently within teams

• Keep your infrastructure code DRY and maintainable

The next step? Try expanding your modules! Add resources like S3 buckets, RDS databases, or Load Balancers and see how modular design keeps your Terraform journey organized and production-ready.

References

If youd like to go deeper, here are the key references I used while crafting this guide:

• Terraform Modules Official HashiCorp Documentation

• env0 Blog Terraform Modules Explained

• Spacelift Blog What Are Terraform Modules and How Do They Work

Lets Connect 🌐

If you enjoyed this blog and want to explore more about DevOps, Terraform, and Cloud automation, feel free to connect with me here:

• 💼 LinkedIn

• 🐦 Twitter / X

• 📺 YouTube

• 📝 Blog Website

If you found this post helpful, share it with your DevOps peers and drop your thoughts in the comments Id love to hear how you use Terraform modules in your projects!

👋 Adios Amigos!

Welcome, Devs 👋 to the exciting world of Infrastructure as Code (IaC) and automation!
If youve been working with Terraform, you already know how powerful it is in spinning up infrastructure in minutes. But with great power comes well, the need for great security.

In todays cloud-first job market, security isnt optionalits a core skill. DevOps engineers are now expected to think like DevSecOps engineers, ensuring that every piece of code, every module, and every configuration follows security best practices. Why? Because misconfigured infrastructure is like leaving your front door unlockedhackers love it.

Thats exactly what were diving into today. In this blog, Ill walk you through Terraform security best practices that will help you safeguard your cloud environments, avoid costly mistakes, and step confidently into the world of DevSecOps.

So, grab your coffee and lets get started! 🚀

💡 Youtube Demonstration

1. Verify Modules and Providers 🔍

When working with Terraform, your providers and modules are just like external dependencies in application development. And just like you wouldnt blindly install a random library in your production code, you shouldnt blindly trust Terraform modules or providers either.

Think of it this way:

• Providers are your bridge to cloud platforms (AWS, Azure, GCP, etc.).

• Modules are reusable building blocks that make your Terraform code cleaner and more scalable.

But since both come from external sources, treating them with security-first practices is a must.

Always Pin the Source and Version

Instead of using a vague provider declaration like this:

provider "aws" {
    region = "us-east-1"
}

Lock it down with a clear source and version:

terraform {  
  required_providers {
    aws = {
      source  = "hashicorp/aws"      
      version = "~> 5.98.0"    
    }  
  }
}

This ensures youre not pulling in an unverified or malicious provider update without realizing it. Think of it as freezing dependencies in your app codepredictability and security go hand in hand.

🏢 For Organizations

If youre working in a team or an enterprise environment, youll likely have a private registry in place. Thats a great way to enforce version control and ensure everyone only uses approved and vetted providers/modules.

Some best practices here:

• Use a filesystem or network mirror to point to an internal artifact repo (similar to how you store container images).

• Implement a minimal module registry API so teams can share trusted Terraform modules across projects.

• Review and approve new modules/providers before they go into production pipelines.

In short: treat Terraform providers and modules like you treat code librariesverify, version, and control them.

2. Dont Store the State File Locally 🔒

One of the most common mistakes Terraform beginners (and sometimes even pros) make is keeping the state file (terraform.tfstate) on their local machine.

Why is this risky?

• Your state file contains sensitive data like resource IDs, configuration details, and even secrets.

• If stored locally or pushed to a public repo (e.g., GitHub), its basically like handing over your cloud blueprint to attackers.

• A leaked state file can lead to account compromise or full infra takeover.

Best Practice: Remote and Encrypted Storage

Instead of keeping state locally, store it in a secure, remote backend that your team and organization can access safely. Make sure its encrypted to protect against unauthorized access.

Heres an example using AWS S3 with encryption and versioning:

terraform {
  backend "s3" {
    bucket       = "my-terraform-state-bucket"
    key          = "my-terraform-state.tfstate"
    region       = "eu-west-1"
    use_lockfile = true
    encrypt      = true
  }
}

🔑 A Note on State Locking

Earlier, Terraform state locking was usually handled with S3 + DynamoDB. But with recent updates, S3 now has the use_lockfile property, making it easier to prevent race conditions when multiple people or pipelines try to update the state at the same time.

🚀 Quick Checklist

• Never commit state files to GitHub (or any VCS).

• Always enable encryption and versioning in your remote backend.

• Use role-based access control (RBAC) to restrict who can read/write to the state.

In short: treat your Terraform state like a password vaultit holds the keys to your entire infrastructure. 🔐

3. Detect Vulnerabilities Early 🕵

Security isnt something you add at the end. With Terraform (and IaC in general), you want to shift security leftmeaning you catch issues while writing code, not after deploying it to production.

Thats where static code analysis tools come in. These tools scan your Terraform configuration and flag potential misconfigurations or risky patterns before they ever touch your cloud environment.

🚨 What They Catch

• Unintended resource exposure (e.g., open security groups, public S3 buckets).

• Weak encryption setups (like missing encrypt = true).

• Misconfigurations that could lead to compliance violations.

🛠 Popular Tools for Terraform Security

• tfsec: Lightweight, easy-to-use, and integrates well with CI/CD pipelines.

• Checkov: Great for policy-as-code, compliance checks, and multi-cloud environments.

• Terrascan: One of the most popular tools, with 500+ built-in policies for Terraform, Kubernetes, and more.

Example: Scanning with Terrascan

Run this command to analyze your Terraform code:

terrascan scan -f /path/to/terraform/code

It will quickly point out vulnerabilities, risky resources, or compliance issuessaving you hours of debugging and reducing security risks before deployment.

👉 Pro tip: Integrate these scans into your CI/CD pipeline so every commit or PR is automatically checked for vulnerabilities. That way, youre not just automating infra deploymentyoure automating secure infra deployment. 🚀

4. Apply the Principle of Least Privilege 🔑

When it comes to securing cloud infrastructure, one golden rule stands out: never give more access than necessary.

This is the Principle of Least Privilege (PoLP)granting only the minimum level of permissions required for a resource or user to function. By following PoLP, you reduce the attack surface and limit potential damage in case of a breach.

Think of it like this: if your friend only needs to borrow your car keys, dont hand over the keys to your house too.

Why It Matters in Terraform

Terraform often interacts with your cloud provider (AWS, Azure, GCP) to spin up or manage resources. If the IAM roles or service accounts used by Terraform are too permissive, attackers can exploit that access to move laterally, exfiltrate data, or even shut down your infra.

📝 Example: Minimal Policy for Reading Terraform State from S3

Heres a sample IAM policy for an EC2 instance that only needs to read the Terraform state file:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowListAndLocation",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketLocation"
      ],
      "Resource": "arn:aws:s3:::your-state-file-bucket"
    },
    {
      "Sid": "AllowReadStateFile",
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::your-state-file-bucket/path/to/your/terraform.tfstate"
    }
  ]
}

Notice how this policy doesnt allow write/delete actionsjust enough access to list and read the state file.

🚀 Best Practices

• Always scope IAM roles/policies to the specific bucket, object, or resource path.

• Avoid using wildcards (*) unless absolutely necessary.

• Regularly audit IAM policies to check for over-privileged accounts.

• Use tools like AWS IAM Access Analyzer to catch unintended permissions.

In short: lock the doors you dont need to open. By sticking to PoLP, you make life harder for attackers and safer for your team.

5. Dont Modify the Terraform State File Manually

Your Terraform state file is the single source of truth for your infrastructure. It keeps track of all the resources Terraform manages. Because of that, manually editing the state file is a recipe for disaster.

Why?

• It can corrupt the state, making Terraform lose track of your infra.

• It introduces configuration driftyour code and your actual infra wont match.

• Worst case, it can lead to unexpected resource destruction (yep, Terraform might just wipe out live resources).

The Wrong Way

Lets say you have this configuration for an EC2 instance:

provider "aws" {
  region = "us-east-1"
}

resource "aws_instance" "web_server" {
  ami           = "ami-0c55b159f2f12217c"
  instance_type = "t2.micro"
}

Now you decide to rename web_server to app_server. You update the code like this:

provider "aws" {
  region = "us-east-1"
}

resource "aws_instance" "app_server" {
  ami           = "ami-0c55b159f2f12217c"
  instance_type = "t2.micro"
}

When you run terraform plan, Terraform sees this as:

• Delete: aws_instance.web_server

• Create: aws_instance.app_server

Meaningyour EC2 instance is destroyed and replaced, causing unnecessary downtime.

The Right Way

Instead of editing the state file or letting Terraform destroy and recreate resources, use the built-in state management command:

terraform state mv aws_instance.web_server aws_instance.app_server

This safely moves the resource address in the state file without touching the actual EC2 instance. Terraform will now recognize the resource under its new name, keeping your infra intact.

🚀 Best Practice Recap

• Never open and edit terraform.tfstate directly.

• Always use Terraform CLI (terraform state mv, terraform state rm, terraform import, etc.) to manage resources.

• Commit your config changes along with proper state moves to avoid drift.

Remember: your state file is like the brain of Terraformdont perform surgery on it without the right tools. 🧠🔧

Awesome pointer 🙌 This is the heart of the blogthe hands-on demonstration that ties everything together. Ill write it step by step in a practical, tutorial-like tone while reinforcing the 5 best practices we already covered. Heres the draft:

6. Practical Demonstration 🛠

Now that weve gone through the 5 Terraform security best practices, lets put them into action. Nothing beats seeing these principles applied in a real-world example.

Well set up a simple Terraform configuration that:

• Pins the provider source and version.

• Stores the state file securely in S3 with encryption + locking.

• Implements Principle of Least Privilege (PoLP) with IAM policies.

• Demonstrates state management commands instead of manual edits.

• Uses Terrascan to detect misconfigurations early.

Ready? Lets dive in 🚀

To make things easy, I have hosted the code of this project on my Github account, just got the following repo:
Best-Practices project
Navigate inside the best-practices dir.

Step 1: Create main.tf and Specify Provider

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "6.14.1"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

Here, weve pinned the provider source and version (hashicorp/aws at 6.14.1) and specified the region. This ensures consistency and avoids pulling in unverified updates.

Step 2: Add an EC2 Instance with PoLP IAM Policy

Now lets create an EC2 instance and enforce Principle of Least Privilege by attaching an IAM policy that allows it to only read the Terraform state file from S3.

# IAM Policy Document for S3 State Access
data "aws_iam_policy_document" "s3_state_access_doc" {
  statement {
    sid    = "AllowListAndLocation"
    effect = "Allow"
    actions = [
      "s3:ListBucket",
      "s3:GetBucketLocation"
    ]
    resources = ["arn:aws:s3:::my-pravesh-terraform-state-bucket-2025"]
  }

  statement {
    sid    = "AllowReadStateFile"
    effect = "Allow"
    actions = [
      "s3:GetObject"
    ]
    resources = ["arn:aws:s3:::my-pravesh-terraform-state-bucket-2025/terraform/terraform.tfstate"]
  }
}

# Create the IAM Policy
resource "aws_iam_policy" "s3_state_access_policy" {
  name        = "EC2-S3-State-Read-Policy"
  description = "Policy for EC2 to read the Terraform state file."
  policy      = data.aws_iam_policy_document.s3_state_access_doc.json
}

# Create the IAM Role
resource "aws_iam_role" "ec2_s3_role" {
  name               = "EC2-S3-State-Reader-Role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Action = "sts:AssumeRole",
        Effect = "Allow",
        Principal = {
          Service = "ec2.amazonaws.com"
        },
        Sid = ""
      },
    ],
  })
}

# Attach Policy to Role
resource "aws_iam_role_policy_attachment" "s3_state_attach" {
  role       = aws_iam_role.ec2_s3_role.name
  policy_arn = aws_iam_policy.s3_state_access_policy.arn
}

# Create the Instance Profile
resource "aws_iam_instance_profile" "ec2_s3_profile" {
  name = "EC2-S3-State-Reader-Profile"
  role = aws_iam_role.ec2_s3_role.name
}

# Get Latest Ubuntu AMI
data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-*-amd64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}

# Create the EC2 Instance
resource "aws_instance" "web_server" {
  ami                  = data.aws_ami.ubuntu.id
  instance_type        = "t2.micro"
  key_name             = "default-ec2" # <<-- Update this with your SSH key
  iam_instance_profile = aws_iam_instance_profile.ec2_s3_profile.name

  tags = {
    Name = "testing-instance"
  }
}

🔑 Notes:

• Make sure you have an SSH key pair (default-ec2) in us-east-1. If not, create one in the AWS Console.

• Ensure your default security group in the default VPC allows inbound traffic on Port 22 (SSH).

Step 3: Configure Backend (backend.tf)

terraform {
  backend "s3" {
    bucket       = "my-pravesh-terraform-state-bucket-2025"
    key          = "terraform/terraform.tfstate"
    region       = "us-east-1"
    use_lockfile = true
    encrypt      = true
  }
}

This stores the state file remotely in S3 with encryption + locking enabled.

Step 4: Create the S3 Bucket

aws s3 mb s3://my-pravesh-terraform-state-bucket-2025

Step 5: Initialise and Deploy

terraform init --upgrade
terraform plan
terraform apply --auto-approve

💡 Terraform should show 5 resources to add. Within 23 minutes, your infrastructure will be ready!

Step 6: Test PoLP in Action

1. Go to your AWS Console EC2 Dashboard Select testing-instance Click Connect.

2. SSH into the instance.

3. Install AWS CLI:

sudo apt update
sudo apt install awscli -y

4. Try listing objects in the S3 bucket (works ):

aws s3 ls s3://my-pravesh-terraform-state-bucket-2025/terraform

5. Try deleting the state file (blocked ):

aws s3 rm s3://my-pravesh-terraform-state-bucket-2025/terraform/terraform.tfstate

Youll get AccessDenied, proving our PoLP IAM policy works! 🔐

Step 7: Scan with Terrascan

Install Terrascan from GitHub.

Then run it against your Terraform code:

terrascan scan -f main.tf

Terrascan will flag any misconfigurations or potential security risks before deployment.

Step 8: Safe State Management

Now lets rename our EC2 resource without downtime.

Instead of changing web_server app_server in code and letting Terraform destroy/recreate the resource, use:

terraform state mv aws_instance.web_server aws_instance.app_server

You can verify the update by downloading the state file from the S3 console. No downtime, no driftjust clean state management.

Step 9: Clean up

Once you are done with the project, make sure to use the following command to delete the resources to avoid incurring charges:

terraform destroy --auto-approve

aws s3 rm s3://my-pravesh-terraform-state-bucket-2025 --recursive
aws s3 rb s3://my-pravesh-terraform-state-bucket-2025

🎯 And thats it!

🏁 Conclusion

In this project, we walked through five essential Terraform security best practices and then implemented them in a hands-on demonstration. From enabling state file encryption, configuring remote backends, enforcing version pinning, and using terrascan for security scanning, to applying the Principle of Least Privilege (PLoP) with IAM roles and policies we covered a complete workflow to secure Terraform in production-grade environments.

Security is never a one-time task; its an ongoing process. As your infrastructure grows, make sure you constantly revisit and update your security practices to minimize risks. With Terraform, its not just about provisioning resources its about doing so securely, scalably, and responsibly.

If you found this guide useful, feel free to connect with me and check out more of my work 👇

🔗 Connect with me

• 🌐 Website/Blog: https://blog.praveshsudha.com

• 💼 LinkedIn: https://www.linkedin.com/in/pravesh-sudha/

• 🐦 Twitter/X: https://x.com/praveshstwt

• 🎥 YouTube: https://www.youtube.com/@pravesh-sudha

👋 Adios, see you in next one!

Welcome to the world of pipelines and automation 🚀. In this guide, were going to uncover an exciting project where we deploy a Node.js + Redis web app on AWS using Terraform and GitHub Actions for seamless integration.

This walkthrough is designed with beginners in mindso even if youre just starting with DevOps or cloud automation, youll be able to follow along step by step.

The application itself is simple but practical: a Request Counter app that tracks the number of visits and stores the data in Redis. But the real magic isnt the appits the way well set up Infrastructure as Code (IaC) with Terraform, integrate it with GitHub Actions CI/CD, and see how everything ties together.

So without further ado, lets roll up our sleeves and get hands-on with some practical knowledge 💡.

🛠 Pre-requisites

Before diving into the project, lets make sure our host system is ready with all the essentials. Heres what youll need:

• AWS Account + IAM User with Admin Access (Only for testing purposes, in prod, always use PLOP)
  Since well be spinning up infrastructure on AWS, you need an IAM user with proper permissions and AWS CLI configured locally.
  👉 If youre new to this, dont worryIve explained the entire setup process (IAM user, AWS CLI, and Terraform installation) in one of my earlier blogs:
  Learn How to Deploy a Three-Tier Application on AWS EKS Using Terraform with Best Practices

• Docker & Docker Compose
  Our application runs in containers, so Docker and Docker Compose are a must. You can install them easily from their official documentation:
  Install Docker

Once these are configured, were all set to start building and automating this project!

💡 Video Demonstration

🚀 Getting Started with the Project

The complete code for this project is available on my GitHub repository:
👉 nginx-node-redis

To begin, fork this repository under your own GitHub username and then clone it locally using the commands below:

git clone https://github.com/<your-username>/nginx-node-redis.git
cd nginx-node-redis/

Why this repo?
The project originally comes from Dockers official dockersamples collection. Its a simple Node.js + Redis request counter application with an NGINX load balancer in front. Every time you refresh the page, the counter increments and you also see the hostname (web1 or web2) that served your request.

Ive taken that base project and leveled it up to make it more attractive and production-friendly.

🔑 Key Highlights of the Project

• web/server.js Basic Node.js app that connects to Redis on port 6379 and returns:

  • The number of visits (increments with every refresh)

  • The hostname (web1 or web2) serving the request

  • Runs on port 5000

  • Now enhanced with a beautiful HTML UI instead of plain text 🎨

• nginx/nginx.conf Custom configuration with:

  • An upstream load balancer pointing to web1:5000 and web2:5000

  • Proxy pass rules to evenly distribute traffic

  • Dockerfile that replaces the default NGINX config with our custom one

• docker-compose.yml Glues everything together:

  • Redis container (database)

  • Two Node.js containers (web1 and web2)

  • NGINX container (acting as a reverse proxy and load balancer)

My Enhancements

• Improved UI in web/server.js

• 📦 Added Terraform configuration (terra-config/) to deploy the app on AWS

• Added GitHub Actions workflow (.github/workflows/main.yml) for CI/CD automation

With these changes, the project is no longer just a demoits a mini production-grade system that you can deploy with one push.

🐳 Testing Locally with Docker Compose

Before deploying this app on AWS with Terraform, lets test it locally to make sure everything works as expected.

Navigate to the project root directory and run the following command:

docker-compose up --build

This will:

• Build Docker images for Redis, Node.js web servers, and NGINX

• Start up all the containers in the correct order

• Expose the application through NGINX on port 80

If everything is set up properly, youll see logs confirming the containers are running. Most importantly, look for this message in your terminal:

The web server is listening on Port 5000.

Now, open your browser and go to 👉 http://localhost:80

🎉 You should see the application live!

• Every time you refresh the page, the counter will increment by +1

• The hostname will change between web1 and web2, showing that load balancing is working perfectly

This step confirms that the Docker + NGINX + Redis setup is solid before we move on to cloud deployment.

Integrating GitHub Actions and Terraform

Now that our app works locally, lets automate deployment with GitHub Actions + Terraform. This will allow us to:

• Provision infrastructure on AWS automatically

• Deploy the app onto an EC2 instance

• Run health checks

• Tear down resources after testing (to save 💰 on AWS bills)

🔑 Step 1: Add AWS Secrets

1. Go to your GitHub Dashboard open the nginx-node-redis repo

2. Navigate to Settings > Secrets and Variables > Actions

   

3. Add the following Repository Secrets (from the IAM user you created earlier):

   • AWS_ACCESS_KEY_ID

   • AWS_SECRET_ACCESS_KEY

🏗 Step 2: Terraform Configuration

Inside the terra-config/main.tf file:

• Uses the default VPC

• Fetches the latest Ubuntu AMI

• Creates a Security Group with port 22 (SSH) and port 80 (HTTP) open

• Provisions a t2.micro EC2 instance using that SG and AMI

• Includes a user-data script that installs Docker, Docker Compose, and runs our app

📌 In short: Terraform handles all infra creation and boots up an EC2 instance that runs our Request Counter app.

Important Change:
In the user_data section of main.tf, update the GitHub repo URL to point to your forked repo:

git clone https://github.com/<your-github-username>/nginx-node-redis.git

Step 3: GitHub Actions Workflow

The .github/workflows/main.yml pipeline is triggered on push or pull requests. Heres what happens step by step:

1. Checkout the repository

2. Setup Terraform

3. 📂 Run terraform init, terraform validate, terraform plan

4. 🚀 Apply infrastructure with terraform apply

5. Wait 90 seconds allows user-data to finish setting up Docker + app

6. 🌍 Fetch the Public IP of the EC2 instance

7. 🔎 Health check ensure the app is live

8. Keep the application running for 5 minutes

9. 🧹 Auto teardown with terraform destroy ensures no unused AWS resources are left behind

Step 4: Trigger the Workflow

Once youve updated your repo URL in main.tf, commit and push changes:

git status
git add terra-config/main.tf
git commit -m "Updated GitHub repo URL"
git push origin main

This push will trigger the GitHub Actions workflow, automatically deploying your app to AWS.

🚀 Testing the Deployment on AWS

Once everything is set up, its time to watch the magic happen .

1. Go to your GitHub Repository Dashboard

2. Click on Actions

   

3. Find your recent commit for example:

    "Updated GitHub repo URL"
   

4. Open the workflow run youll see the terraform job executing step by step.

🟢 Wait until the workflow reaches the Keep App Running Stage. At this point, Terraform has already created your EC2 instance, installed Docker & Docker Compose, and started the application.

👉 Click on the Public URL shown in the logs, and Viola! 🎉
Your Request Counter App is now live on an AWS EC2 instance 🚀.

• You can interact with the app for 5 minutes

• Every refresh increments the counter

• Requests are distributed between web1 and web2 via the Nginx load balancer

After 5 minutes, GitHub Actions will automatically run terraform destroy to clean up resources and avoid unnecessary AWS charges 💸.

🔄 Real-Life Scenario: Updating Your App on the Fly

Now that our app is live, lets simulate a real production change.

Imagine your manager says:

"Hey, can you update the heading from Welcome to Request Counter to Welcome to My Amazing Request Counter?"

Heres how simple it becomes with GitHub Actions + Terraform:

1. Open the web/server.js file in your project

2. Find the <h1> HTML tag and update it:

    <h1>Welcome to My Amazing Request Counter</h1>
   

   

3. Push the changes to your repository:

    git status
    git add web/server.js
    git commit -m "Heading Changed"
    git push origin main
   

   

🎉 Thats it! GitHub Actions will pick up the changes, trigger the workflow, and within 5 minutes your updated heading will be live on the EC2 instance.

This is the true power of CI/CD pipelines no manual SSH into servers, no redeploying by hand. Just push your code and let Terraform + GitHub Actions handle the rest 🚀.

🏁 Conclusion

In this blog, we took a simple Node.js + Redis counter application and supercharged it with Nginx, Docker, Terraform, and GitHub Actions.

What started as a local demo quickly transformed into a cloud-ready, automated CI/CD pipeline:

• 🚀 Docker + Docker Compose handled local testing and containerization

• Terraform provisioned AWS infrastructure seamlessly

• 🔄 GitHub Actions automated deployments and teardown, giving us a clean, cost-effective workflow

• 🎯 And most importantly we saw how easy it is to make real-time changes that go live with just a git push.

This project proves how infrastructure as code + automation can save developers hours of repetitive work and make production-ready workflows more reliable.

Thanks for following along I hope this inspires you to build your own Terraform + GitHub Actions pipelines!

📬 Lets Connect:

• 🌐 Website: praveshsudha.com

• 💼 LinkedIn: Pravesh Sudha

• 🐦 Twitter/X: @praveshstwt

• 📺 YouTube: Pravesh Sudha

Welcome Devs 👋 to the world of AI and Automation.

In this blog, Ill be sharing my experience of building an AI Agent that writes personalized cover letters for my freelance gigs powered by Portia AI and Gemini. As a full-time freelancer, writing cover letters for every single project often feels repetitive and time-consuming. Thats exactly where this idea was born. The journey of making this project was full of ebbs and flows debugging errors, testing prompts, tweaking workflows but in the end, it was worth it. The biggest takeaway? 🤔 If you use AI agents wisely, they can save you countless hours every week while still keeping your work personal and professional. So without further ado, lets dive into how I built this project, what challenges I faced, and how Portia helped me automate one of the most tiring parts of freelancing: writing cover letters.

💡 Youtube Demo

💡 The Problem

As a full-time freelancer, one of the most time-consuming parts of my workflow is writing personalized cover lettersfor each new opportunity.

Dont get me wrong being thoughtful and specific in a proposal is important. It shows effort and increases the chances of landing the gig. But after doing it over and over again, it quickly becomes:

• Repetitive 🌀

• Exhausting 😮💨

• And honestly, inefficient

I found myself spending hours every week just writing cover letters time that could have been used to actually work on projects.

Thats when I realized I needed a smarter solution. A tool that could:

• Read documents containing the job description along with my portfolio.

• Generate a personalized cover letter based on that information.

• Send the cover letter via email directly to the job poster. And if no email is provided, then simply mail it to myself to keep it safe.

This became the spark for building my own AI-powered freelancing assistant 🚀.

🚀 The Journey

After deciding to build my AI-powered Cover Letter Generator, I rolled up my sleeves and dived into the Portia AI docs.

At first, Ill admit it felt overwhelming. Theres a lot to take in when youre just starting with AI agents. But after tinkering around for a while, I started to get the hang of it.

My first step was creating a Google API Key using Google AI Studio for the Gemini-2.0-flash model. I thought Id start simple, so I ran a basic 2 + 2 test. To my surprise it failed. 🙃

The error? Something about missing API keys even though I had provided them correctly. Thats when I reached out to the Portia Team on Discord. And heres the cool part: none other than Emma Burrows, Co-founder & CTO of Portia, personally helped me troubleshoot it. Turns out, it wasnt the API at all it was a dependency issue. Once I fixed that, everything clicked into place. That was my first small win .

Next, I wanted to see how Portia works with Google Apps like Gmail and Calendar. So I went through their official examples on GitHub. That gave me clarity on how to structure tool usage and start building my own agent.

From there, I built the first version of my program (app.py). It had all the pieces I needed:

• Reading the job description doc + my portfolio doc.

• Extracting recipient emails.

• Generating a personalized cover letter.

• Sending it via Gmail automatically.

from dotenv import load_dotenv
import os

from portia import (
    Config,
    DefaultToolRegistry,
    Portia,
    LLMProvider,
    ActionClarification,
    InputClarification,
    MultipleChoiceClarification,
    PlanRunState,
)

load_dotenv(override=True)

job_doc_title = os.getenv("JOB_DOC_TITLE", "Job").strip()
portfolio_doc_title = os.getenv("PORTFOLIO_DOC_TITLE", "Portfolio").strip()
output_doc_title = os.getenv("OUTPUT_DOC_TITLE", "Cover-letter").strip()

self_email = os.getenv("SELF_EMAIL", "").strip()
if not self_email:
    self_email = input("Enter your fallback email (SELF_EMAIL):\n").strip()

socials_block = os.getenv("SOCIALS", "").strip()
if not socials_block:
    parts = []
    for key, label in [
        ("LINKEDIN_URL", "LinkedIn"),
        ("TWITTER_URL", "X/Twitter"),
        ("YOUTUBE_URL", "YouTube"),
        ("WEBSITE_URL", "Website"),
    ]:
        v = os.getenv(key, "").strip()
        if v:
            parts.append(f"{label}: {v}")
    socials_block = "\n".join(parts) if parts else "(Add your socials here)"

constraints: list[str] = []


def task_text() -> str:
    """
    We describe *what* to do. Portia will pick the right tools:
    - Google Drive: search files
    - Google Docs: read contents
    - Gmail: draft/send email
    - (Attempt) create a new Google Doc for the cover letter
    """
    return f"""
You are an expert career coach and outreach copywriter. Use available Google tools to complete the flow below.

Inputs:
- Job doc title: "{job_doc_title}"
- Portfolio doc title: "{portfolio_doc_title}"
- Output Google Doc title: "{output_doc_title}"
- Fallback recipient email (if none in Job doc): "{self_email}"
- Socials block (append after sign-off):
{socials_block}

Heres what I need you to do:  

1. Locate the Google Doc that matches the job title and the one that matches my portfolio. If either is missing or there are multiple, stop and ask me.

2. Read both docs carefully. From the job doc, identify the role, company, and (if available) the recipient email.
    - If no recipient email is found, use the fallback.

3. Write a tailored cover letter (200350 words).
    - Inputs: ONLY the job doc content, portfolio content
    - Begin with a strong opening.
    - Highlight 23 achievements that connect my skills with the job requirements.
    - Keep it professional, concise, and specific.
    - After the closing and my name, add the socials block.

4. Send the cover letter via Gmail.
    - Inputs: the cover letter text + the recipient email.
    - Subject: Cover letter  Pravesh Sudha  AWS Community Builder
    - Body: Use the same cover letter text, then include the Google Doc link at the end if available. 

Rules:  
- When using tools (Drive, Docs, Gmail), respond only with the correct tool call JSON (no explanations, no markdown).  
- If you need my input (e.g., choosing between multiple docs, missing info), pause and ask.  
- If a Google Doc cannot be created, fallback to emailing me a draft with the letter.  
"""


print("\n📋 Generating a plan with Portia ...\n")

config = Config.from_default(
    llm_provider=LLMProvider.GOOGLE,
    default_model="google/gemini-2.0-flash",  # fast + capable for writing
    enforce_schema=True,
)

tool_registry = DefaultToolRegistry(config)
portia = Portia(config=config, tools=tool_registry)

plan = portia.plan(task_text())
print("Here are the steps in the generated plan:\n")
print(plan.pretty_print())

while True:
    yn = input("\nAre you happy with the plan? (y/n):\n").strip().lower()
    if yn == "y":
        break
    extra = input("Any additional guidance for the planner?:\n").strip()
    if extra:
        constraints.append(extra)
    plan = portia.plan(task_text())
    print("\nUpdated plan:\n")
    print(plan.pretty_print())


print("\n🚀 Executing the plan ...\n")
plan_run = portia.run_plan(plan)

while plan_run.state == PlanRunState.NEED_CLARIFICATION:
    for clarification in plan_run.get_outstanding_clarifications():
        if isinstance(clarification, (InputClarification, MultipleChoiceClarification)):
            prompt = clarification.user_guidance or "Please provide a value"
            # Some multiple-choice clarifications expose options
            options = getattr(clarification, "options", None)
            if options:
                prompt += "\n" + "\n".join(str(o) for o in options)
            user_input = input(prompt + "\n")
            plan_run = portia.resolve_clarification(clarification, user_input, plan_run)

        elif isinstance(clarification, ActionClarification):
            # Typically OAuth. Youll get a link to click.
            print("\n🔐 Authorization required. Open this link to continue:")
            print(clarification.action_url)
            print("Waiting for authorization to complete...\n")
            plan_run = portia.wait_for_ready(plan_run)


    plan_run = portia.resume(plan_run)


print("\n Plan run complete. Raw output below:\n")
print(plan_run.model_dump_json(indent=2))

I saved all my sensitive info in a .env file (API keys, fallback emails, socials), making the setup clean and flexible.

PORTIA_API_KEY="your-portia-key"
GOOGLE_API_KEY="your-gemini-key"

# Optional (defaults shown in code)
JOB_DOC_TITLE="Job"
PORTFOLIO_DOC_TITLE="Portfolio"
OUTPUT_DOC_TITLE="Cover-letter"

# Email handling
SELF_EMAIL="example@gmail.com"   # used if Job doc has no email

# Socials  either provide a single SOCIALS block or individual links
LINKEDIN_URL="Your-Linkedin"
TWITTER_URL="Your-Twitter"
YOUTUBE_URL="Your-Youtube"
WEBSITE_URL="Your-Website"

Heres the repo if you want to check out the code: 👉 GitHub agent-hack-2025

Of course, the road wasnt smooth. Along the way, I hit service quota limits, tried different LLMs like OpenAI, Anthropic, and Mistral none of them worked seamlessly. Eventually, I generated a new Google API key on a different account, which fixed the quota issue.

But then came the real grind: getting the task prompt right. Portia is powerful, but it requires clear instructions. I had to refactor the prompt over 24 times before the execution flow was smooth. Finally, the magic happened:

• Portia read the job + portfolio docs,

• Generated a crisp cover letter,

• And emailed it directly to the client.

Seeing that email land in the inbox felt dope. 😎

How It Works

Now that the agent was finally up and running, heres what the flow looks like in action:

1. Input Stage Job & Portfolio Docs 📂
   The agent starts by reading two files:

   • A Job Description doc (the opportunity I want to apply for).

   • My Portfolio doc (skills, past projects, and relevant achievements).

This ensures every cover letter is tailored, not generic.

2. Processing Stage Gemini + Portia Magic

   • Portia passes the docs into the Gemini model.

   • Gemini analyzes the job requirements alongside my portfolio.

   • The output? A personalized cover letter that feels thoughtful, not copy-pasted.

3. Email Handling 📧

   • If the clients email is available in the doc, the cover letter gets sent directly to them.

   • If no email is found, the agent sends the cover letter to my fallback email (saved in .env), so I can reuse it later.

4. Smart Fail-safes 🛡

   • All sensitive data (API keys, fallback emails, socials) are stored securely in .env.

   • If an email fails, the letter isnt lost I still get it in my inbox.

5. Automation in Action 🚀

   • No more copy-pasting job descriptions.

   • No more staring at a blank page wondering how to start a letter.

   • Everything is automated: read generate email done.

Heres a simple visual flow for clarity:

Job Doc + Portfolio Gemini via Portia Cover Letter Gmail Client/Me

What used to take me 3040 minutes per job now takes less than 30 seconds. Thats the kind of efficiency freelancers dream of.

📄 Portfolio Doc

📄 Job Doc

💡 The Impact

This small project ended up creating a huge impact on my workflow as a freelancer.

• What once took me hours every week now happens in seconds.

• Instead of wasting energy on repetitive writing, I can focus more on client work and skill-building.

• The best part? Each cover letter still feels personalized and human, not like a bland AI template.

Its not just about saving time its about working smarter. AI agents like this one give freelancers the leverage to scale their efforts without burning out.

🚀 Conclusion

Building this project with Portia AI and Gemini was more than just a hackathon entry it was a lesson in how AI agents can redefine productivity for freelancers.

As a full-time freelancer, I know the hustle of juggling multiple gigs, proposals, and deadlines. With Portia, Ive turned a time-consuming, repetitive task into something swift, efficient, and stress-free.

This is just the beginning AI agents are not here to replace us, but to empower us. If we learn how to harness them wisely, they can free us from routine work and let us focus on what truly matters: creating, solving, and delivering value.

Thanks for reading my journey! 💙
If you found this project inspiring or want to connect, feel free to reach out to me on:

• 🌐 Website: praveshsudha.com

• 💼 LinkedIn: linkedin.com/in/pravesh-sudha

• 🐦 Twitter/X: x.com/praveshstwt

• 📹 YouTube: @pravesh-sudha

Welcome, Devs, to the world of cloud and automation! 🚀

If youve been following my blogs, you might remember that a few months ago I published an experimental project where I created an Employee Management Application using Django and migrated its database to Amazon RDS.

Recently, after clearing my AWS Solutions Architect Associate exam 🎉, I started reflecting on some of the exam questions around resilient and highly available architectures. That sparked an idea why not take my earlier project and level it up to meet production-grade standards?

So, in this blog, well upgrade the Employee Management App and deploy it on AWS using Terraform. Our end goal? A highly scalable and highly available architecture, following real-world market practices.

Without further ado, lets dive in! 🏗

💡 Pre-Requisites

Before we jump into building our scalable Django setup, lets make sure youve got the essentials ready.

1. AWS Account Youll need an AWS account with an IAM user that has AdministratorAccess permissions (only for the sake of this project in a production environment, you should always follow the principle of least privilege).

2. AWS CLI Installed Download and install the AWS CLI on your local machine.

3. Configure IAM User Run aws configure and provide the IAM users credentials to set up AWS CLI authentication.

4. Terraform CLI Installed Make sure Terraform is installed so we can define and deploy our infrastructure as code.

Once these items are checked off your list, youre all set to start this awesome project. 🚀

💡 Youtube Demonstration

💡 Setting Up Terraform Remote Backend & Secrets

Since this project follows best practices for hosting a highly scalable and highly available application on AWS, we wont be storing our Terraform state file locally. Instead, well store it remotely to ensure team collaboration, backup, and resilience.

For this setup, Im using Amazon S3 (with versioning enabled) to store the state file, and DynamoDB for state locking. Interestingly, while browsing the AWS Docs, I noticed that the S3 + DynamoDB combination is now deprecated in favor of a use_lockfile tag for S3 state locking. However, for some reason, the new method didnt work on my machine so I decided to stick with the tried-and-tested S3 + DynamoDB approach.

The complete Terraform code for this project is available on my GitHub:
https://github.com/Pravesh-Sudha/terra-projects

Navigate to the scripts directory:

cd terra-projects/two-tier-app/scripts

First, well create the S3 bucket and DynamoDB table for our Terraform backend. Ive created a config.sh script for this:

chmod u+x config.sh
./config.sh

This will create:

• Bucket: pravesh-tf-two-tier-bucket

• Table: pravesh-state-table

If you get an error that the bucket name is already taken, simply edit the name in the config.sh file and try again. Just remember to update the same name inside terra-config/backend.tf.

Next, since well be deploying an RDS instance for our Django app, we need to securely store the database credentials. For this, well use AWS Secrets Manager. In the scripts/commands.md file, youll find the exact commands, but heres the process:

Create AWS Secrets for RDS Instance

aws secretsmanager create-secret \
  --name "employee-mgnt/rds-credentials" \
  --secret-string '{"username":"admin","password":"admin1234"}' \
  --region us-east-1

Create Policy for IAM User

aws iam create-policy --policy-name TerraformSecretsRead --policy-document '{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret"
      ],
      "Resource": "arn:aws:secretsmanager:us-east-1:<account-id>:secret:employee-mgnt/rds-credentials-*"
    }
  ]
}'

Attach the Policy to IAM User

aws iam attach-user-policy --user-name <iam-user-name> --policy-arn arn:aws:iam::<account-id>:policy/TerraformSecretsRead

Replace <account-id> and <iam-user-name> with your own values before running the commands.

With this, our Terraform remote backend and secure database credentials are ready, and we can now move on to configuring our infrastructure.

💡Understanding the Terraform Configuration

Before we run any commands, lets first understand what our Terraform code does and how it works under the hood. Ill break down each .tf file so you can see exactly how all the pieces fit together.

provider.tf

provider "aws" {
  region = var.aws_region
}

Were telling Terraform that our cloud provider is AWS and setting the region via the variable aws_region (default: us-east-1).

variables.tf

variable "aws_region" {
  default = "us-east-1"
}
variable "project_name" {
  default = "employee-mgnt"
}
variable "vpc_cidr" {
  default = "10.0.0.0/16"
}
variable "public_subnet_cidrs" {
  default = ["10.0.1.0/24", "10.0.2.0/24"]
}
variable "private_subnet_cidrs" {
  default = ["10.0.11.0/24", "10.0.12.0/24"]
}
variable "instance_type" {
  default = "t3.micro"
}

Here we define key parameters like VPC CIDR range, public/private subnets, EC2 instance type, and project name.

backend.tf

terraform {
  backend "s3" {
    bucket         = "pravesh-tf-two-tier-bucket"
    key            = "terraform/terrform.tfstate"
    region         = var.aws_region
    dynamodb_table = "pravesh-state-table"
    encrypt        = true
  }
}

This is our Terraform remote backend configuration. Were storing the state file in S3 (with encryption) and using DynamoDB for state locking.

vpc.tf

We use the official terraform-aws-vpc module to create:

• VPC with custom CIDR

• Public & private subnets

• NAT Gateway for outbound internet in private subnets

• DNS hostnames & support enabled

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"

  name = var.project_name
  cidr = var.vpc_cidr

  azs             = slice(data.aws_availability_zones.available.names, 0, 2)
  private_subnets = var.private_subnet_cidrs
  public_subnets  = var.public_subnet_cidrs

  enable_nat_gateway   = true
  single_nat_gateway   = false
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Project = var.project_name
  }
}

security_groups.tf

We define three security groups:

1. Load Balancer SG Allows HTTP (80) & HTTPS (443) from the internet.

2. App SG Allows traffic on port 8000 only from the ALB SG.

3. RDS SG Allows MySQL traffic (3306) only from the App SG.

This creates a secure, layered approach only necessary services talk to each other.

# Load-Balancer Security Group
resource "aws_security_group" "alb_sg" {
  name        = "${var.project_name}-alb_sg"
  description = "Allow HTTP/HTTPS access from Internet"
  vpc_id      = module.vpc.vpc_id

  tags = {
    Name = "alb_sg"
  }
}

resource "aws_vpc_security_group_ingress_rule" "alb_sg_http_ipv4" {
  security_group_id = aws_security_group.alb_sg.id
  cidr_ipv4         = "0.0.0.0/0"
  from_port         = 80
  ip_protocol       = "tcp"
  to_port           = 80
}

resource "aws_vpc_security_group_ingress_rule" "alb_sg_https_ipv4" {
  security_group_id = aws_security_group.alb_sg.id
  cidr_ipv4         = "0.0.0.0/0"
  from_port         = 443
  ip_protocol       = "tcp"
  to_port           = 443
}

resource "aws_vpc_security_group_egress_rule" "alb_sg_egress" {
  security_group_id = aws_security_group.alb_sg.id
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "-1"
}

# Frontend (Application) Security Group
resource "aws_security_group" "app_sg" {
  name        = "${var.project_name}-app_sg"
  description = "Allow port 8000 access from ALB only"
  vpc_id      = module.vpc.vpc_id

  tags = {
    Name = "app_sg"
  }
}

resource "aws_vpc_security_group_ingress_rule" "app_sg_from_alb" {
  security_group_id            = aws_security_group.app_sg.id
  referenced_security_group_id = aws_security_group.alb_sg.id
  from_port                    = 8000
  ip_protocol                  = "tcp"
  to_port                      = 8000
}

resource "aws_vpc_security_group_egress_rule" "app_sg_to_rds" {
  security_group_id = aws_security_group.app_sg.id
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "-1"
}

# Database (Backend) Security Group
resource "aws_security_group" "rds_sg" {
  name        = "${var.project_name}-rds_sg"
  description = "Allow port 3306 access from App SG only"
  vpc_id      = module.vpc.vpc_id

  tags = {
    Name = "rds_sg"
  }
}

resource "aws_vpc_security_group_ingress_rule" "rds_sg_from_app" {
  security_group_id            = aws_security_group.rds_sg.id
  referenced_security_group_id = aws_security_group.app_sg.id
  from_port                    = 3306
  ip_protocol                  = "tcp"
  to_port                      = 3306
}

resource "aws_vpc_security_group_egress_rule" "rds_sg_egress" {
  security_group_id = aws_security_group.rds_sg.id
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "-1"
}

data.tf

We fetch:

• Available AZs in the region

• Latest Ubuntu AMI

• Database credentials from AWS Secrets Manager

We also define userdata for EC2 bootstrap configuration.

data "aws_availability_zones" "available" {
  state = "available"
}

data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["amazon"]
  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
  }
}

data "aws_secretsmanager_secret_version" "rds_credentials" {
  secret_id = "employee-mgnt/rds-credentials"  
}

locals {
  userdata = templatefile("${path.module}/userdata.tpl", {
    DB_NAME     = "employee_db"
    DB_USER     = jsondecode(data.aws_secretsmanager_secret_version.rds_credentials.secret_string)["username"]
    DB_PASSWORD = jsondecode(data.aws_secretsmanager_secret_version.rds_credentials.secret_string)["password"]
    DB_PORT     = "3306"
    DB_HOST = aws_db_instance.rds_instance.endpoint
  })
}

alb.tf

Creates:

• Application Load Balancer in public subnets

• Target Group (port 8000, health check at /health)

• Listener on port 80 forwarding requests to the target group

resource "aws_lb" "app_alb" {
  name               = "${var.project_name}-alb"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.alb_sg.id]
  subnets            = module.vpc.public_subnets
}

resource "aws_lb_target_group" "app_tg" {
  name     = "${var.project_name}-tg"
  port     = 8000
  protocol = "HTTP"
  vpc_id   = module.vpc.vpc_id
  health_check {
    path                = "/health"
    protocol            = "HTTP"
    matcher             = "200"
    healthy_threshold   = 2
    unhealthy_threshold = 3
    interval            = 30
    timeout             = 5
  }
}

resource "aws_lb_listener" "app_listener" {
  load_balancer_arn = aws_lb.app_alb.arn
  port              = "80"
  protocol          = "HTTP"

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.app_tg.arn
  }
}

iam.tf

Sets up:

• IAM Role for EC2

• SSM Policy attachment (for remote management)

• Instance Profile for EC2 instances

resource "aws_iam_role" "ec2-role" {
  name = "ec2-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      },
    ]
  })
}

resource "aws_iam_role_policy_attachment" "ssm_policy" {
  role       = aws_iam_role.ec2-role.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}

resource "aws_iam_instance_profile" "instance_profile" {
  name = "${var.project_name}-instance_profile"
  role = aws_iam_role.ec2-role.name
}

asg_launch_template.tf

Defines:

• Launch Template with Ubuntu AMI, instance type, security group, and bootstrap script

• Auto Scaling Group across private subnets with ALB health checks

resource "aws_launch_template" "lt" {
  name_prefix   = "${var.project_name}-lt-"
  image_id      = data.aws_ami.ubuntu.id
  instance_type = var.instance_type

  iam_instance_profile {
    name = aws_iam_instance_profile.instance_profile.name
  }

  network_interfaces {
    security_groups             = [aws_security_group.app_sg.id]
    associate_public_ip_address = false
  }

  user_data = base64encode(local.userdata)

}

resource "aws_autoscaling_group" "app-ag" {
  name                = "${var.project_name}-ag"
  max_size            = 2
  min_size            = 1
  desired_capacity    = 1
  vpc_zone_identifier = module.vpc.private_subnets

  launch_template {
    id      = aws_launch_template.lt.id
    version = "$Latest"
  }
  target_group_arns = [aws_lb_target_group.app_tg.arn]

  # Tie ASG health to ALB checks
  health_check_type         = "ELB"
  health_check_grace_period = 120

  tag {
    key                 = "Name"
    value               = "${var.project_name}-app"
    propagate_at_launch = true
  }
}

userdata.tpl

Bootstrap script that:

• Installs dependencies

• Fetches DB credentials from Secrets Manager

• Creates the database if it doesnt exist

• Clones the Employee Management Django app from GitHub

• Runs migrations

• Starts the app with Gunicorn on port 8000

#!/bin/bash

set -e
export DEBIAN_FRONTEND=noninteractive

# basic deps
apt-get update -y
apt-get install -y git python3 python3-pip python3-venv mysql-client libmysqlclient-dev build-essential pkg-config

# export DB env variables (inherited by processes started below)
export DB_NAME="${DB_NAME}"
export DB_USER="${DB_USER}"
export DB_PASSWORD="${DB_PASSWORD}"
export DB_HOST="${DB_HOST}"
export DB_PORT="3306"

echo "DB_NAME=$DB_NAME" >> /home/ubuntu/userdata.log
echo "DB_USER=$DB_USER" >> /home/ubuntu/userdata.log
echo "DB_PASSWORD=$DB_PASSWORD" >> /home/ubuntu/userdata.log
echo "DB_HOST=$DB_HOST" >> /home/ubuntu/userdata.log
echo "DB_PORT=$DB_PORT" >> /home/ubuntu/userdata.log

# Create database if it doesn't exist
mysql -h "$(echo ${DB_HOST} | cut -d : -f1)" -u "${DB_USER}" -p"${DB_PASSWORD}" -P "${DB_PORT}" -e "CREATE DATABASE IF NOT EXISTS ${DB_NAME};" 2>> /home/ubuntu/userdata.log

# Remove existing /home/ubuntu/app directory if it exists
if [ -d "/home/ubuntu/app" ]; then
  rm -rf /home/ubuntu/app
fi

# Clone & install app
git clone https://github.com/Pravesh-Sudha/employee_management.git /home/ubuntu/app 2>> /home/ubuntu/userdata.log
cd /home/ubuntu/app
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt 2>> /home/ubuntu/userdata.log

chown -R ubuntu:ubuntu /home/ubuntu/app


# Small randomized sleep to reduce concurrent migrations
sleep $((RANDOM % 10))

# Retry migrations (5 attempts)
attempt=0
until [ $attempt -ge 5 ]
do
  attempt=$((attempt+1))
  echo "Running migrations (attempt $attempt)..." tee -a /home/ubuntu/userdata.log
  python manage.py migrate --noinput >> /home/ubuntu/migrate.log 2>&1 && break
  echo "Migrate failed, retrying in 5s..." | tee -a /home/ubuntu/userdata.log
  sleep 5
done

# start gunicorn
nohup /home/ubuntu/app/venv/bin/gunicorn --workers 2 --timeout 60 --access-logfile /home/ubuntu/gunicorn-access.log --error-logfile /home/ubuntu/gunicorn-error.log employee_management.wsgi:application --bind 0.0.0.0:8000 &

# Done
echo "user-data finished"

outputs.tf

Displays:

• ALB DNS Name The public URL of our app

• RDS Endpoint The database connection string

output "alb_dns_name" {
  description = "DNS name of the Application Load Balancer"
  value       = aws_lb.app_alb.dns_name
}


output "rds_endpoint" {
  description = "Endpoint of the RDS instance"
  value       = aws_db_instance.rds_instance.endpoint
}

Running the Terraform Code

Now that we understand the structure, lets run it:

cd terra-projects/two-tier-app/terra-config
terraform init
terraform plan
terraform apply --auto-approve

Provisioning will take 1015 minutes, so grab a coffee while AWS spins up your infrastructure.

Once the architecture is ready, open the alb_dns_name in your web browser youll see the Django Employee Management application up and running.

You can now:

• Add employee information and save it.

• Refresh the browser to confirm the data persists, proving its connected to the RDS MySQL database.

• Check the Target Groups in the Load Balancer console to see your healthy instances in action.

  

With this setup, weve successfully deployed a highly available, scalable, and secure Django Employee Management application on AWS, with credentials securely stored in AWS Secrets Manager.

You can check the application logs by SSM session

Clean-Up to Avoid Charges:

After experimenting, run the following command to delete all AWS resources:

terraform destroy --auto-approve

To remove the S3 bucket and DynamoDB table used for Terraform state, navigate to the scripts directory and run:

chmod u+x delete.sh
./delete.sh

This ensures all infrastructure is cleaned up, preventing unnecessary costs.

💡 Conclusion

In this project, we successfully deployed a Django Employee Management Application on AWS using a highly available, scalable, and secure architecture. By integrating ALB, EC2 Auto Scaling, RDS MySQL, Secrets Manager, and an S3 + DynamoDB backend for Terraform state, we created a production-ready setup that can handle growth while keeping credentials safe.

We also explored how the application persists data in RDS, verified healthy instances via Target Groups, and ensured cost optimization by cleaning up resources with terraform destroy and custom scripts. This hands-on project not only strengthens your AWS and Terraform skills but also gives you a blueprint for hosting real-world applications in the cloud.

If you found this guide helpful, dont forget to connect with me and explore more of my work:

• 🌐 Website: praveshsudha.com

• Blog: blog.praveshsudha.com

• 💼 LinkedIn: linkedin.com/in/pravesh-sudha

• 🐦 Twitter/X: x.com/PraveshStwt

• 📺 YouTube: Pravesh-Sudha

Welcome to the world of cloud computing and certifications!
In this blog, Im excited to share my journey of preparing for and clearing the AWS Certified Solutions Architect Associate (SAA-C03) exam. Ill walk you through the strategies, resources, and practical steps that helped me succeed insights that I believe can serve as a helpful guide for anyone aiming to strengthen their AWS skills and gain hands-on experience with core services.

Whether youre just starting with AWS or looking to validate your cloud knowledge through certification, I hope this post will offer practical direction and boost your confidence.
So, without further ado, lets dive in.

💡 Youtube Demonstration

💡 What is the AWS Solutions Architect Associate (SAA-C03) Certification?

The AWS Certified Solutions Architect Associate (SAA-C03) is a mid-level certification designed for individuals who want to demonstrate their ability to design well-architected, cost-effective, scalable, and secure solutions on AWS. It's ideal for candidates with prior cloud knowledge or strong on-premises IT experience looking to validate their architecture skills in the AWS ecosystem.

One of the key advantages of this certification is that it doesn't require deep programming expertise. However, having a working knowledge of programming concepts (especially related to APIs, automation, and infrastructure-as-code) can certainly help during both the preparation and the exam.

This certification validates your understanding of a broad range of AWS services and how to design architectures that meet specific business and technical requirements. In short, it positions you as someone capable of designing reliable cloud environments at scale.

💡 Before diving into your preparation, I highly recommend reading the official exam guide:
AWS SAA-C03 Exam Guide PDF

📘 Exam Overview

• Number of Questions: 65
  (50 scored + 15 unscored experimental questions all are presented equally, so aim to answer all 65)

• Question Types: Multiple choice and multiple response

• Duration: 130 minutes (You can submit early)

• Cost: $150 + taxes

• Passing Score: 720/1000

🧠 Exam Domains Breakdown

Each domain is deeply tied to AWS best practices, the Well-Architected Framework, and real-world solution design scenarios.

💡 My Preparation Journey: From Community Builder to Certified Architect

My certification journey began with a moment of gratitude and opportunity.

In March 2025, I was honored to be selected as an AWS Community Builder under the Containers category. One of the perks of this program is a 100% discount voucher for any AWS Certification examan amazing benefit AWS offers its builders every year. I knew I wanted to use it wisely.

By the end of June, I made up my mind to take on the AWS Solutions Architect Associate exam. Since this was going to be my first cloud certification, I wanted something both challenging and meaningfulsomething that would push me to understand AWS at a deeper architectural level. And SAA-C03 was exactly that.

📚 Getting Started: The Right Learning Path

I kicked off my preparation with Andrew Browns 50-hour AWS Solutions Architect Associate course on FreeCodeCamps YouTube channel:

This course turned out to be a goldmine. It covered all the essential AWS servicesEC2, S3, VPC, RDS, IAM, CloudFront, and many othersalong with real-world architecture use cases and hands-on labs. I dedicated two hours every day, and in about 25 days, I was able to complete the entire course. I then spent two additional days revisiting my notes, reinforcing the concepts and focusing on my weak areas.

No Mocks? Here's What I Did Instead

Now, heres an important point:

I did not take any full-length mock tests during my preparation.

That said, I strongly recommend that you do! Mock tests are incredibly useful for identifying gaps in your understanding and getting familiar with the exam pattern.

Instead, I practiced with over 350 questions from this curated YouTube playlist:

These questions were aligned with the topics covered in the exam and gave me enough confidence to evaluate my readiness.

🔍 Bridging the Gaps with Tutorials Dojo

Even after all that practice, there were still a few services that seemed overlapping in functionality, and I wasnt entirely sure when to use which. Thats when I turned to one of the most reliable resources in the AWS certification world: Tutorials Dojo.

Their Cheat Sheets helped me quickly compare services, understand real-life use cases, and sharpen my differentiation skills between similar offerings like S3 vs. EFS vs. FSx, or ALB vs. NLB vs. Gateway Load Balancer.

🧠 Visit Tutorials Dojo Highly recommended for last-minute revision and concept clarity.

Here is my CheatSheet:

🗓 The Final Step: Booking the Exam

On August 2nd, feeling confident and well-prepared, I used my AWS-provided voucher and scheduled the exam for the very next daySunday, August 3rd.

With solid prep behind me and a calm mindset, I was ready to put my knowledge to the test.

💡 Exam Day: Nerves, Questions, and a Surprise Ending

The big day had finally arrived Sunday, August 3rd. My AWS Solutions Architect Associate exam was scheduled from 12:30 PM to 3:00 PM. I had already set up Pearson VUEs OnVUE application the night before, so by 12:00 PM, I logged in, completed the identity verification and system checks, and waited as the proctor launched my session.

Right on time, at 12:30 PM, the exam began.

The interface was clean, and I quickly found my rhythm. I worked through all 65 questions, staying focused and managing time carefully. I made sure to flag any questions I wasnt fully confident about, which helped during the review.

With just five minutes left, I had completed the exam and went back to review all the questions once. Even though I felt I had performed well, imposter syndrome started to creep in that familiar self-doubt whispered that maybe Id misunderstood the questions, or worse, wasted my only voucher.

The exam results werent immediate; AWS mentioned it could take up to five business days to receive the score. So, to calm my nerves, I decided to distract myself by watching one of Kevin Harts comedy specials and thankfully, it worked. I was laughing and had finally started to unwind.

But then, at around 5:00 PM, I casually checked my email and there it was.

The first email was from Credly, notifying me that I had earned the badge for AWS Certified Solutions Architect Associate. My heart raced. I immediately logged into the AWS Certification portal using my Community Builder ID, and downloaded the results.

To my surprise and delight, I had scored 800 out of 1000.

It was a proud and fulfilling moment. All those hours of study, practice, and note-taking had paid off. Not only had I cleared the exam, but I had done so with a score that validated my understanding and effort.

💡 Whats Next?

Clearing the AWS Solutions Architect Associate exam has not only boosted my confidence but also deepened my understanding of AWS architecture and services. I now feel more prepared to take the next step in my cloud journey.

Up next on my roadmap is the AWS DevOps Engineer Professional (DOP-C02) certification. This aligns perfectly with my passion for DevOps, automation, and building scalable, reliable infrastructure using AWS-native tools. Im excited to explore how AWS can empower and streamline modern DevOps workflows and of course, Ill be sharing my preparation journey with you along the way.

💡 Conclusion

I hope this blog gave you a clear and practical roadmap to prepare for the AWS Solutions Architect Associate exam. Whether youre just starting out or looking for motivation to get back on track, remember consistency and the right resources make all the difference.

If you found this helpful, please give this blog a thumbs-up, share it with your network, and consider subscribing to my newsletter for more AWS, DevOps, and cloud-related content.

Feel free to connect with me here:
🔗 Website: praveshsudha.com
📝 Blog: blog.praveshsudha.com
🐦 Twitter: praveshstwt
💼 LinkedIn: Pravesh Sudha
📺 YouTube: Pravesh Sudha

Thanks for reading, and best of luck on your AWS certification journey! 🚀

Welcome to the world of Cloud and Infrastructure as Code (IaC)! If you're building infrastructure with Terraform one of the most popular tools in the DevOps ecosystem youve probably come across the mysterious terraform.tfstatefile. This small yet crucial file is the backbone of how Terraform tracks infrastructure resources.

In this blog, well dive into:

• What the terraform.tfstate file is

• Why relying on a local state file can cause issues

• And how to properly store Terraform state remotely using AWS S3 and DynamoDB

Whether you're a student exploring Terraform or a cloud enthusiast looking to follow best practices, this guide will help you understand how state management works and how to secure and scale your infrastructure properly.

So without further ado, lets get started! 🚀

💡 Prerequisites

Before we dive into configuring remote state storage in Terraform, ensure you have the following setup ready:

An AWS Account Youll need access to an AWS account with an IAM user that has at least EC2 Full Access, S3FullAccess and DynamoDBFullAccess permissions. You can grant broader permissions like AdministratorAccess for learning purposes, but its recommended to follow the principle of least privilege in production.

AWS CLI Installed and Configured Make sure youve installed the AWS CLI and configured it with your IAM credentials using:

aws configure

Terraform Installed Download and install Terraform from the official website. Confirm installation by running:

terraform -v

Once all these are in place, you're ready to start working with Terraform state files!

💡 Youtube Video Demo

💡 What is the terraform.tfstate File?

If youve worked with Terraform before and created some resources, you might have noticed that after running terraform init, a file named terraform.tfstate is automatically generated. This file is the heart of your Terraform project.

But what exactly does it do?

The terraform.tfstate file maintains a mapping between the infrastructure code written in your .tf files and the actual resources deployed in your cloud account. It stores the last known state of those resources, including important metadata like resource IDs, attributes, dependencies, and more.

Here's why this is so important:

• When you run terraform plan, Terraform reads this file to compare the desired state (what's in your code) with the actual state (what's currently running in your cloud).

• It then shows you the differences and determines what actions (create, update, delete) are needed to bring the infrastructure in sync with your code.

• When you run terraform apply, Terraform updates the actual infrastructure and then updates the terraform.tfstate file accordingly.

In short, this file is what allows Terraform to track, manage, and orchestrate changes to your infrastructure consistently and reliably.

However, theres a catch especially when it comes to local state.

💡 Problems with Local State File

While the terraform.tfstate file plays a critical role in managing your infrastructure, storing it locally comes with serious drawbacks especially in team environments or production setups.

1. Sensitive Information Exposure

The state file often contains sensitive data like:

• API keys

• Passwords

• Resource metadata and configuration values

If this file is stored locally and shared improperly (e.g., committed to Git), it can lead to security risks and potential breaches.

2. No Single Source of Truth

Lets say two developers are working on the same Terraform project with local state files. If both of them:

• Make changes

• Apply them independently

• And have different versions of the terraform.tfstate file

...this can lead to conflicting updates, resource mismatches, or worse an endless loop of undoing each other's changes. The result? Chaos and an unreliable infrastructure state.

3. Lack of Collaboration and Control

Local state doesnt support features like:

• State Locking: Prevents multiple users from making concurrent changes

• Versioning: Track history of infrastructure changes

• Secure Sharing: Centralized access for teams

• High Availability: No risk of losing state due to a lost local machine

💡 Why Remote State is Recommended

To address these issues, Terraform allows storing state remotely using backends like:

• Amazon S3 (with DynamoDB for state locking)

• Azure Blob Storage

• Google Cloud Storage

• HashiCorp Cloud Platform (HCP) Terraform

Remote backends provide:

• Encrypted storage

• Automatic versioning

• Team-friendly collaboration

• State locking to avoid simultaneous updates

For most real-world projects especially when working in a team or managing production infrastructure configuring a remote backend is not just a best practice, its a necessity.

Next, lets walk through how to do this using AWS S3 and DynamoDB.

💡 Setting Up Remote Terraform Backend with AWS S3 and DynamoDB

Now that we understand the problems with local state, lets see how to properly configure remote state storage using AWS S3 (for storing the state file) and DynamoDB (for state locking).

Instead of manually creating the required AWS resources, well automate the setup using a simple bash script.

🔧 Step 1: Automate Backend Resource Creation

Create a new file named config.sh and paste the following content into it:

#!/bin/bash

set -e

# Environment Variables
AWS_REGION="us-east-1"
S3_BUCKET_NAME="pravesh-terraform-state-bucket-2025"
DYNAMODB_TABLE_NAME="terraform-state-lock" 
STATE_KEY="terraform/terraform.tfstate" 

echo "--- Creating AWS Resources for Terraform Backend ---"
echo ""

# 1. Create S3 Bucket
echo "Creating S3 bucket: $S3_BUCKET_NAME in region $AWS_REGION..."
aws s3api create-bucket \
    --bucket "$S3_BUCKET_NAME" \
    --region "$AWS_REGION"

echo "Enabling versioning on S3 bucket..."
aws s3api put-bucket-versioning \
    --bucket "$S3_BUCKET_NAME" \
    --versioning-configuration Status=Enabled

echo "S3 bucket created and versioning enabled."
echo ""

# 2. Create DynamoDB Table for State Locking
echo "Creating DynamoDB table: $DYNAMODB_TABLE_NAME..."
aws dynamodb create-table \
    --table-name "$DYNAMODB_TABLE_NAME" \
    --attribute-definitions AttributeName=LockID,AttributeType=S \
    --key-schema AttributeName=LockID,KeyType=HASH \
    --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5 \
    --region "$AWS_REGION"

echo "DynamoDB table created for state locking."
echo ""

Make the script executable and run it:

chmod u+x config.sh
./config.sh

This script will create:

• An S3 bucket with versioning enabled to store your terraform.tfstate

• A DynamoDB table for locking and preventing concurrent state operations

🚀 Step 2: Create a Terraform Project to Provision an NGINX Server

Now lets set up a basic Terraform project that provisions an EC2 instance running an NGINX server with a portfolio page.

Create a new directory called basic-terra and inside it, add the following files:

📄 provider.tf

provider "aws" {
  region = "us-east-1"
}

📄 main.tf

resource "aws_security_group" "sg" {
  name        = "Basic-Security Group"
  description = "Allow port 80 for HTTP"

  tags = {
    Name = "Basic-sg"
  }
}

resource "aws_vpc_security_group_egress_rule" "example" {
  security_group_id = aws_security_group.sg.id
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "-1"
}

resource "aws_vpc_security_group_ingress_rule" "example" {
  security_group_id = aws_security_group.sg.id
  cidr_ipv4         = "0.0.0.0/0"
  from_port         = 80
  to_port           = 80
  ip_protocol       = "tcp"
}

resource "aws_instance" "web" {
  ami             = "ami-020cba7c55df1f615"  # Use a valid AMI ID for your region
  instance_type   = "t2.micro"
  security_groups = [aws_security_group.sg.name]
  user_data       = file("userdata.sh")

  tags = {
    Name = "basic-terra"
  }
}

output "instance_public_ip" {
  value       = aws_instance.web.public_ip
  description = "Website is running on this address:"
}

📄 userdata.sh

#!/bin/bash

# Install NGINX
apt update -y
apt install nginx -y

# Start NGINX service
systemctl enable nginx
systemctl start nginx

# Portfolio HTML
cat <<EOF > /var/www/html/index.html
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Pravesh Sudha | Portfolio</title>
  <style>
    body {
      font-family: Arial, sans-serif;
      background-color: #f4f4f4;
      text-align: center;
      padding: 50px;
    }
    h1 { color: #333; }
    p  { font-size: 18px; color: #666; }
    a  { color: #007BFF; text-decoration: none; }
  </style>
</head>
<body>
  <h1>Hi, I'm Pravesh Sudha</h1>
  <p>DevOps  Cloud  Content Creator</p>
  <p>
    <a href="https://blog.praveshsudha.com" target="_blank">Blog</a> |
    <a href="https://x.com/praveshstwt" target="_blank">Twitter</a> |
    <a href="https://www.youtube.com/@pravesh-sudha" target="_blank">YouTube</a> |
    <a href="https://www.linkedin.com/in/pravesh-sudha/" target="_blank">LinkedIn</a>
  </p>
</body>
</html>
EOF

📄 backend.tf

terraform {
  backend "s3" {
    bucket         = "pravesh-terraform-state-bucket-2025"
    key            = "terraform/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-state-lock"
    encrypt        = true
  }
}

Step 3: Initialize, Plan & Apply

Now inside the basic-terra directory, run the following commands:

terraform init
terraform plan
terraform apply --auto-approve

Once the resources are created, Terraform will output the public IP of the EC2 instance. Open it in your browser and youll see your personal portfolio hosted via NGINX! 🎉

📦 Check Your State File

Now visit your S3 console and open the bucket my-terraform-state-bucket-2025. Youll see your terraform.tfstate file stored securely with:

• Encryption enabled

• Versioning in place

• State locking handled by DynamoDB

This setup not only makes your state secure but also production-grade and team-ready!

🧹 Cleaning Up: Tearing the Resources Down

Before we wrap up, lets clean up all the resources we created to avoid unnecessary AWS charges.

Step 1: Destroy Terraform-managed Resources

Navigate to your project directory basic-terra and run the following command:

terraform destroy --auto-approve

This will:

• Terminate the EC2 instance

• Delete the security group and its associated ingress/egress rules

Step 2: Delete the S3 Bucket and DynamoDB Table

The S3 bucket and DynamoDB table were created manually via the config.sh script, so well clean them up using another automation script.

Create a file called delete.sh and paste the following content:

#!/bin/bash

set -e

# Environment Variables
AWS_REGION="us-east-1"
S3_BUCKET_NAME="pravesh-terraform-state-bucket-2025"
DYNAMODB_TABLE_NAME="terraform-state-lock"

echo "--- Deleting AWS Resources for Terraform Backend ---"
echo ""

# Empty the S3 Bucket (including versions and delete markers)
echo "Emptying S3 bucket: $S3_BUCKET_NAME..."

objects_to_delete=$(aws s3api list-object-versions \
    --bucket "$S3_BUCKET_NAME" \
    --output=json \
    --query='{Objects: Versions[].[Key,VersionId],DeleteMarkers:DeleteMarkers[].[Key,VersionId]}' \
    --region "$AWS_REGION")

if [ "$(echo "$objects_to_delete" | jq '.Objects | length')" -gt 0 ] || \
   [ "$(echo "$objects_to_delete" | jq '.DeleteMarkers | length')" -gt 0 ]; then

    delete_payload=$(echo "$objects_to_delete" | jq -c '{Objects: (.Objects + .DeleteMarkers | map({Key: .[0], VersionId: .[1]}) | unique)}')

    aws s3api delete-objects \
        --bucket "$S3_BUCKET_NAME" \
        --delete "$delete_payload" \
        --region "$AWS_REGION"

    echo "S3 bucket emptied."
else
    echo "S3 bucket is already empty."
fi

# Delete the S3 Bucket
echo "Deleting S3 bucket..."
aws s3 rb s3://"$S3_BUCKET_NAME" --region "$AWS_REGION" --force
echo "S3 bucket deleted."

# Delete the DynamoDB Table
echo "Deleting DynamoDB table..."
aws dynamodb delete-table \
    --table-name "$DYNAMODB_TABLE_NAME" \
    --region "$AWS_REGION"
echo "DynamoDB table deleted."

echo " Terraform backend resources deleted successfully."

Make the script executable and run it:

chmod u+x delete.sh
./delete.sh

This script will:

• Empty the S3 bucket including all versions and delete markers

• Delete the bucket itself

• Remove the DynamoDB table

🏁 Conclusion

Managing Terraform state properly is not just a best practice it's essential for building reliable, secure, and scalable infrastructure. In this blog, we explored the importance of the terraform.tfstate file, the risks of keeping it local, and how to overcome those risks by configuring remote state storage using AWS S3 and DynamoDB.

We also took a hands-on approach to:

• Automate backend resource creation with a bash script

• Deploy an EC2 instance running NGINX to host a simple portfolio

• Clean up all resources to avoid charges

Whether youre a beginner exploring Infrastructure as Code or someone working on real-world cloud projects, using remote state and state locking will take your Terraform workflows to the next level. 🌍

If you found this helpful, feel free to connect with me and follow my work:

• 🔗 LinkedIn

• 🐦 Twitter / X

• 📺 YouTube

• Blog

Thanks for reading, and happy Terraforming! 💻🚀

Welcome to the world of cloud computing and automation!
In this blog, well dive into one of AWSs core container orchestration services: Amazon Elastic Container Service (ECS).

Well start by understanding what ECS is and how it works. Then, to bring theory into practice, well deploy a Tetris game that has been containerized using Docker onto an ECS cluster all provisioned and automated with Terraform.

By the end of this walkthrough, youll gain practical experience in:
Writing Terraform configurations to provision AWS infrastructure
Deploying containerized applications on ECS
Understanding the overall workflow of infrastructure as code on AWS

So, without further ado lets get started!

🛠 Pre-requisites

Before we dive into building and deploying, lets make sure your environment is set up correctly.

To follow along, youll need:

An AWS account with an IAM user configured
The IAM user must have sufficient permissions to create and manage ECS, IAM roles, and other resources
AWS CLI installed and configured on your system with your Access Key and Secret Access Key

If youre new to configuring the AWS CLI or IAM users, dont worry Ive explained this step-by-step in another project guide on my blog:
👉 blog.praveshsudha.com

Having these prerequisites ready will help you focus directly on writing Terraform code and deploying the application smoothly.

💡 Youtube Demo

🧩 What is ECS?

AWS Elastic Container Service (ECS) is a fully managed container orchestration service that simplifies the deployment, management, and scaling of containerized applications.
With ECS, you can easily start, stop, and manage Docker containers running on a cluster of EC2 instances or on a serverless architecture using AWS Fargate all without worrying about managing the underlying infrastructure.

How things work with AWS ECS

Lets break down the basic workflow of deploying a containerized application on ECS:

1. Create an ECS Cluster

   • This acts as the logical grouping of your infrastructure.

   • You can choose the infrastructure type (EC2 instances or serverless Fargate) and configure logging, security, and tags.

2. Define a Task Definition Family

   • Think of this as the blueprint for your container.

   • Here you specify:

     • The processor architecture (e.g., AMD64 or ARM64)

     • The Docker image (public image from Docker Hub, an Amazon ECR repository, etc.)

     • Container details such as port mappings, container name, and other runtime configurations.

   • You can also configure multi-AZ deployments for high availability.

3. Create a Service

   • Inside your ECS cluster, you create a service that uses your task definition.

   • The service ensures the desired number of containers are running and handles deployment strategies and scaling.