kacerr/kodekloud-engineer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add AWS certification notes (labs 1-50)

Browse Source
This commit is contained in:
Jan Novak
2026-08-04 23:18:39 +02:00
parent 5f2a867e70
commit 76d4b4d716
15 changed files with 4391 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

867
aws-21-30.md Normal file
View File

@@ -0,0 +1,867 @@
## Task 21
Create an EC2 instance named datacenter-ec2 using any linux AMI like ubuntu, the Instance type must be t2.micro and associate an Elastic IP address with this instance, name it as datacenter-eip.
```bash
# 1. Launch the instance (AL2023, x86_64 for t2.micro)
IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/canonical/ubuntu/server/noble/stable/current/amd64/hvm/ebs-gp3/ami-id \
--instance-type t2.micro \
--region us-east-1 \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=datacenter-ec2}]' \
--query 'Instances[0].InstanceId' \
--output text)
# Ubuntu 24.04 LTS (noble) — safe LTS pick
resolve:ssm:/aws/service/canonical/ubuntu/server/noble/stable/current/amd64/hvm/ebs-gp3/ami-id
# Ubuntu 26.04 LTS — newest, if the grader wants latest
resolve:ssm:/aws/service/canonical/ubuntu/server/26.04/stable/current/amd64/hvm/ebs-gp3/ami-id
# 2. Wait until it's running
aws ec2 wait instance-running --instance-ids "$IID" --region us-east-1
# 3. Allocate a new EIP (VPC scope) and tag it datacenter-eip
ALLOC_ID=$(aws ec2 allocate-address \
--domain vpc \
--region us-east-1 \
--tag-specifications 'ResourceType=elastic-ip,Tags=[{Key=Name,Value=datacenter-eip}]' \
--query 'AllocationId' \
--output text)
# 4. Associate the EIP with the instance
aws ec2 associate-address \
--instance-id "$IID" \
--allocation-id "$ALLOC_ID" \
--region us-east-1
# validate
aws ec2 describe-instances \
--instance-ids "$IID" \
--region us-east-1 \
--query 'Reservations[0].Instances[0].{Type:InstanceType,State:State.Name,EIP:PublicIpAddress}'
aws ec2 describe-addresses \
--filters "Name=tag:Name,Values=datacenter-eip" \
--region us-east-1 \
--query 'Addresses[0].{IP:PublicIp,Instance:InstanceId,Assoc:AssociationId}'
```
## Task 22
The Nautilus DevOps team needs to set up a new EC2 instance that can be accessed securely from their landing host (aws-client). The instance should be of type t2.micro and named xfusion-ec2. A new SSH key with name id_rsa should be created on the aws-client host under the/root/.ssh/ folder, if it doesn't already exist. This key should then be added to the root user's authorised keys on the EC2 instance, allowing passwordless SSH access from the aws-client host.
```bash
# 1. Generate id_rsa only if it doesn't already exist
if [ ! -f /root/.ssh/id_rsa ]; then
ssh-keygen -t rsa -b 4096 -f /root/.ssh/id_rsa -N "" -q
fi
PUBKEY=$(cat /root/.ssh/id_rsa.pub)
# 2. Security group allowing SSH so aws-client can reach the box
VPC_ID=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
--region us-east-1 --query 'Vpcs[0].VpcId' --output text)
SG_ID=$(aws ec2 create-security-group \
--group-name xfusion-ssh-sg \
--description "SSH access for xfusion-ec2 from aws-client" \
--vpc-id "$VPC_ID" --region us-east-1 \
--query 'GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id "$SG_ID" --protocol tcp --port 22 --cidr 0.0.0.0/0 \
--region us-east-1
# 3. Build user-data that plants the pubkey into root's authorized_keys
cat > /tmp/userdata.sh << EOF
#!/bin/bash
mkdir -p /root/.ssh
chmod 700 /root/.ssh
echo "$PUBKEY" >> /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys
# ensure key-based root login is permitted (neutralize any disabling drop-in)
mkdir -p /etc/ssh/sshd_config.d
echo "PermitRootLogin prohibit-password" > /etc/ssh/sshd_config.d/99-root-login.conf
systemctl restart sshd 2>/dev/null || systemctl restart ssh
EOF
# 4. Launch the instance with that user-data
IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
--instance-type t2.micro \
--security-group-ids "$SG_ID" \
--user-data file:///tmp/userdata.sh \
--region us-east-1 \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=xfusion-ec2}]' \
--query 'Instances[0].InstanceId' --output text)
# 5. Wait until running, then grab the public IP
aws ec2 wait instance-running --instance-ids "$IID" --region us-east-1
IP=$(aws ec2 describe-instances --instance-ids "$IID" --region us-east-1 \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
echo "xfusion-ec2 => $IP"
# verify
ssh -i /root/.ssh/id_rsa -o StrictHostKeyChecking=no root@"$IP" hostname
```
## Task 23
Create a New Private S3 Bucket: Name the bucket devops-sync-24571.
Data Migration: Migrate the entire data from the existing devops-s3-28903 bucket to the new devops-sync-24571 bucket.
Ensure Data Consistency: Ensure that both buckets have the same data.
Use AWS CLI: Use the AWS CLI to perform the creation and data migration tasks.
```bash
# 1. Create the new bucket (private by default)
aws s3api create-bucket \
--bucket devops-sync-24571 \
--region us-east-1
# 2. Sync everything from old -> new
aws s3 sync s3://devops-s3-28903 s3://devops-sync-24571 \
--region us-east-1
# 3. Verify parity (see below)
# Object counts should match
echo "source:"; aws s3 ls s3://devops-s3-28903 --recursive --summarize | tail -2
echo "dest:"; aws s3 ls s3://devops-sync-24571 --recursive --summarize | tail -2
# Dry-run reverse sync: if output is empty, buckets are in parity
aws s3 sync s3://devops-s3-28903 s3://devops-sync-24571 --dryrun
```
## Task 24
Set up an Application Load Balancer named xfusion-alb.
Create a target group named xfusion-tg.
Create a security group named xfusion-sg to open port 80 for the public.
Attach this security group to the ALB.
The ALB should route traffic on port 80 to port 80 of the xfusion-ec2 instance.
Make appropriate changes in the default security group attached to the EC2 instance if necessary.
### Solution
**xfusion-sg** opens port 80 to the public (`0.0.0.0/0`) — this is what lets internet traffic reach the ALB.
**ALB (xfusion-alb)** needs 2 subnets in 2 AZs (hard requirement) and wears `xfusion-sg`. It's the internet-facing entry point.
**Target group (xfusion-tg)** defines *where* traffic goes: HTTP:80, instance targets. The ALB itself doesn't know about instances — the TG is the binding layer. You register `xfusion-ec2` into it.
**Listener** ties them together: "traffic hitting ALB:80 → forward to xfusion-tg." Without it, the ALB accepts nothing.
**Instance's default SG** must allow 80 *from the ALB's SG* — because the ALB, not the public, is what actually connects to the instance. Skip this and traffic dies at the instance firewall even though everything upstream is wired.
Flow: `internet → xfusion-sg/ALB:80 → listener → xfusion-tg → ec2:80 (default SG allows ALB)`.
The one non-obvious bit: two separate SG hops. Public→ALB is open to the world; ALB→instance is locked to just the ALB's SG. That's least-privilege, and it's why the task has both a "create sg" step and a "fix the default sg" step.
```bash
REGION=us-east-1
# 0. Default VPC + two subnets in different AZs
VPC_ID=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
read -r SUBNET1 SUBNET2 <<< $(aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=$VPC_ID" \
--region $REGION \
--query 'Subnets[?MapPublicIpOnLaunch==`true`].SubnetId | [0:2]' \
--output text)
# 1. ALB security group — open 80 to the public
ALB_SG=$(aws ec2 create-security-group \
--group-name xfusion-sg \
--description "Public HTTP for xfusion-alb" \
--vpc-id "$VPC_ID" --region $REGION \
--query 'GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id "$ALB_SG" --protocol tcp --port 80 --cidr 0.0.0.0/0 \
--region $REGION
# 2. Create the ALB across both subnets, attach the SG
ALB_ARN=$(aws elbv2 create-load-balancer \
--name xfusion-alb \
--subnets "$SUBNET1" "$SUBNET2" \
--security-groups "$ALB_SG" \
--scheme internet-facing \
--type application \
--region $REGION \
--query 'LoadBalancers[0].LoadBalancerArn' --output text)
# 3. Create the target group (instance targets, HTTP:80, in the VPC)
TG_ARN=$(aws elbv2 create-target-group \
--name xfusion-tg \
--protocol HTTP --port 80 \
--vpc-id "$VPC_ID" \
--target-type instance \
--region $REGION \
--query 'TargetGroups[0].TargetGroupArn' --output text)
# 4. Register xfusion-ec2 into the target group
IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=xfusion-ec2" "Name=instance-state-name,Values=pending,running,stopping,stopped" \
--region $REGION \
--query 'Reservations[0].Instances[0].InstanceId' --output text)
aws elbv2 register-targets \
--target-group-arn "$TG_ARN" \
--targets Id="$IID" \
--region $REGION
# 5. Listener on ALB:80 -> forward to the target group
aws elbv2 create-listener \
--load-balancer-arn "$ALB_ARN" \
--protocol HTTP --port 80 \
--default-actions Type=forward,TargetGroupArn="$TG_ARN" \
--region $REGION
# 6. Open the instance's default SG to the ALB on port 80
DEFAULT_SG=$(aws ec2 describe-security-groups \
--filters "Name=group-name,Values=default" "Name=vpc-id,Values=$VPC_ID" \
--region $REGION \
--query 'SecurityGroups[0].GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id "$DEFAULT_SG" \
--protocol tcp --port 80 \
--source-group "$ALB_SG" \
--region $REGION
# LB waiter
aws elbv2 wait load-balancer-available \
--load-balancer-arns "$ALB_ARN" \
--region us-east-1
# Verify
# ALB provisioned + its DNS name
aws elbv2 describe-load-balancers --names xfusion-alb --region $REGION \
--query 'LoadBalancers[0].{State:State.Code,DNS:DNSName,SGs:SecurityGroups}'
# Target registered
aws elbv2 describe-target-health --target-group-arn "$TG_ARN" --region $REGION \
--query 'TargetHealthDescriptions[].{Id:Target.Id,Health:TargetHealth.State}'
# Listener forwarding 80 -> TG
aws elbv2 describe-listeners --load-balancer-arn "$ALB_ARN" --region $REGION \
--query 'Listeners[].{Port:Port,Action:DefaultActions[0].Type}'
```
## Task 25
The Nautilus DevOps team has been tasked with setting up an EC2 instance for their application. To ensure the application performs optimally, they also need to create a CloudWatch alarm to monitor the instance's CPU utilization. The alarm should trigger if the CPU utilization exceeds 90% for one consecutive 5-minute period. To send notifications, use the SNS topic named xfusion-sns-topic which is already created.
Launch EC2 Instance: Create an EC2 instance named xfusion-ec2 using any appropriate Ubuntu AMI.
Create CloudWatch Alarm: Create a CloudWatch alarm named xfusion-alarm with the following specifications:
Statistic: Average
Metric: CPU Utilization
Threshold: >= 90% for 1 consecutive 5-minute period.
Alarm Actions: Send a notification to xfusion-sns-topic.
```bash
REGION=us-east-1
# 1. Launch Ubuntu instance (24.04 noble LTS via SSM resolver, x86_64 for t2.micro)
IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/canonical/ubuntu/server/noble/stable/current/amd64/hvm/ebs-gp3/ami-id \
--instance-type t2.micro \
--region $REGION \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=xfusion-ec2}]' \
--query 'Instances[0].InstanceId' --output text)
aws ec2 wait instance-running --instance-ids "$IID" --region $REGION
# 2. Resolve the SNS topic ARN by name
SNS_ARN=$(aws sns list-topics --region $REGION \
--query "Topics[?ends_with(TopicArn, ':xfusion-sns-topic')].TopicArn | [0]" \
--output text)
# 3. Create the alarm
aws cloudwatch put-metric-alarm \
--alarm-name xfusion-alarm \
--region $REGION \
--namespace AWS/EC2 \
--metric-name CPUUtilization \
--dimensions Name=InstanceId,Value="$IID" \
--statistic Average \
--period 300 \
--evaluation-periods 1 \
--threshold 90 \
--comparison-operator GreaterThanOrEqualToThreshold \
--alarm-actions "$SNS_ARN"
# Verify
aws cloudwatch describe-alarms \
--alarm-names xfusion-alarm \
--region $REGION \
--query 'MetricAlarms[0].{Name:AlarmName,Metric:MetricName,Stat:Statistic,Threshold:Threshold,Op:ComparisonOperator,Period:Period,Eval:EvaluationPeriods,Actions:AlarmActions,Instance:Dimensions[0].Value}'
```
## Task 26
As a member of the Nautilus DevOps Team, your task is to create an EC2 instance with the following specifications:
Instance Name: The EC2 instance must be named datacenter-ec2.
AMI: Use any available Ubuntu AMI to create this instance.
User Data Script: Configure the instance to run a user data script during its launch. This script should:
Install the Nginx package.
Start the Nginx service.
Security Group: Ensure that the instance allows HTTP traffic on port 80 from the internet.
### Ubuntu based
```bash
REGION=us-east-1
# 0. Default VPC + SG allowing HTTP/80 from the internet
VPC_ID=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
SG_ID=$(aws ec2 create-security-group \
--group-name datacenter-http-sg \
--description "HTTP 80 from internet for datacenter-ec2" \
--vpc-id "$VPC_ID" --region $REGION \
--query 'GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id "$SG_ID" --protocol tcp --port 80 --cidr 0.0.0.0/0 \
--region $REGION
# 1. User-data script (runs as root at first boot)
cat > /tmp/nginx-userdata.sh << 'EOF'
#!/bin/bash
apt-get update -y
apt-get install -y nginx
systemctl enable nginx
systemctl start nginx
EOF
# 2. Launch Ubuntu instance with the user-data + SG
IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/canonical/ubuntu/server/noble/stable/current/amd64/hvm/ebs-gp3/ami-id \
--instance-type t2.micro \
--security-group-ids "$SG_ID" \
--user-data file:///tmp/nginx-userdata.sh \
--region $REGION \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=datacenter-ec2}]' \
--query 'Instances[0].InstanceId' --output text)
aws ec2 wait instance-running --instance-ids "$IID" --region $REGION
IP=$(aws ec2 describe-instances --instance-ids "$IID" --region $REGION \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
echo "datacenter-ec2 => http://$IP"
```
### Amazon linux based
```bash
REGION=us-east-1
# 0. Default VPC + SG allowing HTTP/80 from the internet
VPC_ID=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
SG_ID=$(aws ec2 create-security-group \
--group-name datacenter-http-sg \
--description "HTTP 80 from internet for datacenter-ec2" \
--vpc-id "$VPC_ID" --region $REGION \
--query 'GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id "$SG_ID" --protocol tcp --port 80 --cidr 0.0.0.0/0 \
--region $REGION
# 1. User-data script (runs as root at first boot)
cat > /tmp/nginx-userdata.sh << 'EOF'
#!/bin/bash
dnf update -y
dnf install -y nginx
systemctl enable nginx
systemctl start nginx
EOF
# 2. Launch AL2023 instance with the user-data + SG
IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
--instance-type t2.micro \
--security-group-ids "$SG_ID" \
--user-data file:///tmp/nginx-userdata.sh \
--region $REGION \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=datacenter-ec2}]' \
--query 'Instances[0].InstanceId' --output text)
aws ec2 wait instance-running --instance-ids "$IID" --region $REGION
IP=$(aws ec2 describe-instances --instance-ids "$IID" --region $REGION \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
echo "datacenter-ec2 => http://$IP"
# Validation
aws ec2 describe-security-groups --group-ids "$SG_ID" --region $REGION \
--query 'SecurityGroups[0].IpPermissions'
curl -I "http://$IP"
```
## Task 27
The Nautilus DevOps Team has received a request from the Networking Team to set up a new public VPC to support a set of public-facing services. This VPC will host various resources that need to be accessible over the internet. As part of this setup, you need to ensure the VPC has public subnets with automatic IP assignment for resources. Additionally, a new EC2 instance will be launched within this VPC to host public applications that require SSH access. This setup will enable the Networking Team to deploy and manage public-facing applications.
Create a public VPC named nautilus-pub-vpc, and a subnet named nautilus-pub-subnet under the same, make sure public IP is being auto assigned to resources under this subnet. Further, create an EC2 instance named nautilus-pub-ec2 under this VPC with instance type t2.micro. Make sure SSH port 22 is open for this instance and accessible over the internet.
```bash
REGION=us-east-1
# 1. VPC
VPC_ID=$(aws ec2 create-vpc \
--cidr-block 10.0.0.0/16 \
--region $REGION \
--tag-specifications 'ResourceType=vpc,Tags=[{Key=Name,Value=nautilus-pub-vpc}]' \
--query 'Vpc.VpcId' --output text)
# 2. Internet Gateway + attach to the VPC
IGW_ID=$(aws ec2 create-internet-gateway \
--region $REGION \
--tag-specifications 'ResourceType=internet-gateway,Tags=[{Key=Name,Value=nautilus-pub-igw}]' \
--query 'InternetGateway.InternetGatewayId' --output text)
aws ec2 attach-internet-gateway \
--internet-gateway-id "$IGW_ID" --vpc-id "$VPC_ID" --region $REGION
# 3. Subnet
SUBNET_ID=$(aws ec2 create-subnet \
--vpc-id "$VPC_ID" \
--cidr-block 10.0.1.0/24 \
--region $REGION \
--tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=nautilus-pub-subnet}]' \
--query 'Subnet.SubnetId' --output text)
# 4. Auto-assign public IP on the subnet
aws ec2 modify-subnet-attribute \
--subnet-id "$SUBNET_ID" \
--map-public-ip-on-launch \
--region $REGION
# 5. Route table + default route to IGW + associate with subnet
RTB_ID=$(aws ec2 create-route-table \
--vpc-id "$VPC_ID" \
--region $REGION \
--tag-specifications 'ResourceType=route-table,Tags=[{Key=Name,Value=nautilus-pub-rtb}]' \
--query 'RouteTable.RouteTableId' --output text)
aws ec2 create-route \
--route-table-id "$RTB_ID" \
--destination-cidr-block 0.0.0.0/0 \
--gateway-id "$IGW_ID" \
--region $REGION
aws ec2 associate-route-table \
--route-table-id "$RTB_ID" --subnet-id "$SUBNET_ID" --region $REGION
# 6. Security group allowing SSH/22 from the internet
SG_ID=$(aws ec2 create-security-group \
--group-name nautilus-pub-sg \
--description "SSH from internet for nautilus-pub-ec2" \
--vpc-id "$VPC_ID" --region $REGION \
--query 'GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id "$SG_ID" --protocol tcp --port 22 --cidr 0.0.0.0/0 \
--region $REGION
# 7. Launch the instance in the subnet
IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
--instance-type t2.micro \
--subnet-id "$SUBNET_ID" \
--security-group-ids "$SG_ID" \
--region $REGION \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=nautilus-pub-ec2}]' \
--query 'Instances[0].InstanceId' --output text)
aws ec2 wait instance-running --instance-ids "$IID" --region $REGION
IP=$(aws ec2 describe-instances --instance-ids "$IID" --region $REGION \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
echo "nautilus-pub-ec2 => $IP"
# Verify
# Route table has the IGW route and is associated with the subnet
aws ec2 describe-route-tables --route-table-ids "$RTB_ID" --region $REGION \
--query 'RouteTables[0].{Routes:Routes,Assoc:Associations[].SubnetId}'
# Subnet auto-assigns public IPs
aws ec2 describe-subnets --subnet-ids "$SUBNET_ID" --region $REGION \
--query 'Subnets[0].{Cidr:CidrBlock,AutoIP:MapPublicIpOnLaunch,VPC:VpcId}'
# Instance is up with a public IP
aws ec2 describe-instances --instance-ids "$IID" --region $REGION \
--query 'Reservations[0].Instances[0].{State:State.Name,PubIP:PublicIpAddress,Subnet:SubnetId}'
```
## Task 28
Create a private ECR repository named xfusion-ecr. There is a Dockerfile under /root/pyapp directory on aws-client host, build a docker image using this Dockerfile and push the same to the newly created ECR repo, the image tag must be latest.
```bash
REGION=us-east-1
# 1. Create the private ECR repository
aws ecr create-repository \
--repository-name xfusion-ecr \
--region $REGION
# 2. Resolve your account ID and build the registry URI
ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text)
REGISTRY="${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com"
REPO_URI="${REGISTRY}/xfusion-ecr"
# 3. Authenticate docker to ECR (token valid 12h)
aws ecr get-login-password --region $REGION \
| docker login --username AWS --password-stdin "$REGISTRY"
# 4. Build the image from the Dockerfile
docker build -t xfusion-ecr:latest /root/pyapp
# 5. Tag it with the full ECR URI + latest
docker tag xfusion-ecr:latest "${REPO_URI}:latest"
# 6. Push
docker push "${REPO_URI}:latest"
# Validate
aws ecr describe-images \
--repository-name xfusion-ecr \
--region $REGION \
--query 'imageDetails[].{Tags:imageTags,Pushed:imagePushedAt,SizeMB:imageSizeInBytes}'
```
## Task 29
The Nautilus DevOps team has been tasked with demonstrating the use of VPC Peering to enable communication between two VPCs. One VPC will be a private VPC that contains a private EC2 instance, while the other will be the default public VPC containing a publicly accessible EC2 instance.
1) There is already an existing EC2 instance in the public vpc/subnet:
Name: devops-public-ec2
2) There is already an existing Private VPC:
Name: devops-private-vpc
CIDR: 10.1.0.0/16
3) There is already an existing Subnet in devops-private-vpc:
Name: devops-private-subnet
CIDR: 10.1.1.0/24
4) There is already an existing EC2 instance in the private subnet:
Name: devops-private-ec2
5) Create a Peering Connection between the Default VPC and the Private VPC:
VPC Peering Connection Name: devops-vpc-peering
6) Configure Route Tables to enable communication between the two VPCs.
Ensure the private EC2 instance is accessible from the public EC2 instance.
7) Test the Connection:
Add /root/.ssh/id_rsa.pub public key to the public EC2 instance's ec2-user's authorized_keys to make sure we are able to ssh into this instance from AWS client host. You may also need to update the security group of the private EC2 instance to allow ICMP traffic from the public/default VPC CIDR. This will enable you to ping the private instance from the public instance.
SSH into the public EC2 instance and ensure that you can ping the private EC2 instance.
```bash
REGION=us-east-1
# ── Discover both VPCs and their CIDRs ──────────────────────────────
DEF_VPC=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
DEF_CIDR=$(aws ec2 describe-vpcs --vpc-ids $DEF_VPC \
--region $REGION --query 'Vpcs[0].CidrBlock' --output text)
PRIV_VPC=$(aws ec2 describe-vpcs --filters Name=tag:Name,Values=devops-private-vpc \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
PRIV_CIDR=10.1.0.0/16
# ── 1. Create the peering connection (requester=default, peer=private) ──
PCX=$(aws ec2 create-vpc-peering-connection \
--vpc-id $DEF_VPC \
--peer-vpc-id $PRIV_VPC \
--region $REGION \
--tag-specifications 'ResourceType=vpc-peering-connection,Tags=[{Key=Name,Value=devops-vpc-peering}]' \
--query 'VpcPeeringConnection.VpcPeeringConnectionId' --output text)
# ── 2. Accept it (even same-account starts in pending-acceptance) ──
aws ec2 accept-vpc-peering-connection \
--vpc-peering-connection-id $PCX --region $REGION
# ── 3. Resolve instance IDs, subnets, IPs ──
PUB_IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=devops-public-ec2" "Name=instance-state-name,Values=pending,running,stopping,stopped" \
--region $REGION --query 'Reservations[0].Instances[0].InstanceId' --output text)
PUB_SUBNET=$(aws ec2 describe-instances --instance-ids $PUB_IID --region $REGION \
--query 'Reservations[0].Instances[0].SubnetId' --output text)
PUB_IP=$(aws ec2 describe-instances --instance-ids $PUB_IID --region $REGION \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
PUB_AZ=$(aws ec2 describe-instances --instance-ids $PUB_IID --region $REGION \
--query 'Reservations[0].Instances[0].Placement.AvailabilityZone' --output text)
PUB_SG=$(aws ec2 describe-instances --instance-ids $PUB_IID --region $REGION \
--query 'Reservations[0].Instances[0].SecurityGroups[0].GroupId' --output text)
PRIV_IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=devops-private-ec2" "Name=instance-state-name,Values=pending,running,stopping,stopped" \
--region $REGION --query 'Reservations[0].Instances[0].InstanceId' --output text)
PRIV_SUBNET=$(aws ec2 describe-subnets --filters "Name=tag:Name,Values=devops-private-subnet" \
--region $REGION --query 'Subnets[0].SubnetId' --output text)
PRIV_PRIV_IP=$(aws ec2 describe-instances --instance-ids $PRIV_IID --region $REGION \
--query 'Reservations[0].Instances[0].PrivateIpAddress' --output text)
PRIV_SG=$(aws ec2 describe-instances --instance-ids $PRIV_IID --region $REGION \
--query 'Reservations[0].Instances[0].SecurityGroups[0].GroupId' --output text)
# ── 4. Find the route table serving each subnet (explicit assoc, else main) ──
rtb_for_subnet () { # $1=subnet $2=vpc
local r
r=$(aws ec2 describe-route-tables --filters "Name=association.subnet-id,Values=$1" \
--region $REGION --query 'RouteTables[0].RouteTableId' --output text)
if [ "$r" = "None" ] || [ -z "$r" ]; then
r=$(aws ec2 describe-route-tables \
--filters "Name=vpc-id,Values=$2" "Name=association.main,Values=true" \
--region $REGION --query 'RouteTables[0].RouteTableId' --output text)
fi
echo "$r"
}
DEF_RTB=$(rtb_for_subnet $PUB_SUBNET $DEF_VPC)
PRIV_RTB=$(rtb_for_subnet $PRIV_SUBNET $PRIV_VPC)
# ── 5. Routes BOTH directions through the peering connection ──
aws ec2 create-route --route-table-id $DEF_RTB \
--destination-cidr-block $PRIV_CIDR \
--vpc-peering-connection-id $PCX --region $REGION
aws ec2 create-route --route-table-id $PRIV_RTB \
--destination-cidr-block $DEF_CIDR \
--vpc-peering-connection-id $PCX --region $REGION
# ── 6. Private instance SG: allow ICMP from the default VPC CIDR ──
aws ec2 authorize-security-group-ingress \
--group-id $PRIV_SG --protocol icmp --port -1 \
--cidr $DEF_CIDR --region $REGION
# ── 7a. Make sure public instance SG allows SSH from aws-client ──
aws ec2 authorize-security-group-ingress \
--group-id $PUB_SG --protocol tcp --port 22 \
--cidr 0.0.0.0/0 --region $REGION 2>/dev/null || true
# Push id_rsa.pub via EC2 Instance Connect (valid 60s)
aws ec2-instance-connect send-ssh-public-key \
--instance-id $PUB_IID \
--instance-os-user ec2-user \
--ssh-public-key file:///root/.ssh/id_rsa.pub \
--availability-zone $PUB_AZ \
--region $REGION
# Within 60s: SSH in and make the key permanent
PUBKEY=$(cat /root/.ssh/id_rsa.pub)
ssh -o StrictHostKeyChecking=no -i /root/.ssh/id_rsa ec2-user@$PUB_IP \
"mkdir -p ~/.ssh && chmod 700 ~/.ssh && echo '$PUBKEY' >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
# Now SSH persists. Test the peering by pinging the private instance's PRIVATE IP:
ssh -i /root/.ssh/id_rsa ec2-user@$PUB_IP "ping -c 4 $PRIV_PRIV_IP"
# Verify
# Peering active
aws ec2 describe-vpc-peering-connections --vpc-peering-connection-ids $PCX \
--region $REGION --query 'VpcPeeringConnections[0].Status.Code'
# Both route tables have the pcx route
aws ec2 describe-route-tables --route-table-ids $DEF_RTB $PRIV_RTB --region $REGION \
--query 'RouteTables[].{RTB:RouteTableId,PcxRoutes:Routes[?VpcPeeringConnectionId==`'$PCX'`].DestinationCidrBlock}'
# Private SG has ICMP from default CIDR
aws ec2 describe-security-groups --group-ids $PRIV_SG --region $REGION \
--query 'SecurityGroups[0].IpPermissions[?IpProtocol==`icmp`]'
```
## Task 30
The following components already exist in the environment:
1) A VPC named xfusion-priv-vpc and a private subnet named xfusion-priv-subnet have been created.
2) An EC2 instance named xfusion-priv-ec2 is already running in the private subnet.
3) The EC2 instance is configured with a cron job that uploads a test file to the S3 bucket xfusion-nat-6527 every minute. Upload will only succeed once internet access is established.
Your task is to:
Create a new public subnet named xfusion-pub-subnet in the existing VPC.
Launch a NAT Instance in the public subnet using an Amazon Linux 2023 AMI and name it xfusion-nat-instance. Configure this instance to act as a NAT instance. Make sure to use a custom security group for this instance.
After the configuration, verify that the test file xfusion-test.txt appears in the S3 bucket xfusion-nat-6527. This indicates successful internet access from the private EC2 instance via the NAT Instance.
Note: iptables is not installed by default on Amazon Linux 2023. You will need to install and enable it before configuring NAT setup.
### Claude's rewording:
The final boss — a manual NAT instance, which is a genuinely different beast from a NAT gateway. The concept that makes or breaks this: a NAT instance must have source/dest check disabled. By default EC2 drops any packet whose source or destination IP isn't the instance itself — but a NAT box exists precisely to forward packets belonging to other instances. Leave that check on and every forwarded packet silently dies, no error, nothing. That plus IP forwarding + an iptables MASQUERADE rule is the whole trick, and the routing has to point the private subnet's default route at the NAT instance's ENI.
```bash
REGION=us-east-1
# ── Discover existing pieces ──────────────────────────────────────
VPC_ID=$(aws ec2 describe-vpcs --filters Name=tag:Name,Values=xfusion-priv-vpc \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
VPC_CIDR=$(aws ec2 describe-vpcs --vpc-ids $VPC_ID --region $REGION \
--query 'Vpcs[0].CidrBlock' --output text)
PRIV_SUBNET=$(aws ec2 describe-subnets --filters "Name=tag:Name,Values=xfusion-priv-subnet" \
--region $REGION --query 'Subnets[0].SubnetId' --output text)
PRIV_SUBNET_CIDR=$(aws ec2 describe-subnets --subnet-ids $PRIV_SUBNET --region $REGION \
--query 'Subnets[0].CidrBlock' --output text)
PRIV_AZ=$(aws ec2 describe-subnets --subnet-ids $PRIV_SUBNET --region $REGION \
--query 'Subnets[0].AvailabilityZone' --output text)
# ── Ensure the VPC has an IGW (a "private VPC" usually doesn't) ────
IGW_ID=$(aws ec2 describe-internet-gateways \
--filters "Name=attachment.vpc-id,Values=$VPC_ID" \
--region $REGION --query 'InternetGateways[0].InternetGatewayId' --output text)
if [ "$IGW_ID" = "None" ] || [ -z "$IGW_ID" ]; then
IGW_ID=$(aws ec2 create-internet-gateway --region $REGION \
--tag-specifications 'ResourceType=internet-gateway,Tags=[{Key=Name,Value=xfusion-igw}]' \
--query 'InternetGateway.InternetGatewayId' --output text)
aws ec2 attach-internet-gateway --internet-gateway-id $IGW_ID --vpc-id $VPC_ID --region $REGION
fi
echo "REGION=$REGION
VPC_ID=$VPC_ID
VPC_CIDR=$VPC_CIDR
PRIV_SUBNET=$PRIV_SUBNET
PRIV_SUBNET_CIDR=$PRIV_SUBNET_CIDR
PRIV_AZ=$PRIV_AZ
IGW_ID=$IGW_ID"
# ── Pick a free /24 in the VPC for the public subnet ──────────────
BASE=$(echo $VPC_CIDR | cut -d. -f1-2)
USED=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" \
--region $REGION --query 'Subnets[].CidrBlock' --output text | tr '\t' '\n')
for i in $(seq 1 254); do
CAND="${BASE}.${i}.0/24"
echo "$USED" | grep -Fxq "$CAND" || { PUB_CIDR=$CAND; break; }
done
# ── 1. Create the public subnet + auto-assign public IP ───────────
PUB_SUBNET=$(aws ec2 create-subnet \
--vpc-id $VPC_ID --cidr-block $PUB_CIDR \
--availability-zone $PRIV_AZ \
--region $REGION \
--tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=xfusion-pub-subnet}]' \
--query 'Subnet.SubnetId' --output text)
aws ec2 modify-subnet-attribute --subnet-id $PUB_SUBNET \
--map-public-ip-on-launch --region $REGION
# ── 2. Public route table -> IGW, associate with public subnet ────
PUB_RTB=$(aws ec2 create-route-table --vpc-id $VPC_ID --region $REGION \
--tag-specifications 'ResourceType=route-table,Tags=[{Key=Name,Value=xfusion-pub-rtb}]' \
--query 'RouteTable.RouteTableId' --output text)
aws ec2 create-route --route-table-id $PUB_RTB \
--destination-cidr-block 0.0.0.0/0 --gateway-id $IGW_ID --region $REGION
aws ec2 associate-route-table --route-table-id $PUB_RTB --subnet-id $PUB_SUBNET --region $REGION
# ── 3. Custom SG for the NAT instance ─────────────────────────────
NAT_SG=$(aws ec2 create-security-group \
--group-name xfusion-nat-sg \
--description "NAT instance SG for xfusion" \
--vpc-id $VPC_ID --region $REGION \
--query 'GroupId' --output text)
# allow all traffic from inside the VPC (private instances routing through)
aws ec2 authorize-security-group-ingress --group-id $NAT_SG \
--protocol -1 --cidr $VPC_CIDR --region $REGION
# optional: SSH from anywhere for debugging
aws ec2 authorize-security-group-ingress --group-id $NAT_SG \
--protocol tcp --port 22 --cidr 0.0.0.0/0 --region $REGION 2>/dev/null || true
echo "BASE=$BASE
USED=$USED
PUB_CIDR=$PUB_CIDR
PUB_SUBNET=$PUB_SUBNET
PUB_RTB=$PUB_RTB
NAT_SG=$NAT_SG"
# ── 4. NAT user-data: iptables + IP forwarding + MASQUERADE ───────
cat > /tmp/nat-userdata.sh << 'EOF'
#!/bin/bash
set -e
dnf install -y iptables iptables-services
echo 'net.ipv4.ip_forward = 1' > /etc/sysctl.d/99-nat.conf
sysctl -p /etc/sysctl.d/99-nat.conf
EXT_IF=$(ip -o -4 route show to default | awk '{print $5; exit}')
iptables -t nat -A POSTROUTING -o "$EXT_IF" -j MASQUERADE
iptables -A FORWARD -i "$EXT_IF" -j ACCEPT
systemctl enable iptables
service iptables save
EOF
# ── 5. Launch the NAT instance (AL2023) ───────────────────────────
NAT_IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
--instance-type t2.micro \
--subnet-id $PUB_SUBNET \
--security-group-ids $NAT_SG \
--associate-public-ip-address \
--user-data file:///tmp/nat-userdata.sh \
--region $REGION \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=xfusion-nat-instance}]' \
--query 'Instances[0].InstanceId' --output text)
aws ec2 wait instance-running --instance-ids $NAT_IID --region $REGION
# ── 6. *** THE CRITICAL STEP *** disable source/dest check ────────
aws ec2 modify-instance-attribute --instance-id $NAT_IID \
--no-source-dest-check --region $REGION
# ── 7. Point the private subnet's default route at the NAT instance ─
PRIV_RTB=$(aws ec2 describe-route-tables \
--filters "Name=association.subnet-id,Values=$PRIV_SUBNET" \
--region $REGION --query 'RouteTables[0].RouteTableId' --output text)
if [ "$PRIV_RTB" = "None" ] || [ -z "$PRIV_RTB" ]; then
PRIV_RTB=$(aws ec2 describe-route-tables \
--filters "Name=vpc-id,Values=$VPC_ID" "Name=association.main,Values=true" \
--region $REGION --query 'RouteTables[0].RouteTableId' --output text)
fi
# create (or replace if one already exists)
aws ec2 create-route --route-table-id $PRIV_RTB \
--destination-cidr-block 0.0.0.0/0 --instance-id $NAT_IID --region $REGION \
2>/dev/null || \
aws ec2 replace-route --route-table-id $PRIV_RTB \
--destination-cidr-block 0.0.0.0/0 --instance-id $NAT_IID --region $REGION
echo "NAT instance: $NAT_IID | public subnet: $PUB_SUBNET ($PUB_CIDR) | priv RTB: $PRIV_RTB"
# Verify
# Confirm source/dest check is OFF (must be false)
aws ec2 describe-instances --instance-ids $NAT_IID --region $REGION \
--query 'Reservations[0].Instances[0].SourceDestCheck'
# Poll the bucket for the test file
for n in $(seq 1 6); do
if aws s3 ls s3://xfusion-nat-6527/ --region $REGION | grep -q xfusion-test.txt; then
echo "SUCCESS — file landed:"; aws s3 ls s3://xfusion-nat-6527/ --region $REGION | grep xfusion-test.txt
break
fi
echo "waiting for cron upload... ($n)"; sleep 30
done
```