kacerr/kodekloud-engineer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add AWS certification notes (labs 1-50)

Browse Source
This commit is contained in:
Jan Novak
2026-08-04 23:18:39 +02:00
parent 5f2a867e70
commit 76d4b4d716
15 changed files with 4391 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

284
aws-1-10.md Normal file
View File

@@ -0,0 +1,284 @@
## Task #1
```bash
aws ec2 create-key-pair \
--key-name devops-kp \
--key-type rsa \
--key-format pem \
--region us-east-1 \
--query 'KeyMaterial' \
--output text > devops-kp.pem
chmod 400 devops-kp.pem
```
## Task 2
```bash
# 1. Grab the default VPC ID
VPC_ID=$(aws ec2 describe-vpcs \
--filters "Name=isDefault,Values=true" \
--region us-east-1 \
--query 'Vpcs[0].VpcId' \
--output text)
# 2. Create the SG, capture its ID
SG_ID=$(aws ec2 create-security-group \
--group-name devops-sg \
--description "Security group for Nautilus App Servers" \
--vpc-id "$VPC_ID" \
--region us-east-1 \
--query 'GroupId' \
--output text)
# 3. HTTP ingress
aws ec2 authorize-security-group-ingress \
--group-id "$SG_ID" \
--protocol tcp \
--port 80 \
--cidr 0.0.0.0/0 \
--region us-east-1
# 4. SSH ingress
aws ec2 authorize-security-group-ingress \
--group-id "$SG_ID" \
--protocol tcp \
--port 22 \
--cidr 0.0.0.0/0 \
--region us-east-1
```
## Task 3
For this task, create one subnet named xfusion-subnet under default VPC.
```bash
# Grab default VPC ID
VPC_ID=$(aws ec2 describe-vpcs \
--filters "Name=isDefault,Values=true" \
--region us-east-1 \
--query 'Vpcs[0].VpcId' \
--output text)
# Create subnet + tag in a single call
aws ec2 create-subnet \
--vpc-id "$VPC_ID" \
--cidr-block 172.31.96.0/20 \
--region us-east-1 \
--tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=xfusion-subnet}]' \
--query 'Subnet.{Id:SubnetId,Cidr:CidrBlock,AZ:AvailabilityZone}'
# list / describe subnets
aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=$VPC_ID" \
--region us-east-1
aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=$VPC_ID" \
--region us-east-1 \
--query 'Subnets[].{Id:SubnetId,Cidr:CidrBlock,AZ:AvailabilityZone,Name:Tags[?Key==`Name`]|[0].Value}' \
--output table
```
## Task 4
The s3 bucket name is devops-s3-15697, enable versioning for this bucket.
```bash
aws s3api put-bucket-versioning \
--bucket devops-s3-15697 \
--versioning-configuration Status=Enabled \
--region us-east-1
```
## Task 5
Create a volume with the following requirements:
- Name of the volume should be nautilus-volume.
- Volume type must be gp3.
- Volume size must be 2 GiB.
```bash
aws ec2 create-volume \
--volume-type gp3 \
--size 2 \
--availability-zone us-east-1a \
--region us-east-1 \
--tag-specifications 'ResourceType=volume,Tags=[{Key=Name,Value=nautilus-volume}]' \
--query '{Id:VolumeId,Type:VolumeType,Size:Size,AZ:AvailabilityZone}'
# verify
aws ec2 describe-volumes \
--filters "Name=tag:Name,Values=nautilus-volume" \
--region us-east-1 \
--query 'Volumes[0].{Id:VolumeId,Type:VolumeType,Size:Size,AZ:AvailabilityZone,State:State,Name:Tags[?Key==`Name`]|[0].Value}'
```
## Task 6
For this task, create an EC2 instance with following requirements:
1) The name of the instance must be xfusion-ec2.
2) You can use the Amazon Linux AMI to launch this instance.
3) The Instance type must be t2.micro.
4) Create a new RSA key pair named xfusion-kp.
5) Attach the default (available by default) security group.
```bash
# 1. Create the new RSA key pair
aws ec2 create-key-pair \
--key-name xfusion-kp \
--key-type rsa \
--key-format pem \
--region us-east-1 \
--query 'KeyMaterial' \
--output text > xfusion-kp.pem
chmod 400 xfusion-kp.pem
# 2. Grab the default VPC's default security group ID
DEFAULT_SG=$(aws ec2 describe-security-groups \
--filters "Name=group-name,Values=default" "Name=vpc-id,Values=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true --region us-east-1 --query 'Vpcs[0].VpcId' --output text)" \
--region us-east-1 \
--query 'SecurityGroups[0].GroupId' \
--output text)
# 3. Launch the instance
aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
--instance-type t2.micro \
--key-name xfusion-kp \
--security-group-ids "$DEFAULT_SG" \
--region us-east-1 \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=xfusion-ec2}]' \
--query 'Instances[0].{Id:InstanceId,AMI:ImageId,Type:InstanceType,State:State.Name}'
# verify
aws ec2 describe-instances \
--filters "Name=tag:Name,Values=xfusion-ec2" \
--region us-east-1 \
--query 'Reservations[0].Instances[0].{Id:InstanceId,Type:InstanceType,State:State.Name,Key:KeyName,SG:SecurityGroups[0].GroupName,AMI:ImageId}'
```
## Task 7
1) Change the instance type from t2.micro to t2.nano for devops-ec2 instance.
2) Make sure the ec2 instance devops-ec2 is in running state after the change.
```bash
# 0. Resolve instance ID from the Name tag
IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=devops-ec2" "Name=instance-state-name,Values=pending,running,stopping,stopped" \
--region us-east-1 \
--query 'Reservations[0].Instances[0].InstanceId' \
--output text)
# 1. Stop it
aws ec2 stop-instances --instance-ids "$IID" --region us-east-1
# 2. Block until fully stopped (not just 'stopping')
aws ec2 wait instance-stopped --instance-ids "$IID" --region us-east-1
# 3. Change the type
aws ec2 modify-instance-attribute \
--instance-id "$IID" \
--instance-type t2.nano \
--region us-east-1
# 4. Start it back up
aws ec2 start-instances --instance-ids "$IID" --region us-east-1
# 5. Block until running
aws ec2 wait instance-running --instance-ids "$IID" --region us-east-1
# Verify
aws ec2 describe-instances \
--instance-ids "$IID" \
--region us-east-1 \
--query 'Reservations[0].Instances[0].{Type:InstanceType,State:State.Name}'
```
## Task 8
There is an EC2 instance named xfusion-ec2 under us-east-1 region, enable the stop protection for this instance.
```bash
IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=xfusion-ec2" "Name=instance-state-name,Values=pending,running,stopping,stopped" \
--region us-east-1 \
--query 'Reservations[0].Instances[0].InstanceId' \
--output text)
aws ec2 modify-instance-attribute \
--instance-id "$IID" \
--disable-api-stop \
--region us-east-1
# Verify
aws ec2 describe-instance-attribute \
--instance-id "$IID" \
--attribute disableApiStop \
--region us-east-1 \
--query 'DisableApiStop.Value'
```
## Task 9
```bash
IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=devops-ec2" "Name=instance-state-name,Values=pending,running,stopping,stopped" \
--region us-east-1 \
--query 'Reservations[0].Instances[0].InstanceId' \
--output text)
aws ec2 modify-instance-attribute \
--instance-id "$IID" \
--disable-api-termination \
--region us-east-1
# Verify:
aws ec2 describe-instance-attribute \
--instance-id "$IID" \
--attribute disableApiTermination \
--region us-east-1 \
--query 'DisableApiTermination.Value'
```
## Task 10
There is an instance named nautilus-ec2 and an elastic-ip named nautilus-ec2-eip in us-east-1 region. Attach the nautilus-ec2-eip elastic-ip to the nautilus-ec2 instance.
```bash
# Instance ID from its Name tag
IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=nautilus-ec2" "Name=instance-state-name,Values=pending,running,stopping,stopped" \
--region us-east-1 \
--query 'Reservations[0].Instances[0].InstanceId' \
--output text)
# EIP allocation ID from its Name tag
ALLOC_ID=$(aws ec2 describe-addresses \
--filters "Name=tag:Name,Values=nautilus-ec2-eip" \
--region us-east-1 \
--query 'Addresses[0].AllocationId' \
--output text)
# Associate
aws ec2 associate-address \
--instance-id "$IID" \
--allocation-id "$ALLOC_ID" \
--region us-east-1
# Validate
aws ec2 describe-addresses \
--filters "Name=tag:Name,Values=nautilus-ec2-eip" \
--region us-east-1 \
--query 'Addresses[0].{IP:PublicIp,Instance:InstanceId,Assoc:AssociationId}'
```

289
aws-11-20.md Normal file
View File

@@ -0,0 +1,289 @@
## Task 11
An instance named xfusion-ec2 and an elastic network interface named xfusion-eni already exists in us-east-1 region.
Attach the xfusion-eni network interface to the xfusion-ec2 instance.
Make sure status is attached before submitting the task.
```bash
# Instance ID from Name tag
IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=xfusion-ec2" "Name=instance-state-name,Values=pending,running,stopping,stopped" \
--region us-east-1 \
--query 'Reservations[0].Instances[0].InstanceId' \
--output text)
# ENI ID from Name tag
ENI_ID=$(aws ec2 describe-network-interfaces \
--filters "Name=tag:Name,Values=xfusion-eni" \
--region us-east-1 \
--query 'NetworkInterfaces[0].NetworkInterfaceId' \
--output text)
# Attach at device index 1 (0 is taken by the primary ENI)
aws ec2 attach-network-interface \
--network-interface-id "$ENI_ID" \
--instance-id "$IID" \
--device-index 1 \
--region us-east-1
# validate
aws ec2 describe-network-interfaces \
--network-interface-ids "$ENI_ID" \
--region us-east-1 \
--query 'NetworkInterfaces[0].Attachment.Status'
```
## Task 12
An instance named datacenter-ec2 and a volume named datacenter-volume already exists in us-east-1 region. Attach the datacenter-volume volume to the datacenter-ec2 instance, make sure to set the device name to /dev/sdb while attaching the volume.
```bash
# Instance ID from Name tag
IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=datacenter-ec2" "Name=instance-state-name,Values=pending,running,stopping,stopped" \
--region us-east-1 \
--query 'Reservations[0].Instances[0].InstanceId' \
--output text)
# Volume ID from Name tag
VOL_ID=$(aws ec2 describe-volumes \
--filters "Name=tag:Name,Values=datacenter-volume" \
--region us-east-1 \
--query 'Volumes[0].VolumeId' \
--output text)
# Attach at /dev/sdb
aws ec2 attach-volume \
--volume-id "$VOL_ID" \
--instance-id "$IID" \
--device /dev/sdb \
--region us-east-1
# waiter
aws ec2 wait volume-in-use --volume-id "$VOL_ID" --region us-east-1
# validate
aws ec2 describe-volumes \
--volume-ids "$VOL_ID" \
--region us-east-1 \
--query 'Volumes[0].{State:State,Device:Attachments[0].Device,Instance:Attachments[0].InstanceId,AttachState:Attachments[0].State}'
```
## Task 13
For this task, create an AMI from an existing EC2 instance named nautilus-ec2 with the following requirement:
Name of the AMI should be nautilus-ec2-ami, make sure AMI is in available state.
```bash
# Instance ID from Name tag
IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=nautilus-ec2" "Name=instance-state-name,Values=pending,running,stopping,stopped" \
--region us-east-1 \
--query 'Reservations[0].Instances[0].InstanceId' \
--output text)
# Create the AMI
AMI_ID=$(aws ec2 create-image \
--instance-id "$IID" \
--name nautilus-ec2-ami \
--region us-east-1 \
--query 'ImageId' \
--output text)
echo "AMI: $AMI_ID"
# waiter
aws ec2 wait image-available --image-ids "$AMI_ID" --region us-east-1
# verify
aws ec2 describe-images \
--image-ids "$AMI_ID" \
--region us-east-1 \
--query 'Images[0].{Name:Name,State:State,Id:ImageId}'
```
## Task 14
1) Delete the ec2 instance named datacenter-ec2 present in us-east-1 region.
2) Before submitting your task, make sure instance is in terminated state.
```bash
# Instance ID from Name tag
IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=datacenter-ec2" "Name=instance-state-name,Values=pending,running,stopping,stopped" \
--region us-east-1 \
--query 'Reservations[0].Instances[0].InstanceId' \
--output text)
# Clear termination protection if it's set (harmless if it wasn't)
aws ec2 modify-instance-attribute \
--instance-id "$IID" \
--no-disable-api-termination \
--region us-east-1
# Terminate
aws ec2 terminate-instances --instance-ids "$IID" --region us-east-1
# Block until fully terminated
aws ec2 wait instance-terminated --instance-ids "$IID" --region us-east-1
# Verify
aws ec2 describe-instances \
--instance-ids "$IID" \
--region us-east-1 \
--query 'Reservations[0].Instances[0].State.Name'
```
## Task 15
Create a snapshot of an existing volume named nautilus-vol in us-east-1 region.
1) The name of the snapshot must be nautilus-vol-ss.
2) The description must be nautilus Snapshot.
3) Make sure the snapshot status is completed before submitting the task.
```bash
# Volume ID from Name tag
VOL_ID=$(aws ec2 describe-volumes \
--filters "Name=tag:Name,Values=nautilus-vol" \
--region us-east-1 \
--query 'Volumes[0].VolumeId' \
--output text)
# Create snapshot: description as field, name as tag
SNAP_ID=$(aws ec2 create-snapshot \
--volume-id "$VOL_ID" \
--description "nautilus Snapshot" \
--region us-east-1 \
--tag-specifications 'ResourceType=snapshot,Tags=[{Key=Name,Value=nautilus-vol-ss}]' \
--query 'SnapshotId' \
--output text)
echo "Snapshot: $SNAP_ID"
# Verify
aws ec2 describe-snapshots \
--snapshot-ids "$SNAP_ID" \
--region us-east-1 \
--query 'Snapshots[0].{Id:SnapshotId,State:State,Desc:Description,Name:Tags[?Key==`Name`]|[0].Value}'
```
## Task 16
create an IAM user named iamuser_anita.
```bash
aws iam create-user --user-name iamuser_anita
# Validate
aws iam get-user --user-name iamuser_anita \
--query 'User.{Name:UserName,Arn:Arn,Created:CreateDate}'
```
## Task 17
Create an IAM group named iamgroup_ravi.
```bash
aws iam create-group --group-name iamgroup_ravi
# Verify
aws iam get-group --group-name iamgroup_ravi \
--query 'Group.{Name:GroupName,Arn:Arn,Created:CreateDate}'
```
## Task 18
Create an IAM policy named iampolicy_anita in us-east-1 region, it must allow read-only access to the EC2 console, i.e this policy must allow users to view all instances, AMIs, and snapshots in the Amazon EC2 console.
```bash
cat > /tmp/iampolicy_anita.json << 'EOF'
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "ec2:Describe*",
"Resource": "*"
}
]
}
EOF
aws iam create-policy \
--policy-name iampolicy_anita \
--policy-document file:///tmp/iampolicy_anita.json
# Verify
aws iam list-policies \
--scope Local \
--query 'Policies[?PolicyName==`iampolicy_anita`].{Name:PolicyName,Arn:Arn,Attachments:AttachmentCount}' \
--output table
```
## Task 19
An IAM user named iamuser_kirsty and a policy named iampolicy_kirsty already exist. Attach the IAM policy iampolicy_kirsty to the IAM user iamuser_kirsty.
```bash
# Resolve the policy ARN by name
POLICY_ARN=$(aws iam list-policies \
--scope Local \
--query 'Policies[?PolicyName==`iampolicy_kirsty`].Arn' \
--output text)
# Attach it to the user
aws iam attach-user-policy \
--user-name iamuser_kirsty \
--policy-arn "$POLICY_ARN"
# Verify
aws iam list-attached-user-policies \
--user-name iamuser_kirsty \
--query 'AttachedPolicies[].{Name:PolicyName,Arn:PolicyArn}' \
--output table
```
## Task 20
Create an IAM role as below:
1) IAM role name must be iamrole_john.
2) Entity type must be AWS Service and use case must be EC2.
3) Attach a policy named iampolicy_john.
```bash
# 1. Trust policy: allow the EC2 service to assume this role
cat > /tmp/trust-ec2.json << 'EOF'
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": { "Service": "ec2.amazonaws.com" },
"Action": "sts:AssumeRole"
}
]
}
EOF
# 2. Create the role with that trust policy
aws iam create-role \
--role-name iamrole_john \
--assume-role-policy-document file:///tmp/trust-ec2.json
# 3. Resolve the policy ARN, then attach
POLICY_ARN=$(aws iam list-policies \
--scope Local \
--query 'Policies[?PolicyName==`iampolicy_john`].Arn' \
--output text)
aws iam attach-role-policy \
--role-name iamrole_john \
--policy-arn "$POLICY_ARN"
```

867
aws-21-30.md Normal file
View File

@@ -0,0 +1,867 @@
## Task 21
Create an EC2 instance named datacenter-ec2 using any linux AMI like ubuntu, the Instance type must be t2.micro and associate an Elastic IP address with this instance, name it as datacenter-eip.
```bash
# 1. Launch the instance (AL2023, x86_64 for t2.micro)
IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/canonical/ubuntu/server/noble/stable/current/amd64/hvm/ebs-gp3/ami-id \
--instance-type t2.micro \
--region us-east-1 \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=datacenter-ec2}]' \
--query 'Instances[0].InstanceId' \
--output text)
# Ubuntu 24.04 LTS (noble) — safe LTS pick
resolve:ssm:/aws/service/canonical/ubuntu/server/noble/stable/current/amd64/hvm/ebs-gp3/ami-id
# Ubuntu 26.04 LTS — newest, if the grader wants latest
resolve:ssm:/aws/service/canonical/ubuntu/server/26.04/stable/current/amd64/hvm/ebs-gp3/ami-id
# 2. Wait until it's running
aws ec2 wait instance-running --instance-ids "$IID" --region us-east-1
# 3. Allocate a new EIP (VPC scope) and tag it datacenter-eip
ALLOC_ID=$(aws ec2 allocate-address \
--domain vpc \
--region us-east-1 \
--tag-specifications 'ResourceType=elastic-ip,Tags=[{Key=Name,Value=datacenter-eip}]' \
--query 'AllocationId' \
--output text)
# 4. Associate the EIP with the instance
aws ec2 associate-address \
--instance-id "$IID" \
--allocation-id "$ALLOC_ID" \
--region us-east-1
# validate
aws ec2 describe-instances \
--instance-ids "$IID" \
--region us-east-1 \
--query 'Reservations[0].Instances[0].{Type:InstanceType,State:State.Name,EIP:PublicIpAddress}'
aws ec2 describe-addresses \
--filters "Name=tag:Name,Values=datacenter-eip" \
--region us-east-1 \
--query 'Addresses[0].{IP:PublicIp,Instance:InstanceId,Assoc:AssociationId}'
```
## Task 22
The Nautilus DevOps team needs to set up a new EC2 instance that can be accessed securely from their landing host (aws-client). The instance should be of type t2.micro and named xfusion-ec2. A new SSH key with name id_rsa should be created on the aws-client host under the/root/.ssh/ folder, if it doesn't already exist. This key should then be added to the root user's authorised keys on the EC2 instance, allowing passwordless SSH access from the aws-client host.
```bash
# 1. Generate id_rsa only if it doesn't already exist
if [ ! -f /root/.ssh/id_rsa ]; then
ssh-keygen -t rsa -b 4096 -f /root/.ssh/id_rsa -N "" -q
fi
PUBKEY=$(cat /root/.ssh/id_rsa.pub)
# 2. Security group allowing SSH so aws-client can reach the box
VPC_ID=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
--region us-east-1 --query 'Vpcs[0].VpcId' --output text)
SG_ID=$(aws ec2 create-security-group \
--group-name xfusion-ssh-sg \
--description "SSH access for xfusion-ec2 from aws-client" \
--vpc-id "$VPC_ID" --region us-east-1 \
--query 'GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id "$SG_ID" --protocol tcp --port 22 --cidr 0.0.0.0/0 \
--region us-east-1
# 3. Build user-data that plants the pubkey into root's authorized_keys
cat > /tmp/userdata.sh << EOF
#!/bin/bash
mkdir -p /root/.ssh
chmod 700 /root/.ssh
echo "$PUBKEY" >> /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys
# ensure key-based root login is permitted (neutralize any disabling drop-in)
mkdir -p /etc/ssh/sshd_config.d
echo "PermitRootLogin prohibit-password" > /etc/ssh/sshd_config.d/99-root-login.conf
systemctl restart sshd 2>/dev/null || systemctl restart ssh
EOF
# 4. Launch the instance with that user-data
IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
--instance-type t2.micro \
--security-group-ids "$SG_ID" \
--user-data file:///tmp/userdata.sh \
--region us-east-1 \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=xfusion-ec2}]' \
--query 'Instances[0].InstanceId' --output text)
# 5. Wait until running, then grab the public IP
aws ec2 wait instance-running --instance-ids "$IID" --region us-east-1
IP=$(aws ec2 describe-instances --instance-ids "$IID" --region us-east-1 \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
echo "xfusion-ec2 => $IP"
# verify
ssh -i /root/.ssh/id_rsa -o StrictHostKeyChecking=no root@"$IP" hostname
```
## Task 23
Create a New Private S3 Bucket: Name the bucket devops-sync-24571.
Data Migration: Migrate the entire data from the existing devops-s3-28903 bucket to the new devops-sync-24571 bucket.
Ensure Data Consistency: Ensure that both buckets have the same data.
Use AWS CLI: Use the AWS CLI to perform the creation and data migration tasks.
```bash
# 1. Create the new bucket (private by default)
aws s3api create-bucket \
--bucket devops-sync-24571 \
--region us-east-1
# 2. Sync everything from old -> new
aws s3 sync s3://devops-s3-28903 s3://devops-sync-24571 \
--region us-east-1
# 3. Verify parity (see below)
# Object counts should match
echo "source:"; aws s3 ls s3://devops-s3-28903 --recursive --summarize | tail -2
echo "dest:"; aws s3 ls s3://devops-sync-24571 --recursive --summarize | tail -2
# Dry-run reverse sync: if output is empty, buckets are in parity
aws s3 sync s3://devops-s3-28903 s3://devops-sync-24571 --dryrun
```
## Task 24
Set up an Application Load Balancer named xfusion-alb.
Create a target group named xfusion-tg.
Create a security group named xfusion-sg to open port 80 for the public.
Attach this security group to the ALB.
The ALB should route traffic on port 80 to port 80 of the xfusion-ec2 instance.
Make appropriate changes in the default security group attached to the EC2 instance if necessary.
### Solution
**xfusion-sg** opens port 80 to the public (`0.0.0.0/0`) — this is what lets internet traffic reach the ALB.
**ALB (xfusion-alb)** needs 2 subnets in 2 AZs (hard requirement) and wears `xfusion-sg`. It's the internet-facing entry point.
**Target group (xfusion-tg)** defines *where* traffic goes: HTTP:80, instance targets. The ALB itself doesn't know about instances — the TG is the binding layer. You register `xfusion-ec2` into it.
**Listener** ties them together: "traffic hitting ALB:80 → forward to xfusion-tg." Without it, the ALB accepts nothing.
**Instance's default SG** must allow 80 *from the ALB's SG* — because the ALB, not the public, is what actually connects to the instance. Skip this and traffic dies at the instance firewall even though everything upstream is wired.
Flow: `internet → xfusion-sg/ALB:80 → listener → xfusion-tg → ec2:80 (default SG allows ALB)`.
The one non-obvious bit: two separate SG hops. Public→ALB is open to the world; ALB→instance is locked to just the ALB's SG. That's least-privilege, and it's why the task has both a "create sg" step and a "fix the default sg" step.
```bash
REGION=us-east-1
# 0. Default VPC + two subnets in different AZs
VPC_ID=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
read -r SUBNET1 SUBNET2 <<< $(aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=$VPC_ID" \
--region $REGION \
--query 'Subnets[?MapPublicIpOnLaunch==`true`].SubnetId | [0:2]' \
--output text)
# 1. ALB security group — open 80 to the public
ALB_SG=$(aws ec2 create-security-group \
--group-name xfusion-sg \
--description "Public HTTP for xfusion-alb" \
--vpc-id "$VPC_ID" --region $REGION \
--query 'GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id "$ALB_SG" --protocol tcp --port 80 --cidr 0.0.0.0/0 \
--region $REGION
# 2. Create the ALB across both subnets, attach the SG
ALB_ARN=$(aws elbv2 create-load-balancer \
--name xfusion-alb \
--subnets "$SUBNET1" "$SUBNET2" \
--security-groups "$ALB_SG" \
--scheme internet-facing \
--type application \
--region $REGION \
--query 'LoadBalancers[0].LoadBalancerArn' --output text)
# 3. Create the target group (instance targets, HTTP:80, in the VPC)
TG_ARN=$(aws elbv2 create-target-group \
--name xfusion-tg \
--protocol HTTP --port 80 \
--vpc-id "$VPC_ID" \
--target-type instance \
--region $REGION \
--query 'TargetGroups[0].TargetGroupArn' --output text)
# 4. Register xfusion-ec2 into the target group
IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=xfusion-ec2" "Name=instance-state-name,Values=pending,running,stopping,stopped" \
--region $REGION \
--query 'Reservations[0].Instances[0].InstanceId' --output text)
aws elbv2 register-targets \
--target-group-arn "$TG_ARN" \
--targets Id="$IID" \
--region $REGION
# 5. Listener on ALB:80 -> forward to the target group
aws elbv2 create-listener \
--load-balancer-arn "$ALB_ARN" \
--protocol HTTP --port 80 \
--default-actions Type=forward,TargetGroupArn="$TG_ARN" \
--region $REGION
# 6. Open the instance's default SG to the ALB on port 80
DEFAULT_SG=$(aws ec2 describe-security-groups \
--filters "Name=group-name,Values=default" "Name=vpc-id,Values=$VPC_ID" \
--region $REGION \
--query 'SecurityGroups[0].GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id "$DEFAULT_SG" \
--protocol tcp --port 80 \
--source-group "$ALB_SG" \
--region $REGION
# LB waiter
aws elbv2 wait load-balancer-available \
--load-balancer-arns "$ALB_ARN" \
--region us-east-1
# Verify
# ALB provisioned + its DNS name
aws elbv2 describe-load-balancers --names xfusion-alb --region $REGION \
--query 'LoadBalancers[0].{State:State.Code,DNS:DNSName,SGs:SecurityGroups}'
# Target registered
aws elbv2 describe-target-health --target-group-arn "$TG_ARN" --region $REGION \
--query 'TargetHealthDescriptions[].{Id:Target.Id,Health:TargetHealth.State}'
# Listener forwarding 80 -> TG
aws elbv2 describe-listeners --load-balancer-arn "$ALB_ARN" --region $REGION \
--query 'Listeners[].{Port:Port,Action:DefaultActions[0].Type}'
```
## Task 25
The Nautilus DevOps team has been tasked with setting up an EC2 instance for their application. To ensure the application performs optimally, they also need to create a CloudWatch alarm to monitor the instance's CPU utilization. The alarm should trigger if the CPU utilization exceeds 90% for one consecutive 5-minute period. To send notifications, use the SNS topic named xfusion-sns-topic which is already created.
Launch EC2 Instance: Create an EC2 instance named xfusion-ec2 using any appropriate Ubuntu AMI.
Create CloudWatch Alarm: Create a CloudWatch alarm named xfusion-alarm with the following specifications:
Statistic: Average
Metric: CPU Utilization
Threshold: >= 90% for 1 consecutive 5-minute period.
Alarm Actions: Send a notification to xfusion-sns-topic.
```bash
REGION=us-east-1
# 1. Launch Ubuntu instance (24.04 noble LTS via SSM resolver, x86_64 for t2.micro)
IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/canonical/ubuntu/server/noble/stable/current/amd64/hvm/ebs-gp3/ami-id \
--instance-type t2.micro \
--region $REGION \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=xfusion-ec2}]' \
--query 'Instances[0].InstanceId' --output text)
aws ec2 wait instance-running --instance-ids "$IID" --region $REGION
# 2. Resolve the SNS topic ARN by name
SNS_ARN=$(aws sns list-topics --region $REGION \
--query "Topics[?ends_with(TopicArn, ':xfusion-sns-topic')].TopicArn | [0]" \
--output text)
# 3. Create the alarm
aws cloudwatch put-metric-alarm \
--alarm-name xfusion-alarm \
--region $REGION \
--namespace AWS/EC2 \
--metric-name CPUUtilization \
--dimensions Name=InstanceId,Value="$IID" \
--statistic Average \
--period 300 \
--evaluation-periods 1 \
--threshold 90 \
--comparison-operator GreaterThanOrEqualToThreshold \
--alarm-actions "$SNS_ARN"
# Verify
aws cloudwatch describe-alarms \
--alarm-names xfusion-alarm \
--region $REGION \
--query 'MetricAlarms[0].{Name:AlarmName,Metric:MetricName,Stat:Statistic,Threshold:Threshold,Op:ComparisonOperator,Period:Period,Eval:EvaluationPeriods,Actions:AlarmActions,Instance:Dimensions[0].Value}'
```
## Task 26
As a member of the Nautilus DevOps Team, your task is to create an EC2 instance with the following specifications:
Instance Name: The EC2 instance must be named datacenter-ec2.
AMI: Use any available Ubuntu AMI to create this instance.
User Data Script: Configure the instance to run a user data script during its launch. This script should:
Install the Nginx package.
Start the Nginx service.
Security Group: Ensure that the instance allows HTTP traffic on port 80 from the internet.
### Ubuntu based
```bash
REGION=us-east-1
# 0. Default VPC + SG allowing HTTP/80 from the internet
VPC_ID=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
SG_ID=$(aws ec2 create-security-group \
--group-name datacenter-http-sg \
--description "HTTP 80 from internet for datacenter-ec2" \
--vpc-id "$VPC_ID" --region $REGION \
--query 'GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id "$SG_ID" --protocol tcp --port 80 --cidr 0.0.0.0/0 \
--region $REGION
# 1. User-data script (runs as root at first boot)
cat > /tmp/nginx-userdata.sh << 'EOF'
#!/bin/bash
apt-get update -y
apt-get install -y nginx
systemctl enable nginx
systemctl start nginx
EOF
# 2. Launch Ubuntu instance with the user-data + SG
IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/canonical/ubuntu/server/noble/stable/current/amd64/hvm/ebs-gp3/ami-id \
--instance-type t2.micro \
--security-group-ids "$SG_ID" \
--user-data file:///tmp/nginx-userdata.sh \
--region $REGION \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=datacenter-ec2}]' \
--query 'Instances[0].InstanceId' --output text)
aws ec2 wait instance-running --instance-ids "$IID" --region $REGION
IP=$(aws ec2 describe-instances --instance-ids "$IID" --region $REGION \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
echo "datacenter-ec2 => http://$IP"
```
### Amazon linux based
```bash
REGION=us-east-1
# 0. Default VPC + SG allowing HTTP/80 from the internet
VPC_ID=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
SG_ID=$(aws ec2 create-security-group \
--group-name datacenter-http-sg \
--description "HTTP 80 from internet for datacenter-ec2" \
--vpc-id "$VPC_ID" --region $REGION \
--query 'GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id "$SG_ID" --protocol tcp --port 80 --cidr 0.0.0.0/0 \
--region $REGION
# 1. User-data script (runs as root at first boot)
cat > /tmp/nginx-userdata.sh << 'EOF'
#!/bin/bash
dnf update -y
dnf install -y nginx
systemctl enable nginx
systemctl start nginx
EOF
# 2. Launch AL2023 instance with the user-data + SG
IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
--instance-type t2.micro \
--security-group-ids "$SG_ID" \
--user-data file:///tmp/nginx-userdata.sh \
--region $REGION \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=datacenter-ec2}]' \
--query 'Instances[0].InstanceId' --output text)
aws ec2 wait instance-running --instance-ids "$IID" --region $REGION
IP=$(aws ec2 describe-instances --instance-ids "$IID" --region $REGION \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
echo "datacenter-ec2 => http://$IP"
# Validation
aws ec2 describe-security-groups --group-ids "$SG_ID" --region $REGION \
--query 'SecurityGroups[0].IpPermissions'
curl -I "http://$IP"
```
## Task 27
The Nautilus DevOps Team has received a request from the Networking Team to set up a new public VPC to support a set of public-facing services. This VPC will host various resources that need to be accessible over the internet. As part of this setup, you need to ensure the VPC has public subnets with automatic IP assignment for resources. Additionally, a new EC2 instance will be launched within this VPC to host public applications that require SSH access. This setup will enable the Networking Team to deploy and manage public-facing applications.
Create a public VPC named nautilus-pub-vpc, and a subnet named nautilus-pub-subnet under the same, make sure public IP is being auto assigned to resources under this subnet. Further, create an EC2 instance named nautilus-pub-ec2 under this VPC with instance type t2.micro. Make sure SSH port 22 is open for this instance and accessible over the internet.
```bash
REGION=us-east-1
# 1. VPC
VPC_ID=$(aws ec2 create-vpc \
--cidr-block 10.0.0.0/16 \
--region $REGION \
--tag-specifications 'ResourceType=vpc,Tags=[{Key=Name,Value=nautilus-pub-vpc}]' \
--query 'Vpc.VpcId' --output text)
# 2. Internet Gateway + attach to the VPC
IGW_ID=$(aws ec2 create-internet-gateway \
--region $REGION \
--tag-specifications 'ResourceType=internet-gateway,Tags=[{Key=Name,Value=nautilus-pub-igw}]' \
--query 'InternetGateway.InternetGatewayId' --output text)
aws ec2 attach-internet-gateway \
--internet-gateway-id "$IGW_ID" --vpc-id "$VPC_ID" --region $REGION
# 3. Subnet
SUBNET_ID=$(aws ec2 create-subnet \
--vpc-id "$VPC_ID" \
--cidr-block 10.0.1.0/24 \
--region $REGION \
--tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=nautilus-pub-subnet}]' \
--query 'Subnet.SubnetId' --output text)
# 4. Auto-assign public IP on the subnet
aws ec2 modify-subnet-attribute \
--subnet-id "$SUBNET_ID" \
--map-public-ip-on-launch \
--region $REGION
# 5. Route table + default route to IGW + associate with subnet
RTB_ID=$(aws ec2 create-route-table \
--vpc-id "$VPC_ID" \
--region $REGION \
--tag-specifications 'ResourceType=route-table,Tags=[{Key=Name,Value=nautilus-pub-rtb}]' \
--query 'RouteTable.RouteTableId' --output text)
aws ec2 create-route \
--route-table-id "$RTB_ID" \
--destination-cidr-block 0.0.0.0/0 \
--gateway-id "$IGW_ID" \
--region $REGION
aws ec2 associate-route-table \
--route-table-id "$RTB_ID" --subnet-id "$SUBNET_ID" --region $REGION
# 6. Security group allowing SSH/22 from the internet
SG_ID=$(aws ec2 create-security-group \
--group-name nautilus-pub-sg \
--description "SSH from internet for nautilus-pub-ec2" \
--vpc-id "$VPC_ID" --region $REGION \
--query 'GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id "$SG_ID" --protocol tcp --port 22 --cidr 0.0.0.0/0 \
--region $REGION
# 7. Launch the instance in the subnet
IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
--instance-type t2.micro \
--subnet-id "$SUBNET_ID" \
--security-group-ids "$SG_ID" \
--region $REGION \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=nautilus-pub-ec2}]' \
--query 'Instances[0].InstanceId' --output text)
aws ec2 wait instance-running --instance-ids "$IID" --region $REGION
IP=$(aws ec2 describe-instances --instance-ids "$IID" --region $REGION \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
echo "nautilus-pub-ec2 => $IP"
# Verify
# Route table has the IGW route and is associated with the subnet
aws ec2 describe-route-tables --route-table-ids "$RTB_ID" --region $REGION \
--query 'RouteTables[0].{Routes:Routes,Assoc:Associations[].SubnetId}'
# Subnet auto-assigns public IPs
aws ec2 describe-subnets --subnet-ids "$SUBNET_ID" --region $REGION \
--query 'Subnets[0].{Cidr:CidrBlock,AutoIP:MapPublicIpOnLaunch,VPC:VpcId}'
# Instance is up with a public IP
aws ec2 describe-instances --instance-ids "$IID" --region $REGION \
--query 'Reservations[0].Instances[0].{State:State.Name,PubIP:PublicIpAddress,Subnet:SubnetId}'
```
## Task 28
Create a private ECR repository named xfusion-ecr. There is a Dockerfile under /root/pyapp directory on aws-client host, build a docker image using this Dockerfile and push the same to the newly created ECR repo, the image tag must be latest.
```bash
REGION=us-east-1
# 1. Create the private ECR repository
aws ecr create-repository \
--repository-name xfusion-ecr \
--region $REGION
# 2. Resolve your account ID and build the registry URI
ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text)
REGISTRY="${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com"
REPO_URI="${REGISTRY}/xfusion-ecr"
# 3. Authenticate docker to ECR (token valid 12h)
aws ecr get-login-password --region $REGION \
| docker login --username AWS --password-stdin "$REGISTRY"
# 4. Build the image from the Dockerfile
docker build -t xfusion-ecr:latest /root/pyapp
# 5. Tag it with the full ECR URI + latest
docker tag xfusion-ecr:latest "${REPO_URI}:latest"
# 6. Push
docker push "${REPO_URI}:latest"
# Validate
aws ecr describe-images \
--repository-name xfusion-ecr \
--region $REGION \
--query 'imageDetails[].{Tags:imageTags,Pushed:imagePushedAt,SizeMB:imageSizeInBytes}'
```
## Task 29
The Nautilus DevOps team has been tasked with demonstrating the use of VPC Peering to enable communication between two VPCs. One VPC will be a private VPC that contains a private EC2 instance, while the other will be the default public VPC containing a publicly accessible EC2 instance.
1) There is already an existing EC2 instance in the public vpc/subnet:
Name: devops-public-ec2
2) There is already an existing Private VPC:
Name: devops-private-vpc
CIDR: 10.1.0.0/16
3) There is already an existing Subnet in devops-private-vpc:
Name: devops-private-subnet
CIDR: 10.1.1.0/24
4) There is already an existing EC2 instance in the private subnet:
Name: devops-private-ec2
5) Create a Peering Connection between the Default VPC and the Private VPC:
VPC Peering Connection Name: devops-vpc-peering
6) Configure Route Tables to enable communication between the two VPCs.
Ensure the private EC2 instance is accessible from the public EC2 instance.
7) Test the Connection:
Add /root/.ssh/id_rsa.pub public key to the public EC2 instance's ec2-user's authorized_keys to make sure we are able to ssh into this instance from AWS client host. You may also need to update the security group of the private EC2 instance to allow ICMP traffic from the public/default VPC CIDR. This will enable you to ping the private instance from the public instance.
SSH into the public EC2 instance and ensure that you can ping the private EC2 instance.
```bash
REGION=us-east-1
# ── Discover both VPCs and their CIDRs ──────────────────────────────
DEF_VPC=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
DEF_CIDR=$(aws ec2 describe-vpcs --vpc-ids $DEF_VPC \
--region $REGION --query 'Vpcs[0].CidrBlock' --output text)
PRIV_VPC=$(aws ec2 describe-vpcs --filters Name=tag:Name,Values=devops-private-vpc \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
PRIV_CIDR=10.1.0.0/16
# ── 1. Create the peering connection (requester=default, peer=private) ──
PCX=$(aws ec2 create-vpc-peering-connection \
--vpc-id $DEF_VPC \
--peer-vpc-id $PRIV_VPC \
--region $REGION \
--tag-specifications 'ResourceType=vpc-peering-connection,Tags=[{Key=Name,Value=devops-vpc-peering}]' \
--query 'VpcPeeringConnection.VpcPeeringConnectionId' --output text)
# ── 2. Accept it (even same-account starts in pending-acceptance) ──
aws ec2 accept-vpc-peering-connection \
--vpc-peering-connection-id $PCX --region $REGION
# ── 3. Resolve instance IDs, subnets, IPs ──
PUB_IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=devops-public-ec2" "Name=instance-state-name,Values=pending,running,stopping,stopped" \
--region $REGION --query 'Reservations[0].Instances[0].InstanceId' --output text)
PUB_SUBNET=$(aws ec2 describe-instances --instance-ids $PUB_IID --region $REGION \
--query 'Reservations[0].Instances[0].SubnetId' --output text)
PUB_IP=$(aws ec2 describe-instances --instance-ids $PUB_IID --region $REGION \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
PUB_AZ=$(aws ec2 describe-instances --instance-ids $PUB_IID --region $REGION \
--query 'Reservations[0].Instances[0].Placement.AvailabilityZone' --output text)
PUB_SG=$(aws ec2 describe-instances --instance-ids $PUB_IID --region $REGION \
--query 'Reservations[0].Instances[0].SecurityGroups[0].GroupId' --output text)
PRIV_IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=devops-private-ec2" "Name=instance-state-name,Values=pending,running,stopping,stopped" \
--region $REGION --query 'Reservations[0].Instances[0].InstanceId' --output text)
PRIV_SUBNET=$(aws ec2 describe-subnets --filters "Name=tag:Name,Values=devops-private-subnet" \
--region $REGION --query 'Subnets[0].SubnetId' --output text)
PRIV_PRIV_IP=$(aws ec2 describe-instances --instance-ids $PRIV_IID --region $REGION \
--query 'Reservations[0].Instances[0].PrivateIpAddress' --output text)
PRIV_SG=$(aws ec2 describe-instances --instance-ids $PRIV_IID --region $REGION \
--query 'Reservations[0].Instances[0].SecurityGroups[0].GroupId' --output text)
# ── 4. Find the route table serving each subnet (explicit assoc, else main) ──
rtb_for_subnet () { # $1=subnet $2=vpc
local r
r=$(aws ec2 describe-route-tables --filters "Name=association.subnet-id,Values=$1" \
--region $REGION --query 'RouteTables[0].RouteTableId' --output text)
if [ "$r" = "None" ] || [ -z "$r" ]; then
r=$(aws ec2 describe-route-tables \
--filters "Name=vpc-id,Values=$2" "Name=association.main,Values=true" \
--region $REGION --query 'RouteTables[0].RouteTableId' --output text)
fi
echo "$r"
}
DEF_RTB=$(rtb_for_subnet $PUB_SUBNET $DEF_VPC)
PRIV_RTB=$(rtb_for_subnet $PRIV_SUBNET $PRIV_VPC)
# ── 5. Routes BOTH directions through the peering connection ──
aws ec2 create-route --route-table-id $DEF_RTB \
--destination-cidr-block $PRIV_CIDR \
--vpc-peering-connection-id $PCX --region $REGION
aws ec2 create-route --route-table-id $PRIV_RTB \
--destination-cidr-block $DEF_CIDR \
--vpc-peering-connection-id $PCX --region $REGION
# ── 6. Private instance SG: allow ICMP from the default VPC CIDR ──
aws ec2 authorize-security-group-ingress \
--group-id $PRIV_SG --protocol icmp --port -1 \
--cidr $DEF_CIDR --region $REGION
# ── 7a. Make sure public instance SG allows SSH from aws-client ──
aws ec2 authorize-security-group-ingress \
--group-id $PUB_SG --protocol tcp --port 22 \
--cidr 0.0.0.0/0 --region $REGION 2>/dev/null || true
# Push id_rsa.pub via EC2 Instance Connect (valid 60s)
aws ec2-instance-connect send-ssh-public-key \
--instance-id $PUB_IID \
--instance-os-user ec2-user \
--ssh-public-key file:///root/.ssh/id_rsa.pub \
--availability-zone $PUB_AZ \
--region $REGION
# Within 60s: SSH in and make the key permanent
PUBKEY=$(cat /root/.ssh/id_rsa.pub)
ssh -o StrictHostKeyChecking=no -i /root/.ssh/id_rsa ec2-user@$PUB_IP \
"mkdir -p ~/.ssh && chmod 700 ~/.ssh && echo '$PUBKEY' >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
# Now SSH persists. Test the peering by pinging the private instance's PRIVATE IP:
ssh -i /root/.ssh/id_rsa ec2-user@$PUB_IP "ping -c 4 $PRIV_PRIV_IP"
# Verify
# Peering active
aws ec2 describe-vpc-peering-connections --vpc-peering-connection-ids $PCX \
--region $REGION --query 'VpcPeeringConnections[0].Status.Code'
# Both route tables have the pcx route
aws ec2 describe-route-tables --route-table-ids $DEF_RTB $PRIV_RTB --region $REGION \
--query 'RouteTables[].{RTB:RouteTableId,PcxRoutes:Routes[?VpcPeeringConnectionId==`'$PCX'`].DestinationCidrBlock}'
# Private SG has ICMP from default CIDR
aws ec2 describe-security-groups --group-ids $PRIV_SG --region $REGION \
--query 'SecurityGroups[0].IpPermissions[?IpProtocol==`icmp`]'
```
## Task 30
The following components already exist in the environment:
1) A VPC named xfusion-priv-vpc and a private subnet named xfusion-priv-subnet have been created.
2) An EC2 instance named xfusion-priv-ec2 is already running in the private subnet.
3) The EC2 instance is configured with a cron job that uploads a test file to the S3 bucket xfusion-nat-6527 every minute. Upload will only succeed once internet access is established.
Your task is to:
Create a new public subnet named xfusion-pub-subnet in the existing VPC.
Launch a NAT Instance in the public subnet using an Amazon Linux 2023 AMI and name it xfusion-nat-instance. Configure this instance to act as a NAT instance. Make sure to use a custom security group for this instance.
After the configuration, verify that the test file xfusion-test.txt appears in the S3 bucket xfusion-nat-6527. This indicates successful internet access from the private EC2 instance via the NAT Instance.
Note: iptables is not installed by default on Amazon Linux 2023. You will need to install and enable it before configuring NAT setup.
### Claude's rewording:
The final boss — a manual NAT instance, which is a genuinely different beast from a NAT gateway. The concept that makes or breaks this: a NAT instance must have source/dest check disabled. By default EC2 drops any packet whose source or destination IP isn't the instance itself — but a NAT box exists precisely to forward packets belonging to other instances. Leave that check on and every forwarded packet silently dies, no error, nothing. That plus IP forwarding + an iptables MASQUERADE rule is the whole trick, and the routing has to point the private subnet's default route at the NAT instance's ENI.
```bash
REGION=us-east-1
# ── Discover existing pieces ──────────────────────────────────────
VPC_ID=$(aws ec2 describe-vpcs --filters Name=tag:Name,Values=xfusion-priv-vpc \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
VPC_CIDR=$(aws ec2 describe-vpcs --vpc-ids $VPC_ID --region $REGION \
--query 'Vpcs[0].CidrBlock' --output text)
PRIV_SUBNET=$(aws ec2 describe-subnets --filters "Name=tag:Name,Values=xfusion-priv-subnet" \
--region $REGION --query 'Subnets[0].SubnetId' --output text)
PRIV_SUBNET_CIDR=$(aws ec2 describe-subnets --subnet-ids $PRIV_SUBNET --region $REGION \
--query 'Subnets[0].CidrBlock' --output text)
PRIV_AZ=$(aws ec2 describe-subnets --subnet-ids $PRIV_SUBNET --region $REGION \
--query 'Subnets[0].AvailabilityZone' --output text)
# ── Ensure the VPC has an IGW (a "private VPC" usually doesn't) ────
IGW_ID=$(aws ec2 describe-internet-gateways \
--filters "Name=attachment.vpc-id,Values=$VPC_ID" \
--region $REGION --query 'InternetGateways[0].InternetGatewayId' --output text)
if [ "$IGW_ID" = "None" ] || [ -z "$IGW_ID" ]; then
IGW_ID=$(aws ec2 create-internet-gateway --region $REGION \
--tag-specifications 'ResourceType=internet-gateway,Tags=[{Key=Name,Value=xfusion-igw}]' \
--query 'InternetGateway.InternetGatewayId' --output text)
aws ec2 attach-internet-gateway --internet-gateway-id $IGW_ID --vpc-id $VPC_ID --region $REGION
fi
echo "REGION=$REGION
VPC_ID=$VPC_ID
VPC_CIDR=$VPC_CIDR
PRIV_SUBNET=$PRIV_SUBNET
PRIV_SUBNET_CIDR=$PRIV_SUBNET_CIDR
PRIV_AZ=$PRIV_AZ
IGW_ID=$IGW_ID"
# ── Pick a free /24 in the VPC for the public subnet ──────────────
BASE=$(echo $VPC_CIDR | cut -d. -f1-2)
USED=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" \
--region $REGION --query 'Subnets[].CidrBlock' --output text | tr '\t' '\n')
for i in $(seq 1 254); do
CAND="${BASE}.${i}.0/24"
echo "$USED" | grep -Fxq "$CAND" || { PUB_CIDR=$CAND; break; }
done
# ── 1. Create the public subnet + auto-assign public IP ───────────
PUB_SUBNET=$(aws ec2 create-subnet \
--vpc-id $VPC_ID --cidr-block $PUB_CIDR \
--availability-zone $PRIV_AZ \
--region $REGION \
--tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=xfusion-pub-subnet}]' \
--query 'Subnet.SubnetId' --output text)
aws ec2 modify-subnet-attribute --subnet-id $PUB_SUBNET \
--map-public-ip-on-launch --region $REGION
# ── 2. Public route table -> IGW, associate with public subnet ────
PUB_RTB=$(aws ec2 create-route-table --vpc-id $VPC_ID --region $REGION \
--tag-specifications 'ResourceType=route-table,Tags=[{Key=Name,Value=xfusion-pub-rtb}]' \
--query 'RouteTable.RouteTableId' --output text)
aws ec2 create-route --route-table-id $PUB_RTB \
--destination-cidr-block 0.0.0.0/0 --gateway-id $IGW_ID --region $REGION
aws ec2 associate-route-table --route-table-id $PUB_RTB --subnet-id $PUB_SUBNET --region $REGION
# ── 3. Custom SG for the NAT instance ─────────────────────────────
NAT_SG=$(aws ec2 create-security-group \
--group-name xfusion-nat-sg \
--description "NAT instance SG for xfusion" \
--vpc-id $VPC_ID --region $REGION \
--query 'GroupId' --output text)
# allow all traffic from inside the VPC (private instances routing through)
aws ec2 authorize-security-group-ingress --group-id $NAT_SG \
--protocol -1 --cidr $VPC_CIDR --region $REGION
# optional: SSH from anywhere for debugging
aws ec2 authorize-security-group-ingress --group-id $NAT_SG \
--protocol tcp --port 22 --cidr 0.0.0.0/0 --region $REGION 2>/dev/null || true
echo "BASE=$BASE
USED=$USED
PUB_CIDR=$PUB_CIDR
PUB_SUBNET=$PUB_SUBNET
PUB_RTB=$PUB_RTB
NAT_SG=$NAT_SG"
# ── 4. NAT user-data: iptables + IP forwarding + MASQUERADE ───────
cat > /tmp/nat-userdata.sh << 'EOF'
#!/bin/bash
set -e
dnf install -y iptables iptables-services
echo 'net.ipv4.ip_forward = 1' > /etc/sysctl.d/99-nat.conf
sysctl -p /etc/sysctl.d/99-nat.conf
EXT_IF=$(ip -o -4 route show to default | awk '{print $5; exit}')
iptables -t nat -A POSTROUTING -o "$EXT_IF" -j MASQUERADE
iptables -A FORWARD -i "$EXT_IF" -j ACCEPT
systemctl enable iptables
service iptables save
EOF
# ── 5. Launch the NAT instance (AL2023) ───────────────────────────
NAT_IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
--instance-type t2.micro \
--subnet-id $PUB_SUBNET \
--security-group-ids $NAT_SG \
--associate-public-ip-address \
--user-data file:///tmp/nat-userdata.sh \
--region $REGION \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=xfusion-nat-instance}]' \
--query 'Instances[0].InstanceId' --output text)
aws ec2 wait instance-running --instance-ids $NAT_IID --region $REGION
# ── 6. *** THE CRITICAL STEP *** disable source/dest check ────────
aws ec2 modify-instance-attribute --instance-id $NAT_IID \
--no-source-dest-check --region $REGION
# ── 7. Point the private subnet's default route at the NAT instance ─
PRIV_RTB=$(aws ec2 describe-route-tables \
--filters "Name=association.subnet-id,Values=$PRIV_SUBNET" \
--region $REGION --query 'RouteTables[0].RouteTableId' --output text)
if [ "$PRIV_RTB" = "None" ] || [ -z "$PRIV_RTB" ]; then
PRIV_RTB=$(aws ec2 describe-route-tables \
--filters "Name=vpc-id,Values=$VPC_ID" "Name=association.main,Values=true" \
--region $REGION --query 'RouteTables[0].RouteTableId' --output text)
fi
# create (or replace if one already exists)
aws ec2 create-route --route-table-id $PRIV_RTB \
--destination-cidr-block 0.0.0.0/0 --instance-id $NAT_IID --region $REGION \
2>/dev/null || \
aws ec2 replace-route --route-table-id $PRIV_RTB \
--destination-cidr-block 0.0.0.0/0 --instance-id $NAT_IID --region $REGION
echo "NAT instance: $NAT_IID | public subnet: $PUB_SUBNET ($PUB_CIDR) | priv RTB: $PRIV_RTB"
# Verify
# Confirm source/dest check is OFF (must be false)
aws ec2 describe-instances --instance-ids $NAT_IID --region $REGION \
--query 'Reservations[0].Instances[0].SourceDestCheck'
# Poll the bucket for the test file
for n in $(seq 1 6); do
if aws s3 ls s3://xfusion-nat-6527/ --region $REGION | grep -q xfusion-test.txt; then
echo "SUCCESS — file landed:"; aws s3 ls s3://xfusion-nat-6527/ --region $REGION | grep xfusion-test.txt
break
fi
echo "waiting for cron upload... ($n)"; sleep 30
done
```

316
aws-31-34.md Normal file
View File

@@ -0,0 +1,316 @@
## Task 31
As a member of the Nautilus DevOps Team, your task is to perform the following:
Provision a Private RDS Instance: Create a new private RDS instance named devops-rds using the Full configuration database creation method, and select the Free tier template. Further, it must be a db.t3.micro type instance.
Engine Configuration: Use the MySQL engine with version 8.4.x.
Enable Storage Autoscaling: Enable storage autoscaling and set the threshold value to 50GB. Keep the rest of the configurations as default.
Instance Availability: Ensure the instance is in the available state before submitting this task.
```bash
REGION=us-east-1
aws rds create-db-instance \
--db-instance-identifier devops-rds \
--engine mysql \
--engine-version 8.4.10 \
--db-instance-class db.t3.micro \
--allocated-storage 20 \
--max-allocated-storage 50 \
--master-username admin \
--master-user-password 'ChangeMe_Str0ng!23' \
--no-publicly-accessible \
--no-multi-az \
--backup-retention-period 7 \
--region $REGION
# waiter
aws rds wait db-instance-available \
--db-instance-identifier devops-rds \
--region $REGION
# status poller
watch -n 10 'aws rds describe-db-instances \
--db-instance-identifier devops-rds \
--query "DBInstances[0].DBInstanceStatus" --output text'
# validation
aws rds describe-db-instances \
--db-instance-identifier devops-rds \
--region $REGION \
--query 'DBInstances[0].{Status:DBInstanceStatus,Engine:Engine,Version:EngineVersion,Class:DBInstanceClass,Alloc:AllocatedStorage,MaxAlloc:MaxAllocatedStorage,Public:PubliclyAccessible,MultiAZ:MultiAZ}'
```
## Task 32
As a member of the Nautilus DevOps Team, your task is to perform the following:
Take a Snapshot: Take a snapshot of the xfusion-rds RDS instance and name it xfusion-snapshot (please wait xfusion-rds instance to be in available state).
Restore the Snapshot: Restore the snapshot to a new RDS instance named xfusion-snapshot-restore.
Configure the New RDS Instance: Ensure that the new RDS instance has a class of db.t3.micro.
Verify the New RDS Instance: The new RDS instance must be in the Available state upon completion of the restoration process.
```bash
REGION=us-east-1
# ── 0. Wait for the source to be available before snapshotting ──
# `aws rds wait db-instance-available` blocks silently for up to ~40 min
# with zero feedback, so swap in a polling loop that prints the state
# on every check (original waiter kept below for reference):
# aws rds wait db-instance-available \
# --db-instance-identifier xfusion-rds --region $REGION
until [ "$STATUS" = "available" ]; do
STATUS=$(aws rds describe-db-instances \
--db-instance-identifier xfusion-rds --region $REGION \
--query 'DBInstances[0].DBInstanceStatus' --output text)
echo "xfusion-rds: $STATUS"
[ "$STATUS" = "available" ] || sleep 10
done
# ── 1. Take the snapshot ──
aws rds create-db-snapshot \
--db-instance-identifier xfusion-rds \
--db-snapshot-identifier xfusion-snapshot \
--region $REGION
# Wait for the snapshot to finish (creating -> available)
# aws rds wait db-snapshot-available \
# --db-snapshot-identifier xfusion-snapshot --region $REGION
unset STATUS
until [ "$STATUS" = "available" ]; do
STATUS=$(aws rds describe-db-snapshots \
--db-snapshot-identifier xfusion-snapshot --region $REGION \
--query 'DBSnapshots[0].Status' --output text)
echo "xfusion-snapshot: $STATUS"
[ "$STATUS" = "available" ] || sleep 10
done
# ── 2 + 3. Restore into a NEW instance, force db.t3.micro ──
aws rds restore-db-instance-from-db-snapshot \
--db-instance-identifier xfusion-snapshot-restore \
--db-snapshot-identifier xfusion-snapshot \
--db-instance-class db.t3.micro \
--region $REGION
# ── 4. Wait for the restored instance to be available ──
# aws rds wait db-instance-available \
# --db-instance-identifier xfusion-snapshot-restore --region $REGION
unset STATUS
until [ "$STATUS" = "available" ]; do
STATUS=$(aws rds describe-db-instances \
--db-instance-identifier xfusion-snapshot-restore --region $REGION \
--query 'DBInstances[0].DBInstanceStatus' --output text)
echo "xfusion-snapshot-restore: $STATUS"
[ "$STATUS" = "available" ] || sleep 10
done
# verify
REGION=us-east-1
aws rds describe-db-instances \
--db-instance-identifier xfusion-snapshot-restore \
--region $REGION \
--query 'DBInstances[0].{Status:DBInstanceStatus,Class:DBInstanceClass,Engine:Engine,Version:EngineVersion}'
aws rds describe-db-snapshots \
--db-snapshot-identifier xfusion-snapshot \
--region $REGION \
--query 'DBSnapshots[0].{Id:DBSnapshotIdentifier,Status:Status,Source:DBInstanceIdentifier}'
```
## Task 33
The Nautilus DevOps team is embracing serverless architecture by integrating AWS Lambda into their operational tasks. They have decided to deploy a simple Lambda function that will return a custom greeting to demonstrate serverless capabilities effectively. This function is crucial for showcasing rapid deployment and easy scalability features of AWS Lambda to the team.
Create Lambda Function: Create a Lambda function named xfusion-lambda.
Runtime: Use the Runtime Python.
Deploy: The function should print the body Welcome to KKE AWS Labs!.
Status Code: Ensure the status code is 200.
IAM Role: Create and use the IAM role named lambda_execution_role.
Use the AWS Console to complete this task.
# AWS Lambda Task — `xfusion-lambda`
Runtime pinned: latest is **Python 3.14** (Lambda added it Nov 18, 2025 as the latest LTS release). Use `python3.14` — or 3.13 if the grader's env lags; both are current. Avoid 3.10/3.11 (on the AL2 deprecation track tied to the June 30, 2026 EOL).
Task says use the console, so here's the click-path — with the CLI equivalent after, for reproducibility.
## Console steps
First the IAM role (Lambda needs it to exist before/at function creation):
1. **IAM → Roles → Create role.** Trusted entity type: **AWS service**. Use case: **Lambda**. Next.
2. Attach **`AWSLambdaBasicExecutionRole`** (managed policy — grants CloudWatch Logs write, the minimum a Lambda needs). Next.
3. Role name: **`lambda_execution_role`**. Create role.
Then the function:
4. **Lambda → Create function → Author from scratch.**
5. Function name: **`xfusion-lambda`**. Runtime: **Python 3.14**. Architecture: leave x86_64.
6. Expand **Change default execution role → Use an existing role →** pick **`lambda_execution_role`**. Create function.
7. In the code editor, replace `lambda_function.py` with the handler below, then **Deploy** (Ctrl+S / Deploy button).
8. **Test** → create a test event (any name, default empty `{}` payload) → Run. Confirm the result shows `statusCode: 200` and the body string.
The handler:
```python
def lambda_handler(event, context):
return {
'statusCode': 200,
'body': 'Welcome to KKE AWS Labs!'
}
```
That's the whole function — returns the exact body and a 200. The task's "print the body" maps to the returned `body` field (what an invocation/API-Gateway-style response surfaces), not a `print()` to stdout. Returning the dict with `statusCode: 200` satisfies both requirements 3 and 4.
## CLI equivalent
In case you'd rather, or to verify:
```bash
REGION=us-east-1
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
# 1. Trust policy + role
cat > /tmp/lambda-trust.json << 'EOF'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"lambda.amazonaws.com"},"Action":"sts:AssumeRole"}]}
EOF
aws iam create-role \
--role-name lambda_execution_role \
--assume-role-policy-document file:///tmp/lambda-trust.json
aws iam attach-role-policy \
--role-name lambda_execution_role \
--policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
# 2. Package the handler
mkdir -p /tmp/fn && cat > /tmp/fn/lambda_function.py << 'EOF'
def lambda_handler(event, context):
return {'statusCode': 200, 'body': 'Welcome to KKE AWS Labs!'}
EOF
(cd /tmp/fn && zip -q function.zip lambda_function.py)
# 3. Create the function (IAM role propagation lags ~10s; retry if it errors)
aws lambda create-function \
--function-name xfusion-lambda \
--runtime python3.14 \
--role "arn:aws:iam::${ACCOUNT_ID}:role/lambda_execution_role" \
--handler lambda_function.lambda_handler \
--zip-file fileb:///tmp/fn/function.zip \
--region $REGION
```
## Things worth knowing (either path)
- **`--handler lambda_function.lambda_handler`** = `<filename without .py>.<function name>`. The file must be `lambda_function.py` and the function `lambda_handler`, or Lambda can't find the entry point. This is the #1 "works in console, breaks on deploy" mismatch.
- **IAM role propagation is eventually consistent** — a freshly-created role sometimes isn't visible to Lambda for a few seconds, so CLI `create-function` can throw `The role defined for the function cannot be assumed`. Just re-run; it settles. The console usually masks this because you click through slower than the propagation.
## Verify
```bash
aws lambda invoke --function-name xfusion-lambda --region $REGION /tmp/out.json \
&& cat /tmp/out.json
```
Want `{"statusCode": 200, "body": "Welcome to KKE AWS Labs!"}`. That's the invocation returning the exact body with a 200. Done.
## Task 34
The Nautilus DevOps team continues to explore serverless architecture by setting up another Lambda function. This time, the task must be completed using the AWS Console to familiarize the team with the web interface. The function will return a custom greeting and demonstrate the capabilities of AWS Lambda effectively.
Create Python Script: Create a Python script named lambda_function.py with a function that returns the body Welcome to KKE AWS Labs! and status code 200.
Zip the Python Script: Zip the script into a file named function.zip.
Create Lambda Function: Create a Lambda function named xfusion-lambda-cli using the zipped file and specify Python as the runtime.
IAM Role: Use the IAM role named lambda_execution_role.
```bash
REGION=us-east-1
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
# --- script + zip ---
mkdir -p ~/xfusion-lambda-cli && cd ~/xfusion-lambda-cli
cat > lambda_function.py << 'EOF'
def lambda_handler(event, context):
return {'statusCode': 200, 'body': 'Welcome to KKE AWS Labs!'}
EOF
zip function.zip lambda_function.py
# --- IAM role ---
cat > /tmp/lambda-trust.json << 'EOF'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"lambda.amazonaws.com"},"Action":"sts:AssumeRole"}]}
EOF
aws iam create-role \
--role-name lambda_execution_role \
--assume-role-policy-document file:///tmp/lambda-trust.json
aws iam attach-role-policy \
--role-name lambda_execution_role \
--policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
# --- give IAM a moment to propagate, then create the function ---
sleep 10
aws lambda create-function \
--function-name xfusion-lambda-cli \
--runtime python3.14 \
--role "arn:aws:iam::${ACCOUNT_ID}:role/lambda_execution_role" \
--handler lambda_function.lambda_handler \
--zip-file fileb://function.zip \
--region $REGION
# Verify
aws lambda invoke --function-name xfusion-lambda-cli --region $REGION /tmp/out.json \
&& cat /tmp/out.json
```
## Task 36
```bash
```
## Task 37
```bash
```
## Task 38
```bash
```
## Task 39
```bash
```
## Task 40
```bash
```

217
aws-35.md Normal file
View File

@@ -0,0 +1,217 @@
## Task 35
The Nautilus DevOps team needs a new private RDS instance for their application. They need to set up a MySQL database and ensure that their existing EC2 instance can connect to it. This will help in managing their database needs efficiently and securely.
1) Task Details:
Create a private RDS instance named xfusion-rds using a sandbox template.
The engine type must be MySQL v8.4.5, and it must be a db.t3.micro type instance.
The master username must be xfusion_admin with an appropriate password.
The RDS storage type must be gp2, and the storage size must be 5GiB.
Create a database named xfusion_db.
Keep the rest of the configurations as default. Ensure the instance is in available state.
Adjust the security groups so that the xfusion-ec2 instance can connect to the RDS on port 3306 and also open port 80 for the instance.
2) An EC2 instance named xfusion-ec2 exists. Connect to this instance from the AWS console. Create an SSH key (/root/.ssh/id_rsa) on the aws-client host if it doesn't already exist. Add the public key to the authorized keys of the root user on the EC2 instance for password-less SSH access.
3) There is a file named index.php under the /root directory on the aws-client host. Copy this file to the xfusion-ec2 instance under the /var/www/html/ directory. Make the appropriate changes in the file to connect to the RDS.
4) You should see a Connected successfully message in the browser once you access the instance using the public IP.
### Solution
# xfusion-rds + EC2 + PHP Task
Big multi-part one — RDS + SSH bootstrap + a PHP app wired to the DB. The engine version is **pinned by the task to 8.4.5**, so use exactly that (no "latest" substitution — the grader wants `8.4.5`, which is a valid 8.4 minor).
Same CLI-template caveat as before: "sandbox" isn't a CLI flag — it's a console wizard preset that maps to single-AZ / minimal / no deletion-protection. You reproduce it with the individual flags.
## Phase 1 — Discover the EC2, build the RDS security group
```bash
REGION=us-east-1
# EC2 identity, its SG, VPC, AZ, public IP
EC2_IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=xfusion-ec2" "Name=instance-state-name,Values=pending,running,stopping,stopped" \
--region $REGION --query 'Reservations[0].Instances[0].InstanceId' --output text)
EC2_SG=$(aws ec2 describe-instances --instance-ids $EC2_IID --region $REGION \
--query 'Reservations[0].Instances[0].SecurityGroups[0].GroupId' --output text)
VPC_ID=$(aws ec2 describe-instances --instance-ids $EC2_IID --region $REGION \
--query 'Reservations[0].Instances[0].VpcId' --output text)
EC2_AZ=$(aws ec2 describe-instances --instance-ids $EC2_IID --region $REGION \
--query 'Reservations[0].Instances[0].Placement.AvailabilityZone' --output text)
EC2_IP=$(aws ec2 describe-instances --instance-ids $EC2_IID --region $REGION \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
# Dedicated SG for RDS, allow 3306 ONLY from the EC2's SG
RDS_SG=$(aws ec2 create-security-group \
--group-name xfusion-rds-sg \
--description "MySQL 3306 from xfusion-ec2" \
--vpc-id $VPC_ID --region $REGION \
--query 'GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id $RDS_SG --protocol tcp --port 3306 \
--source-group $EC2_SG --region $REGION
# Open port 80 on the EC2 itself (for the web page)
aws ec2 authorize-security-group-ingress \
--group-id $EC2_SG --protocol tcp --port 80 --cidr 0.0.0.0/0 \
--region $REGION 2>/dev/null || true
```
The **`--source-group $EC2_SG`** is the clean pattern: RDS accepts 3306 only from instances wearing the EC2's SG, not the whole internet. That's the "adjust SGs so xfusion-ec2 can connect to RDS on 3306" requirement, least-privilege.
## Phase 2 — Create the private RDS instance
```bash
RDS_PASS='Xfusion_Adm1n#2026' # no / @ " or spaces
aws rds create-db-instance \
--db-instance-identifier xfusion-rds \
--engine mysql \
--engine-version 8.4.5 \
--db-instance-class db.t3.micro \
--storage-type gp2 \
--allocated-storage 5 \
--master-username xfusion_admin \
--master-user-password "$RDS_PASS" \
--db-name xfusion_db \
--vpc-security-group-ids $RDS_SG \
--no-publicly-accessible \
--no-multi-az \
--region $REGION
# RDS is slow — wait it out
# aws rds wait db-instance-available \
# --db-instance-identifier xfusion-rds --region $REGION
STATUS=""
until [ "$STATUS" = "available" ]; do
STATUS=$(aws rds describe-db-instances \
--db-instance-identifier xfusion-rds --region $REGION \
--query 'DBInstances[0].DBInstanceStatus' --output text)
echo "xfusion-rds: $STATUS"
[ "$STATUS" = "available" ] || sleep 10
done
# Grab the endpoint for the PHP config
RDS_ENDPOINT=$(aws rds describe-db-instances \
--db-instance-identifier xfusion-rds --region $REGION \
--query 'DBInstances[0].Endpoint.Address' --output text)
echo "RDS endpoint: $RDS_ENDPOINT"
```
Requirement to flag mapping:
- **`--db-name xfusion_db`** creates the initial database at provision time (req #5 — don't skip it, adding a DB later means a separate mysql connection).
- **`--storage-type gp2 --allocated-storage 5`** = gp2 / 5 GiB.
- **`--no-publicly-accessible`** = private.
- **`--no-multi-az`** + minimal flags = the sandbox template shape.
- **`--vpc-security-group-ids $RDS_SG`** attaches the SG from Phase 1 so the EC2 can reach it.
## Phase 3 — SSH key + passwordless root access to the EC2
On `aws-client`:
```bash
# 1. Generate id_rsa only if missing
[ -f /root/.ssh/id_rsa ] || ssh-keygen -t rsa -b 4096 -f /root/.ssh/id_rsa -N "" -q
PUBKEY=$(cat /root/.ssh/id_rsa.pub)
# 2. Push the key to ec2-user via EC2 Instance Connect (60s TTL)
aws ec2-instance-connect send-ssh-public-key \
--instance-id $EC2_IID \
--instance-os-user ec2-user \
--ssh-public-key file:///root/.ssh/id_rsa.pub \
--availability-zone $EC2_AZ \
--region $REGION
# 3. Within 60s: hop in as ec2-user, plant the key into ROOT's authorized_keys
ssh -o StrictHostKeyChecking=no -i /root/.ssh/id_rsa ec2-user@$EC2_IP "
sudo mkdir -p /root/.ssh && sudo chmod 700 /root/.ssh
echo '$PUBKEY' | sudo tee -a /root/.ssh/authorized_keys >/dev/null
sudo chmod 600 /root/.ssh/authorized_keys
echo 'PermitRootLogin prohibit-password' | sudo tee /etc/ssh/sshd_config.d/99-root.conf >/dev/null
sudo systemctl restart sshd
"
# 4. Now passwordless root SSH works:
ssh -i /root/.ssh/id_rsa root@$EC2_IP hostname
```
"Connect from the AWS console" = EC2 Instance Connect, which is exactly what `send-ssh-public-key` does programmatically — pushes a short-lived key so you can get on the box the first time, then you persist your real key into **root's** authorized_keys (the task specifically wants root, not ec2-user).
## Phase 4 — Install the web stack, deploy & wire index.php
The EC2 needs Apache + PHP + the MySQL PHP driver, or the page won't render or connect. Do it over the root SSH you just set up:
```bash
ssh -i /root/.ssh/id_rsa root@$EC2_IP "
apt-get update -y
apt-get install -y apache2 php libapache2-mod-php php-mysql
systemctl enable --now apache2
"
# Stage a local copy of index.php and look at what placeholders it uses
cp /root/index.php /tmp/index.php
cat /tmp/index.php
```
KodeKloud's `index.php` uses `$dbhost`, `$dbuser`, `$dbpass`, `$dbname` as single-quoted placeholders (`'<dbhost>'`, etc.). Patch the staged copy in-place — then upload the finished file:
```bash
sed -i \
-e "s/\$dbhost *= *.*/\$dbhost = '$RDS_ENDPOINT';/" \
-e "s/\$dbuser *= *.*/\$dbuser = 'xfusion_admin';/" \
-e "s/\$dbpass *= *.*/\$dbpass = '$RDS_PASS';/" \
-e "s/\$dbname *= *.*/\$dbname = 'xfusion_db';/" \
/tmp/index.php
# Upload the already-patched file — no remote editing needed
scp -i /root/.ssh/id_rsa /tmp/index.php root@$EC2_IP:/var/www/html/index.php
ssh -i /root/.ssh/id_rsa root@$EC2_IP "systemctl restart apache2"
```
If the file's variable names differ, don't fight sed — just rewrite the connection block directly. A known-good minimal version matching this file's actual structure:
```php
<?php
$dbname = 'xfusion_db';
$dbuser = 'xfusion_admin';
$dbpass = 'REPLACE_RDS_PASS';
$dbhost = 'REPLACE_RDS_ENDPOINT';
$link = mysqli_connect($dbhost, $dbuser, $dbpass) or die("Unable to Connect to '$dbhost'");
mysqli_select_db($link, $dbname) or die("Could not open the db '$dbname'");
echo "Connected successfully<br />\n";
?>
```
Keep whatever surrounding HTML the original had; only the connection params must be real.
## Verify
```bash
# RDS up, correct spec
aws rds describe-db-instances --db-instance-identifier xfusion-rds --region $REGION \
--query 'DBInstances[0].{Status:DBInstanceStatus,Ver:EngineVersion,Class:DBInstanceClass,Storage:StorageType,Size:AllocatedStorage,Public:PubliclyAccessible,DB:DBName}'
# The actual end-to-end proof — hit the page
curl -s "http://$EC2_IP/index.php"
```
Want the describe showing `available`, `8.4.5`, `db.t3.micro`, `gp2`, `5`, `Public: false`, `DB: xfusion_db` — and `curl` returning **`Connected successfully`**. That last string is the whole task's success signal: it means PHP on the EC2 reached the private RDS over 3306 through the SG you wired. Open `http://<EC2_IP>` in a browser for the same.
## Failure-mode debug order
If you *don't* see "Connected successfully":
1. **`Connection failed` with a timeout** → SG issue: confirm `RDS_SG` allows 3306 from `EC2_SG` and RDS actually has `RDS_SG` attached (`describe-db-instances ... VpcSecurityGroups`).
2. **`Connection failed` with access-denied** → wrong username/password in index.php, or you fat-fingered `$RDS_PASS`.
3. **Blank page / PHP source shown** → `php`/`php-mysqlnd` not installed or httpd not restarted after install.
4. **Can't reach the page at all** → port 80 not open on `EC2_SG`, or httpd not running.
Wired as above it resolves clean.

728
aws-36-x.md Normal file
View File

@@ -0,0 +1,728 @@
## Task 36
```bash
REGION=us-east-1
# 0. Default VPC, its default SG (goes on the ALB), two public subnets in 2 AZs
VPC_ID=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
DEFAULT_SG=$(aws ec2 describe-security-groups \
--filters "Name=group-name,Values=default" "Name=vpc-id,Values=$VPC_ID" \
--region $REGION --query 'SecurityGroups[0].GroupId' --output text)
read -r SUBNET1 SUBNET2 <<< $(aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=$VPC_ID" "Name=map-public-ip-on-launch,Values=true" \
--region $REGION --query 'Subnets[].SubnetId | [0:2]' --output text)
# 1. nautilus-sg (for the instance): allow 80 FROM the default SG (the ALB's SG)
NAUT_SG=$(aws ec2 create-security-group \
--group-name nautilus-sg \
--description "HTTP 80 from ALB default SG" \
--vpc-id $VPC_ID --region $REGION \
--query 'GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id $NAUT_SG --protocol tcp --port 80 \
--source-group $DEFAULT_SG --region $REGION
# 2. Make sure the default SG (on the ALB) allows 80 from the internet
aws ec2 authorize-security-group-ingress \
--group-id $DEFAULT_SG --protocol tcp --port 80 --cidr 0.0.0.0/0 \
--region $REGION 2>/dev/null || true
# 3. User-data: install + start Nginx (Ubuntu)
cat > /tmp/nginx-ud.sh << 'EOF'
#!/bin/bash
apt-get update -y
apt-get install -y nginx
systemctl enable nginx
systemctl start nginx
EOF
# 4. Launch nautilus-ec2 (Ubuntu 24.04 noble, amd64 for t2.micro) with nautilus-sg
IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/canonical/ubuntu/server/noble/stable/current/amd64/hvm/ebs-gp3/ami-id \
--instance-type t2.micro \
--security-group-ids $NAUT_SG \
--subnet-id $SUBNET1 \
--user-data file:///tmp/nginx-ud.sh \
--region $REGION \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=nautilus-ec2}]' \
--query 'Instances[0].InstanceId' --output text)
# aws ec2 wait instance-running --instance-ids $IID --region $REGION
STATE=""
until [ "$STATE" = "running" ]; do
STATE=$(aws ec2 describe-instances --instance-ids $IID --region $REGION \
--query 'Reservations[0].Instances[0].State.Name' --output text)
echo "$IID: $STATE"
[ "$STATE" = "running" ] || sleep 5
done
# 5. ALB across both subnets, wearing the default SG
ALB_ARN=$(aws elbv2 create-load-balancer \
--name nautilus-alb \
--subnets $SUBNET1 $SUBNET2 \
--security-groups $DEFAULT_SG \
--scheme internet-facing --type application \
--region $REGION \
--query 'LoadBalancers[0].LoadBalancerArn' --output text)
# 6. Target group (HTTP:80, instance targets, in the VPC)
TG_ARN=$(aws elbv2 create-target-group \
--name nautilus-tg \
--protocol HTTP --port 80 \
--vpc-id $VPC_ID --target-type instance \
--region $REGION \
--query 'TargetGroups[0].TargetGroupArn' --output text)
# 7. Register the instance
aws elbv2 register-targets \
--target-group-arn $TG_ARN --targets Id=$IID --region $REGION
# 8. Listener 80 -> forward to TG
aws elbv2 create-listener \
--load-balancer-arn $ALB_ARN \
--protocol HTTP --port 80 \
--default-actions Type=forward,TargetGroupArn=$TG_ARN \
--region $REGION
# 9. Wait for ALB to provision
# aws elbv2 wait load-balancer-available \
# --load-balancer-arns $ALB_ARN --region $REGION
STATE=""
until [ "$STATE" = "active" ]; do
STATE=$(aws elbv2 describe-load-balancers --load-balancer-arns $ALB_ARN --region $REGION \
--query 'LoadBalancers[0].State.Code' --output text)
echo "$ALB_ARN: $STATE"
[ "$STATE" = "active" ] || sleep 5
done
ALB_DNS=$(aws elbv2 describe-load-balancers --load-balancer-arns $ALB_ARN \
--region $REGION --query 'LoadBalancers[0].DNSName' --output text)
echo "ALB DNS: http://$ALB_DNS"
# Verify
# Poll target health until healthy
for n in $(seq 1 10); do
H=$(aws elbv2 describe-target-health --target-group-arn $TG_ARN --region $REGION \
--query 'TargetHealthDescriptions[0].TargetHealth.State' --output text)
echo "target: $H"
[ "$H" = "healthy" ] && break
sleep 20
done
# Hit the ALB DNS — should return Nginx's page
curl -I "http://$ALB_DNS"
```
## Task 37
Task:
1) EC2 Instance Setup:
An instance named devops-ec2 already exists.
The instance requires access to an S3 bucket.
2) Setup SSH Keys:
Create new SSH key pair (id_rsa and id_rsa.pub) on the aws-client host and add the public key to the root user's authorized keys on the EC2 instance.
3) Create a Private S3 Bucket:
Name the bucket devops-s3-046473746767.
Ensure the bucket is private.
4) Create an IAM Policy and Role:
Create an IAM policy allowing s3:PutObject, s3:ListBucket and s3:GetObject access to devops-s3-046473746767.
Create an IAM role named devops-role.
Attach the policy to the IAM role.
Attach this role to the devops-ec2 instance.
5) Test the Access:
SSH into the EC2 instance and try to upload a file to devops-s3-046473746767 bucket using following command:
aws s3 cp <your-file> s3://devops-s3-046473746767/
Now run following command to list the upload file:
aws s3 ls s3://devops-s3-046473746767/
### Solution:
# EC2 IAM Role → S3 Access Task
The IAM-role-to-EC2 task — the concept here is **instance profiles**. A role can't attach directly to an EC2 instance; it goes through an *instance profile*, a wrapper the CLI makes you create explicitly (the console hides this). And the S3 policy has a subtlety most people get wrong: **`ListBucket` acts on the bucket ARN, `GetObject`/`PutObject` act on the object ARN** (`/*`) — mixing those up makes the policy silently not work.
Phased, run on `aws-client`.
## Phase 1 — Private S3 bucket
```bash
REGION=us-east-1
BUCKET=devops-s3-046473746767
# us-east-1: NO LocationConstraint (special case)
aws s3api create-bucket --bucket $BUCKET --region $REGION
```
Private by default — since 2023 all new buckets ship with Block Public Access fully on and ACLs disabled. Nothing extra needed. (Any other region would require `--create-bucket-configuration LocationConstraint=<region>`; us-east-1 errors if you add it.)
## Phase 2 — IAM policy + role + instance profile
```bash
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
# Policy — note the TWO resource ARNs: bucket for ListBucket, /* for object ops
cat > /tmp/devops-s3-policy.json << EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:ListBucket"],
"Resource": "arn:aws:s3:::$BUCKET"
},
{
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:PutObject"],
"Resource": "arn:aws:s3:::$BUCKET/*"
}
]
}
EOF
POLICY_ARN=$(aws iam create-policy \
--policy-name devops-s3-policy \
--policy-document file:///tmp/devops-s3-policy.json \
--query 'Policy.Arn' --output text)
# Role with EC2 trust
cat > /tmp/ec2-trust.json << 'EOF'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}
EOF
aws iam create-role --role-name devops-role \
--assume-role-policy-document file:///tmp/ec2-trust.json
aws iam attach-role-policy --role-name devops-role --policy-arn $POLICY_ARN
# Instance profile — the wrapper that lets a role attach to EC2
aws iam create-instance-profile --instance-profile-name devops-role
aws iam add-role-to-instance-profile \
--instance-profile-name devops-role --role-name devops-role
```
The **ListBucket vs GetObject/PutObject ARN split** is the crux: `ListBucket` is a *bucket-level* action (it lists the bucket's contents), so its resource is the bucket ARN with no `/*`. `GetObject`/`PutObject` act on *objects inside* the bucket, so their resource needs the `/*` suffix. Put `ListBucket` on the `/*` ARN and your `aws s3 ls` fails with AccessDenied even though everything looks right. Two separate statements, two different ARNs.
## Phase 3 — Attach the instance profile to devops-ec2
```bash
IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=devops-ec2" "Name=instance-state-name,Values=running,stopped" \
--region $REGION --query 'Reservations[0].Instances[0].InstanceId' --output text)
# Give IAM a few seconds to propagate the profile, then associate
sleep 8
aws ec2 associate-iam-instance-profile \
--instance-id $IID \
--iam-instance-profile Name=devops-role \
--region $REGION
```
`associate-iam-instance-profile` attaches by the **instance-profile** name (`devops-role` here — profile named the same as the role, common convention). No reboot needed; the credentials appear via the instance metadata service (IMDS) within a minute or so. If the instance *already* has a profile attached, use `replace-iam-instance-profile-association` instead.
## Phase 4 — SSH key + passwordless root
```bash
[ -f /root/.ssh/id_rsa ] || ssh-keygen -t rsa -b 4096 -f /root/.ssh/id_rsa -N "" -q
PUBKEY=$(cat /root/.ssh/id_rsa.pub)
EC2_AZ=$(aws ec2 describe-instances --instance-ids $IID --region $REGION \
--query 'Reservations[0].Instances[0].Placement.AvailabilityZone' --output text)
EC2_IP=$(aws ec2 describe-instances --instance-ids $IID --region $REGION \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
# Bootstrap via EC2 Instance Connect (60s TTL), then persist into root's keys
# Ubuntu AMI -> default user is "ubuntu", not "ec2-user"
aws ec2-instance-connect send-ssh-public-key \
--instance-id $IID --instance-os-user ubuntu \
--ssh-public-key file:///root/.ssh/id_rsa.pub \
--availability-zone $EC2_AZ --region $REGION
ssh -o StrictHostKeyChecking=no -i /root/.ssh/id_rsa ubuntu@$EC2_IP "
set -e
sudo mkdir -p /root/.ssh && sudo chmod 700 /root/.ssh
echo '$PUBKEY' | sudo tee -a /root/.ssh/authorized_keys >/dev/null
sudo chmod 600 /root/.ssh/authorized_keys
echo 'PermitRootLogin prohibit-password' | sudo tee /etc/ssh/sshd_config.d/99-root.conf >/dev/null
sudo systemctl restart sshd
"
```
(If port 22 isn't open, this SSH leg won't work — use SSM `send-command` to plant the key instead, or open 22 to aws-client's `/32`.)
## Phase 5 — Test the S3 access from the instance
```bash
ssh -i /root/.ssh/id_rsa root@$EC2_IP "
echo 'hello from devops-ec2' > /tmp/testfile.txt
aws s3 cp /tmp/testfile.txt s3://$BUCKET/
aws s3 ls s3://$BUCKET/
"
```
Both commands should succeed — the `cp` uploads (exercises `PutObject`), the `ls` lists (exercises `ListBucket`). No `aws configure` needed on the instance: the CLI picks up temporary credentials automatically from IMDS via the attached role. That's the whole point of the role — **no static keys on the box.**
## Verify
```bash
# Role attached to the instance?
aws ec2 describe-instances --instance-ids $IID --region $REGION \
--query 'Reservations[0].Instances[0].IamInstanceProfile.Arn'
# File actually landed?
aws s3 ls s3://$BUCKET/ --region $REGION
```
Want the instance showing an `IamInstanceProfile.Arn` ending in `devops-role`, and `testfile.txt` in the bucket listing. The successful `cp`+`ls` from *inside* the instance is the task's proof — it confirms the role → profile → instance chain delivered S3 permissions via IMDS.
## Debug order if the S3 commands fail from the instance
1. **`Unable to locate credentials`** → profile not attached yet (IMDS propagation) or association failed. Re-check Phase 3's verify; wait a minute.
2. **`AccessDenied` on `ls` but `cp` works (or vice versa)** → the ListBucket/object ARN split is wrong. `ls` needs `ListBucket` on the bare bucket ARN; `cp` needs `PutObject` on `/*`.
3. **`AccessDenied` on both** → policy didn't attach to the role, or wrong bucket name in the ARNs.
Wired as above it works clean.
## Task 38
The Nautilus DevOps team is tasked with deploying a containerized application using Amazon's container services. They need to create a private Amazon Elastic Container Registry (ECR) to store their Docker images and use Amazon Elastic Container Service (ECS) to deploy the application. The process involves building a Docker image from a given Dockerfile, pushing it to the ECR, and then setting up an ECS cluster to run the application.
Create a Private ECR Repository:
Create a private ECR repository named datacenter-ecr to store Docker images.
Build and Push Docker Image:
Use the Dockerfile located at /root/pyapp on the aws-client host.
Build a Docker image using this Dockerfile.
Tag the image with latest tag.
Push the Docker image to the datacenter-ecr repository.
Create and Configure ECS cluster:
Create an ECS cluster named datacenter-cluster using the Fargate launch type.
Create an ECS Task Definition:
Define a task named datacenter-taskdefinition using the Docker image from the datacenter-ecr ECR repository.
Specify necessary CPU and memory resources.
Deploy the Application Using ECS Service:
Create a service named datacenter-service on the datacenter-cluster to run the task.
Ensure the service runs at least one task.
### Solution
# ECS on Fargate Deployment Task (datacenter)
The full ECS-on-Fargate pipeline — the most involved task in the set because it chains ECR → cluster → task def → service, and Fargate has hard requirements the others don't: **an execution role**, **awsvpc networking with explicit subnets/SG**, and **CPU/memory from a fixed valid-combinations table** (you can't pick arbitrary values).
## Phase 1 — ECR repo + build + push
On `aws-client`:
```bash
REGION=us-east-1
aws ecr create-repository --repository-name datacenter-ecr --region $REGION
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
REGISTRY="${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com"
REPO_URI="${REGISTRY}/datacenter-ecr"
# Auth docker to ECR (12h token)
aws ecr get-login-password --region $REGION \
| docker login --username AWS --password-stdin "$REGISTRY"
# Build, tag latest, push
docker build -t datacenter-ecr:latest /root/pyapp
docker tag datacenter-ecr:latest "${REPO_URI}:latest"
docker push "${REPO_URI}:latest"
```
## Phase 2 — ECS cluster (Fargate)
```bash
aws ecs create-cluster \
--cluster-name datacenter-cluster \
--capacity-providers FARGATE \
--region $REGION
```
`--capacity-providers FARGATE` registers the cluster for the Fargate (serverless) launch type — no EC2 container instances to manage. Satisfies "Fargate launch type" at the cluster level; also specified per-task later.
## Phase 3 — Execution role (the Fargate prerequisite people miss)
Fargate needs a **task execution role** — the role ECS itself assumes to pull the image from ECR and write logs to CloudWatch. Without it, the task fails at launch with a pull error. AWS has a canonical managed policy for exactly this:
```bash
cat > /tmp/ecs-trust.json << 'EOF'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"ecs-tasks.amazonaws.com"},"Action":"sts:AssumeRole"}]}
EOF
aws iam create-role --role-name ecsTaskExecutionRole \
--assume-role-policy-document file:///tmp/ecs-trust.json 2>/dev/null || true
aws iam attach-role-policy --role-name ecsTaskExecutionRole \
--policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
EXEC_ROLE_ARN=$(aws iam get-role --role-name ecsTaskExecutionRole \
--query 'Role.Arn' --output text)
```
The trust principal is **`ecs-tasks.amazonaws.com`** (not `ec2` or `ecs`). `AmazonECSTaskExecutionRolePolicy` grants exactly ECR-pull + CloudWatch-logs-write — the two things ECS needs to stand up your container. This role often already exists in an account (`|| true` handles that).
## Phase 4 — Task definition
```bash
cat > /tmp/datacenter-taskdef.json << EOF
{
"family": "datacenter-taskdefinition",
"networkMode": "awsvpc",
"requiresCompatibilities": ["FARGATE"],
"cpu": "256",
"memory": "512",
"executionRoleArn": "$EXEC_ROLE_ARN",
"containerDefinitions": [
{
"name": "datacenter-container",
"image": "${REPO_URI}:latest",
"essential": true,
"portMappings": [
{ "containerPort": 80, "protocol": "tcp" }
]
}
]
}
EOF
TASKDEF_ARN=$(aws ecs register-task-definition \
--cli-input-json file:///tmp/datacenter-taskdef.json \
--region $REGION \
--query 'taskDefinition.taskDefinitionArn' --output text)
```
The Fargate-specific requirements baked in here:
- **`networkMode: awsvpc`** is *mandatory* for Fargate — every task gets its own ENI. `bridge`/`host` modes are EC2-launch-type only. Skip this and registration fails.
- **`cpu`/`memory` must be a valid pair from Fargate's table** — not arbitrary. `256` CPU (.25 vCPU) pairs only with `512`/`1024`/`2048` MiB memory. Other valid CPU values: 512, 1024, 2048, 4096. `256/512` is the smallest, cheapest, and fine for "specify necessary CPU and memory." Pick a mismatched pair (e.g. cpu 256 / memory 4096) and it rejects.
- **`executionRoleArn`** — the role from Phase 3. Required for Fargate to pull from ECR.
- **Values are strings** (`"256"`, not `256`) in the JSON — ECS is picky about that.
- **`image` points at the ECR URI** with `:latest` — the image you just pushed.
## Phase 5 — ECS service (needs subnets + SG for awsvpc)
Because it's `awsvpc`, the service launch requires an explicit network config — subnets and a security group:
```bash
VPC_ID=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
read -r SUBNET1 SUBNET2 <<< $(aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=$VPC_ID" "Name=map-public-ip-on-launch,Values=true" \
--region $REGION --query 'Subnets[].SubnetId | [0:2]' --output text)
# SG for the task (allow inbound 80 if the app serves HTTP)
TASK_SG=$(aws ec2 create-security-group \
--group-name datacenter-task-sg \
--description "ECS task SG" --vpc-id $VPC_ID --region $REGION \
--query 'GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id $TASK_SG --protocol tcp --port 80 --cidr 0.0.0.0/0 --region $REGION
# Create the service — desired count 1
aws ecs create-service \
--cluster datacenter-cluster \
--service-name datacenter-service \
--task-definition datacenter-taskdefinition \
--desired-count 1 \
--launch-type FARGATE \
--network-configuration "awsvpcConfiguration={subnets=[$SUBNET1,$SUBNET2],securityGroups=[$TASK_SG],assignPublicIp=ENABLED}" \
--region $REGION
```
The service-level essentials:
- **`--desired-count 1`** = "runs at least one task" (the requirement). ECS keeps one task running, restarting it if it dies.
- **`--network-configuration ... awsvpcConfiguration`** is required *because* of awsvpc mode — the task's ENI needs to land in specific subnets with a specific SG. Omit it and service creation errors.
- **`assignPublicIp=ENABLED`** is critical on Fargate in **public subnets**: without a public IP, the task can't reach ECR/internet to pull the image (unless you have a NAT gateway), and it'll fail to launch. In public subnets this is the simplest working config. (Private-subnet Fargate would use a NAT gateway instead and set this DISABLED.)
- **`--launch-type FARGATE`** — reaffirms Fargate at the service level.
## Verify
```bash
# Image pushed
aws ecr describe-images --repository-name datacenter-ecr --region $REGION \
--query 'imageDetails[].imageTags'
# Cluster active
aws ecs describe-clusters --clusters datacenter-cluster --region $REGION \
--query 'clusters[0].{Name:clusterName,Status:status}'
# Service + running task count (poll until runningCount=1)
for n in $(seq 1 12); do
RC=$(aws ecs describe-services --cluster datacenter-cluster \
--services datacenter-service --region $REGION \
--query 'services[0].runningCount' --output text)
echo "runningCount: $RC"
[ "$RC" = "1" ] && break
sleep 15
done
```
Want: ECR showing `["latest"]`, cluster `ACTIVE`, and the service's `runningCount` reaching `1`. That last one is the task's success signal — a task actually running means the whole chain (image pull via exec role → Fargate ENI in the subnet → container start) worked.
## Debug order if `runningCount` stays `0`
1. **`aws ecs describe-services ... --query 'services[0].events[0].message'`** — ECS posts the failure reason here in plain English. Read it first.
2. **`describe-tasks`** on the stopped task → `stoppedReason`. Common ones: `CannotPullContainerError` (public IP missing, or exec role can't reach ECR — check `assignPublicIp=ENABLED` and the exec role), `ResourceInitializationError` (SG blocking egress, or subnet has no internet path).
3. **Task def cpu/memory rejected at registration** → invalid Fargate pair; fix to `256/512`.
The two things that trip everyone on first Fargate deploy: forgetting the **execution role** (image pull fails) and forgetting **`assignPublicIp=ENABLED`** in a public subnet (image pull also fails, different error). Both covered above.
## Task 39
The Nautilus DevOps team has been tasked with creating an internal information portal for public access. As part of this project, they need to host a static website on AWS using an S3 bucket. The S3 bucket must be configured for public access to allow external users to access the static website directly via the S3 website URL.
Task Requirements:
Create an S3 bucket named nautilus-web-205059040.
Configure the S3 bucket for static website hosting with index.html as the index document.
Allow public access to the bucket so that the website is publicly accessible.
Upload the index.html file from the /root/ directory of the AWS client host to the S3 bucket.
Verify that the website is accessible directly through the S3 website URL.
### Solution
# S3 Static Website Hosting Task (nautilus-web)
S3 static website hosting — this one fights the modern defaults hard. Since 2023 every new bucket ships with **Block Public Access fully ON and ACLs disabled**, but static website hosting requires the *opposite*: public reads via a bucket policy. So the real work is deliberately *undoing* the safe defaults — disable BPA, then attach a public-read policy. That's the crux; the hosting config itself is trivial.
Run on `aws-client`:
```bash
REGION=us-east-1
BUCKET=nautilus-web-205059040
# 1. Create the bucket (us-east-1: no LocationConstraint)
aws s3api create-bucket --bucket $BUCKET --region $REGION
# 2. Disable Block Public Access (the safe default that blocks website hosting)
aws s3api put-public-access-block \
--bucket $BUCKET \
--public-access-block-configuration \
"BlockPublicAcls=false,IgnorePublicAcls=false,BlockPublicPolicy=false,RestrictPublicBuckets=false" \
--region $REGION
# 3. Attach a public-read bucket policy (this is what makes objects world-readable)
cat > /tmp/web-policy.json << EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::$BUCKET/*"
}
]
}
EOF
aws s3api put-bucket-policy \
--bucket $BUCKET \
--policy file:///tmp/web-policy.json \
--region $REGION
# 4. Enable static website hosting with index.html
aws s3 website s3://$BUCKET/ --index-document index.html
# 5. Upload index.html from /root
aws s3 cp /root/index.html s3://$BUCKET/index.html --region $REGION
```
## Why each step is shaped this way
- **Step 2 must come before step 3.** `BlockPublicPolicy=true` (the default) actively *rejects* any attempt to attach a public bucket policy — you'd get AccessDenied on the `put-bucket-policy` call. So you disable BPA first, *then* the policy takes. All four flags to `false` fully opens it; `BlockPublicPolicy` and `RestrictPublicBuckets` are the two that specifically gate policies, but for website hosting you clear all four.
- **The bucket policy — not ACLs — is what grants public read.** The old way was `--acl public-read`, but ACLs are disabled by default now (bucket-owner-enforced) and AWS steers you away from them. The `s3:GetObject` policy with `Principal: "*"` on `arn:.../*` (object-level, note the `/*`) is the current correct mechanism. `GetObject` on `/*` = anyone can read any object = the website serves.
- **`aws s3 website` is the high-level shortcut** for enabling hosting. Equivalent to `s3api put-bucket-website` with a config block, but one line. `--index-document index.html` is the required index doc; you could add `--error-document error.html` but the task doesn't ask.
- **`create-bucket` in us-east-1** takes no `LocationConstraint` — the special-region rule again.
## The website URL — a specific endpoint format
This is NOT the regular bucket URL. Two different hostnames:
- REST/object URL: `https://nautilus-web-205059040.s3.amazonaws.com/index.html`
- **Website endpoint** (what the task wants): `http://nautilus-web-205059040.s3-website-us-east-1.amazonaws.com`
The website endpoint uses `s3-website-<region>` in the hostname and is **HTTP only** (no HTTPS on native S3 website hosting — you'd need CloudFront for TLS). That's the URL that serves `index.html` at the root.
## Verify
```bash
# Confirm hosting config
aws s3api get-bucket-website --bucket $BUCKET --region $REGION
# Confirm BPA is off
aws s3api get-public-access-block --bucket $BUCKET --region $REGION \
--query 'PublicAccessBlockConfiguration'
# Build the website URL and hit it
WEB_URL="http://${BUCKET}.s3-website-${REGION}.amazonaws.com"
echo "$WEB_URL"
curl -I "$WEB_URL"
```
Want the website config showing `index.html`, all four BPA flags `false`, and `curl -I` returning `HTTP/1.1 200 OK`. The 200 at the `s3-website-` URL is the task's success proof — it means public read + hosting + the uploaded index all line up.
## Debug order if it doesn't serve
1. **`403 Forbidden`** → either BPA still blocking (re-check step 2 actually applied) or the bucket policy didn't attach. These are the two public-access gates; one is still closed.
2. **`404 Not Found`** → `index.html` didn't upload, or hosting isn't enabled. Check `aws s3 ls s3://$BUCKET/` shows the file and `get-bucket-website` returns a config.
3. **Wrong URL entirely** → you hit the `.s3.amazonaws.com` REST endpoint instead of `.s3-website-<region>.amazonaws.com`. Only the website endpoint renders `index.html` at root.
Wired in this order it serves clean.
## Task 40
The Nautilus Development Team recently deployed a new web application hosted on an EC2 instance within a public VPC named nautilus-vpc. The application, running on an Nginx server, should be accessible from the internet on port 80. Despite configuring the security group nautilus-sg to allow traffic on port 80 and verifying the EC2 instance settings, the application remains inaccessible from the internet. The team suspects that the issue might be related to the VPC configuration, as all other components appear to be set up correctly. The DevOps team has been asked to troubleshoot and resolve the issue to ensure the application is accessible to external users.
As a member of the Nautilus DevOps Team, your task is to perform the following:
Verify VPC Configuration: Ensure that the VPC nautilus-vpc is properly configured to allow internet access.
Ensure Accessibility: Make sure the EC2 instance nautilus-ec2 running the Nginx server is accessible from the internet on port 80.
### Solution
# VPC Internet Access Troubleshooting Task (nautilus-vpc / nautilus-ec2)
A **troubleshooting** task, not a build-from-scratch — so the approach is diagnose-then-fix, not blindly recreate. The scenario is textbook: SG allows 80, instance is fine, "suspect the VPC." The near-certain culprit in a public-VPC-that-doesn't-work is one of the **route/IGW** links being broken — either no IGW attached, or the subnet's route table lacks the `0.0.0.0/0 → IGW` route, or the instance has no public IP.
Run on `aws-client`.
## Diagnose — gather the full picture first
```bash
REGION=us-east-1
VPC_ID=$(aws ec2 describe-vpcs --filters Name=tag:Name,Values=nautilus-vpc \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
VPC_CIDR=$(aws ec2 describe-vpcs --vpc-ids $VPC_ID --region $REGION \
--query 'Vpcs[0].CidrBlock' --output text)
IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=nautilus-ec2" "Name=instance-state-name,Values=running,stopped" \
--region $REGION --query 'Reservations[0].Instances[0].InstanceId' --output text)
SUBNET_ID=$(aws ec2 describe-instances --instance-ids $IID --region $REGION \
--query 'Reservations[0].Instances[0].SubnetId' --output text)
PUB_IP=$(aws ec2 describe-instances --instance-ids $IID --region $REGION \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
echo "VPC=$VPC_ID CIDR=$VPC_CIDR instance=$IID subnet=$SUBNET_ID publicIP=$PUB_IP"
# ── DIAGNOSE #1: Is an IGW attached to this VPC? ──
IGW_ID=$(aws ec2 describe-internet-gateways \
--filters "Name=attachment.vpc-id,Values=$VPC_ID" \
--region $REGION --query 'InternetGateways[0].InternetGatewayId' --output text)
echo "IGW: $IGW_ID"
# ── DIAGNOSE #2: What route table serves the instance's subnet, and does it route to the IGW? ──
RTB_ID=$(aws ec2 describe-route-tables \
--filters "Name=association.subnet-id,Values=$SUBNET_ID" \
--region $REGION --query 'RouteTables[0].RouteTableId' --output text)
# if no explicit association, it's using the main RTB:
if [ "$RTB_ID" = "None" ] || [ -z "$RTB_ID" ]; then
RTB_ID=$(aws ec2 describe-route-tables \
--filters "Name=vpc-id,Values=$VPC_ID" "Name=association.main,Values=true" \
--region $REGION --query 'RouteTables[0].RouteTableId' --output text)
fi
echo "Subnet's RTB: $RTB_ID"
aws ec2 describe-route-tables --route-table-ids $RTB_ID --region $REGION \
--query 'RouteTables[0].Routes'
```
That output tells you exactly which link is broken.
## Fix — each possible fault (idempotent, only creates what's missing)
```bash
# ── FIX #1: No IGW → create + attach one ──
if [ "$IGW_ID" = "None" ] || [ -z "$IGW_ID" ]; then
IGW_ID=$(aws ec2 create-internet-gateway --region $REGION \
--tag-specifications 'ResourceType=internet-gateway,Tags=[{Key=Name,Value=nautilus-igw}]' \
--query 'InternetGateway.InternetGatewayId' --output text)
aws ec2 attach-internet-gateway --internet-gateway-id $IGW_ID --vpc-id $VPC_ID --region $REGION
echo "Created + attached IGW: $IGW_ID"
fi
# ── FIX #2: Missing default route → add 0.0.0.0/0 -> IGW to the subnet's RTB ──
# (create-route errors if it exists; replace-route as fallback)
aws ec2 create-route --route-table-id $RTB_ID \
--destination-cidr-block 0.0.0.0/0 --gateway-id $IGW_ID --region $REGION 2>/dev/null || \
aws ec2 replace-route --route-table-id $RTB_ID \
--destination-cidr-block 0.0.0.0/0 --gateway-id $IGW_ID --region $REGION
# ── FIX #3: Subnet doesn't auto-assign public IPs (for future launches) ──
aws ec2 modify-subnet-attribute --subnet-id $SUBNET_ID \
--map-public-ip-on-launch --region $REGION
# ── FIX #4: The RUNNING instance has NO public IP ──
# map-public-ip-on-launch only affects NEW launches — an already-running instance
# without a public IP needs an Elastic IP associated.
if [ "$PUB_IP" = "None" ] || [ -z "$PUB_IP" ]; then
ALLOC_ID=$(aws ec2 allocate-address --domain vpc --region $REGION \
--query 'AllocationId' --output text)
aws ec2 associate-address --instance-id $IID --allocation-id $ALLOC_ID --region $REGION
PUB_IP=$(aws ec2 describe-instances --instance-ids $IID --region $REGION \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
echo "Associated EIP: $PUB_IP"
fi
```
## The reasoning behind each fault
- **The #1 cause of "public VPC, SG's fine, still unreachable" is the IGW+route.** SG allowing 80 is necessary but not sufficient — it governs the instance's firewall, but if the *subnet* has no route to an IGW, packets from the internet never reach the instance in the first place. The SG check passing is exactly why the team is confused: they verified the wrong layer.
- **`map-public-ip-on-launch` is retroactively useless.** Setting it fixes *future* launches but does nothing for the already-running `nautilus-ec2`. If the running instance lacks a public IP, the only fix is associating an **Elastic IP** — that's FIX #4, and it's the subtle one people miss after correctly fixing the routing.
- **Explicit vs main route table matters.** If the subnet has no explicit RTB association, it silently uses the VPC's *main* route table — and if someone added the IGW route to a *custom* RTB that the subnet isn't associated with, it looks configured but isn't. The diagnose step resolves which RTB actually serves the subnet, so you fix the right one.
- **Also worth a glance: the NACL.** Security groups are stateful, but **network ACLs** are subnet-level, stateless, and could block 80 or the ephemeral return ports. Default NACLs allow all, so it's rarely the culprit, but if everything above checks out and it's still dead, that's the next place to look:
```bash
aws ec2 describe-network-acls \
--filters "Name=association.subnet-id,Values=$SUBNET_ID" \
--region $REGION --query 'NetworkAcls[0].Entries'
```
One thing to also confirm — that **Nginx is actually running** on the box (the task says it is, but "verify EC2 settings" is part of it). If you have SSH: `systemctl status nginx`. If not, the curl below tells you regardless.
## Verify the fix end to end
```bash
# Full chain re-check
aws ec2 describe-route-tables --route-table-ids $RTB_ID --region $REGION \
--query 'RouteTables[0].Routes[?DestinationCidrBlock==`0.0.0.0/0`]'
echo "Testing http://$PUB_IP ..."
curl -I --max-time 10 "http://$PUB_IP"
```
Want the route table showing `0.0.0.0/0` with a `GatewayId` of `igw-...` and `State: active`, and `curl -I` returning `HTTP/1.1 200 OK` with an `nginx` Server header. That 200 from the public IP is the resolution — external reachability on port 80 restored.
## Most likely single fix
In order of probability for this scenario: **missing `0.0.0.0/0 → IGW` route** (or the IGW not attached at all) is the classic "public VPC that isn't actually public." The EIP fix (#4) is the second-most-common gotcha. Run the diagnose block first to see which it actually is rather than applying all fixes blind — though the fix block is safe to run wholesale since each piece is guarded.

189
aws-41-50.md Normal file
View File

@@ -0,0 +1,189 @@
## Task 41
The Nautilus DevOps team is focusing on improving their data security by using AWS KMS. Your task is to create a KMS key and manage the encryption and decryption of a pre-existing sensitive file using the KMS key.
Specific Requirements:
Create a symmetric KMS key named nautilus-KMS-Key to manage encryption and decryption.
Encrypt the provided SensitiveData.txt file (located in /root/), base64 decode the ciphertext, and save the encrypted version as EncryptedData.bin in the /root/ directory.
Try to decrypt the same and verify that the decrypted data matches the original file.
Make sure that the KMS key is correctly configured. The validation script will test your configuration by decrypting the EncryptedData.bin file using the KMS key you created.
### Solution
# AWS KMS Encryption/Decryption Task (nautilus-KMS-Key)
The concept that matters: **KMS naming works differently.** A KMS key has no "name" attribute at all — what you set is an **alias** (`alias/nautilus-KMS-Key`), a friendly pointer to the key's real ID. And there's a hard **4KB size limit** on direct `kms encrypt`, which is why this task is fine for a small file but you'd need envelope encryption for anything bigger. The base64-decode step exists because the CLI returns ciphertext base64-encoded in JSON — you decode it back to raw bytes for the `.bin`.
Run on `aws-client`:
```bash
REGION=us-east-1
# 1. Create the symmetric KMS key (symmetric + ENCRYPT_DECRYPT are the defaults)
KEY_ID=$(aws kms create-key \
--description "nautilus KMS key for sensitive data" \
--key-usage ENCRYPT_DECRYPT \
--key-spec SYMMETRIC_DEFAULT \
--region $REGION \
--query 'KeyMetadata.KeyId' --output text)
# Give it the "name" — which in KMS means an alias
aws kms create-alias \
--alias-name alias/nautilus-KMS-Key \
--target-key-id $KEY_ID \
--region $REGION
# 2. Encrypt SensitiveData.txt -> base64-decode -> EncryptedData.bin
aws kms encrypt \
--key-id alias/nautilus-KMS-Key \
--plaintext fileb:///root/SensitiveData.txt \
--output text \
--query CiphertextBlob \
--region $REGION \
| base64 --decode > /root/EncryptedData.bin
# 3. Decrypt EncryptedData.bin and compare to the original
aws kms decrypt \
--ciphertext-blob fileb:///root/EncryptedData.bin \
--output text \
--query Plaintext \
--region $REGION \
| base64 --decode > /root/DecryptedData.txt
# Verify round-trip integrity
diff /root/SensitiveData.txt /root/DecryptedData.txt && echo "MATCH ✔" || echo "MISMATCH <20>’"
```
## Every non-obvious piece explained
- **Alias, not name.** `create-key` returns only a `KeyId` (a UUID) and ARN — no name field exists. `create-alias` with the `alias/` prefix (mandatory prefix) is how you give it the human label `nautilus-KMS-Key`. Everywhere else you can then reference `alias/nautilus-KMS-Key` instead of the UUID. The validation script decrypting "using the KMS key you created" works because the ciphertext blob has the key ID embedded in it — decrypt doesn't even need `--key-id`.
- **`SYMMETRIC_DEFAULT` + `ENCRYPT_DECRYPT`** are both the defaults, so you could omit those flags — set explicitly since the task says "symmetric." Symmetric = same key encrypts and decrypts, AES-256-GCM under the hood, key material never leaves KMS.
- **`fileb://` not `file://`** for `--plaintext` — the `b` = binary. This reads the file's raw bytes. `file://` would try to interpret it as text/UTF-8 and can mangle binary or trailing content. Critical for a clean round-trip.
- **The base64 dance.** `kms encrypt` returns `CiphertextBlob` as base64 text inside JSON. `--query CiphertextBlob --output text` extracts just that base64 string; `base64 --decode` converts it to the raw binary ciphertext that becomes `EncryptedData.bin`. This is exactly the task's "base64 decode the ciphertext" step — the `.bin` must be raw bytes, not base64 text, or the validation script's decrypt fails.
- **Decrypt needs no `--key-id`.** This surprises people: `kms decrypt` for a symmetric key figures out which key to use because the **key ID is baked into the ciphertext blob itself**. You just hand it the blob. (Asymmetric keys *would* require `--key-id`.) Same base64-decode on the way out to recover the original plaintext bytes.
- **`fileb://` on decrypt's `--ciphertext-blob`** too — reading the raw `.bin` bytes.
## Verify the key + alias config (what the validation script cares about)
```bash
# Alias points at an enabled key
aws kms list-aliases --region $REGION \
--query "Aliases[?AliasName=='alias/nautilus-KMS-Key']"
aws kms describe-key --key-id alias/nautilus-KMS-Key --region $REGION \
--query 'KeyMetadata.{Id:KeyId,State:KeyState,Usage:KeyUsage,Spec:KeySpec,Enabled:Enabled}'
# The round-trip already proved it, but confirm the .bin exists and is binary
file /root/EncryptedData.bin
ls -l /root/EncryptedData.bin
```
Want: the alias resolving to your key; `describe-key` showing `State: Enabled`, `Usage: ENCRYPT_DECRYPT`, `Spec: SYMMETRIC_DEFAULT`; the `diff` printing `MATCH`; and `EncryptedData.bin` existing as binary data. The successful decrypt + matching diff is the task's proof — it means the key encrypts and decrypts correctly and the `.bin` is in the raw format the validator expects.
## Debug order if something's off
1. **`diff` shows MISMATCH** → almost always a `file://` vs `fileb://` mix-up mangling bytes, or you skipped a `base64 --decode`. Both the write and read paths need the binary treatment.
2. **Validation script can't decrypt the `.bin`** → the `.bin` is probably still base64-encoded text instead of raw bytes (missing the `base64 --decode` on encrypt). `file /root/EncryptedData.bin` should say "data," not "ASCII text."
3. **`AccessDenied` on encrypt/decrypt** → your CLI identity lacks kms permissions, or (rare) the key policy is restrictive — a freshly created key grants the creating account root full access by default, so this is unusual in a lab.
## Task 42
The Nautilus DevOps team is developing a simple 'To-Do' application using DynamoDB to store and manage tasks efficiently. The team needs to create a DynamoDB table to hold tasks, each identified by a unique task ID. Each task will have a description and a status, which indicates the progress of the task (e.g., 'completed' or 'in-progress').
Your task is to:
Create a DynamoDB table named nautilus-tasks with a primary key called taskId (string).
Insert the following tasks into the table:
Task 1: taskId: '1', description: 'Learn DynamoDB', status: 'completed'
Task 2: taskId: '2', description: 'Build To-Do App', status: 'in-progress'
Verify that Task 1 has a status of 'completed' and Task 2 has a status of 'in-progress'.
Ensure the DynamoDB table is created successfully and that both tasks are inserted correctly with the appropriate statuses.
### Solution
# DynamoDB To-Do Table Task (nautilus-tasks)
DynamoDB — the concepts that matter here: DynamoDB is **schemaless except for the key**. You only declare `taskId` at table creation; `description` and `status` are *not* defined up front — they're just attributes you attach per-item at insert time. And every value carries a **type descriptor** (`S` for string, `N` for number, etc.) in the API, which is why the item JSON looks verbose. Also **on-demand billing** (`PAY_PER_REQUEST`) is the right pick for a lab — no capacity planning, no idle cost.
Run on `aws-client`:
## 1. Create the table
```bash
REGION=us-east-1
aws dynamodb create-table \
--table-name nautilus-tasks \
--attribute-definitions AttributeName=taskId,AttributeType=S \
--key-schema AttributeName=taskId,KeyType=HASH \
--billing-mode PAY_PER_REQUEST \
--region $REGION
# Table creation is async — wait until ACTIVE
aws dynamodb wait table-exists --table-name nautilus-tasks --region $REGION
```
## 2. Insert the two tasks
```bash
aws dynamodb put-item \
--table-name nautilus-tasks \
--item '{
"taskId": {"S": "1"},
"description": {"S": "Learn DynamoDB"},
"status": {"S": "completed"}
}' \
--region $REGION
aws dynamodb put-item \
--table-name nautilus-tasks \
--item '{
"taskId": {"S": "2"},
"description": {"S": "Build To-Do App"},
"status": {"S": "in-progress"}
}' \
--region $REGION
```
## Every non-obvious piece explained
- **Only the key attribute is declared at creation.** `--attribute-definitions` lists *only* `taskId` — you do NOT declare `description` or `status`. DynamoDB is schemaless beyond the primary key; non-key attributes materialize when you write items. Declaring extra attributes here would actually error unless they're part of a key or index. This trips people coming from relational DBs.
- **`KeyType=HASH` = partition key.** `taskId` as `HASH` makes it the partition (primary) key — the single-attribute simple primary key the task wants. (A `RANGE` key would add a sort key for a composite key; not needed here.)
- **The `{"S": "..."}` type descriptors are mandatory** in the low-level API. Every attribute value is wrapped with its type: `S` string, `N` number, `BOOL`, `M` map, `L` list, etc. Even though `taskId` "looks" numeric (`"1"`), the task says it's a *string*, so `S` — and the values are quoted strings. Get this wrong (e.g. `N` for taskId) and it mismatches the key schema.
- **`PAY_PER_REQUEST`** = on-demand mode: no read/write capacity units to provision, you pay per request. For a lab with 2 items this is free-tier-friendly and zero-config. The alternative `PROVISIONED` mode needs `--provisioned-throughput` numbers.
- **`wait table-exists`** — table creation goes `CREATING → ACTIVE` asynchronously. You can't `put-item` until it's `ACTIVE`, so the waiter prevents a race where inserts fire against a still-creating table.
## 3. Verify both items and their statuses
```bash
# Fetch each item by key, checking the status
aws dynamodb get-item \
--table-name nautilus-tasks \
--key '{"taskId": {"S": "1"}}' \
--region $REGION \
--query 'Item.status.S' # expect: "completed"
aws dynamodb get-item \
--table-name nautilus-tasks \
--key '{"taskId": {"S": "2"}}' \
--region $REGION \
--query 'Item.status.S' # expect: "in-progress"
# Or dump both items at once to eyeball everything
aws dynamodb scan --table-name nautilus-tasks --region $REGION \
--query 'Items[].{ID:taskId.S, Desc:description.S, Status:status.S}' \
--output table
```
Want the two `get-item` calls returning `"completed"` and `"in-progress"` respectively, and the scan table showing both rows with correct descriptions and statuses. That confirms the task: table `ACTIVE`, both items inserted, statuses correct.
**`get-item` vs `scan`:** `get-item` is a direct key lookup (fast, cheap, single item) — the right way to verify a *specific* task by its `taskId`. `scan` reads the whole table (fine for 2 items, but avoid on large tables — it's a full sweep). Used both here: get-item for the targeted status checks, scan for the at-a-glance dump.
## Debug order if verification fails
1. **`get-item` returns nothing / null** → wrong key type or value. The key in `get-item` must match exactly: `{"taskId": {"S": "1"}}` — `S` type, string `"1"`. An `N` type or unquoted value won't match.
2. **`put-item` errored on insert** → usually a JSON quoting issue in the `--item` blob, or a type mismatch against the key schema (taskId declared `S` but sent as `N`).
3. **`create-table` errored** → likely declared a non-key attribute in `--attribute-definitions`. Only `taskId` belongs there.

112
aws-43.md Normal file
View File

@@ -0,0 +1,112 @@
## Task 43
The Nautilus DevOps team has been tasked with preparing the infrastructure for a new Kubernetes-based application that will be deployed using Amazon EKS. The team is in the process of setting up an EKS cluster that meets their internal security and scalability standards. They require that the cluster be provisioned using the latest stable Kubernetes version to take advantage of new features and security improvements.
To minimize external exposure, the EKS cluster endpoint must be kept private. Additionally, the cluster needs to use the default VPC with availability zones a, b, and c to ensure high availability across different physical locations.
Your task is to create an EKS cluster named nautilus-eks, with Custom configuration, use IAM role for the cluster named eksClusterRole. Additionally, ensure that EKS Auto Mode is disabled and that the cluster endpoint access is set to private.
Finally, verify that the EKS cluster is successfully created with the correct configuration and is ready for workloads.
### Solution
# EKS Private Cluster Task (nautilus-eks)
Version pinned: **latest EKS Kubernetes version is 1.36** (`1.36-eks-3` platform version, released June 2, 2026; EKS actively supports 1.36, 1.35, 1.34, 1.33). So `--kubernetes-version 1.36` — swap to 1.35 if the grader's env lags, but 1.36 is "latest stable."
The concepts that matter for EKS: the cluster control plane needs an **IAM role it assumes** (trust principal `eks.amazonaws.com`, managed policy `AmazonEKSClusterPolicy`); the control plane spans **subnets in the AZs you pass** (task wants a/b/c for HA); and **"EKS Auto Mode disabled"** is an explicit flag now (Auto Mode is the newer AWS-manages-compute option — disabling it means classic control-plane-only). "Private endpoint" flips the API server's reachability so it's only reachable from inside the VPC.
Run on `aws-client`.
## Phase 1 — Cluster IAM role
```bash
REGION=us-east-1
cat > /tmp/eks-trust.json << 'EOF'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"eks.amazonaws.com"},"Action":"sts:AssumeRole"}]}
EOF
aws iam create-role --role-name eksClusterRole \
--assume-role-policy-document file:///tmp/eks-trust.json 2>/dev/null || true
aws iam attach-role-policy --role-name eksClusterRole \
--policy-arn arn:aws:iam::aws:policy/AmazonEKSClusterPolicy
ROLE_ARN=$(aws iam get-role --role-name eksClusterRole --query 'Role.Arn' --output text)
```
Trust principal is **`eks.amazonaws.com`** (the control-plane service, not `eks-tasks` or `ec2`). `AmazonEKSClusterPolicy` is the single managed policy the control plane needs to manage AWS resources on your behalf (ENIs, load balancers, etc.).
## Phase 2 — Gather default VPC subnets in AZs a, b, c
```bash
VPC_ID=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
# One subnet per AZ for us-east-1a/b/c
SUBNET_A=$(aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=$VPC_ID" "Name=availability-zone,Values=${REGION}a" \
--region $REGION --query 'Subnets[0].SubnetId' --output text)
SUBNET_B=$(aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=$VPC_ID" "Name=availability-zone,Values=${REGION}b" \
--region $REGION --query 'Subnets[0].SubnetId' --output text)
SUBNET_C=$(aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=$VPC_ID" "Name=availability-zone,Values=${REGION}c" \
--region $REGION --query 'Subnets[0].SubnetId' --output text)
echo "subnets: $SUBNET_A $SUBNET_B $SUBNET_C"
```
EKS requires subnets in **at least two** AZs; the task asks for three (a/b/c) for higher availability. Explicitly filtering by AZ guarantees you land one subnet in each of the three, rather than accidentally grabbing two in the same AZ.
## Phase 3 — Create the cluster (private endpoint, Auto Mode off)
```bash
aws eks create-cluster \
--name nautilus-eks \
--kubernetes-version 1.36 \
--role-arn "$ROLE_ARN" \
--resources-vpc-config "subnetIds=$SUBNET_A,$SUBNET_B,$SUBNET_C,endpointPublicAccess=false,endpointPrivateAccess=true" \
--compute-config enabled=false \
--kubernetes-network-config '{"elasticLoadBalancing":{"enabled":false}}' \
--storage-config '{"blockStorage":{"enabled":false}}' \
--access-config authenticationMode=API_AND_CONFIG_MAP \
--region $REGION
# EKS control plane provisioning is SLOW (~10-15 min). Wait it out.
# aws eks wait cluster-active --name nautilus-eks --region $REGION
STATUS=""
until [ "$STATUS" = "ACTIVE" ]; do
STATUS=$(aws eks describe-cluster --name nautilus-eks --region $REGION \
--query 'cluster.status' --output text)
echo "nautilus-eks: $STATUS"
[ "$STATUS" = "ACTIVE" ] || sleep 30
done
```
The requirement-to-flag mapping — the important ones:
- **`endpointPublicAccess=false,endpointPrivateAccess=true`** = the "private endpoint" requirement. This is the crux: the API server becomes reachable *only* from within the VPC, not the internet. Note the implication — you can no longer `kubectl` from outside the VPC after this; you'd need a bastion/VPN inside the VPC. That's the intended security posture.
- **`--compute-config enabled=false`** + **`elasticLoadBalancing.enabled=false`** + **`blockStorage.enabled=false`** = **EKS Auto Mode disabled.** Auto Mode is the newer bundle where EKS auto-manages compute, load balancing, and storage; all three sub-toggles being `false` is what "Auto Mode disabled / Custom configuration" means. Setting `compute enabled=true` would flip Auto Mode on — the opposite of the task.
- **`--kubernetes-version 1.36`** = latest stable.
- **`--role-arn`** = the `eksClusterRole` from Phase 1.
- **`--resources-vpc-config subnetIds=...`** = the three AZ subnets for HA.
## Verify
```bash
aws eks describe-cluster --name nautilus-eks --region $REGION \
--query 'cluster.{Name:name,Status:status,Version:version,Role:roleArn,
PublicAccess:resourcesVpcConfig.endpointPublicAccess,
PrivateAccess:resourcesVpcConfig.endpointPrivateAccess,
Subnets:resourcesVpcConfig.subnetIds}'
```
Want: `Status: ACTIVE`, `Version: 1.36`, `Role` ending in `eksClusterRole`, `PublicAccess: false`, `PrivateAccess: true`, and three subnet IDs. `ACTIVE` with those endpoint flags is the task's success condition — cluster up, private-only, correct role and version.
## Notes / gotchas
- **Provisioning is the slowest wait in the whole AWS set** — EKS control plane takes ~10-15 minutes to go `CREATING → ACTIVE`. The `wait cluster-active` waiter (polls every 30s, generous timeout) covers it; don't submit while `CREATING`.
- **Private-only endpoint means no external kubectl.** Once `endpointPublicAccess=false`, you can't reach the API server from `aws-client` unless `aws-client` is inside the VPC. That's expected for this task (it only asks the cluster be created + private + ACTIVE, not that you run workloads through it).
- **Auto Mode flag names can drift** across CLI versions. If `--compute-config`/`--storage-config` are rejected by an older CLI, the console "Custom configuration" path with Auto Mode toggled off is the equivalent; or update the CLI. The intent is: compute/LB/storage auto-management all OFF.
- **No node group here.** The task is control-plane only — it doesn't ask for worker nodes. A functioning cluster for workloads would need a managed node group or Fargate profile added after, but that's beyond this task's scope.

190
aws-44.md Normal file
View File

@@ -0,0 +1,190 @@
## Task 44
The DevOps team is tasked with setting up a highly available web application using AWS. To achieve this, they plan to use an Auto Scaling Group (ASG) to ensure that the required number of EC2 instances are always running, and an Application Load Balancer (ALB) to distribute traffic across these instances. The goal of this task is to set up an ASG that automatically scales EC2 instances based on CPU utilization, and an ALB that directs incoming traffic to the instances. The EC2 instances should have Nginx installed and running to serve web traffic.
Create an EC2 launch template named xfusion-launch-template that specifies the configuration for the EC2 instances, including the Amazon Linux 2 AMI, t2.micro instance type, and a security group that allows HTTP traffic on port 80.
Add a User Data script to the launch template to install Nginx on the EC2 instances when they are launched. The script should install Nginx, start the Nginx service, and enable it to start on boot.
Create an Auto Scaling Group named xfusion-asg that uses the launch template and ensures a minimum of 1 instance, desired capacity is 1 instance and a maximum of 2 instances are running based on CPU utilization. Set the target CPU utilization to 50%.
Create a target group named xfusion-tg, an Application Load Balancer named xfusion-alb and configure it to listen on port 80. Ensure the ALB is associated with the Auto Scaling Group and distributes traffic across the instances.
Configure health checks on the ALB to ensure it routes traffic only to healthy instances.
Verify that the ALB's DNS name is accessible and that it displays the default Nginx page served by the EC2 instances.
### Solution
# ASG + ALB Auto-Scaling Task (xfusion)
The full auto-scaling stack — launch template → ASG with target-tracking → ALB/TG wired to the ASG. The concept that ties it together: you **don't register instances into the target group manually**; you attach the TG to the ASG (`--target-group-arns`), and the ASG auto-registers every instance it launches (and deregisters on scale-in). And scaling "based on CPU at 50%" = a **target-tracking scaling policy** on the `ASGAverageCPUUtilization` metric — not a manual CloudWatch alarm.
## Version note — Amazon Linux 2 is EOL
The task says "Amazon Linux 2 AMI," but **AL2 hit end-of-life June 30, 2026** (already past). The AMIs are still *launchable* (they don't vanish at EOL, they just stop getting patches), so if the grader specifically checks for AL2, use the AL2 alias below. But for anything real you'd use AL2023. Both aliases given — pick per what the grader wants. On AL2 the Nginx install uses `amazon-linux-extras`; on AL2023 it's `dnf`. The user-data below is written for **AL2** to match the task literally.
Run on `aws-client`.
## Phase 1 — Security group (HTTP 80)
```bash
REGION=us-east-1
VPC_ID=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
SG_ID=$(aws ec2 create-security-group \
--group-name xfusion-web-sg \
--description "HTTP 80 for xfusion ASG" \
--vpc-id $VPC_ID --region $REGION \
--query 'GroupId' --output text)
aws ec2 authorize-security-group-ingress \
--group-id $SG_ID --protocol tcp --port 80 --cidr 0.0.0.0/0 --region $REGION
```
## Phase 2 — Launch template (with base64 user-data)
```bash
# AL2 AMI via SSM alias (EOL but still launchable). AL2023 alternative commented.
AMI_ID=$(aws ssm get-parameters \
--names /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2 \
--region $REGION --query 'Parameters[0].Value' --output text)
# AL2023: /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64
# User-data for AL2 (amazon-linux-extras) — must be base64 for launch templates
USERDATA=$(base64 -w0 << 'EOF'
#!/bin/bash
amazon-linux-extras install nginx1 -y
systemctl start nginx
systemctl enable nginx
EOF
)
# (AL2023 user-data would be: dnf install -y nginx; systemctl enable --now nginx)
aws ec2 create-launch-template \
--launch-template-name xfusion-launch-template \
--region $REGION \
--launch-template-data "{
\"ImageId\": \"$AMI_ID\",
\"InstanceType\": \"t2.micro\",
\"SecurityGroupIds\": [\"$SG_ID\"],
\"UserData\": \"$USERDATA\"
}"
```
**Key point:** in a launch template, `UserData` must be **base64-encoded** (unlike `run-instances --user-data file://` which encodes for you). `base64 -w0` produces a single line with no wrapping — critical, since embedded newlines break the JSON.
## Phase 3 — Target group + ALB + listener
```bash
read -r SUBNET1 SUBNET2 <<< $(aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=$VPC_ID" "Name=map-public-ip-on-launch,Values=true" \
--region $REGION --query 'Subnets[].SubnetId | [0:2]' --output text)
# Target group with explicit health check on "/"
TG_ARN=$(aws elbv2 create-target-group \
--name xfusion-tg \
--protocol HTTP --port 80 \
--vpc-id $VPC_ID --target-type instance \
--health-check-protocol HTTP \
--health-check-path / \
--health-check-interval-seconds 30 \
--healthy-threshold-count 2 \
--unhealthy-threshold-count 2 \
--region $REGION \
--query 'TargetGroups[0].TargetGroupArn' --output text)
# ALB across two AZs, wearing the web SG
ALB_ARN=$(aws elbv2 create-load-balancer \
--name xfusion-alb \
--subnets $SUBNET1 $SUBNET2 \
--security-groups $SG_ID \
--scheme internet-facing --type application \
--region $REGION \
--query 'LoadBalancers[0].LoadBalancerArn' --output text)
# Listener on 80 -> forward to TG
aws elbv2 create-listener \
--load-balancer-arn $ALB_ARN \
--protocol HTTP --port 80 \
--default-actions Type=forward,TargetGroupArn=$TG_ARN \
--region $REGION
```
The **health check** (task req #5) is configured on the target group: `HTTP GET /` every 30s, healthy after 2 passes, unhealthy after 2 fails. The ALB only routes to targets the TG reports healthy — that's the "route traffic only to healthy instances" requirement.
## Phase 4 — Auto Scaling Group (attached to the TG)
```bash
aws autoscaling create-auto-scaling-group \
--auto-scaling-group-name xfusion-asg \
--launch-template "LaunchTemplateName=xfusion-launch-template,Version=\$Latest" \
--min-size 1 --max-size 2 --desired-capacity 1 \
--vpc-zone-identifier "$SUBNET1,$SUBNET2" \
--target-group-arns $TG_ARN \
--health-check-type ELB \
--health-check-grace-period 90 \
--region $REGION
```
The two crucial bits here:
- **`--target-group-arns $TG_ARN`** is what "associates the ALB with the ASG." The ASG now auto-registers each instance it launches into the TG (and deregisters on scale-in). You never call `register-targets` manually — that's the whole point of ASG+ALB integration.
- **`--health-check-type ELB`** makes the ASG use the *load balancer's* health check to decide instance health, not just the EC2 status check. So an instance whose Nginx died gets replaced even though the VM is "running." `--health-check-grace-period 90` gives user-data time to install Nginx before health checks start counting (otherwise the ASG kills the instance mid-bootstrap).
## Phase 5 — Target-tracking scaling policy (CPU 50%)
```bash
cat > /tmp/tt-config.json << 'EOF'
{
"PredefinedMetricSpecification": {
"PredefinedMetricType": "ASGAverageCPUUtilization"
},
"TargetValue": 50.0
}
EOF
aws autoscaling put-scaling-policy \
--auto-scaling-group-name xfusion-asg \
--policy-name xfusion-cpu-target-tracking \
--policy-type TargetTrackingScaling \
--target-tracking-configuration file:///tmp/tt-config.json \
--region $REGION
```
**Target tracking** is the modern way to do "scale based on CPU at 50%." It auto-creates the CloudWatch alarms behind the scenes and adjusts capacity to keep average CPU near 50% — scale out above, scale in below, within the min/max (1–2) bounds. Far cleaner than manually wiring step-scaling policies to CloudWatch alarms (the old way).
## Verify
```bash
# ASG has the instance + correct sizing
aws autoscaling describe-auto-scaling-groups \
--auto-scaling-group-names xfusion-asg --region $REGION \
--query 'AutoScalingGroups[0].{Min:MinSize,Max:MaxSize,Desired:DesiredCapacity,
Instances:Instances[].{Id:InstanceId,Health:HealthStatus,State:LifecycleState},
TGs:TargetGroupARNs}'
# Poll target health until healthy
for n in $(seq 1 12); do
H=$(aws elbv2 describe-target-health --target-group-arn $TG_ARN --region $REGION \
--query 'TargetHealthDescriptions[0].TargetHealth.State' --output text)
echo "target: $H"; [ "$H" = "healthy" ] && break; sleep 20
done
# Hit the ALB DNS — should return the Nginx page
ALB_DNS=$(aws elbv2 describe-load-balancers --load-balancer-arns $ALB_ARN \
--region $REGION --query 'LoadBalancers[0].DNSName' --output text)
echo "http://$ALB_DNS"
curl -I "http://$ALB_DNS"
```
Want: ASG showing min1/max2/desired1 with one instance `Healthy`/`InService`, the TG target reaching `healthy`, and `curl -I` on the ALB DNS returning `HTTP/1.1 200 OK` with an `nginx` Server header. That 200 through the ALB DNS is the task's success proof — the whole chain (ASG launches instance → user-data installs Nginx → TG health check passes → ALB routes to it) works.
## Timing & debug
Two stacked waits: user-data installing Nginx (~60-90s) plus the TG health-check threshold (2 × 30s). Even after the ALB is `active`, give it ~3 min for the target to go healthy. If it stays `unhealthy`:
1. **`Target.Timeout`** → SG not allowing 80 from the ALB, or Nginx not up. Since ALB and instances share `xfusion-web-sg` here and it allows 80 from `0.0.0.0/0`, the ALB can reach the instance; if still failing, Nginx didn't install — check `/var/log/cloud-init-output.log`.
2. **`Target.ResponseCodeMismatch`** → Nginx serving something other than 200 on `/`, or user-data failed (on AL2023, `amazon-linux-extras` doesn't exist — wrong user-data for the AMI).
3. **ASG launches then immediately terminates instances** → health-check grace period too short; the ELB health check failed a bootstrapping instance. Raise `--health-check-grace-period`.
## Note on launch template vs launch configuration
This uses a **launch template**, not the deprecated launch configuration. AWS ended new-account access to launch configurations; templates are the current standard and support versioning (`Version=$Latest`), mixed instances, and newer features. Always reach for `create-launch-template`.

185
aws-45.md Normal file
View File

@@ -0,0 +1,185 @@
## Task 45
The Nautilus DevOps team is tasked with enabling internet access for an EC2 instance running in a private subnet. This instance should be able to upload a test file to a public S3 bucket once it can access the internet. To achieve this, the team must set up a NAT Gateway in a public subnet within the same VPC.
1) A VPC named xfusion-priv-vpc and a private subnet xfusion-priv-subnet have already been created.
2) An EC2 instance named xfusion-priv-ec2 is already running in the private subnet.
3) The EC2 instance is configured with a cron job that uploads a test file to a bucket xfusion-nat-285654546 once internet is accessible.
Your task is to:
Create a public subnet named xfusion-pub-subnet in the same VPC.
Create an Internet Gateway and attach it to the VPC.
Create a route table xfusion-pub-rt and associate it with the public subnet.
Allocate an Elastic IP and create a NAT Gateway named xfusion-natgw.
Update the private route table to route 0.0.0.0/0 traffic via the NAT Gateway.
Once complete, verify that the EC2 instance can reach the internet by confirming the presence of the test file in the S3 bucket xfusion-nat-285654546. After completing all the configuration, please wait a few minutes for the test file to appear in the bucket, as it may take 2–3 minutes.
### Solution
# NAT Gateway Task (xfusion-priv-vpc)
The **managed NAT Gateway** version — contrast with the earlier NAT *instance* task, which needed a self-managed EC2 box with source/dest-check disabled, IP forwarding, and iptables MASQUERADE. A NAT **Gateway** is a fully AWS-managed resource: no instance, no OS config, no source/dest check, auto-scaling and HA within its AZ. You just place it in a public subnet, give it an EIP, and route the private subnet's default route at it. Much less to get wrong.
The architecture: private subnet → its route table `0.0.0.0/0 → NAT GW` → NAT GW (in public subnet) → public subnet's route table `0.0.0.0/0 → IGW` → internet → S3. Two route tables, two different default routes.
Run on `aws-client`.
## Phase 1 — Discover VPC + private subnet
```bash
REGION=us-east-1
VPC_ID=$(aws ec2 describe-vpcs --filters Name=tag:Name,Values=xfusion-priv-vpc \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
VPC_CIDR=$(aws ec2 describe-vpcs --vpc-ids $VPC_ID --region $REGION \
--query 'Vpcs[0].CidrBlock' --output text)
PRIV_SUBNET=$(aws ec2 describe-subnets --filters "Name=tag:Name,Values=xfusion-priv-subnet" \
--region $REGION --query 'Subnets[0].SubnetId' --output text)
PRIV_AZ=$(aws ec2 describe-subnets --subnet-ids $PRIV_SUBNET --region $REGION \
--query 'Subnets[0].AvailabilityZone' --output text)
```
## Phase 2 — Public subnet + Internet Gateway
```bash
# Pick a free /24 in the VPC for the public subnet
BASE=$(echo $VPC_CIDR | cut -d. -f1-2)
USED=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" \
--region $REGION --query 'Subnets[].CidrBlock' --output text | tr '\t' '\n')
for i in $(seq 1 254); do
CAND="${BASE}.${i}.0/24"
echo "$USED" | grep -Fxq "$CAND" || { PUB_CIDR=$CAND; break; }
done
# Public subnet (same AZ as private keeps NAT traffic in-AZ, avoids cross-AZ charges)
PUB_SUBNET=$(aws ec2 create-subnet \
--vpc-id $VPC_ID --cidr-block $PUB_CIDR \
--availability-zone $PRIV_AZ --region $REGION \
--tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=xfusion-pub-subnet}]' \
--query 'Subnet.SubnetId' --output text)
aws ec2 modify-subnet-attribute --subnet-id $PUB_SUBNET \
--map-public-ip-on-launch --region $REGION
# Internet Gateway (reuse if the VPC already has one)
IGW_ID=$(aws ec2 describe-internet-gateways \
--filters "Name=attachment.vpc-id,Values=$VPC_ID" \
--region $REGION --query 'InternetGateways[0].InternetGatewayId' --output text)
if [ "$IGW_ID" = "None" ] || [ -z "$IGW_ID" ]; then
IGW_ID=$(aws ec2 create-internet-gateway --region $REGION \
--tag-specifications 'ResourceType=internet-gateway,Tags=[{Key=Name,Value=xfusion-igw}]' \
--query 'InternetGateway.InternetGatewayId' --output text)
aws ec2 attach-internet-gateway --internet-gateway-id $IGW_ID --vpc-id $VPC_ID --region $REGION
fi
```
## Phase 3 — Public route table (→ IGW), associate with public subnet
```bash
PUB_RT=$(aws ec2 create-route-table --vpc-id $VPC_ID --region $REGION \
--tag-specifications 'ResourceType=route-table,Tags=[{Key=Name,Value=xfusion-pub-rt}]' \
--query 'RouteTable.RouteTableId' --output text)
aws ec2 create-route --route-table-id $PUB_RT \
--destination-cidr-block 0.0.0.0/0 --gateway-id $IGW_ID --region $REGION
aws ec2 associate-route-table --route-table-id $PUB_RT --subnet-id $PUB_SUBNET --region $REGION
```
This route table makes `xfusion-pub-subnet` genuinely public — its `0.0.0.0/0` points at the IGW. **The NAT Gateway must live in this subnet** so its own outbound traffic (forwarding on behalf of the private instance) can reach the internet via the IGW.
## Phase 4 — Elastic IP + NAT Gateway
```bash
# Allocate an EIP for the NAT GW
NAT_EIP_ALLOC=$(aws ec2 allocate-address --domain vpc --region $REGION \
--tag-specifications 'ResourceType=elastic-ip,Tags=[{Key=Name,Value=xfusion-nat-eip}]' \
--query 'AllocationId' --output text)
# Create the NAT Gateway IN THE PUBLIC SUBNET
NATGW_ID=$(aws ec2 create-nat-gateway \
--subnet-id $PUB_SUBNET \
--allocation-id $NAT_EIP_ALLOC \
--region $REGION \
--tag-specifications 'ResourceType=natgateway,Tags=[{Key=Name,Value=xfusion-natgw}]' \
--query 'NatGateway.NatGatewayId' --output text)
# NAT GW takes ~1-2 min to become available — wait for it
# aws ec2 wait nat-gateway-available --nat-gateway-ids $NATGW_ID --region $REGION
STATE=""
until [ "$STATE" = "available" ]; do
STATE=$(aws ec2 describe-nat-gateways --nat-gateway-ids $NATGW_ID --region $REGION \
--query 'NatGateways[0].State' --output text)
echo "$NATGW_ID: $STATE"
[ "$STATE" = "available" ] || sleep 10
done
```
Two must-get-right points:
- **NAT GW goes in the PUBLIC subnet, not the private one.** This trips people constantly. The NAT GW needs a path to the internet for the traffic it forwards, so it sits in the public subnet (which routes to the IGW). The *private* instance then routes *to* the NAT GW. Putting the NAT GW in the private subnet creates a routing loop with no internet path — nothing works.
- **A NAT GW requires an EIP** (public NAT GWs). `allocate-address` → pass its allocation ID to `create-nat-gateway`. The EIP is the NAT GW's public-facing address that S3 sees.
## Phase 5 — Private route table → NAT Gateway
```bash
# Find the route table serving the private subnet (explicit assoc, else main)
PRIV_RT=$(aws ec2 describe-route-tables \
--filters "Name=association.subnet-id,Values=$PRIV_SUBNET" \
--region $REGION --query 'RouteTables[0].RouteTableId' --output text)
if [ "$PRIV_RT" = "None" ] || [ -z "$PRIV_RT" ]; then
PRIV_RT=$(aws ec2 describe-route-tables \
--filters "Name=vpc-id,Values=$VPC_ID" "Name=association.main,Values=true" \
--region $REGION --query 'RouteTables[0].RouteTableId' --output text)
fi
# Point the private default route at the NAT GW (create, or replace if one exists)
aws ec2 create-route --route-table-id $PRIV_RT \
--destination-cidr-block 0.0.0.0/0 --nat-gateway-id $NATGW_ID --region $REGION 2>/dev/null || \
aws ec2 replace-route --route-table-id $PRIV_RT \
--destination-cidr-block 0.0.0.0/0 --nat-gateway-id $NATGW_ID --region $REGION
```
This is the step that actually gives the private instance internet access: its subnet's `0.0.0.0/0` now flows to the NAT GW (`--nat-gateway-id`, not `--gateway-id` which is for IGWs). The instance keeps *no* public IP — outbound-only internet via NAT, which is exactly the private-instance pattern.
## Verify
```bash
# NAT GW available
aws ec2 describe-nat-gateways --nat-gateway-ids $NATGW_ID --region $REGION \
--query 'NatGateways[0].{State:State,Subnet:SubnetId,EIP:NatGatewayAddresses[0].PublicIp}'
# Private route table points at the NAT GW
aws ec2 describe-route-tables --route-table-ids $PRIV_RT --region $REGION \
--query 'RouteTables[0].Routes[?DestinationCidrBlock==`0.0.0.0/0`]'
# Poll the bucket for the cron-uploaded test file (2-3 min after config)
for n in $(seq 1 8); do
FILES=$(aws s3 ls s3://xfusion-nat-285654546/ --region $REGION 2>/dev/null)
if [ -n "$FILES" ]; then echo "SUCCESS — bucket contents:"; echo "$FILES"; break; fi
echo "waiting for cron upload... ($n)"; sleep 30
done
```
Want: NAT GW `State: available` with its EIP; the private route table showing `0.0.0.0/0 → nat-...`; and a file appearing in the bucket within 2–3 minutes. **The file appearing is the end-to-end proof** — the private instance (no public IP) reached S3 entirely through the NAT Gateway.
## NAT Gateway vs NAT Instance (the contrast)
| | NAT Gateway (this task) | NAT Instance (earlier task) |
|---|---|---|
| Management | Fully AWS-managed | You run/patch an EC2 box |
| Source/dest check | N/A (managed) | Must disable manually |
| iptables / IP forwarding | None | You configure MASQUERADE + `ip_forward` |
| HA / scaling | Automatic within AZ | Single instance, you handle HA |
| Cost | Higher hourly + data processing | Just the EC2 instance |
The NAT Gateway is the production-standard choice; the NAT instance exists mostly for cost-sensitive or learning scenarios. This task is the "right way."
## Debug if the file doesn't appear
1. **NAT GW stuck in `pending`** → wait longer (up to 2 min), or it `failed` (usually the EIP was already in use — allocate a fresh one).
2. **Private route wrong** → confirm `0.0.0.0/0` points at `nat-...` (NAT GW), not `igw-...`. A private subnet routing to an IGW directly doesn't work without a public IP on the instance.
3. **Public subnet's route table missing the IGW route** → the NAT GW itself can't reach the internet, so forwarded traffic dies. Confirm `xfusion-pub-rt` has `0.0.0.0/0 → igw-...` and is associated with the public subnet the NAT GW lives in.
4. **Give it the full 2–3 min** — the cron runs on an interval; the file won't appear instantly even once networking is correct.

222
aws-46.md Normal file
View File

@@ -0,0 +1,222 @@
## Task 46
The DevOps team is working on automating file management between two S3 buckets. The task is to create a public S3 bucket for file uploads and a private S3 bucket for securely storing the files. A Lambda function will be triggered automatically whenever a file is uploaded to the public S3 bucket, which will copy the file to the private bucket. Additionally, logs of the operation will be stored in a DynamoDB table. The logs should include details such as the source bucket, destination bucket, and the object key of the file that was copied. This will help the team maintain better security and visibility for file transfers.
Create a public S3 bucket named datacenter-public-8776. Ensure that the bucket allows public access to its objects.
Create a private S3 bucket named datacenter-private-12032. Ensure that the bucket does not allow public access.
Create a Lambda function named datacenter-copyfunction. This function should be triggered by uploads to the public S3 bucket and should copy the uploaded file to the private bucket. Create the necessary policies and a role named lambda_execution_role. Attach these policies to the role, and then link this role to the Lambda function.
lambda-function.py is already present under the /root/ directory on AWS client host, replace REPLACE-WITH-YOUR-DYNAMODB-TABLE and REPLACE-WITH-YOUR-PRIVATE-BUCKET values.
Create a DynamoDB table named datacenter-S3CopyLogs with a partition key LogID (string). This table will store logs generated by the Lambda function, including details such as source bucket name, destination bucket name, and object key.
For testing upload the file sample.zip located in the /root directory on the client host to the public S3 bucket. The Lambda function should trigger and copy the file to the private bucket.
Verify that the file has been successfully copied to the private bucket by checking the private bucket in the S3 console.
Verify that a log entry has been created in the DynamoDB table containing the file copy details.
### Solution
# S3 → Lambda → S3 + DynamoDB Event-Driven Copy Task (datacenter)
An event-driven serverless pipeline — the most concept-dense task in the set. The crux is the **S3 → Lambda trigger**, which is a *two-part* wiring most people get half-right: (1) a **resource-based permission** on the Lambda (`add-permission`) letting the S3 service invoke it, AND (2) the **event notification** on the bucket pointing at the function. Miss either and uploads silently don't trigger anything. Plus the Lambda's role needs four distinct permission sets: read public S3, write private S3, write DynamoDB, write logs.
## ⚠️ Filename gotcha — read this first
The task says the code file is `lambda-function.py` (**hyphen**). Python module names **cannot contain hyphens**, so a handler of `lambda-function.lambda_handler` will fail at import with `Runtime.ImportModuleError`. The fix: **rename to `lambda_function.py` (underscore) when zipping**, and set the handler to `lambda_function.lambda_handler`. Handled below.
Run on `aws-client`.
## Phase 1 — The two S3 buckets
```bash
REGION=us-east-1
PUB_BUCKET=datacenter-public-8776
PRIV_BUCKET=datacenter-private-12032
# --- Public bucket: allow public access ---
aws s3api create-bucket --bucket $PUB_BUCKET --region $REGION
aws s3api put-public-access-block --bucket $PUB_BUCKET \
--public-access-block-configuration \
"BlockPublicAcls=false,IgnorePublicAcls=false,BlockPublicPolicy=false,RestrictPublicBuckets=false" \
--region $REGION
cat > /tmp/pub-policy.json << EOF
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "PublicRead",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::$PUB_BUCKET/*"
}]
}
EOF
aws s3api put-bucket-policy --bucket $PUB_BUCKET \
--policy file:///tmp/pub-policy.json --region $REGION
# --- Private bucket: keep the secure defaults (public access blocked) ---
aws s3api create-bucket --bucket $PRIV_BUCKET --region $REGION
# New buckets are private by default (BPA fully on) — nothing to do.
```
The public bucket needs BPA disabled + a public-read policy (same pattern as the static-website task). The private bucket just uses the default locked-down state — no action needed, which *is* the "does not allow public access" requirement.
## Phase 2 — DynamoDB table
```bash
aws dynamodb create-table \
--table-name datacenter-S3CopyLogs \
--attribute-definitions AttributeName=LogID,AttributeType=S \
--key-schema AttributeName=LogID,KeyType=HASH \
--billing-mode PAY_PER_REQUEST \
--region $REGION
aws dynamodb wait table-exists --table-name datacenter-S3CopyLogs --region $REGION
```
Only `LogID` (string partition key) is declared — the source/dest/objectKey attributes are written per-item by the Lambda, schemaless.
## Phase 3 — IAM role + policy (four permission sets)
```bash
REGION=us-east-1
PUB_BUCKET=datacenter-public-8776
PRIV_BUCKET=datacenter-private-12032
# --- STEP 1: Does the role already exist (lab-provided)? ---
EXISTING=$(aws iam get-role --role-name lambda_execution_role \
--query 'Role.Arn' --output text 2>/dev/null || echo "MISSING")
echo "Role status: [$EXISTING]"
# --- STEP 2: If MISSING, create it (trust policy only — that's allowed) ---
if [ "$EXISTING" = "MISSING" ]; then
cat > /tmp/lambda-trust.json << 'EOF'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"lambda.amazonaws.com"},"Action":"sts:AssumeRole"}]}
EOF
aws iam create-role --role-name lambda_execution_role \
--assume-role-policy-document file:///tmp/lambda-trust.json
fi
# --- STEP 3: See what's already attached (lab may have pre-attached perms) ---
echo "=== attached managed policies ==="
aws iam list-attached-role-policies --role-name lambda_execution_role
echo "=== inline policies ==="
aws iam list-role-policies --role-name lambda_execution_role
# --- STEP 4: Attach AWS-managed policies (permitted where custom creation isn't) ---
# S3 (read public + write private), DynamoDB (write logs), Logs (basic execution)
for ARN in \
arn:aws:iam::aws:policy/AmazonS3FullAccess \
arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess \
arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole ; do
aws iam attach-role-policy --role-name lambda_execution_role --policy-arn "$ARN" \
2>/dev/null && echo "attached: $ARN" || echo "skip/denied: $ARN"
done
ROLE_ARN=$(aws iam get-role --role-name lambda_execution_role --query 'Role.Arn' --output text)
echo "ROLE_ARN=[$ROLE_ARN]"
```
Four permission sets, each essential: **GetObject** on the public bucket (read the upload), **PutObject** on the private bucket (write the copy), **PutItem** on the table (write the log), and **AWSLambdaBasicExecutionRole** for CloudWatch Logs (or the function can't even log). Note the S3 ARNs use `/*` (object-level) since Get/Put act on objects.
## Phase 4 — Prep the code, zip (with the rename), deploy
```bash
# Replace placeholders; write out with UNDERSCORE filename (hyphen breaks Python import)
mkdir -p /tmp/fn
sed \
-e "s/REPLACE-WITH-YOUR-DYNAMODB-TABLE/datacenter-S3CopyLogs/g" \
-e "s/REPLACE-WITH-YOUR-PRIVATE-BUCKET/$PRIV_BUCKET/g" \
/root/lambda-function.py > /tmp/fn/lambda_function.py
(cd /tmp/fn && zip -q function.zip lambda_function.py)
# Deploy with KodeKloud-compliant timeout (10s) and memory (256 MB)
sleep 10 # let role propagate
aws lambda create-function \
--function-name datacenter-copyfunction \
--runtime python3.13 \
--role "$ROLE_ARN" \
--handler lambda_function.lambda_handler \
--zip-file fileb:///tmp/fn/function.zip \
--timeout 10 \
--memory-size 256 \
--region $REGION
FUNC_ARN=$(aws lambda get-function --function-name datacenter-copyfunction \
--region $REGION --query 'Configuration.FunctionArn' --output text)
echo "FUNC_ARN=[$FUNC_ARN]"
```
- **The `sed` handles step 4** — swaps `REPLACE-WITH-YOUR-DYNAMODB-TABLE` → `datacenter-S3CopyLogs` and `REPLACE-WITH-YOUR-PRIVATE-BUCKET` → the private bucket name, writing the result out as `lambda_function.py` (underscore) to dodge the hyphen import problem.
- **`--handler lambda_function.lambda_handler`** matches the underscore filename + the function inside.
- **`--timeout 30`** — the default 3s can be too short for an S3 copy + DynamoDB write; bump it.
- **`python3.13`** — current supported runtime (3.14 also valid; the code was written for whatever the lab targets, 3.13 is safe).
## Phase 5 — Wire the S3 trigger (the two-part gotcha)
```bash
# PART 1: Let the S3 service invoke this Lambda (resource-based permission)
aws lambda add-permission \
--function-name datacenter-copyfunction \
--statement-id s3-invoke-permission \
--action lambda:InvokeFunction \
--principal s3.amazonaws.com \
--source-arn arn:aws:s3:::$PUB_BUCKET \
--region $REGION
# PART 2: Configure the public bucket to notify Lambda on object creation
cat > /tmp/notif.json << EOF
{
"LambdaFunctionConfigurations": [
{
"LambdaFunctionArn": "$FUNC_ARN",
"Events": ["s3:ObjectCreated:*"]
}
]
}
EOF
aws s3api put-bucket-notification-configuration \
--bucket $PUB_BUCKET \
--notification-configuration file:///tmp/notif.json \
--region $REGION
```
**This ordering is mandatory.** `put-bucket-notification-configuration` *validates at config time* that S3 can actually invoke the target Lambda — so the `add-permission` (Part 1) must exist first, or Part 2 fails with `Unable to validate the following destination configurations`. The permission is a resource-based policy on the Lambda; the notification is config on the bucket. Both halves = a working trigger; either alone = silent no-op.
## Phase 6 — Test: upload sample.zip
```bash
aws s3 cp /root/sample.zip s3://$PUB_BUCKET/sample.zip --region $REGION
# Give the async trigger + copy + log-write a few seconds
sleep 10
```
## Phase 7 & 8 — Verify copy + log entry
```bash
# File copied to private bucket?
echo "=== private bucket ==="
aws s3 ls s3://$PRIV_BUCKET/ --region $REGION
# Log entry in DynamoDB?
echo "=== DynamoDB logs ==="
aws dynamodb scan --table-name datacenter-S3CopyLogs --region $REGION \
--query 'Items[].{LogID:LogID.S,Source:sourceBucket.S,Dest:destinationBucket.S,Key:objectKey.S}' \
--output table
```
Want `sample.zip` present in the private bucket, and at least one log item in the table showing source (`datacenter-public-8776`), destination (`datacenter-private-12032`), and object key (`sample.zip`). Both appearing is the end-to-end proof: upload → S3 event → Lambda invoked → copied + logged.
> Note: the exact DynamoDB attribute names (`sourceBucket`, `destinationBucket`, `objectKey`) depend on what the provided `lambda-function.py` writes — adjust the `--query` to match its actual keys if they differ. A plain `aws dynamodb scan --table-name datacenter-S3CopyLogs` with no query shows the raw item so you can see the real attribute names.
## Debug order if nothing copies
1. **Nothing in private bucket, nothing in DynamoDB** → the trigger didn't fire. Check the two-part wiring: `aws lambda get-policy --function-name datacenter-copyfunction` should show the s3.amazonaws.com permission, and `aws s3api get-bucket-notification-configuration --bucket $PUB_BUCKET` should show the Lambda config. If either's missing, redo Phase 5 (in order).
2. **Trigger fired but errored** → read the Lambda's logs: `aws logs tail /aws/lambda/datacenter-copyfunction --region $REGION`. Common causes: `AccessDenied` (role missing a permission — check Phase 3), `ImportModuleError` (the hyphen filename problem — confirm the zip has `lambda_function.py` with underscore), or `ResourceNotFoundException` (placeholders not replaced — the private bucket / table name still says `REPLACE-WITH-...`).
3. **Copy works but no log** → DynamoDB `PutItem` permission missing from the role, or the table name placeholder wasn't substituted.
4. **`ImportModuleError: No module named 'lambda-function'`** → the classic — you zipped/handler'd with the hyphen. Re-zip as `lambda_function.py`, update handler, `update-function-code`.
## The core concept
The whole task hinges on **event-driven decoupling**: S3 doesn't "call" Lambda directly — it emits an event that the Lambda service consumes, gated by a resource-based permission. This is the fundamental AWS serverless pattern (S3/SNS/SQS/EventBridge → Lambda all work this way). The two-part permission+notification wiring is the thing to internalize; it shows up on the DevOps Pro exam constantly.

265
aws-47.md Normal file
View File

@@ -0,0 +1,265 @@
## Task 47
The Nautilus DevOps team needs to implement priority queuing using Amazon SQS and SNS. The goal is to create a system where messages with different priorities are handled accordingly. You are required to use AWS CloudFormation to deploy the necessary resources in your AWS account. The CloudFormation template should be created on the AWS client host at /root/datacenter-priority-stack.yml, the stack name must be datacenter-priority-stack and it should create the following resources:
Two SQS queues named datacenter-High-Priority-Queue and datacenter-Low-Priority-Queue.
An SNS topic named datacenter-Priority-Queues-Topic.
A Lambda function named datacenter-priorities-queue-function that will consume messages from the SQS queues. The Lambda function code is provided in /root/index.py on the AWS client host.
An IAM role named lambda_execution_role that provides the necessary permissions for the Lambda function to interact with SQS and SNS.
Once the stack is deployed, to test the same you can publish messages to the SNS topic, invoke the Lambda function and observe the order in which they are processed by the Lambda function. The high-priority message must be processed first.
topicarn=$(aws sns list-topics --query "Topics[?contains(TopicArn, 'datacenter-Priority-Queues-Topic')].TopicArn" --output text)
aws sns publish --topic-arn $topicarn --message 'High Priority message 1' --message-attributes '{"priority" : { "DataType":"String", "StringValue":"high"}}'
aws sns publish --topic-arn $topicarn --message 'High Priority message 2' --message-attributes '{"priority" : { "DataType":"String", "StringValue":"high"}}'
aws sns publish --topic-arn $topicarn --message 'Low Priority message 1' --message-attributes '{"priority" : { "DataType":"String", "StringValue":"low"}}'
aws sns publish --topic-arn $topicarn --message 'Low Priority message 2' --message-attributes '{"priority" : { "DataType":"String", "StringValue":"low"}}'
### Solution
# SQS/SNS Priority Queue via CloudFormation Task (datacenter) — KodeKloud-Constrained
A CloudFormation task — deploy SNS → (filtered fan-out) → two SQS queues, plus a Lambda that drains high-priority first. The priority logic is: SNS **message-attribute filter policies** route `priority=high` messages to the high queue and `priority=low` to the low queue; the Lambda (provided `index.py`) reads the high queue before the low queue on invocation.
> ⚠️ **KodeKloud + CloudFormation caveat (HIGHLIGHT):** KodeKloud's general FAQ warns that arbitrary CloudFormation/Terraform "will almost certainly not work due to IAM restrictions." **This task explicitly requires CloudFormation**, which means the lab has curated the IAM permissions for *this specific* scenario. So it works here — but it's why the IAM role below uses **managed policies, not inline** (inline `PutRolePolicy` is the restricted path). Don't add custom/inline policies to the role.
## Where KodeKloud limits are specifically applied
| Limit | Where it lands in this task | Highlighted below |
|---|---|---|
| **Lambda timeout ≤ 10s** | `Timeout: 10` on the function | ⚠️ HIGHLIGHT |
| **Lambda memory ≤ 256 MB** | `MemorySize: 256` on the function | ⚠️ HIGHLIGHT |
| **IAM: managed policies only, no inline/custom** | Role uses `ManagedPolicyArns` | ⚠️ HIGHLIGHT |
| **IAM: can't exceed login user's scope** | AWS-managed policies (SQS/SNS/Logs Full) are within scope | ⚠️ HIGHLIGHT |
| **Lambda config set at create, not updated later** | All props set in the template | ⚠️ HIGHLIGHT |
| SQS/SNS "basic operations only" | Standard queues, standard topic — nothing advanced | (implicit) |
## Step 1 — Write the template (everything except the embedded code)
The Lambda code from `/root/index.py` gets **injected** into the template (inline `ZipFile`) in Step 2, so here we write the template *up to* the `ZipFile: |` line, with the Lambda as the last resource and `Code` its last property.
```bash
cat > /root/datacenter-priority-stack.yml << 'YAML'
AWSTemplateFormatVersion: '2010-09-09'
Description: Priority queuing with SNS filter-policy fan-out to SQS + Lambda consumer
Resources:
HighPriorityQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: datacenter-High-Priority-Queue
LowPriorityQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: datacenter-Low-Priority-Queue
PriorityTopic:
Type: AWS::SNS::Topic
Properties:
TopicName: datacenter-Priority-Queues-Topic
HighSubscription:
Type: AWS::SNS::Subscription
Properties:
TopicArn: !Ref PriorityTopic
Protocol: sqs
Endpoint: !GetAtt HighPriorityQueue.Arn
RawMessageDelivery: true
FilterPolicy:
priority:
- high
LowSubscription:
Type: AWS::SNS::Subscription
Properties:
TopicArn: !Ref PriorityTopic
Protocol: sqs
Endpoint: !GetAtt LowPriorityQueue.Arn
RawMessageDelivery: true
FilterPolicy:
priority:
- low
# SPLIT: one QueuePolicy per queue — SQS requires exactly one resource per statement
HighQueuePolicy:
Type: AWS::SQS::QueuePolicy
Properties:
Queues:
- !Ref HighPriorityQueue
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal: "*"
Action: sqs:SendMessage
Resource: !GetAtt HighPriorityQueue.Arn
Condition:
ArnEquals:
aws:SourceArn: !Ref PriorityTopic
LowQueuePolicy:
Type: AWS::SQS::QueuePolicy
Properties:
Queues:
- !Ref LowPriorityQueue
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal: "*"
Action: sqs:SendMessage
Resource: !GetAtt LowPriorityQueue.Arn
Condition:
ArnEquals:
aws:SourceArn: !Ref PriorityTopic
LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
RoleName: lambda_execution_role
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonSQSFullAccess
- arn:aws:iam::aws:policy/AmazonSNSFullAccess
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
PriorityLambda:
Type: AWS::Lambda::Function
Properties:
FunctionName: datacenter-priorities-queue-function
Handler: index.lambda_handler
Runtime: python3.13
Role: !GetAtt LambdaExecutionRole.Arn
Timeout: 10
MemorySize: 256
Environment:
Variables:
high_priority_queue: !Ref HighPriorityQueue
low_priority_queue: !Ref LowPriorityQueue
Code:
ZipFile: |
YAML
```
⚠️ **HIGHLIGHTED limit applications in that template:**
- **`Timeout: 10`** — KodeKloud auto-resets Lambda timeouts > 10s to 3s and suspends the session for > 30s. 10 is the ceiling. **This is a hard limit you must set here.**
- **`MemorySize: 256`** — the lab cap; > 256 gets reset to 128. Set it explicitly.
- **`ManagedPolicyArns` instead of inline `Policies`** — the lab restricts custom/inline policy creation (your `PutRolePolicy` denial earlier). Attaching AWS-managed policies is permitted because they don't exceed your login user's scope. `AmazonSQSFullAccess` + `AmazonSNSFullAccess` + `AWSLambdaBasicExecutionRole` cover "interact with SQS and SNS" + logging.
- All function config is set **at create time in the template** — KodeKloud blocks post-hoc `update-function-configuration`, so getting it right in the stack matters.
## Step 2 — Inject `/root/index.py` into the template (indented under `ZipFile: |`)
The inline `ZipFile` block scalar needs each code line indented to 10 spaces (deeper than the `ZipFile: |` at 8). `sed` handles it; appending is all that's needed since the Lambda is the last resource.
```bash
sed 's/^/ /' /root/index.py >> /root/datacenter-priority-stack.yml
# Sanity-check the tail of the template shows your indented code
tail -20 /root/datacenter-priority-stack.yml
```
Two things to verify:
- **Handler match:** the template uses `Handler: index.lambda_handler`. If the function inside `/root/index.py` is named something other than `lambda_handler`, change the handler to `index.<actual_function_name>`. Check with `grep '^def ' /root/index.py`.
- **Inline code size limit:** CloudFormation inline `ZipFile` caps at ~4096 bytes. `wc -c /root/index.py` — if it's over 4 KB, you can't inline it; you'd zip it to S3 and use `Code: {S3Bucket, S3Key}` instead. A simple priority processor is almost always well under 4 KB.
## Step 3 — Deploy the stack
```bash
REGION=us-east-1
aws cloudformation deploy \
--template-file /root/datacenter-priority-stack.yml \
--stack-name datacenter-priority-stack \
--capabilities CAPABILITY_NAMED_IAM \
--region $REGION
```
⚠️ **`--capabilities CAPABILITY_NAMED_IAM` is mandatory** — the stack creates an IAM role with an *explicit name* (`lambda_execution_role`). CloudFormation refuses to create named IAM resources without this acknowledgment (`CAPABILITY_IAM` alone isn't enough for *named* ones). Omit it and the deploy fails with an `InsufficientCapabilities` error.
## Step 4 — Test (publish to SNS, invoke Lambda)
```bash
topicarn=$(aws sns list-topics \
--query "Topics[?contains(TopicArn, 'datacenter-Priority-Queues-Topic')].TopicArn" \
--output text)
aws sns publish --topic-arn $topicarn --message 'High Priority message 1' \
--message-attributes '{"priority":{"DataType":"String","StringValue":"high"}}'
aws sns publish --topic-arn $topicarn --message 'High Priority message 2' \
--message-attributes '{"priority":{"DataType":"String","StringValue":"high"}}'
aws sns publish --topic-arn $topicarn --message 'Low Priority message 1' \
--message-attributes '{"priority":{"DataType":"String","StringValue":"low"}}'
aws sns publish --topic-arn $topicarn --message 'Low Priority message 2' \
--message-attributes '{"priority":{"DataType":"String","StringValue":"low"}}'
# Give SNS a couple seconds to fan out to the queues, then invoke the Lambda
sleep 5
aws lambda invoke --function-name datacenter-priorities-queue-function \
--region $REGION /tmp/out.json
cat /tmp/out.json
```
⚠️ **Don't loop the invoke** — KodeKloud deletes any Lambda invoked **> 300 times in an hour**. A handful of test invokes is fine; don't script a hammer loop.
The `priority` message attribute is what the SNS **filter policy** keys on: `high` messages match `HighSubscription`'s filter and land in the high queue; `low` in the low queue. The Lambda reads high first, so high-priority messages process ahead of low — the task's success condition.
## Verify
```bash
# Stack created cleanly
aws cloudformation describe-stacks --stack-name datacenter-priority-stack \
--region $REGION --query 'Stacks[0].StackStatus'
# All resources present
aws cloudformation describe-stack-resources --stack-name datacenter-priority-stack \
--region $REGION --query 'StackResources[].{Type:ResourceType,Status:ResourceStatus,Id:PhysicalResourceId}' \
--output table
# Messages fanned out to the queues (before Lambda drains them)
for Q in datacenter-High-Priority-Queue datacenter-Low-Priority-Queue; do
URL=$(aws sqs get-queue-url --queue-name $Q --region $REGION --query QueueUrl --output text)
echo "$Q:"; aws sqs get-queue-attributes --queue-url $URL \
--attribute-names ApproximateNumberOfMessages --region $REGION \
--query 'Attributes.ApproximateNumberOfMessages' --output text
done
# Lambda processing order — check its logs
aws logs tail /aws/lambda/datacenter-priorities-queue-function --region $REGION --since 5m
```
Want: stack `CREATE_COMPLETE`, all resources `CREATE_COMPLETE`, messages present in both queues after publish, and the Lambda logs showing **high-priority messages processed before low**. That ordering in the logs is the task's proof.
## Debug order
1. **Deploy fails `InsufficientCapabilities`** → missing `--capabilities CAPABILITY_NAMED_IAM`.
2. **Deploy fails on the IAM role (`AccessDenied`/`not authorized`)** → the lab didn't grant role creation for this scenario, *or* you left inline `Policies` in the template. Confirm you're using `ManagedPolicyArns` only. If it still fails, the role may be pre-created — remove the role resource and reference the existing `lambda_execution_role` ARN in the function's `Role`.
3. **Messages don't reach the queues** → SNS filter policy attribute mismatch (must be `priority`), or the `QueuePolicy` didn't allow SNS `SendMessage`. Also `RawMessageDelivery: true` matters so the SQS message body is the raw message, not SNS-wrapped JSON — check what `index.py` expects.
4. **Lambda errors on invoke** → `ImportModuleError` (handler/filename mismatch — must be `index.lambda_handler` for `index.py`), or `AccessDenied` (role missing SQS/SNS perms — the managed policies cover it).
5. **Inline code rejected at deploy** → `index.py` > 4096 bytes; switch to S3-packaged `Code`.
## Full KodeKloud compliance checklist
| Constraint | Applied |
|---|---|
| Region us-east-1 | ✅ |
| Lambda timeout ≤ 10s | ⚠️✅ `Timeout: 10` |
| Lambda memory ≤ 256 MB | ⚠️✅ `MemorySize: 256` |
| IAM managed policies (no inline/custom) | ⚠️✅ `ManagedPolicyArns` |
| IAM within login-user scope | ⚠️✅ AWS-managed policies |
| Config at create, not updated later | ⚠️✅ all in template |
| SQS/SNS basic ops only (standard) | ✅ standard queue/topic |
| ≤ 300 Lambda invocations/hr | ⚠️✅ single test invoke |
| CFN named-IAM capability | ✅ `CAPABILITY_NAMED_IAM` |

128
aws-48.md Normal file
View File

@@ -0,0 +1,128 @@
## Task 48
The Nautilus DevOps team needs to implement a Lambda function using a CloudFormation stack. Create a CloudFormation template named /root/devops-lambda.yml on the AWS client host and configure it to create the following components. The stack name must be devops-lambda-app.
Create a Lambda function named devops-lambda.
Use the Runtime Python.
The function should print the body Welcome to KKE AWS Labs!.
Ensure the status code is 200.
Create and use the IAM role named lambda_execution_role.
### Solution
# Lambda via CloudFormation Task (devops-lambda-app) — KodeKloud-Constrained
A simple single-Lambda CloudFormation stack. The only real substance is the two-resource template (role + function) with inline code, plus the KodeKloud limits baked in from the start.
## KodeKloud limits applied to this task
| Limit | Where | Why |
|---|---|---|
| **Lambda timeout ≤ 10s** | `Timeout: 10` | > 10s auto-reset to 3s; > 30s suspends session |
| **Lambda memory ≤ 256 MB** | `MemorySize: 256` | > 256 auto-reset to 128 |
| **Config set at CREATE time** | all in the template | post-hoc Lambda config updates are blocked |
| **IAM: managed policies only** | role uses `ManagedPolicyArns` | inline/custom policy creation is denied |
| **IAM within login scope** | `AWSLambdaBasicExecutionRole` (AWS-managed) | can't exceed the `kk_labs_user` scope |
| **Region us-east-1** | deploy `--region us-east-1` | only allowed region |
| **CFN named-IAM capability** | `CAPABILITY_NAMED_IAM` | stack creates a *named* IAM role |
## Step 1 — Write the template
```bash
cat > /root/devops-lambda.yml << 'YAML'
AWSTemplateFormatVersion: '2010-09-09'
Description: devops-lambda function with execution role
Resources:
# IAM role — MANAGED policy only (KodeKloud-safe), named as the task requires
LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
RoleName: lambda_execution_role
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
# Lambda function — inline code, KodeKloud timeout/memory caps
DevopsLambda:
Type: AWS::Lambda::Function
Properties:
FunctionName: devops-lambda
Handler: index.lambda_handler
Runtime: python3.13
Role: !GetAtt LambdaExecutionRole.Arn
Timeout: 10
MemorySize: 256
Code:
ZipFile: |
def lambda_handler(event, context):
return {
'statusCode': 200,
'body': 'Welcome to KKE AWS Labs!'
}
Outputs:
FunctionName:
Value: !Ref DevopsLambda
RoleArn:
Value: !GetAtt LambdaExecutionRole.Arn
YAML
```
Every requirement mapped:
- **`FunctionName: devops-lambda`** — req 1.
- **`Runtime: python3.13`** — req 2 ("Python"). 3.13 is current/supported; 3.14 is the newest if the grader wants absolute latest, but 3.13 is the safe pick. (Avoid 3.10/3.11 — AL2 deprecation track.)
- **The inline handler returns `statusCode: 200` + `body: 'Welcome to KKE AWS Labs!'`** — reqs 3 & 4. "Print the body" maps to the returned `body` field (the invocation response payload), exactly matching the earlier console-Lambda task.
- **`LambdaExecutionRole` named `lambda_execution_role`** with `AWSLambdaBasicExecutionRole` attached — req 5. That managed policy grants CloudWatch Logs write (all a bare Lambda needs), and being AWS-managed it complies with the KodeKloud "no inline/custom, stay within login scope" rule.
- **`Handler: index.lambda_handler`** — CloudFormation's inline `ZipFile` is written to a file named `index.py` internally, so the handler is `index.<function>`. The function is `lambda_handler`, hence `index.lambda_handler`. (No hyphen problem here — inline code, not an uploaded `lambda-function.py`.)
- **`Timeout: 10` / `MemorySize: 256`** — KodeKloud caps, set at create.
## Step 2 — Deploy
```bash
REGION=us-east-1
aws cloudformation deploy \
--template-file /root/devops-lambda.yml \
--stack-name devops-lambda-app \
--capabilities CAPABILITY_NAMED_IAM \
--region $REGION
```
⚠️ **`--capabilities CAPABILITY_NAMED_IAM` is mandatory** — the stack creates an IAM role with an explicit name (`lambda_execution_role`). Without it, deploy fails `InsufficientCapabilities`.
## Step 3 — Verify
```bash
# Stack created cleanly
aws cloudformation describe-stacks --stack-name devops-lambda-app \
--region $REGION --query 'Stacks[0].StackStatus'
# Function exists with correct config
aws lambda get-function-configuration --function-name devops-lambda \
--region $REGION \
--query '{Name:FunctionName,Runtime:Runtime,Timeout:Timeout,Memory:MemorySize,Role:Role}'
# Invoke and confirm the response body + status code
aws lambda invoke --function-name devops-lambda --region $REGION /tmp/out.json
cat /tmp/out.json
```
Want: stack `CREATE_COMPLETE`; the function showing `python3.13`, `Timeout: 10`, `Memory: 256`, role ending `lambda_execution_role`; and the invoke returning `{"statusCode": 200, "body": "Welcome to KKE AWS Labs!"}`. That payload is the task's proof.
## Debug order
1. **Deploy `InsufficientCapabilities`** → add `--capabilities CAPABILITY_NAMED_IAM`.
2. **Stack `ROLLBACK_COMPLETE` on the role** → the lab may pre-create `lambda_execution_role`, causing a name collision (`already exists`). Check `aws iam get-role --role-name lambda_execution_role`; if it exists, remove the `LambdaExecutionRole` resource from the template and set the function's `Role` to that role's ARN directly. Then — since `ROLLBACK_COMPLETE` is terminal — `aws cloudformation delete-stack --stack-name devops-lambda-app && aws cloudformation wait stack-delete-complete ...` before redeploying.
3. **Timeout became 3s / memory 128** → you exceeded the caps somewhere; confirm `Timeout: 10` and `MemorySize: 256`.
4. **Invoke returns wrong body** → typo in the inline string; must be exactly `Welcome to KKE AWS Labs!`.
```

271
aws-49.md Normal file
View File

@@ -0,0 +1,271 @@
## Task 49
The Nautilus DevOps team needs to build a secure and scalable log aggregation setup within their AWS environment. The goal is to gather log files from an internal EC2 instance running in a private VPC, transfer them securely to another EC2 instance in a public VPC, and then push those logs to a secure S3 bucket.
1) A VPC named nautilus-priv-vpc already exists with a private subnet named nautilus-priv-subnet, a route table named nautilus-priv-rt, and an EC2 instance named nautilus-priv-ec2 (using ubuntu image). This instance uses the SSH key pair nautilus-key.pem already available on the AWS client host at /root/.ssh/.
2) Your task is to:
Create a new VPC named nautilus-pub-vpc.
Create a subnet named nautilus-pub-subnet and a route table named nautilus-pub-rt under this public VPC.
Attach an internet gateway to nautilus-pub-vpc and configure the public route table to enable internet access.
Launch an EC2 instance named nautilus-pub-ec2 into the public subnet using the same key pair as the private instance.
Create an IAM role named nautilus-s3-role with PutObject permission to an S3 bucket and attach it to the public EC2 instance.
Create a new private S3 bucket named nautilus-s3-logs-29442.
Configure a VPC Peering named nautilus-vpc-peering between the private and public VPCs.
Modify both nautilus-priv-rt and nautilus-pub-rt to route each other's CIDR blocks through the peering connection.
On the private instance, configure a cron job to push the /var/log/boots.log file to the public instance (using scp or rsync).
On the public instance, configure a cron job to push that same file to the created S3 bucket.
The uploaded file must be stored in the S3 bucket under the path nautilus-priv-vpc/boot/boots.log.
### Solution
# Log Aggregation Pipeline Task (devops) — KodeKloud-Constrained, jump-host + aws-cli-validated
Full cross-VPC log pipeline: `devops-priv-ec2` (private VPC, **no public IP / no internet**) → **scp over VPC peering** → `devops-pub-ec2` (public VPC) → **aws s3 cp via IAM role** → private S3 bucket.
Because the private instance has no public IP, all *setup* access to it goes **through `devops-pub-ec2` as a jump host** over peering:
`aws-client → (public IP) → devops-pub-ec2 → (private IP via peering) → devops-priv-ec2`
The actual pipeline hops are independent of the jump: private→public scp runs over peering; public→S3 runs via the instance role.
## KodeKloud limits applied
EC2 t2.micro + Standard credits · IAM managed policy (`AmazonS3FullAccess`, no custom) · check-for-pre-created-role · us-east-1 · S3 private by default.
---
## Phase 1 — Discover the private side
```bash
REGION=us-east-1
PRIV_VPC=$(aws ec2 describe-vpcs --filters Name=tag:Name,Values=devops-priv-vpc \
--region $REGION --query 'Vpcs[0].VpcId' --output text)
PRIV_CIDR=$(aws ec2 describe-vpcs --vpc-ids $PRIV_VPC --region $REGION \
--query 'Vpcs[0].CidrBlock' --output text)
PRIV_RT=$(aws ec2 describe-route-tables --filters "Name=tag:Name,Values=devops-priv-rt" \
--region $REGION --query 'RouteTables[0].RouteTableId' --output text)
PRIV_IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=devops-priv-ec2" "Name=instance-state-name,Values=running,stopped" \
--region $REGION --query 'Reservations[0].Instances[0].InstanceId' --output text)
PRIV_EC2_IP=$(aws ec2 describe-instances --instance-ids $PRIV_IID --region $REGION \
--query 'Reservations[0].Instances[0].PrivateIpAddress' --output text)
PRIV_SG=$(aws ec2 describe-instances --instance-ids $PRIV_IID --region $REGION \
--query 'Reservations[0].Instances[0].SecurityGroups[0].GroupId' --output text)
echo "PRIV_VPC=$PRIV_VPC CIDR=$PRIV_CIDR RT=$PRIV_RT instance=$PRIV_IID privIP=$PRIV_EC2_IP SG=$PRIV_SG"
```
## Phase 2 — Build the public VPC (non-overlapping CIDR)
```bash
PUB_CIDR=10.1.0.0/16
[ "$PRIV_CIDR" = "10.1.0.0/16" ] && PUB_CIDR=10.2.0.0/16
PUB_VPC=$(aws ec2 create-vpc --cidr-block $PUB_CIDR --region $REGION \
--tag-specifications 'ResourceType=vpc,Tags=[{Key=Name,Value=devops-pub-vpc}]' \
--query 'Vpc.VpcId' --output text)
PUB_SUBNET=$(aws ec2 create-subnet --vpc-id $PUB_VPC --cidr-block 10.1.1.0/24 --region $REGION \
--tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=devops-pub-subnet}]' \
--query 'Subnet.SubnetId' --output text)
aws ec2 modify-subnet-attribute --subnet-id $PUB_SUBNET --map-public-ip-on-launch --region $REGION
IGW=$(aws ec2 create-internet-gateway --region $REGION \
--tag-specifications 'ResourceType=internet-gateway,Tags=[{Key=Name,Value=devops-pub-igw}]' \
--query 'InternetGateway.InternetGatewayId' --output text)
aws ec2 attach-internet-gateway --internet-gateway-id $IGW --vpc-id $PUB_VPC --region $REGION
PUB_RT=$(aws ec2 create-route-table --vpc-id $PUB_VPC --region $REGION \
--tag-specifications 'ResourceType=route-table,Tags=[{Key=Name,Value=devops-pub-rt}]' \
--query 'RouteTable.RouteTableId' --output text)
aws ec2 create-route --route-table-id $PUB_RT --destination-cidr-block 0.0.0.0/0 \
--gateway-id $IGW --region $REGION
aws ec2 associate-route-table --route-table-id $PUB_RT --subnet-id $PUB_SUBNET --region $REGION
```
> Adjust the subnet block to fit your chosen `$PUB_CIDR`.
## Phase 3 — Private S3 bucket
```bash
BUCKET=devops-s3-logs-3826
aws s3api create-bucket --bucket $BUCKET --region $REGION
# Private by default (BPA on).
```
## Phase 4 — IAM role devops-s3-role (managed policy, KK-safe) + instance profile
```bash
EXISTING=$(aws iam get-role --role-name devops-s3-role --query 'Role.Arn' --output text 2>/dev/null || echo MISSING)
if [ "$EXISTING" = "MISSING" ]; then
cat > /tmp/ec2-trust.json << 'EOF'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}
EOF
aws iam create-role --role-name devops-s3-role --assume-role-policy-document file:///tmp/ec2-trust.json
fi
aws iam attach-role-policy --role-name devops-s3-role \
--policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess
aws iam create-instance-profile --instance-profile-name devops-s3-role 2>/dev/null || true
aws iam add-role-to-instance-profile --instance-profile-name devops-s3-role --role-name devops-s3-role 2>/dev/null || true
```
> KK: task wants "PutObject" — `AmazonS3FullAccess` (AWS-managed, allowed) includes it. Custom scoped `s3:PutObject` policy is usually blocked by `CreatePolicy`; use it only if your lab permits and the grader demands scoped least-privilege.
## Phase 5 — Launch the public EC2 (t2.micro, same key, instance profile, jump-capable SG)
```bash
PUB_SG=$(aws ec2 create-security-group --group-name devops-pub-sg \
--description "public jump + log-agg" --vpc-id $PUB_VPC --region $REGION --query 'GroupId' --output text)
# SSH from anywhere so this box can serve as your jump host
aws ec2 authorize-security-group-ingress --group-id $PUB_SG --protocol tcp --port 22 \
--cidr 0.0.0.0/0 --region $REGION
sleep 8
PUB_IID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/canonical/ubuntu/server/noble/stable/current/amd64/hvm/ebs-gp3/ami-id \
--instance-type t2.micro --key-name devops-key --subnet-id $PUB_SUBNET \
--security-group-ids $PUB_SG --iam-instance-profile Name=devops-s3-role --region $REGION \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=devops-pub-ec2}]' \
--query 'Instances[0].InstanceId' --output text)
aws ec2 wait instance-running --instance-ids $PUB_IID --region $REGION
PUB_EC2_PUBIP=$(aws ec2 describe-instances --instance-ids $PUB_IID --region $REGION \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
PUB_EC2_PRIVIP=$(aws ec2 describe-instances --instance-ids $PUB_IID --region $REGION \
--query 'Reservations[0].Instances[0].PrivateIpAddress' --output text)
echo "public: pub=$PUB_EC2_PUBIP priv=$PUB_EC2_PRIVIP"
```
- **`--key-name devops-key`** — same pair as the private instance (`.pem` is `devops-key.pem`).
- **`--iam-instance-profile Name=devops-s3-role`** — role at launch for the S3 push.
- **t2.micro, Standard credits** — KK compliant.
## Phase 6 — Peering + routes both directions
```bash
PCX=$(aws ec2 create-vpc-peering-connection --vpc-id $PUB_VPC --peer-vpc-id $PRIV_VPC --region $REGION \
--tag-specifications 'ResourceType=vpc-peering-connection,Tags=[{Key=Name,Value=devops-vpc-peering}]' \
--query 'VpcPeeringConnection.VpcPeeringConnectionId' --output text)
aws ec2 accept-vpc-peering-connection --vpc-peering-connection-id $PCX --region $REGION
aws ec2 create-route --route-table-id $PUB_RT --destination-cidr-block $PRIV_CIDR \
--vpc-peering-connection-id $PCX --region $REGION
aws ec2 create-route --route-table-id $PRIV_RT --destination-cidr-block $PUB_CIDR \
--vpc-peering-connection-id $PCX --region $REGION
```
**Routes on BOTH tables — non-negotiable.**
## Phase 6.5 — Open private SG for the jump + define the ProxyCommand
```bash
# Private instance must accept SSH from the public VPC CIDR (jump traffic over peering)
aws ec2 authorize-security-group-ingress --group-id $PRIV_SG \
--protocol tcp --port 22 --cidr $PUB_CIDR --region $REGION 2>/dev/null || echo "already allowed"
KEY=/root/.ssh/devops-key.pem
JUMP="ssh -i $KEY -o StrictHostKeyChecking=no -W %h:%p ubuntu@$PUB_EC2_PUBIP"
```
## Phase 7 — Private instance (via jump): place key + install scp cron
```bash
# 7a: copy the key onto the private instance THROUGH the jump
scp -i $KEY -o StrictHostKeyChecking=no -o ProxyCommand="$JUMP" \
$KEY ubuntu@$PRIV_EC2_IP:/home/ubuntu/.ssh/devops-key.pem
# 7b: install the scp cron (targets the public instance's PRIVATE IP over peering)
ssh -i $KEY -o StrictHostKeyChecking=no -o ProxyCommand="$JUMP" ubuntu@$PRIV_EC2_IP bash -s << EOF
chmod 600 /home/ubuntu/.ssh/devops-key.pem
echo "* * * * * scp -i /home/ubuntu/.ssh/devops-key.pem -o StrictHostKeyChecking=no /var/log/boots.log ubuntu@$PUB_EC2_PRIVIP:/home/ubuntu/boots.log" | crontab -
echo "--- private crontab ---"; crontab -l
EOF
```
> If `/var/log/boots.log` doesn't exist on the private instance, the pipeline moves nothing. Confirm it's there (the standard Ubuntu file is `boot.log` singular — the task uses `boots.log`, so it may need creating): `ls -l /var/log/boots.log`.
## Phase 8 — Public instance: VALIDATE + install aws CLI, then the S3 cron
This is the step that bit us before — the cron uses `aws s3 cp`, so aws CLI **must** be present, and cron's minimal PATH must find it.
```bash
ssh -i $KEY -o StrictHostKeyChecking=no ubuntu@$PUB_EC2_PUBIP bash -s << 'EOF'
# --- VALIDATE aws CLI; install only if missing ---
if command -v aws >/dev/null 2>&1; then
echo "aws CLI already present: $(aws --version 2>&1)"
else
echo "aws CLI missing — installing..."
sudo apt-get update -y
sudo apt-get install -y awscli
# fallback if the apt package is unavailable on this Ubuntu release:
if ! command -v aws >/dev/null 2>&1; then
sudo apt-get install -y unzip curl
curl -s "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o /tmp/awscliv2.zip
cd /tmp && unzip -q awscliv2.zip && sudo ./aws/install
fi
fi
AWS_BIN=$(command -v aws)
echo "aws resolved at: $AWS_BIN"
EOF
```
Then install the cron **using the full binary path + an explicit PATH line** (cron's PATH is minimal — `/usr/bin:/bin` — and won't find a `/usr/local/bin/aws` from the v2 installer otherwise):
```bash
ssh -i $KEY -o StrictHostKeyChecking=no ubuntu@$PUB_EC2_PUBIP bash -s << EOF
AWS_BIN=\$(command -v aws)
( echo "PATH=/usr/local/bin:/usr/bin:/bin"; \
echo "* * * * * \$AWS_BIN s3 cp /home/ubuntu/boots.log s3://$BUCKET/devops-priv-vpc/boot/boots.log --region $REGION" ) | crontab -
echo "--- public crontab ---"; crontab -l
EOF
```
- **No `aws configure`** — role creds come from IMDS.
- **S3 key path `devops-priv-vpc/boot/boots.log`** set literally (S3 "folders" are prefixes).
## Phase 8.5 — Prove both hops manually before waiting on cron
```bash
# Hop 1 delivered? (file present on public instance)
ssh -i $KEY -o StrictHostKeyChecking=no ubuntu@$PUB_EC2_PUBIP \
"ls -l /home/ubuntu/boots.log; echo exit=\$?"
# Hop 2 works? (manual upload, confirms aws CLI + role + path)
ssh -i $KEY -o StrictHostKeyChecking=no ubuntu@$PUB_EC2_PUBIP \
"aws s3 cp /home/ubuntu/boots.log s3://$BUCKET/devops-priv-vpc/boot/boots.log --region $REGION; echo exit=\$?"
```
Both `exit=0` means the crons will succeed too. If hop 1's `ls` shows no file, the private→public scp isn't working (check peering route on the private RT + the key on the private box).
## Verify
```bash
aws ec2 describe-vpc-peering-connections --vpc-peering-connection-ids $PCX --region $REGION \
--query 'VpcPeeringConnections[0].Status.Code'
aws ec2 describe-route-tables --route-table-ids $PUB_RT $PRIV_RT --region $REGION \
--query 'RouteTables[].Routes[?VpcPeeringConnectionId!=null].DestinationCidrBlock'
aws ec2 describe-instances --instance-ids $PUB_IID --region $REGION \
--query 'Reservations[0].Instances[0].IamInstanceProfile.Arn'
for n in $(seq 1 6); do
aws s3 ls s3://$BUCKET/devops-priv-vpc/boot/boots.log --region $REGION 2>/dev/null && { echo "SUCCESS"; break; }
echo "waiting for cron pipeline... ($n)"; sleep 30
done
```
Want peering `active`, both peering routes present, the public instance showing `devops-s3-role`, and the object at `s3://devops-s3-logs-3826/devops-priv-vpc/boot/boots.log`.
## Debug order
1. **`aws: command not found` in public cron** → aws CLI missing or not in cron's PATH. Phase 8's validate/install + the `PATH=` line + full `$AWS_BIN` path fix both. Verify with `grep CRON /var/log/syslog | tail` on the public box.
2. **Hop 1 file absent on public instance** → private RT missing `$PUB_CIDR → pcx`, public SG not allowing 22 from `$PRIV_CIDR` (it allows 0.0.0.0/0 here, so usually the route), or `devops-key.pem` not on the private box / wrong perms.
3. **S3 cp AccessDenied** → role not attached (`IamInstanceProfile.Arn` null) or managed policy not attached to the role.
4. **`/var/log/boots.log` missing** on private instance → nothing to move; create it.
5. **Jump SSH fails** → test hops separately: `ssh -i $KEY ubuntu@$PUB_EC2_PUBIP` must work first; then the ProxyCommand hop needs the private SG open to `$PUB_CIDR` + the peering routes.
6. **Wrong user** → Ubuntu image = `ubuntu@` everywhere.
## Skills used
`kodekloud-aws-limits` — drove t2.micro/Standard-credits, managed-policy IAM (`AmazonS3FullAccess` over a scoped custom policy since KK blocks `CreatePolicy`), check-for-pre-created-role, us-east-1, S3-private-by-default. The jump-host ProxyCommand and the aws-CLI validate/install are standard ops patterns, not skill-driven.

128
aws-50.md Normal file
View File

@@ -0,0 +1,128 @@
## Task 50
The Nautilus DevOps Team has recently been informed by the Development Team that their EC2 instance is running out of storage space. This instance, crucial for development activities, is named xfusion-ec2 and currently has an attached volume of 8 GiB. To accommodate the increasing data requirements, the storage needs to be expanded to 12 GiB. This change should ensure that the expanded space is immediately available for use within the instance without disrupting ongoing activities.
Identify Volume: Find the volume attached to the xfusion-ec2 instance.
Expand Volume: Increase the volume size from 8 GiB to 12 GiB.
Reflect Changes: Ensure the root (/) partition within the instance reflects the expanded size from 8 GiB to 12 GiB.
SSH Access: Use the key pair located at /root/xfusion-keypair.pem on the aws-client host to SSH into the EC2 instance.
### Solution
# EBS Volume Expansion Task (xfusion-ec2) — 8 GiB → 12 GiB
EBS volume expansion — the concept that trips everyone: **growing the EBS volume in AWS is only half the job.** The block device gets bigger, but the OS doesn't know until you grow the *partition* and then the *filesystem* on top of it. Three layers, three separate operations: **EBS volume → partition table → filesystem.** Miss the last two and `df -h` still shows 8 GiB despite the volume being 12.
Run on `aws-client`.
## Phase 1 — Identify the volume + resize the EBS layer
```bash
REGION=us-east-1
# Instance + its root volume
IID=$(aws ec2 describe-instances \
--filters "Name=tag:Name,Values=xfusion-ec2" "Name=instance-state-name,Values=running,stopped" \
--region $REGION --query 'Reservations[0].Instances[0].InstanceId' --output text)
VOL_ID=$(aws ec2 describe-instances --instance-ids $IID --region $REGION \
--query 'Reservations[0].Instances[0].BlockDeviceMappings[0].Ebs.VolumeId' --output text)
PUB_IP=$(aws ec2 describe-instances --instance-ids $IID --region $REGION \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
echo "instance=$IID volume=$VOL_ID ip=$PUB_IP"
# Grow the EBS volume 8 -> 12 GiB (online, no detach, no stop)
aws ec2 modify-volume --volume-id $VOL_ID --size 12 --region $REGION
# Wait for the modification to finish optimizing
aws ec2 describe-volumes-modifications --volume-ids $VOL_ID --region $REGION \
--query 'VolumesModifications[0].{State:ModificationState,Progress:Progress,Size:TargetSize}'
```
Key points on the EBS layer:
- **`modify-volume` is online** — no detach, no stop, no disruption. Satisfies the task's "without disrupting ongoing activities." The volume grows live while the instance runs.
- **The modification goes through states** `modifying → optimizing → completed`. The **new size is usable as soon as it hits `optimizing`** (usually seconds) — you don't have to wait for `completed` (which can take a while as it re-lays-out storage in the background). Poll `describe-volumes-modifications` until `State` is `optimizing` or `completed`.
- **6-hour cooldown:** you can't modify the same volume again for 6 hours after a change. Not relevant here, but it exists.
## Phase 2 — SSH in and grow the partition + filesystem
SSH using the specified key:
```bash
chmod 400 /root/xfusion-keypair.pem
ssh -i /root/xfusion-keypair.pem -o StrictHostKeyChecking=no ubuntu@$PUB_IP
```
> User is `ubuntu` for an Ubuntu image; `ec2-user` for Amazon Linux. Check what the instance runs and swap if needed.
Once on the instance, inspect the mismatch:
```bash
lsblk # disk shows 12G, but its partition still 8G
df -h / # / still 8G
```
`lsblk` shows the disk (e.g. `nvme0n1` or `xvda`) at **12G** but its partition (`nvme0n1p1` / `xvda1`) still at **8G**. Two grows needed:
```bash
# --- Grow the PARTITION to fill the disk ---
# Nitro/newer instances: device is /dev/nvme0n1, partition 1
sudo growpart /dev/nvme0n1 1
# (older Xen instances: sudo growpart /dev/xvda 1)
# --- Grow the FILESYSTEM to fill the partition ---
# ext4 (most common):
sudo resize2fs /dev/nvme0n1p1
# XFS (some AMIs, incl. many Amazon Linux): sudo xfs_growfs /
```
Confirm:
```bash
df -h / # / should now show ~12G
```
The three layers explained (this is the whole task):
- **`growpart`** rewrites the partition table so partition 1 extends to the end of the now-larger disk. Note the **space between device and partition number** — it's `growpart /dev/nvme0n1 1`, not `growpart /dev/nvme0n1p1`. Common syntax error.
- **`resize2fs` (ext4) / `xfs_growfs` (xfs)** grows the filesystem to fill the enlarged partition. This is the step that actually makes `df` report the new size. **Pick the right tool for your filesystem** — run `df -T /` or `lsblk -f` to see the fstype. `resize2fs` on an XFS volume (or vice versa) just errors.
- Both are **online operations** — no unmount, no reboot. The `/` filesystem grows live.
## Verify
From the instance:
```bash
lsblk # disk, partition, and fs all ~12G
df -h / # / shows ~12G
```
From `aws-client`, confirm the EBS side settled:
```bash
aws ec2 describe-volumes --volume-ids $VOL_ID --region $REGION \
--query 'Volumes[0].{Size:Size,State:State}'
aws ec2 describe-volumes-modifications --volume-ids $VOL_ID --region $REGION \
--query 'VolumesModifications[0].ModificationState'
```
Want: EBS `Size: 12`, modification `optimizing`/`completed`, `lsblk` partition at 12G, and `df -h /` showing `/` at ~12G. That last one — `/` reflecting 12G — is the task's actual success condition (step 3).
## Debug order
1. **`df` still shows 8G after `modify-volume`** → you only did the EBS layer. Do `growpart` + `resize2fs`/`xfs_growfs` on the instance.
2. **`growpart: NOCHANGE` / "already the maximum"** → partition already grown (ran twice), or the EBS modify hasn't reached `optimizing` yet — check `describe-volumes-modifications`, wait, retry.
3. **`resize2fs: Bad magic number`** → wrong filesystem tool; it's XFS, use `xfs_growfs /`.
4. **Wrong device name** → `nvme0n1` on Nitro, `xvda` on older Xen. `lsblk` shows which; use the actual name.
5. **SSH refused** → wrong user (`ubuntu` vs `ec2-user`), key perms (`chmod 400`), or SG doesn't allow 22 from aws-client.
## KodeKloud note
Nothing here trips a KodeKloud limit — no new resource, no IAM, no instance-type change; you're just growing an existing volume. The `kodekloud-aws-limits` constraints don't apply to a pure resize op (region is already us-east-1).