Files
Honza Novak fb1db7c9ea feat: multi-distro support and tagged golden snapshots
Add Alpine, Debian, and Ubuntu rootfs support to `init [distro]`.
Golden snapshots are now namespaced under `golden/<tag>/` so multiple
baselines can coexist. `spawn [tag] [N]` selects which snapshot to
clone from. Systemd-based distros (Debian, Ubuntu) get a fc-net-init
systemd unit; Alpine keeps its inittab-based init.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 20:48:43 +00:00

26 KiB
Raw Permalink Blame History

fc-orch Command Reference

fc-orch is a Firecracker microVM snapshot orchestrator. It creates a single "golden" VM, snapshots it at a stable boot state, then rapidly restores arbitrarily many clones from that snapshot. Clones share the golden memory file via Linux kernel MAP_PRIVATE copy-on-write, so the incremental cost of each clone is only its dirty pages and a private copy of the VM state file.

All commands require root privileges (sudo).


Table of Contents


Global Configuration

All tunables are set via environment variables. Every variable has a default; none are required unless you want non-default behavior.

Variable Default Description
FC_BIN firecracker Path or name of the Firecracker binary (resolved via $PATH)
FC_BASE_DIR /tmp/fc-orch Root working directory for all state
FC_KERNEL $FC_BASE_DIR/vmlinux Path to the kernel image
FC_KERNEL_URL Pinned Firecracker CI build (vmlinux-6.1.166) URL to download the kernel if FC_KERNEL is missing
FC_ROOTFS $FC_BASE_DIR/rootfs.ext4 Path to the base ext4 rootfs image
FC_VCPUS 1 Number of vCPUs per VM
FC_MEM_MIB 128 Memory per VM in MiB
FC_BRIDGE fcbr0 Host bridge name. Set to none to disable all networking
FC_BRIDGE_CIDR 172.30.0.1/24 IP address and prefix assigned to the host bridge
FC_GUEST_PREFIX 172.30.0 IP prefix for guest address allocation (used with FC_AUTO_NET_CONFIG)
FC_GUEST_GW 172.30.0.1 Default gateway advertised to guests (used with FC_AUTO_NET_CONFIG)
FC_AUTO_NET_CONFIG (unset) Set to 1 to automatically assign guest IPs via MMDS on clone start

Kernel boot arguments are hardcoded and not user-configurable:

console=ttyS0 reboot=k panic=1 pci=off i8042.noaux quiet loglevel=0

Directory Layout

After running all commands, $FC_BASE_DIR (/tmp/fc-orch by default) contains:

/tmp/fc-orch/
├── vmlinux                    # kernel image (shared, immutable)
├── rootfs.ext4                # base Alpine rootfs (shared, immutable)
├── golden/
│   ├── default/               # "default" tag directory
│   │   ├── api.sock           # Firecracker API socket (golden VM, transient)
│   │   ├── rootfs.ext4        # COW copy of base rootfs used by golden VM
│   │   ├── mem                # memory snapshot (read by all clones, never written)
│   │   └── vmstate            # VM state snapshot (golden reference)
│   └── <tag>/                 # other tagged snapshots
├── clones/
│   ├── 1/
│   │   ├── api.sock           # Firecracker API socket (clone 1)
│   │   ├── rootfs.ext4        # private COW copy of golden rootfs
│   │   └── vmstate            # private copy of golden vmstate
│   ├── 2/
│   │   └── ...
│   └── N/
│       └── ...
└── pids/
    ├── golden.pid             # PID of golden Firecracker process (transient)
    ├── clone-1.pid
    ├── clone-2.pid
    └── ...

Network Topology

When FC_BRIDGE is not none (the default), a Linux bridge and per-VM TAP devices are used:

  Internet
     │
  Host NIC (e.g. eth0)
     │
  iptables NAT MASQUERADE
     │
  Bridge: fcbr0 (172.30.0.1/24)
     ├── fctap0  (golden VM — exists only during golden command)
     ├── fctap1  (clone 1)
     ├── fctap2  (clone 2)
     └── fctapN  (clone N)

Each clone receives a unique TAP device and MAC address (AA:FC:00:00:XX:XX). The host-side bridge has NAT masquerading enabled so guests can reach the internet through the host's default route.

Set FC_BRIDGE=none to skip all network configuration. VMs will boot without a network interface.

Guest IP assignment

The rootfs init script brings eth0 up at the link layer only. Guests have no IP address by default. There are two ways to configure networking inside a VM:

Manual configuration (inside the VM console)

# Pick an unused IP in the bridge subnet — e.g. .11 for clone 1, .12 for clone 2
ip addr add 172.30.0.11/24 dev eth0
ip route add default via 172.30.0.1
echo "nameserver 1.1.1.1" > /etc/resolv.conf
ping 1.1.1.1   # verify

Manual config is ephemeral — it is lost when the clone is stopped. Use the automatic option below for persistent configuration.

Automatic configuration via MMDS (FC_AUTO_NET_CONFIG=1)

When FC_AUTO_NET_CONFIG=1 is set, the orchestrator uses the Firecracker Microvm Metadata Service (MMDS) to inject per-clone network config immediately after the VM starts. A small background daemon embedded in the rootfs (/sbin/fc-net-init) polls 169.254.169.254 and applies the config automatically — no manual steps needed.

IPs are assigned deterministically from FC_GUEST_PREFIX:

clone 1  → 172.30.0.11/24
clone 2  → 172.30.0.12/24
…
clone N  → 172.30.0.(10+N)/24

Usage:

sudo FC_AUTO_NET_CONFIG=1 ./fc-orch start

Within ~12 seconds of clone start, eth0 inside the VM will have the assigned IP, default route, and DNS (1.1.1.1) configured.

Note: FC_AUTO_NET_CONFIG requires fc-orch init and fc-orch golden to have been run (or re-run) after this feature was added, so that the fc-net-init daemon is present in the golden snapshot.


init

Purpose

Downloads the Linux kernel image and builds a minimal filesystem (Alpine, Debian, or Ubuntu). This command only needs to run once per distro; both artifacts are reused by golden invocations. init is idempotent — it skips any artifact that already exists on disk.

Usage

sudo ./fc-orch init [distro]

Where [distro] can be alpine (default), debian, or ubuntu.

Optional overrides:

sudo FC_KERNEL_URL=https://example.com/vmlinux FC_BASE_DIR=/data/fc ./fc-orch init

Prerequisites

  • dd, mkfs.ext4 (e2fsprogs), mount/umount (util-linux), tar must be in $PATH
  • Internet access to download kernel and Alpine tarball
  • Root privileges (required for mount)

Step-by-step execution

  1. Create base directory

    mkdir -p /tmp/fc-orch
    
  2. Download kernel (skipped if /tmp/fc-orch/vmlinux already exists)

    GET https://s3.amazonaws.com/spec.ccfc.min/firecracker-ci/20260408-ce2a467895c1-0/x86_64/vmlinux-6.1.166
    → /tmp/fc-orch/vmlinux
    
  3. Create empty ext4 image (skipped if /tmp/fc-orch/rootfs.ext4 already exists)

    dd if=/dev/zero of=/tmp/fc-orch/rootfs.ext4 bs=1M count=512 status=none
    
  4. Format as ext4

    mkfs.ext4 -qF /tmp/fc-orch/rootfs.ext4
    
  5. Mount the image

    mkdir -p /tmp/fc-orch/mnt
    mount -o loop /tmp/fc-orch/rootfs.ext4 /tmp/fc-orch/mnt
    
  6. Download Alpine minirootfs tarball

    GET https://dl-cdn.alpinelinux.org/alpine/v3.20/releases/x86_64/alpine-minirootfs-3.20.0-x86_64.tar.gz
    → /tmp/fc-orch/alpine-minirootfs-3.20.0-x86_64.tar.gz
    
  7. Extract Alpine into the mounted image

    tar xzf /tmp/fc-orch/alpine-minirootfs-3.20.0-x86_64.tar.gz -C /tmp/fc-orch/mnt
    
  8. Write /etc/init.d/rcS inside the mounted image

    #!/bin/sh
    mount -t proc proc /proc
    mount -t sysfs sys /sys
    mount -t devtmpfs devtmpfs /dev
    ip link set eth0 up 2>/dev/null
    
  9. Write /etc/inittab inside the mounted image

    ::sysinit:/etc/init.d/rcS
    ttyS0::respawn:/bin/sh
    

    This causes the guest to launch a shell on the serial console (ttyS0) and respawn it if it exits.

  10. Unmount the image

    umount /tmp/fc-orch/mnt
    

Outputs

Path Description
/tmp/fc-orch/vmlinux Linux kernel image for Firecracker
/tmp/fc-orch/rootfs.ext4 512 MiB Alpine Linux ext4 image

Error conditions

Error Cause Resolution
download kernel: ... Network failure or bad FC_KERNEL_URL Check connectivity; verify the URL
download alpine: ... Network failure downloading Alpine tarball Check connectivity
build rootfs: ... dd, mkfs.ext4, mount, or tar failed Ensure the tools are installed and you are running as root

golden

Purpose

Boots a fresh Firecracker VM from the base artifacts produced by init, waits 3 seconds for the guest to finish its init sequence, pauses the VM, and takes a snapshot of the entire machine state (memory + VM state). The snapshot artifacts are the input to every spawn invocation. The golden VM process is terminated after snapshotting — only the artifacts on disk are kept.

This command always recreates the golden directory from scratch, discarding any previous snapshot.

Usage

sudo ./fc-orch golden [tag] [distro]

Where [tag] identifies the snapshot baseline name (default default), and [distro] dictates the source .ext4 image to use (default: alpine).

Optional overrides:

sudo FC_MEM_MIB=256 FC_VCPUS=2 ./fc-orch golden ubuntu

Prerequisites

  • init must have been run (kernel and rootfs must exist)
  • firecracker binary must be in $PATH (or set via FC_BIN)
  • ip, iptables, sysctl must be in $PATH (when networking is enabled)

Step-by-step execution

  1. Verify prerequisites

    Checks that FC_KERNEL and FC_ROOTFS exist. Exits with an error if either is missing and directs the user to run init.

  2. Recreate golden directory

    rm -rf /tmp/fc-orch/golden/<tag>
    mkdir -p /tmp/fc-orch/golden/<tag> /tmp/fc-orch/pids
    
  3. COW copy of base rootfs

    cp --reflink=always /tmp/fc-orch/rootfs.ext4 /tmp/fc-orch/golden/<tag>/rootfs.ext4
    

    On filesystems that do not support reflinks (e.g. ext4), this falls back to a regular byte-for-byte copy via io.Copy. On btrfs or xfs, the reflink is instant and consumes no additional space until the VM writes to the disk.

  4. Network setup (skipped when FC_BRIDGE=none)

    a. Create bridge (idempotent — skipped if fcbr0 already exists):

    ip link add fcbr0 type bridge
    ip addr add 172.30.0.1/24 dev fcbr0
    ip link set fcbr0 up
    

    b. Enable IP forwarding and NAT:

    ip -4 route show default          # detect egress interface, e.g. "eth0"
    sysctl -qw net.ipv4.ip_forward=1
    iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
    

    c. Create and attach the golden TAP device:

    ip tuntap add dev fctap0 mode tap
    ip link set fctap0 up
    ip link set fctap0 master fcbr0
    
  5. Build Firecracker machine configuration (passed to the SDK in memory):

    SocketPath:      /tmp/fc-orch/golden/<tag>/api.sock
    KernelImagePath: /tmp/fc-orch/vmlinux
    KernelArgs:      console=ttyS0 reboot=k panic=1 pci=off i8042.noaux quiet loglevel=0
    MachineCfg:
      VcpuCount:       1
      MemSizeMib:      128
      TrackDirtyPages: true      ← required for snapshot support
    Drives:
      - DriveID:      rootfs
        PathOnHost:   /tmp/fc-orch/golden/<tag>/rootfs.ext4
        IsRootDevice: true
        IsReadOnly:   false
    NetworkInterfaces:
      - MacAddress:  AA:FC:00:00:00:01
        HostDevName: fctap0
    
  6. Launch Firecracker process

    The Firecracker Go SDK spawns:

    firecracker --api-sock /tmp/fc-orch/golden/<tag>/api.sock
    

    The SDK then applies the machine configuration via HTTP calls to the Firecracker API socket.

  7. Boot the VM

    m.Start(ctx)   // SDK call — PUT /actions {"action_type": "InstanceStart"}
    

    The golden VM PID is written to /tmp/fc-orch/pids/golden.pid.

  8. Wait for guest init to settle

    time.Sleep(3 * time.Second)
    

    This is a fixed delay. The guest's rcS script mounts pseudo-filesystems and brings up eth0. 3 seconds is conservative enough for the Alpine init sequence to complete.

  9. Pause the VM

    m.PauseVM(ctx)   // SDK call — PATCH /vm {"state": "Paused"}
    

    The VM's vCPUs are frozen. No guest code runs after this point.

  10. Create snapshot

    m.CreateSnapshot(ctx,
        "/tmp/fc-orch/golden/<tag>/mem",
        "/tmp/fc-orch/golden/<tag>/vmstate",
    )
    // SDK call — PUT /snapshot/create
    // {
    //   "mem_file_path":      "/tmp/fc-orch/golden/<tag>/mem",
    //   "snapshot_path":      "/tmp/fc-orch/golden/<tag>/vmstate",
    //   "snapshot_type":      "Full"
    // }
    
    • mem: full dump of guest physical memory (~128 MiB). All clones map this file read-only; the kernel's MAP_PRIVATE gives each clone copy-on-write semantics over it.
    • vmstate: serialized vCPU and device state (typically a few hundred KiB). Each clone gets its own copy.

    Sizes of both files are logged.

  11. Terminate golden VM

    m.StopVMM()   // SDK call — PUT /actions {"action_type": "SendCtrlAltDel"}
    
  12. Destroy golden TAP device

    ip link del fctap0
    

Outputs

Path Description
/tmp/fc-orch/golden/<tag>/mem Full memory snapshot (~FC_MEM_MIB MiB)
/tmp/fc-orch/golden/<tag>/vmstate VM state snapshot (vCPU registers, device state)
/tmp/fc-orch/golden/<tag>/rootfs.ext4 COW copy of base rootfs (not needed after snapshotting, kept for reference)

Error conditions

Error Cause Resolution
kernel not found — run init first FC_KERNEL path does not exist Run init first
rootfs not found — run init first Ext4 file does not exist Run init [distro] first
firecracker binary not found FC_BIN not in $PATH Install Firecracker or set FC_BIN
create bridge: ... ip link add failed Check if another bridge with the same name exists with incompatible config
start golden VM: ... Firecracker failed to boot Check Firecracker logs; verify kernel and rootfs are valid
pause VM: ... VM did not reach a pauseable state in 3s Increase settle time in source or investigate guest crash via serial console
create snapshot: ... Snapshot write failed Check disk space in FC_BASE_DIR

spawn

Purpose

Restores one or more VM clones from the golden snapshot. Each clone is an independent Firecracker process that resumes execution from exactly the paused state captured by golden. Clones differ only in their TAP device, MAC address, rootfs COW layer, and vmstate file; they all map the same golden/mem file.

Clone IDs are auto-incremented: if clones 13 already exist, the next spawn 2 creates clones 4 and 5. Spawn can be called multiple times to add more clones incrementally.

Usage

sudo ./fc-orch spawn               # spawn 1 clone from the "default" golden snapshot
sudo ./fc-orch spawn ubuntu 10     # spawn 10 clones from the "ubuntu" golden snapshot

Prerequisites

  • golden must have been run (golden/vmstate and golden/mem must exist)
  • firecracker binary must be available
  • Sufficient disk space for per-clone rootfs copies (each is a copy of the ~512 MiB golden rootfs, but reflinks are instant on btrfs/xfs)

Step-by-step execution (per clone)

The following steps are performed once for each requested clone. Let {id} be the auto-assigned clone number (e.g. 1, 2, ...).

  1. Verify golden artifacts exist

    Checks for both /tmp/fc-orch/golden/<tag>/vmstate and /tmp/fc-orch/golden/<tag>/mem. Exits with an error if either is missing.

  2. Create directories

    mkdir -p /tmp/fc-orch/clones /tmp/fc-orch/pids
    mkdir -p /tmp/fc-orch/clones/{id}
    
  3. Setup bridge (idempotent, skipped if bridge already exists or FC_BRIDGE=none)

    Same sequence as step 4 of golden. No-op if fcbr0 is already up.

  4. COW copy of golden rootfs

    cp --reflink=always /tmp/fc-orch/golden/<tag>/rootfs.ext4 /tmp/fc-orch/clones/{id}/rootfs.ext4
    

    Falls back to a full copy if reflinks are unsupported.

  5. Shared memory reference (no copy)

    The clone's Firecracker config will point directly at /tmp/fc-orch/golden/<tag>/mem. No file operation is needed here — the kernel's MAP_PRIVATE ensures each clone's writes are private.

  6. Copy vmstate

    # implemented as io.Copy in Go
    cp /tmp/fc-orch/golden/<tag>/vmstate /tmp/fc-orch/clones/{id}/vmstate
    

    The vmstate file is small (typically < 1 MiB), so a full copy is cheap.

  7. Create and attach TAP device (skipped when FC_BRIDGE=none)

    ip tuntap add dev fctap{id} mode tap
    ip link set fctap{id} up
    ip link set fctap{id} master fcbr0
    

    MAC address is derived from the clone ID:

    AA:FC:00:00:{id/256 in hex}:{id%256 in hex}
    # e.g. clone 1  → AA:FC:00:00:00:01
    #      clone 2  → AA:FC:00:00:00:02
    #      clone 256 → AA:FC:00:00:01:00
    
  8. Build Firecracker snapshot-restore configuration (in memory):

    SocketPath: /tmp/fc-orch/clones/{id}/api.sock
    MachineCfg:
      VcpuCount:  1
      MemSizeMib: 128
    NetworkInterfaces:
      - MacAddress:  AA:FC:00:00:00:{id:02X}
        HostDevName: fctap{id}
    Snapshot:
      MemFilePath:  /tmp/fc-orch/golden/<tag>/mem      ← shared, read-only mapping
      SnapshotPath: /tmp/fc-orch/clones/{id}/vmstate
      ResumeVM:     true                          ← restore instead of fresh boot
    

    Note: KernelImagePath and Drives are omitted when restoring from a snapshot — Firecracker uses the snapshot state instead.

  9. Launch Firecracker process

    firecracker --api-sock /tmp/fc-orch/clones/{id}/api.sock
    
  10. Restore and resume VM

    m.Start(ctx)
    // SDK call — POST /snapshot/load
    // {
    //   "mem_file_path":      "/tmp/fc-orch/golden/<tag>/mem",
    //   "snapshot_path":      "/tmp/fc-orch/clones/{id}/vmstate",
    //   "resume_vm":          true
    // }
    

    Restoration time (from m.Start call to return) is measured and logged.

  11. Inject network config via MMDS (only when FC_AUTO_NET_CONFIG=1 and networking is enabled)

    Immediately after the snapshot is restored, the orchestrator configures the MMDS for this clone via two API calls to the clone's Firecracker socket:

    PUT /mmds/config
    {"version": "V1", "network_interfaces": ["1"]}
    
    PUT /mmds
    {"ip": "172.30.0.{10+id}/24", "gw": "172.30.0.1", "dns": "1.1.1.1"}
    

    The fc-net-init daemon already running inside the guest (started during golden VM boot, captured in the snapshot) polls 169.254.169.254 via a link-local route and applies the config to eth0 within ~1 second of clone resume.

  12. Record PID

    echo {pid} > /tmp/fc-orch/pids/clone-{id}.pid
    
  13. Register clone in memory

    The running clone is tracked in an in-process map keyed by clone ID, holding the Firecracker SDK handle, context cancel function, and TAP device name. This allows kill to cleanly terminate clones started in the same process invocation.

After all clones are spawned, Status() is called automatically to display the running clone table.

Outputs

For each clone {id}:

Path Description
/tmp/fc-orch/clones/{id}/rootfs.ext4 Private COW copy of the golden rootfs
/tmp/fc-orch/clones/{id}/vmstate Private copy of golden vmstate
/tmp/fc-orch/clones/{id}/api.sock Firecracker API socket (live while clone is running)
/tmp/fc-orch/pids/clone-{id}.pid PID of this clone's Firecracker process
fctap{id} Host TAP network device attached to fcbr0

Error conditions

Error Cause Resolution
golden vmstate not found — run golden first golden has not been run Run golden first
golden mem not found — run golden first Same as above Run golden first
firecracker not found Binary missing Install Firecracker or set FC_BIN
copy rootfs: ... Disk full or source missing Check disk space; re-run golden
restore clone {id}: ... Firecracker failed to load snapshot Check that golden/mem is not corrupted; re-run golden
create tap {name}: ... TAP device already exists or ip failed Run kill to clean up stale TAPs, then retry
Individual clone failure Per-clone errors are logged but do not abort the batch Check logs; surviving clones continue running

status

Purpose

Displays a table of all clones that have been spawned, along with their PIDs and liveness. Liveness is determined by checking whether /proc/{pid} exists — a process that has exited will no longer appear in /proc.

This command does not require the clones to have been started in the current process invocation; it reads PID files written to disk by spawn.

Usage

sudo ./fc-orch status

Prerequisites

None. Can be run at any time, even with no clones running.

Step-by-step execution

  1. Read PID directory

    Lists all files in /tmp/fc-orch/pids/.

  2. Filter for clone PID files

    Only files whose names start with clone- are considered (excludes golden.pid).

  3. Check liveness

    For each file:

    # read pid from clone-{id}.pid
    test -d /proc/{pid}   # alive if directory exists
    
  4. Print table

    === Running clones ===
      clone-1       pid=12345     alive
      clone-2       pid=12346     alive
      clone-3       pid=12347     DEAD
    

Outputs

Prints to stdout only. No files are modified.


kill

Purpose

Terminates all running Firecracker VM processes and removes all TAP devices. Handles two cases:

  • In-memory clones: clones started in the same fc-orch process invocation, tracked via the in-process clone map.
  • Orphaned clones: clones from a previous fc-orch spawn invocation, whose PIDs are recorded in PID files.

Both cases are always handled, so kill works correctly even after a restart or crash of the orchestrator process.

The command does not remove snapshot files or the clones/ directory — use cleanup for full teardown.

Usage

sudo ./fc-orch kill

Prerequisites

None. Safe to run even if no VMs are running.

Step-by-step execution

  1. Stop in-memory clones (clones started in this process invocation)

    For each clone tracked in the in-process map:

    clone.Machine.StopVMM()   // SDK: PUT /actions {"action_type": "SendCtrlAltDel"}
    clone.Cancel()            // cancels the clone's context
    
    ip link del fctap{id}
    

    The clone is removed from the in-memory map.

  2. Kill orphaned processes from PID files

    For each file in /tmp/fc-orch/pids/:

    // equivalent to:
    kill -9 {pid}
    

    The PID file is then deleted:

    rm /tmp/fc-orch/pids/{file}
    
  3. Destroy stale TAP devices

    ip -o link show
    

    Each line containing fctap is parsed to extract the device name, then:

    ip link del {tapname}
    

    This cleans up any TAP devices left over from previous runs that were not removed by steps 1 or 2.

Outputs

All Firecracker processes are terminated. All fctap* network interfaces are removed. PID files are deleted. The clones/, golden/, and pids/ directories themselves remain on disk.

Error conditions

Errors from individual StopVMM, kill, or ip link del calls are logged but do not abort the rest of the kill sequence. The command always attempts to clean up everything it finds.


cleanup

Purpose

Performs a full teardown: kills all VMs (same as kill), removes all working directories under $FC_BASE_DIR, and tears down the host bridge. After cleanup, the system is in the same state as before golden was run. The base kernel and rootfs files are not removed — only golden/, clones/, and pids/ are deleted.

Usage

sudo ./fc-orch cleanup

Prerequisites

None. Safe to run at any time.

Step-by-step execution

  1. Kill all VMs

    Performs the full kill sequence (see kill above).

  2. Remove working directories

    rm -rf /tmp/fc-orch/clones
    rm -rf /tmp/fc-orch/golden
    rm -rf /tmp/fc-orch/pids
    
  3. Tear down bridge (skipped when FC_BRIDGE=none)

    ip link del fcbr0
    

    This also implicitly removes all addresses and routes associated with the bridge. The iptables NAT rule added during golden/spawn is not removed automatically — remove it manually if needed:

    iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
    

Outputs

After cleanup:

  • All Firecracker processes are terminated
  • /tmp/fc-orch/clones/, /tmp/fc-orch/golden/, /tmp/fc-orch/pids/ are deleted
  • fcbr0 bridge and all associated fctap* devices are removed
  • /tmp/fc-orch/vmlinux and /tmp/fc-orch/rootfs.ext4 remain intact

To also remove the kernel and rootfs:

sudo ./fc-orch cleanup
rm -f /tmp/fc-orch/vmlinux /tmp/fc-orch/rootfs.ext4

Error conditions

Errors from rm -rf or ip link del are ignored (logged only). Cleanup always completes all steps.


Typical Workflow

# One-time setup: download kernel and build rootfs
sudo ./fc-orch init

# Create golden snapshot (re-run any time you want a fresh baseline)
sudo ./fc-orch golden

# Spawn clones
sudo ./fc-orch spawn 10

# Check what's running
sudo ./fc-orch status

# Add more clones to the existing set
sudo ./fc-orch spawn 5

# Tear down all clones (keeps golden snapshot)
sudo ./fc-orch kill

# Full reset (keeps kernel and rootfs)
sudo ./fc-orch cleanup