0. Lab environment: a Linux VM on your Mac

You can do most of this guide natively on macOS with Docker Desktop (see Prerequisites, path B). The recommended path is a dedicated Ubuntu VM on your Mac that holds the whole project: Docker, kind, every CLI, the Orbit repo, and all its clusters. This section builds that VM with a script, so you can delete and recreate it at any time.

0.1 Why a Linux VM instead of native macOS

Lima (a CNCF project, brew install lima) runs Linux VMs on macOS with Apple's Virtualization.framework (vz). It forwards guest ports to your Mac's localhost automatically, and it's scriptable. Alternatives are in §0.8.

0.2 Size the VM

Disk: 100 GiB. It's a sparse file, so it only uses what is actually written.

On Apple Silicon the VM is arm64. Everything in this guide is multi-arch (distroless, Postgres, Redis, Envoy, Grafana stack, Argo). For the odd amd64-only image, the bootstrap enables QEMU emulation through binfmt.

0.3 Create the VM

In the wiki, the lab scripts ship next to this guide in K8s-docker/lab/ (create-vm.sh, bootstrap.sh, extract-blocks.py). They're identical to the listings below, so you can create the VM before the Orbit repo exists. Later, copy lab/ into the repo (the layout above shows it there). Run this script on the Mac, from this directory:

#!/usr/bin/env bash
# lab/create-vm.sh: [mac] create and provision the Orbit lab VM
set -euo pipefail
VM=${VM:-orbit}
CPUS=${CPUS:-6}
MEMORY=${MEMORY:-16}      # GiB
DISK=${DISK:-100}         # GiB
HERE=$(cd "$(dirname "$0")" && pwd)

command -v limactl >/dev/null || brew install lima

if ! limactl list -q | grep -qx "$VM"; then
  limactl create --name="$VM" \
    --cpus="$CPUS" --memory="$MEMORY" --disk="$DISK" \
    --vm-type=vz --mount-type=virtiofs \
    --containerd=none \
    --tty=false \
    template:ubuntu-24.04
fi
limactl start "$VM"

limactl copy "$HERE/bootstrap.sh" "$VM":/tmp/bootstrap.sh
limactl shell "$VM" bash /tmp/bootstrap.sh

echo
echo "Done. Enter the lab with:   limactl shell $VM"
echo "VS Code Remote-SSH host:    add 'Include ~/.lima/$VM/ssh.config' to ~/.ssh/config, connect to 'lima-$VM'"

chmod +x lab/create-vm.sh && ./lab/create-vm.sh        # [mac] ~5-10 min the first time
limactl list                                            # [mac] NAME orbit, STATUS Running
limactl shell orbit                                     # [mac] -> you are now in Ubuntu

Choices in that command:

• --vm-type=vz + virtiofs: Apple's native hypervisor, the fastest option on Apple Silicon and recent Intel Macs.
• --containerd=none: Lima's templates can start their own rootless containerd/nerdctl. We install the real Docker Engine instead (kind and the guide expect docker), so we turn Lima's off to avoid two runtimes.
• By default Lima mounts your Mac home directory read-only in the VM. Keep the Orbit repo on the VM's own disk (~/orbit inside the VM). It's faster, and it avoids UID/permission mismatches with bind mounts.

⚠️ Caveat (verify): the flags above (--cpus, --memory, --disk, --vm-type, --mount-type, --containerd=none, --tty=false) and template:ubuntu-24.04 were checked against Lima 2.2.1 (limactl create --help, --list-templates). The VM itself hasn't been booted with them yet. Older Lima 1.x spells templates template://ubuntu-24.04. If limactl create rejects something on your version, run limactl create --help and limactl create --list-templates, or put the same settings in a YAML file (cpus, memory, disk, vmType: vz, containerd: {system: false, user: false}) and run limactl create --name=orbit orbit.yaml. The ~/.lima/<vm>/ssh.config path is where current Lima versions write it; check with limactl list --format '{{.Dir}}'.

0.4 Bootstrap the toolchain inside the VM

lab/bootstrap.sh is idempotent: safe to re-run to upgrade tools. It detects the architecture, so the same script works on Intel Macs and on Linux cloud VMs.

#!/usr/bin/env bash
# lab/bootstrap.sh: run INSIDE the VM as your normal user (uses sudo). Idempotent.
set -euo pipefail
ARCH=$(dpkg --print-architecture)                  # arm64 | amd64
BIN=/usr/local/bin
log() { printf '\n\033[1;34m>> %s\033[0m\n' "$*"; }
gh_latest() { curl -fsSL "https://api.github.com/repos/$1/releases/latest" | jq -r .tag_name; }
install_url() { sudo curl -fsSLo "$BIN/$1" "$2" && sudo chmod +x "$BIN/$1"; }

log "base packages"
sudo apt-get update -y
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y \
  ca-certificates curl git jq make unzip build-essential bash-completion \
  postgresql-client redis-tools dnsutils iproute2 tcpdump netcat-openbsd

log "kernel settings (multi-node kind, Loki/Tempo, chaos)"
sudo tee /etc/sysctl.d/99-orbit-lab.conf >/dev/null <<'EOF'
fs.inotify.max_user_watches=524288
fs.inotify.max_user_instances=512
vm.max_map_count=262144
EOF
sudo sysctl --system >/dev/null

log "Docker Engine (CE)"
if ! command -v docker >/dev/null; then
  curl -fsSL https://get.docker.com | sudo sh
fi
sudo usermod -aG docker "$USER"
# emulate amd64 for the rare single-arch image (registration does not survive a reboot; re-run bootstrap)
sudo docker run --privileged --rm tonistiigi/binfmt --install amd64 >/dev/null

log "Go"
GO_VERSION=${GO_VERSION:-$(curl -fsSL 'https://go.dev/VERSION?m=text' | head -1)}
if [ "$(/usr/local/go/bin/go version 2>/dev/null | awk '{print $3}')" != "$GO_VERSION" ]; then
  sudo rm -rf /usr/local/go
  curl -fsSL "https://go.dev/dl/${GO_VERSION}.linux-${ARCH}.tar.gz" | sudo tar -C /usr/local -xz
fi
grep -q '/usr/local/go/bin' ~/.bashrc || echo 'export PATH=/usr/local/go/bin:$HOME/go/bin:$PATH' >> ~/.bashrc
export PATH=/usr/local/go/bin:$HOME/go/bin:$PATH

log "kubectl + helm"
install_url kubectl "https://dl.k8s.io/release/$(curl -fsSL https://dl.k8s.io/release/stable.txt)/bin/linux/${ARCH}/kubectl"
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

log "Go-installable CLIs"
for pkg in \
  sigs.k8s.io/kind@latest \
  sigs.k8s.io/cloud-provider-kind@latest \
  sigs.k8s.io/kustomize/kustomize/v5@latest \
  github.com/stern/stern@latest \
  github.com/ahmetb/kubectx/cmd/kubectx@latest \
  github.com/ahmetb/kubectx/cmd/kubens@latest \
  github.com/mikefarah/yq/v4@latest \
  github.com/yannh/kubeconform/cmd/kubeconform@latest \
  github.com/pressly/goose/v3/cmd/goose@latest \
  go.k6.io/k6@latest ; do
  echo "   go install $pkg"; go install "$pkg"
done

log "release-binary CLIs"
install_url argocd                "https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-${ARCH}"
install_url kubectl-argo-rollouts "https://github.com/argoproj/argo-rollouts/releases/latest/download/kubectl-argo-rollouts-linux-${ARCH}"
install_url cosign                "https://github.com/sigstore/cosign/releases/latest/download/cosign-linux-${ARCH}"
curl -fsSL "https://github.com/derailed/k9s/releases/latest/download/k9s_Linux_${ARCH}.tar.gz" | sudo tar -C $BIN -xz k9s
curl -fsSL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sudo sh -s -- -b $BIN
V=$(gh_latest vmware-tanzu/velero)
curl -fsSL "https://github.com/vmware-tanzu/velero/releases/download/${V}/velero-${V}-linux-${ARCH}.tar.gz" \
  | sudo tar -C $BIN -xz --strip-components=1 "velero-${V}-linux-${ARCH}/velero"
curl -fsSL https://github.com/cloudnative-pg/cloudnative-pg/raw/main/hack/install-cnpg-plugin.sh | sudo sh -s -- -b $BIN

log "cloud-provider-kind as a service (LoadBalancer IPs for kind)"
sudo install -m 0755 "$HOME/go/bin/cloud-provider-kind" $BIN/
sudo tee /etc/systemd/system/cloud-provider-kind.service >/dev/null <<'EOF'
[Unit]
Description=cloud-provider-kind: LoadBalancer services for kind clusters
After=docker.service
Requires=docker.service

[Service]
ExecStart=/usr/local/bin/cloud-provider-kind
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now cloud-provider-kind

log "shell conveniences"
grep -q 'orbit-lab' ~/.bashrc || cat >> ~/.bashrc <<'EOF'
# orbit-lab
source <(kubectl completion bash); alias k=kubectl; complete -o default -F __start_kubectl k
source <(helm completion bash)
export KUBE_EDITOR=vim
EOF

log "versions"
docker version --format 'docker {{.Server.Version}}' 2>/dev/null || sudo docker version --format 'docker {{.Server.Version}}'
go version; kubectl version --client | head -1; helm version --short; kind version
echo "Open a NEW shell (limactl shell orbit) so the docker group and PATH apply."

Check it from a new shell:

limactl shell orbit                          # [mac]
docker run --rm hello-world                  # Docker works without sudo
kind version && kubectl version --client && k9s version --short
systemctl status cloud-provider-kind --no-pager | head -3

⚠️ Caveat (verify): the bootstrap pulls latest releases, which is convenient for a lab but not reproducible. Once it works, pin versions (write the output of the "versions" step into lab/versions.env and install from it). Things to check: release asset names (k9s_Linux_arm64.tar.gz, argocd-linux-arm64, Velero's tarball layout); get-helm-3 (Helm 4 has its own installer if you move to Helm 4); and go install paths for tools that restructure their modules. Building k6 with go install takes a few minutes. If it fails, use Grafana's apt repo or docker run --rm -i --network host grafana/k6 run - < load/orders.js.

0.5 Daily workflow in the lab

Editing code. Use VS Code Remote-SSH into lima-orbit and open ~/orbit. The editor UI runs on the Mac; files, terminals, Go tooling and Docker run in the VM. (Alternatively, edit on the Mac in a writable Lima mount. It's slower for large trees and has UID quirks.)

Reaching services from your Mac browser. Lima forwards any port the VM listens on at 127.0.0.1/0.0.0.0 to your Mac's localhost. kubectl port-forward inside the VM is therefore enough:

# inside the VM (run each in its own terminal, or append & to background them)
kubectl -n monitoring port-forward svc/kps-grafana 3000:80          # Mac: http://localhost:3000
kubectl -n argocd port-forward svc/argocd-server 8088:80            # Mac: http://localhost:8088
kubectl -n argo-rollouts port-forward svc/argo-rollouts-dashboard 3100:3100
GW_SVC=$(kubectl -n envoy-gateway-system get svc \
  -l gateway.envoyproxy.io/owning-gateway-name=public -o jsonpath='{.items[0].metadata.name}')
kubectl -n envoy-gateway-system port-forward svc/$GW_SVC 8443:443   # Mac: https://orbit.localtest.me:8443

Inside the VM, LoadBalancer IPs from cloud-provider-kind work directly: curl -sk --resolve orbit.localtest.me:443:$GW https://orbit.localtest.me/api/products (§25.3).

Lifecycle.

limactl stop orbit            # [mac] pause the lab (frees RAM); kind clusters come back with Docker on start
limactl start orbit           # [mac]
limactl delete orbit && ./lab/create-vm.sh     # [mac] full reset: ~10 min, the VM is cattle too
kind delete cluster --name orbit               # [vm]  reset just the cluster

If a kind cluster misbehaves after a VM restart (stale node IPs, CoreDNS crash-looping), delete and recreate it. Because everything is in Git and driven by Argo CD (§36), that takes minutes. This is §39's "clusters are cattle" principle applied to your laptop.

⚠️ Caveat (verify): limactl snapshot has historically worked only with the QEMU VM type, not vz. Lima 2.x also has limactl clone (stop the VM, then limactl clone orbit orbit-golden) to keep a known-good copy, but test it before relying on it. The scripts plus Git are the dependable reset path.

0.6 Verifying this guide as you go

Most content in this guide is unexecuted until you run it. The lab is where you check it, in four layers, cheapest first:

1. Static checks. Extract the fenced code blocks and validate each with its own tool:

#!/usr/bin/env python3
# lab/extract-blocks.py: dump every fenced block of the guide into lab/blocks/NNN.<lang>
import pathlib, re, sys
src = pathlib.Path(sys.argv[1]).read_text()
out = pathlib.Path("lab/blocks"); out.mkdir(parents=True, exist_ok=True)
for i, (lang, body) in enumerate(re.findall(r"^```(\w*)\n(.*?)^```", src, re.S | re.M)):
    (out / f"{i:03d}.{lang or 'txt'}").write_text(body)
print(f"wrote {i + 1} blocks to {out}")

python3 lab/extract-blocks.py ~/wiki/K8s-docker/real-life-k8s-guide.md
# Kubernetes YAML (including CRDs, via the community CRD schema catalog)
kubeconform -strict -summary -ignore-missing-schemas \
  -schema-location default \
  -schema-location 'https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json' \
  lab/blocks/*.yaml
# Go: assemble the snippets into ~/orbit as you go through §4, §11, §29, then:
cd ~/orbit && go mod tidy && go build ./... && go vet ./...
# Dockerfiles
docker build --check . ; docker run --rm -i hadolint/hadolint < Dockerfile
# Prometheus rules (take .spec out of each PrometheusRule)
yq '.spec' rule.yaml > /tmp/rules.yaml && docker run --rm -v /tmp:/w --entrypoint promtool prom/prometheus check rules /w/rules.yaml
# Collector configs (take the `config:` block out of the Helm values)
yq '.config' platform/otel-gateway/values.yaml > /tmp/otel.yaml && \
  docker run --rm -v /tmp:/w otel/opentelemetry-collector-contrib validate --config=/w/otel.yaml
# Helm values -> rendered manifests -> schema check
helm template kps prometheus-community/kube-prometheus-stack -f platform/kube-prometheus-stack/values.yaml | kubeconform -strict -ignore-missing-schemas
# Compose, Terraform
docker compose config -q ; (cd infra/envs/prod && terraform init -backend=false && terraform validate)

(Many YAML blocks are excerpts with # ... placeholders, so expect some failures that are just incomplete snippets. The complete manifests you assemble in ~/orbit/deploy are what must pass.)

2. Server-side dry runs. Once a CRD's operator is installed, kubectl apply --dry-run=server -f file.yaml checks the YAML against the exact CRD schema and admission webhooks in your cluster. That is the definitive check for every ⚠️ box about CRD fields.

3. Behavior. Run the milestone checks in §0.7 and the acceptance tests in §43.2. For anything involving metrics, check that the query returns data in Grafana Explore before you trust an alert, a KEDA trigger, or a canary analysis built on it.

4. Sources. For each ⚠️ box, compare with the project's own docs or release notes for the version you pinned, and record the result in lab/VERIFIED.md (date, version, pass/fix). Re-run layer 1 in CI on the repo so regressions show up when you bump versions.

0.7 Lab runbook: build the full project, milestone by milestone

Everything below runs inside the VM. Each milestone has a hard "done when" check. Don't move on until it passes.

M12 note: GitHub Actions builds the images on GitHub's runners and pushes them to GHCR. kind pulls them from there (make the packages public, or create an imagePullSecret, §16.3). For a fully offline lab, run a local registry next to kind (kind's docs: Local Registry) and point Argo CD at a Git server in the cluster (e.g. Gitea), but the GitHub path is closer to real life.

0.8 Alternatives to Lima