Shell Scripting & Automation

Loops & Iteration

18 min Lesson 4 of 28

Loops & Iteration

Production scripts almost never do one thing once. They deploy to ten servers, rotate thirty log files, validate every line of a config, or restart a service until it becomes healthy. Loops are what transform a one-shot command into a general-purpose automation. This lesson covers the four loop patterns you will reach for every day as a DevOps engineer: the C-style for loop, the list-based for-in loop, the while loop, and reading files line by line — plus the idiomatic glob loop that processes entire directory trees safely.

The for-in Loop: Iterating Over a List

The most common Bash loop iterates over a whitespace-separated list of words. Its form is for variable in list; do ... done. The list can be literal words, a command substitution, a brace expansion, or a glob pattern.

#!/usr/bin/env bash
set -euo pipefail

# --- literal list ---
ENVS=(staging canary production)
for env in "${ENVS[@]}"; do
    echo "Deploying to: ${env}"
    # ./deploy.sh "${env}"
done

# --- brace expansion (sequences) ---
for i in {1..5}; do
    echo "Attempt ${i}"
done

# --- command substitution: loop over kubectl namespaces ---
for ns in $(kubectl get namespaces -o jsonpath='{.items[*].metadata.name}'); do
    echo "Namespace: ${ns}"
done

Word splitting trap: Never use for item in $(command) when output can contain spaces or newlines inside a single logical item. For example, filenames with spaces will be split into separate iterations. Use while IFS= read -r (covered below) or mapfile when the input is newline-delimited.

The while Loop: Condition-Driven Iteration

A while loop runs as long as its condition evaluates to true (exit status zero). It is the right tool when you do not know up front how many iterations you need — waiting for a service to become healthy, polling an API, or draining a queue.

#!/usr/bin/env bash
set -euo pipefail

# --- wait until a service is healthy (production pattern) ---
SERVICE_URL="http://localhost:8080/health"
MAX_WAIT=120   # seconds
INTERVAL=5
elapsed=0

echo "Waiting for service at ${SERVICE_URL}..."

while ! curl -sf "${SERVICE_URL}" >/dev/null; do
    if (( elapsed >= MAX_WAIT )); then
        echo "ERROR: service did not become healthy within ${MAX_WAIT}s" >&2
        exit 1
    fi
    echo "  not ready yet — retrying in ${INTERVAL}s (${elapsed}s elapsed)"
    sleep "${INTERVAL}"
    (( elapsed += INTERVAL ))
done

echo "Service is healthy after ${elapsed}s."

Two details matter here. First, the ! negates the exit status of curl -sf, so the loop continues while the service is not responding. Second, the timeout guard with an explicit exit 1 prevents the script from looping forever if something is truly broken — an essential safety net for CI pipelines and on-call automation.

Pro pattern — exponential backoff: For loops that retry network calls, use exponential backoff instead of a fixed interval: sleep $(( INTERVAL * 2 ** attempt )). This avoids hammering a degraded service. Libraries like retry (a small shell function you paste into your script) implement this pattern cleanly.

Reading Files Line by Line

A very common DevOps task is to process a list stored in a file: a list of hosts, a manifest of S3 objects, a CSV of usernames. The canonical and safe pattern is while IFS= read -r line. Do not use for line in $(cat file) — it breaks on spaces, tabs, and leading/trailing whitespace.

#!/usr/bin/env bash
set -euo pipefail

HOSTS_FILE="./hosts.txt"

# Safe line-by-line reading
# IFS= prevents stripping leading/trailing whitespace
# -r prevents backslash escaping
while IFS= read -r host; do
    # Skip blank lines and comment lines
    [[ -z "${host}" || "${host}" == \#* ]] && continue

    echo "Checking SSH on: ${host}"
    ssh -o ConnectTimeout=5 -o BatchMode=yes "${host}" "uptime" \
        && echo "  OK" \
        || echo "  FAILED — adding to alert queue"

done < "${HOSTS_FILE}"

The redirection < "${HOSTS_FILE}" feeds the file into the while loop's stdin. This is more efficient than piping through cat (which spawns a subprocess) and keeps the loop in the current shell so variable assignments inside the loop remain visible after it ends — a subtle but important difference.

Subprocess scope issue: When you write cat file | while read -r line; do VAR=something; done, the loop body runs in a subshell because of the pipe. Any variable you set inside is lost when the loop exits. The while ... done < file redirect avoids this because no pipe is involved.

Glob Loops: Processing Files Safely

When you need to act on every file matching a pattern — all .log files in a directory, every *.yaml config — Bash glob expansion is both safer and faster than parsing ls output. The shell expands the glob before the loop runs, so each filename is a separate, properly quoted word even if it contains spaces.

#!/usr/bin/env bash
set -euo pipefail

LOG_DIR="/var/log/myapp"
ARCHIVE_DIR="/mnt/cold-storage/logs"
CUTOFF_DAYS=7

mkdir -p "${ARCHIVE_DIR}"

# Glob over all .log files modified more than CUTOFF_DAYS ago
# 'shopt -s nullglob' makes the loop skip if no files match (critical!)
shopt -s nullglob

for logfile in "${LOG_DIR}"/*.log; do
    if [[ $(find "${logfile}" -mtime "+${CUTOFF_DAYS}" 2>/dev/null) ]]; then
        echo "Archiving: ${logfile}"
        gzip --best --keep "${logfile}"
        mv "${logfile}.gz" "${ARCHIVE_DIR}/"
        rm -f "${logfile}"
    fi
done

shopt -u nullglob   # restore default behaviour

The shopt -s nullglob line is critical. Without it, if no files match the pattern, Bash passes the literal string /var/log/myapp/*.log as the first (and only) iteration — causing your script to attempt to archive a file that does not exist. With nullglob enabled, a matching set of zero files simply causes the loop body to never execute. Always set it before a glob loop that might match nothing.

Four Bash loop patterns — their ideal use case and the pitfall each carries if used incorrectly.

Loop Control: break, continue, and Exit Codes

Two built-in keywords modify loop execution. break exits the innermost loop immediately; continue skips the remainder of the current iteration and starts the next one. Both accept an optional integer argument to target outer loops in nested structures.

#!/usr/bin/env bash
set -euo pipefail

# Process a queue of jobs; stop if a critical job fails
declare -a JOBS=("migrate-db" "seed-cache" "warm-cdn" "deploy-app")

for job in "${JOBS[@]}"; do
    echo "Running job: ${job}"

    if [[ "${job}" == "seed-cache" ]]; then
        echo "  Skipping non-critical seed in prod"
        continue   # skip this iteration only
    fi

    # Simulate running the job (replace with real command)
    ./run-job.sh "${job}" || {
        echo "CRITICAL: ${job} failed — halting pipeline" >&2
        break      # stop all remaining jobs
    }

    echo "  ${job} completed successfully"
done

Loop exit codes: A loop's own exit code is the exit code of the last command that ran inside it. If you need to propagate a failure caught inside a break, set a flag variable before breaking — PIPELINE_FAILED=1 — then check it after the loop and exit 1 if set. With set -e active this distinction matters: a failed command in a loop with a trailing || true does not abort the script, giving you control over which failures are fatal.

Practical Production Example: Multi-Server Log Rotation

Combining everything in this lesson: a script that reads a host list from a file, iterates over each host, uses a glob loop remotely to archive old logs, and retries failed hosts once before alerting.

#!/usr/bin/env bash
set -euo pipefail

HOSTS_FILE="${1:-/etc/myapp/hosts.txt}"
LOG_DIR="/var/log/myapp"
FAILED_HOSTS=()

shopt -s nullglob

while IFS= read -r host; do
    [[ -z "${host}" || "${host}" == \#* ]] && continue

    echo "=== ${host} ==="
    if ! ssh -o ConnectTimeout=5 -o BatchMode=yes "${host}" \
        "find ${LOG_DIR} -name '*.log' -mtime +7 -exec gzip -f {} \;"; then
        echo "  WARNING: log rotation failed on ${host}" >&2
        FAILED_HOSTS+=("${host}")
    fi

done < "${HOSTS_FILE}"

shopt -u nullglob

if (( ${#FAILED_HOSTS[@]} > 0 )); then
    echo "ALERT: rotation failed on: ${FAILED_HOSTS[*]}" >&2
    # ./notify.sh "log-rotation" "${FAILED_HOSTS[*]}"
    exit 1
fi

echo "Log rotation complete on all hosts."

This script is production-ready in shape: it reads input safely, skips blank and comment lines, accumulates failures rather than aborting on the first one, and exits non-zero only if any host failed — exactly the behavior a monitoring system or CI pipeline expects.