bashintermediate

Bash Parallel Jobs — Run Tasks Concurrently

Execute multiple shell commands in parallel with job control, exit code checking, and max concurrency.

bash
#!/usr/bin/env bash
set -euo pipefail

# --- Simple background jobs ---
npm run lint &
npm run typecheck &
npm run test &
wait  # wait for all background jobs
echo "All checks passed!"

# --- Parallel with max concurrency ---
MAX_JOBS=4
SERVICES=(api web worker cron scheduler)
PIDS=()
FAILED=0

for svc in "${SERVICES[@]}"; do
  # Wait if at max concurrency
  while (( ${#PIDS[@]} >= MAX_JOBS )); do
    for i in "${!PIDS[@]}"; do
      if ! kill -0 "${PIDS[$i]}" 2>/dev/null; then
        wait "${PIDS[$i]}" || ((FAILED++))
        unset 'PIDS[$i]'
      fi
    done
    PIDS=("${PIDS[@]}")
    sleep 0.1
  done

  echo "Building $svc..."
  docker build -t "myapp-$svc" "./services/$svc" &
  PIDS+=("$!")
done

# Wait for remaining jobs
for pid in "${PIDS[@]}"; do
  wait "$pid" || ((FAILED++))
done

if (( FAILED > 0 )); then
  echo "❌ $FAILED job(s) failed"
  exit 1
fi
echo "✅ All $((${#SERVICES[@]})) services built"

Use Cases

  • Running CI checks in parallel for faster builds
  • Building multiple Docker images concurrently
  • Batch processing files with controlled parallelism

Tags

Related Snippets

Similar patterns you can reuse in the same workflow.