Jenkins & Enterprise CI/CD

Declarative Pipelines

18 min Lesson 3 of 28

Declarative Pipelines

Declarative Pipeline is the opinionated, structured DSL that Jenkins ships as the recommended way to define a CI/CD pipeline in code. It wraps Groovy in a strict grammar that makes pipelines readable, auditable, and maintainable at scale — exactly what big-tech teams need when a Jenkinsfile lives in every microservice repository and is reviewed like application code. This lesson dissects every first-class construct: pipeline, agent, stages/steps, post, and when.

The Mandatory Skeleton

A Declarative Pipeline must follow a precise top-level structure. Any deviation is a parse error at load time — which is actually a feature, because it surfaces misconfiguration before a build is ever triggered.

// Jenkinsfile — minimal valid declarative pipeline
pipeline {
    agent any            // where to run — required at top level

    stages {             // ordered list of stages — required
        stage('Build') {
            steps {      // at least one step — required
                echo 'Compiling...'
            }
        }
    }

    post {               // cleanup / notifications — optional
        always {
            echo 'Pipeline finished.'
        }
    }
}

The parser enforces the grammar: you cannot place bare Groovy at the top level the way Scripted Pipeline allows. This constraint is the trade-off — you gain instant syntax feedback and tooling support (Blue Ocean, Replay), and you lose a small amount of raw flexibility that you rarely need anyway.

The `agent` Directive

The agent block answers one question: on which executor should this pipeline (or stage) run? Jenkins resolves the agent before executing any steps, so getting this right is foundational.

agent any — schedule on any available executor. Fine for small setups; not recommended in production where you want reproducible build environments.
agent none — no top-level agent; each stage must declare its own. This is the production pattern: different stages run on different agents (e.g., build on a Docker image, deploy on a privileged agent with cloud credentials).
agent { label 'linux && docker' } — schedule on any node whose tags match the label expression. Labels are how you route stages to the right hardware or OS.
agent { docker { image 'node:20-alpine' } } — spin up a fresh Docker container per stage; the workspace is mounted inside automatically. This is the recommended pattern for build stages in modern Jenkins installs.
agent { kubernetes { yaml '...' } } — provision a Pod in Kubernetes as the executor. The Jenkins Kubernetes Plugin makes this the de-facto standard at cloud-native companies.

Production pattern — agent none + per-stage agents: declare agent none at the top level and assign explicit agents per stage. This prevents the controller from holding an executor slot idle while slow stages (integration tests, publishing) run.

pipeline {
    agent none   // no global executor slot held

    stages {
        stage('Build') {
            agent { docker { image 'gradle:8-jdk21' } }
            steps {
                sh './gradlew assemble'
                stash name: 'jars', includes: 'build/libs/**/*.jar'
            }
        }

        stage('Test') {
            agent { docker { image 'gradle:8-jdk21' } }
            steps {
                unstash 'jars'
                sh './gradlew test'
            }
            post {
                always {
                    junit 'build/test-results/**/*.xml'
                }
            }
        }

        stage('Deploy to Staging') {
            agent { label 'deploy' }   // node with cloud credentials
            steps {
                unstash 'jars'
                sh './scripts/deploy.sh staging'
            }
        }
    }
}

Stages and Steps

Every meaningful unit of work lives inside a stage. Stages are the granularity level that Blue Ocean visualises and that your team reasons about: Build, Unit Test, Integration Test, Security Scan, Publish, Deploy. Within a stage, steps is the block where you call step functions.

Key built-in steps you will use constantly:

sh 'command' — run a shell command on Linux/macOS agents.
bat 'command' — run a batch command on Windows agents.
echo 'message' — print to build log.
checkout scm — check out the source that triggered this build (implicit in Multibranch, explicit otherwise).
stash / unstash — pass artifacts between stages that run on different agents.
withCredentials([...]) — inject secrets into the environment for the duration of the block.
dir('path') — change working directory for nested steps.
timeout(time: 10, unit: 'MINUTES') — fail a step or stage if it hangs.

Declarative Pipeline top-level blocks and their relationship. Every stage can carry its own agent and when; post runs after all stages complete.

The `post` Section

The post block defines actions that execute after the pipeline (or individual stage) finishes, regardless of outcome. It can appear at the top level and inside any stage. The available conditions map to every meaningful final state:

always — unconditional; use for cleanup and log archival.
success — only when the build passes; use for publishing artifacts or notifications.
failure — only on failure; use for alerting on-call, rolling back.
unstable — build completed but test failures set the result to UNSTABLE.
changed — fires when the current result differs from the previous run; excellent for "back to green" Slack messages.
fixed — shorthand for success-after-failure.
regression — shorthand for failure-after-success.
aborted — the build was manually stopped.
cleanup — runs last of all, after every other post condition; use for workspace cleanup so it always happens even if a notification step fails.

Tip — separate cleanup from notifications: put deleteDir() or container teardown in cleanup rather than always. If a Slack notification throws, always would abort before reaching cleanup; cleanup is guaranteed to run after everything else in post.

The `when` Directive

The when block lets you skip a stage entirely based on conditions evaluated before the stage's agent is allocated. This is critical for efficient pipelines: why spin up a deployment container on every feature branch commit when you only deploy from main?

Common built-in conditions:

branch 'main' — matches the branch name (glob supported: 'release/*').
tag 'v*' — matches a Git tag.
environment name: 'DEPLOY_ENV', value: 'production' — inspects an environment variable.
expression { return params.RUN_INTEGRATION_TESTS } — arbitrary Groovy expression returning a boolean.
changeRequest() — true when the build is a Pull Request.
not { branch 'main' } — negation.
allOf { branch 'main'; environment name: 'CI', value: 'true' } — logical AND.
anyOf { branch 'main'; branch 'staging' } — logical OR.

pipeline {
    agent none

    environment {
        REGISTRY = 'registry.example.com'
        IMAGE    = "${REGISTRY}/myapp"
    }

    stages {
        stage('Build & Push Image') {
            agent { docker { image 'docker:24-dind' } }
            steps {
                sh "docker build -t ${IMAGE}:${env.BUILD_NUMBER} ."
                withCredentials([usernamePassword(
                    credentialsId: 'registry-creds',
                    usernameVariable: 'USER',
                    passwordVariable: 'PASS'
                )]) {
                    sh "echo $PASS | docker login ${REGISTRY} -u $USER --password-stdin"
                    sh "docker push ${IMAGE}:${env.BUILD_NUMBER}"
                }
            }
        }

        stage('Deploy to Staging') {
            agent { label 'deploy' }
            when {
                anyOf {
                    branch 'main'
                    branch 'release/*'
                }
            }
            steps {
                sh "./scripts/deploy.sh staging ${IMAGE}:${env.BUILD_NUMBER}"
            }
        }

        stage('Deploy to Production') {
            agent { label 'deploy' }
            when {
                allOf {
                    branch 'main'
                    tag pattern: 'v\\d+\\.\\d+\\.\\d+', comparator: 'REGEXP'
                }
            }
            steps {
                timeout(time: 5, unit: 'MINUTES') {
                    input message: 'Deploy to production?', ok: 'Ship it'
                }
                sh "./scripts/deploy.sh production ${IMAGE}:${env.BUILD_NUMBER}"
            }
            post {
                success {
                    slackSend channel: '#deployments',
                              color: 'good',
                              message: "Deployed ${IMAGE}:${env.BUILD_NUMBER} to production"
                }
                failure {
                    slackSend channel: '#deployments',
                              color: 'danger',
                              message: "Production deploy FAILED for ${IMAGE}:${env.BUILD_NUMBER}"
                }
            }
        }
    }

    post {
        always {
            echo 'Pipeline complete.'
        }
        cleanup {
            deleteDir()
        }
    }
}

Environment Variables and Options

Two more first-class directives complete the declarative grammar. The environment block injects variables into env for the pipeline or stage scope. The options block configures pipeline-level behaviors:

timeout(time: 30, unit: 'MINUTES') — kill the whole pipeline if it exceeds the budget.
retry(3) — retry the entire pipeline on failure (use per-stage retry for finer control).
buildDiscarder(logRotator(numToKeepStr: '10')) — prevent disk exhaustion by capping retained builds.
disableConcurrentBuilds() — ensures only one instance of this pipeline runs at a time; critical for deploy pipelines.
skipDefaultCheckout() — disables the automatic source checkout so you control it explicitly with checkout scm.

Production pitfall — missing buildDiscarder: a high-velocity service with no log rotation will fill the Jenkins controller disk within days. Every Jenkinsfile should define buildDiscarder(logRotator(numToKeepStr: '20', artifactNumToKeepStr: '5')) as a baseline.

Common Failure Modes

Understanding how the grammar fails is as important as knowing how to use it:

Parse-time errors: Groovy syntax mistakes, undeclared variables, and invalid directive placements are caught when Jenkins loads the Jenkinsfile — before any executor is allocated. Always use the Replay feature to iterate quickly without committing.
Agent allocation timeout: if no matching agent is available, Jenkins will queue the build indefinitely unless you set options { timeout(...) }.
when evaluated before agent: by default when is evaluated before the stage agent is allocated — good for efficiency. If you need the agent already running (e.g., to inspect the workspace), add beforeAgent false inside the when block.
stash/unstash across agents: stash is stored on the controller; for large artifacts (hundreds of MB) it becomes a bottleneck. Use an external artifact store (Artifactory, S3, Nexus) and pass only coordinates between stages.

Declarative Pipeline's strict grammar is not a limitation — it is the contract that makes Jenkinsfiles auditable, diff-able, and maintainable across hundreds of repositories. Master this grammar before reaching for Scripted Pipeline's escape hatches.

Declarative Pipelines

Declarative Pipelines

The Mandatory Skeleton

The agent Directive

Stages and Steps

The post Section

The when Directive

Environment Variables and Options

Common Failure Modes

The `agent` Directive

The `post` Section

The `when` Directive