Jenkins & Enterprise CI/CD

Shared Libraries

18 min Lesson 6 of 28

Shared Libraries

Every engineering organization that runs Jenkins at scale eventually confronts the same problem: dozens of teams each maintain their own Jenkinsfile, and the files start to look disturbingly similar. The Docker login step is copy-pasted across forty repositories. A change to the corporate vulnerability-scan policy requires editing thirty pipelines by hand. A new security team requirement lands at 5 PM on a Friday and every team must patch their pipeline before Monday's release train. This is the pipeline duplication problem, and it is the direct motivation for Jenkins Shared Libraries.

A Shared Library is a Git repository that holds reusable Groovy code — functions, classes, and entire pipeline templates — that any pipeline on the Jenkins controller can import with a single @Library annotation. The library is version-controlled independently, loaded once at runtime, and consumed identically by every Jenkinsfile that declares it. The pipeline is now configuration; the organization's standards live in code.

Key idea: A Shared Library is not a convenience shortcut for lazy engineers — it is the mechanism by which a platform team enforces organization-wide CI/CD standards without owning every individual pipeline. Policy changes deploy once to the library, and every consuming pipeline picks them up on the next build without any Jenkinsfile edits.

Repository Layout

Jenkins enforces a strict directory convention for shared library repositories. Deviating from it causes silent load failures, so internalize the structure before writing a single line of code.

my-shared-library/           # Git repository root
├── vars/                    # Global variables — one file = one pipeline step
│   ├── buildDockerImage.groovy
│   ├── runSecurityScan.groovy
│   └── deployToKubernetes.groovy
├── src/                     # Groovy classes — full OOP, compiled at load time
│   └── com/
│       └── acme/
│           ├── Docker.groovy
│           └── Kubernetes.groovy
├── resources/               # Non-Groovy files: shell scripts, JSON templates, YAML snippets
│   └── com/
│       └── acme/
│           └── deploy-template.yaml
└── README.md

The vars/ directory is where most teams start. Each .groovy file in vars/ becomes a global variable available to every pipeline. If the file defines a call() method, the variable name itself becomes a callable step. The src/ directory follows standard Java/Groovy package conventions and provides full class hierarchy, inheritance, and static methods for complex logic that does not fit a single function.

Registering the Library in Jenkins

Before any pipeline can use a library, a Jenkins administrator must register it under Manage Jenkins → System → Global Pipeline Libraries. Key settings to understand in production:

Name — the identifier used in @Library('my-lib'). Use a stable, namespaced name (e.g., acme-platform-lib). Changing it later requires editing every consuming Jenkinsfile.
Default version — the branch, tag, or commit SHA loaded when a pipeline says @Library('acme-platform-lib') without a version qualifier. In production, point this to a release tag or a protected main branch — never to an unprotected feature branch.
Load implicitly — if enabled, every pipeline loads the library without an annotation. Useful for enforcing mandatory steps (security scans, audit logging) that no team can skip. Use with care: implicit libraries run code in every build, including multi-branch discovery builds, so any bug in the library breaks the entire Jenkins instance.
Allow default version to be overridden — lets individual pipelines pin to a specific version via @Library('acme-platform-lib@v2.1.0'). Essential for library upgrades — teams can test against the new version before it becomes the default.

The library can be sourced from any SCM Jenkins supports: GitHub, GitLab, Bitbucket, or a plain Git URL. In large organizations, the library repository usually has branch protection rules, mandatory code review, and its own CI pipeline that runs unit tests using the jenkins-pipeline-unit framework.

Writing a vars/ Step

The most common pattern is a vars/ step that wraps a multi-step action teams would otherwise duplicate. Here is a production-quality example: a Docker build-and-push step that enforces the organization's registry naming convention, injects credentials from Jenkins Credential Store, and tags with both the commit SHA and a semantic version label.

// vars/buildAndPushImage.groovy

def call(Map config = [:]) {
    // Require mandatory parameters — fail fast with a clear error
    if (!config.imageName) {
        error "buildAndPushImage: 'imageName' is required"
    }

    def registry    = config.registry    ?: 'registry.acme.internal'
    def imageTag    = config.imageTag    ?: env.GIT_COMMIT.take(8)
    def credId      = config.credId      ?: 'docker-registry-creds'
    def dockerfile  = config.dockerfile  ?: 'Dockerfile'
    def buildArgs   = config.buildArgs   ?: [:]

    def fullImage = "${registry}/${config.imageName}:${imageTag}"
    def buildArgStr = buildArgs.collect { k, v -> "--build-arg ${k}=${v}" }.join(' ')

    docker.withRegistry("https://${registry}", credId) {
        def img = docker.build(fullImage, "${buildArgStr} -f ${dockerfile} .")
        img.push()
        img.push('latest')  // also tag latest for convenience
        echo "Pushed ${fullImage}"
        return fullImage
    }
}

A pipeline in any repository can now consume this with minimal boilerplate. The organization's registry, credential, and tagging convention are enforced in one place — the library — not scattered across forty Jenkinsfiles.

// Jenkinsfile in an application repository
@Library('acme-platform-lib@v3.2.1') _

pipeline {
    agent { label 'docker-builder' }

    stages {
        stage('Build & Push') {
            steps {
                buildAndPushImage(
                    imageName: 'payments-service',
                    buildArgs: [APP_VERSION: env.BUILD_NUMBER]
                )
            }
        }

        stage('Deploy') {
            steps {
                deployToKubernetes(
                    cluster: 'prod-us-east-1',
                    namespace: 'payments',
                    image: "registry.acme.internal/payments-service:${env.GIT_COMMIT.take(8)}"
                )
            }
        }
    }
}

Pipeline Templates: Whole-Pipeline Reuse

The most powerful use of shared libraries is not individual steps but entire pipeline templates. A platform team defines a complete, opinionated pipeline for a project type (e.g., "Java microservice", "React SPA", "Helm chart") inside the library, and application teams call it as a single function. The application Jenkinsfile collapses to three lines.

// vars/javaMicroservicePipeline.groovy  (pipeline template in the library)

def call(Map config = [:]) {
    pipeline {
        agent { label 'jdk-21' }

        options {
            timeout(time: 30, unit: 'MINUTES')
            buildDiscarder(logRotator(numToKeepStr: '20'))
            disableConcurrentBuilds()
        }

        environment {
            MAVEN_OPTS = '-Xmx1g -Xms256m'
            SONAR_URL  = 'https://sonar.acme.internal'
        }

        stages {
            stage('Compile') {
                steps { sh 'mvn -B compile' }
            }
            stage('Test') {
                steps { sh 'mvn -B verify' }
                post { always { junit 'target/surefire-reports/*.xml' } }
            }
            stage('SonarQube Analysis') {
                when { branch 'main' }
                steps {
                    withSonarQubeEnv('acme-sonar') {
                        sh 'mvn sonar:sonar'
                    }
                    timeout(time: 5, unit: 'MINUTES') {
                        waitForQualityGate abortPipeline: true
                    }
                }
            }
            stage('Build & Push Image') {
                steps {
                    buildAndPushImage(imageName: config.serviceName)
                }
            }
            stage('Deploy to Staging') {
                when { branch 'main' }
                steps {
                    deployToKubernetes(
                        cluster: 'staging',
                        namespace: config.namespace ?: config.serviceName,
                        image: "registry.acme.internal/${config.serviceName}:${env.GIT_COMMIT.take(8)}"
                    )
                }
            }
        }

        post {
            failure { slackSend channel: config.slackChannel ?: '#ci-alerts', message: "FAILED: ${env.JOB_NAME} #${env.BUILD_NUMBER}" }
        }
    }
}

An application team's entire Jenkinsfile is now:

@Library('acme-platform-lib@v3.2.1') _
javaMicroservicePipeline(serviceName: 'payments-service', slackChannel: '#payments-team')

Architecture: How the Library Loads

The diagram below illustrates the runtime relationship between a Jenkinsfile, the shared library repository, and the Jenkins controller.

Jenkins loads and compiles the shared library from its own Git repository before executing any pipeline step.

Production Pitfalls & Best Practices

Production pitfall — Groovy CPS serialization: Jenkins pipelines run in a Continuation Passing Style (CPS) interpreter, not a standard JVM. Every pipeline variable must be serializable to disk (for pause/resume). If you call a non-serializable Java type (e.g., a java.io.File) in a vars/ step without wrapping it in a @NonCPS method, the build fails with a cryptic NotSerializableException. The rule: any method that deals with non-serializable objects must be annotated @NonCPS, but then it cannot call CPS-transformed pipeline steps. Keep a hard boundary between @NonCPS utility methods and CPS-aware step methods.

Pro practice — version pinning strategy: The library's default version in Jenkins global config should always be a released semver tag (e.g., v3.2.1), never main or HEAD. Teams pin to a specific version in their @Library annotation during upgrade testing, then drop the pin once the new default is set. This gives you a rolling upgrade path with zero forced-flag-day migrations. Tag your library releases from CI — a library with no tests and no tags is a liability, not an asset.

Key idea — sandboxing: Jenkins runs library code in a Groovy sandbox by default. The sandbox blocks certain method calls (file I/O, reflection, external HTTP) unless an administrator explicitly approves them in Manage Jenkins → Script Approval. Libraries from trusted SCM sources can be marked Trusted in Global Pipeline Libraries settings, which bypasses the sandbox. Only mark libraries as trusted if they are owned by the platform team and have mandatory code review — a trusted library runs with full Jenkins controller privileges.