Configure Jenkins with JCasC and Reproducible Bootstrap
Jenkins Configuration as Code turns controller settings into reviewable YAML, but it does not automatically capture every plugin, job, credential secret or host dependency. A reproducible controller needs a versioned plugin set, external secret injection and a tested bootstrap order.
Start from a known controller checkpoint
Use the accepted controller and agent checkpoint from the prior lesson. Record the Jenkins version, Java runtime, active configuration, plugin inventory and current Git revision before changing this boundary.
java -version
sudo systemctl is-active jenkins
sudo journalctl -u jenkins -n 50 --no-pager
curl -fsS http://127.0.0.1:8080/login >/dev/nullUnderstand the operating boundary
| Decision | Implementation | Evidence |
|---|---|---|
| Scope | Name the controller, folder, job, node and environment | The selected boundary is visible |
| Input | Use reviewed source and scoped credentials | Revision and credential ID are attributable |
| Execution | Set label, timeout and concurrency policy | Queue and node evidence match intent |
| Recovery | Preserve the last known working state | Rollback is rehearsed before promotion |
Prepare the lab
Install the Configuration as Code plugin on a disposable controller, view the current export and remove generated secrets or environment-specific values before committing anything.
sudo install -d -o jenkins -g jenkins -m 0750 /etc/jenkins/casc
sudo install -m 0640 -o jenkins -g jenkins /dev/null /etc/jenkins/casc/jenkins.yaml
sudo systemctl edit jenkins
# add Environment="CASC_JENKINS_CONFIG=/etc/jenkins/casc/jenkins.yaml"
sudo systemctl daemon-reloadImplement it step by step
- Record the exact core and plugin versions that understand the exported attributes.
- Split YAML by ownership only when all files form one unambiguous configuration.
- Reference secrets through environment or supported secret sources; never export encrypted controller values into Git.
- Use the JCasC check endpoint or UI validation before reload.
- Provision jobs through reviewed job definitions or organization discovery rather than undocumented clicks.
- Rebuild a clean lab controller and compare effective configuration before production use.
The export is a starting observation, not automatically a portable desired state. Plugin-specific sections disappear when their plugin is absent and can become invalid when attributes change. The plugin catalog and JCasC revision therefore form one release unit.
jenkins:
systemMessage: 'Managed by JCasC'
numExecutors: 0
mode: EXCLUSIVE
securityRealm:
local:
allowsSignup: false
users:
- id: admin-lab
password: "${ADMIN_LAB_PASSWORD}"
authorizationStrategy:
loggedInUsersCanDoAnything:
allowAnonymousRead: false
unclassified:
location:
url: 'https://jenkins.example.test/'Bootstrap order is operationally significant. Install the supported core and the pinned plugin catalog before loading attributes owned by those plugins. Make secret sources available before interpolation, but restrict their files or workload identity to the controller process. Load JCasC, provision discoverable jobs, connect agents and only then admit production triggers. A controller that starts with half its policy missing is not ready merely because the login page responds.
Detect drift by comparing the effective export with reviewed source, while filtering values that are generated or intentionally external. Decide whether emergency UI changes are prohibited or allowed temporarily. If allowed, require an incident record and a follow-up source commit; otherwise the next reload silently removes the repair.
Verify the positive path
Run validation with the same plugin set that will load the file. Reload on the disposable controller, inspect system message, executor count, URL and authorization, then restart to prove bootstrap persistence.
sudo systemctl restart jenkins
sudo journalctl -u jenkins -n 200 --no-pager
curl -fsS https://jenkins.example.test/login >/dev/null
git -C /srv/jenkins-casc rev-parse HEAD
sha256sum /etc/jenkins/casc/jenkins.yaml| Check | Expected evidence | Reject when |
|---|---|---|
| Configuration | Matches reviewed source | UI drift or unresolved placeholder remains |
| Execution | Runs on the intended isolated node | Controller or wrong trust zone executes code |
| Evidence | Revision, result and outputs are retained | Green status has no attributable output |
| Recovery | Known state can be restored and verified | Recovery depends on an improvised manual edit |
Reject a plugin-owned unknown attribute
Add a misspelled or removed attribute in the lab. Validation must fail before production reload. Retain the error showing its path, restore the prior Git revision and prove that a restart loads the accepted configuration.
cp /etc/jenkins/casc/jenkins.yaml /tmp/jenkins.yaml.bad
printf '\n unknownAttribute: true\n' >> /tmp/jenkins.yaml.bad
# Submit only to the lab JCasC validation endpoint or UI
git -C /srv/jenkins-casc diff --exit-codeTroubleshoot by failed layer
| Symptom | Inspect | Correction |
|---|---|---|
| Queued or unavailable | Label, executor, node and network | Repair the failed scheduling or transport layer |
| Configuration rejected | Controller log, syntax and plugin ownership | Correct source; do not bypass validation |
| Job fails unexpectedly | First causal console error and agent logs | Fix one layer and rerun the smallest scope |
| Second run differs | Mutable dependency, workspace or UI drift | Pin inputs and remove hidden retained state |
Unsafe shortcuts
- Unsafe: granting administrator access to avoid designing a narrow permission removes accountability.
- Unsafe: binding protected secrets around untrusted repository code permits exfiltration despite masking.
- Unsafe: changing controller state without a verified backup and rollback turns a small error into an outage.
Operate, recover and retain evidence
Own the configuration, plugin and credential dependencies explicitly. Record the controller version, Git revision, immutable tool or artifact identity, initiator, approver and acceptance result. Rehearse the failure path on a disposable controller before adopting it as production procedure.
sudo systemctl stop jenkins
sudo rsync -aHAX --numeric-ids /backup/jenkins-known-good/ /var/lib/jenkins/
sudo restorecon -RF /var/lib/jenkins 2>/dev/null || true
sudo systemctl start jenkins
sudo journalctl -u jenkins -n 150 --no-pagerWorked use cases
| Situation | Design choice | Acceptance |
|---|---|---|
| Lab rollout | Apply to one disposable controller or folder | Positive, negative and recovery results are retained |
| Team rollout | Promote the same reviewed revision | Permissions and behavior remain consistent |
| Production change | Use backup, change window and acceptance | Failure stays bounded and rollback is tested |
Knowledge checks
Independent lab
- Capture the starting version, configuration and plugin evidence.
- Implement the change on a disposable controller or folder.
- Run one successful case and one controlled failure.
- Restore the accepted state and prove service behavior, not only process state.
- Repeat from a clean source checkout without relying on remembered UI actions.