Lesson 362 · AWS Learning Path

AWS 362: Buildspec phases, variables, secrets, caching, reports, and artifacts

· Published · 6 min read

Labelled process diagram for AWS 362: Versioned intent to Automated validation to Controlled AWS change to Observed result and retained evidence, with decision, proof and rejection evidence.

Why this lesson matters

The buildspec is executable release policy. It selects commands, runtimes, secrets, caches, reports, and artifacts. A small indentation or path error can skip evidence, publish the wrong files, or expose credentials.

Buildspec 0.2 model

The major sections include version, optional run-as, env, proxy, batch, phases, reports, artifacts, and cache. Buildspec 0.2 keeps commands in the same shell context within a phase, unlike the older 0.1 command isolation behavior.

Phases are install, pre_build, build, and post_build. Each can have commands and finally; supported phases can define failure behavior. finally runs after commands in its phase even if a command fails, but cannot help if the host is forcibly terminated and should not hide the original failure.

Complete starting buildspec

version: 0.2

env:
  shell: bash
  variables:
    APP_NAME: aws-read-inventory
  parameter-store:
    QUALITY_PROFILE: /p20/build/quality-profile
  secrets-manager:
    PRIVATE_INDEX_TOKEN: /p20/build/package-token:token
  exported-variables:
    - ARTIFACT_SHA256

phases:
  install:
    runtime-versions:
      python: 3.12
    commands:
      - set -Eeuo pipefail
      - python -m pip install --requirement requirements-lock.txt
  pre_build:
    commands:
      - set -Eeuo pipefail
      - python -m compileall -q src tests
      - mkdir -p reports dist
      - pytest --junitxml=reports/junit.xml
    finally:
      - printf 'phase=pre_build complete\n'
  build:
    commands:
      - set -Eeuo pipefail
      - python -m build
      - test "$(find dist -maxdepth 1 -type f | wc -l)" -ge 2
      - sha256sum dist/* > dist/SHA256SUMS
      - ARTIFACT_SHA256=$(sha256sum dist/SHA256SUMS | awk '{print $1}')
  post_build:
    commands:
      - set -Eeuo pipefail
      - test -s reports/junit.xml
      - test -s dist/SHA256SUMS

reports:
  unit-tests:
    files:
      - junit.xml
    base-directory: reports
    file-format: JUNITXML

artifacts:
  files:
    - '**/*'
  base-directory: dist
  discard-paths: no
  name: aws-read-inventory-$CODEBUILD_RESOLVED_SOURCE_VERSION

cache:
  paths:
    - '/root/.cache/pip/**/*'

Treat it as reviewed source, not universal copy/paste. The image must contain/build-install the build package, secret use must not appear in commands/logs, report naming must match the project, and cache path/user varies by image.

YAML and shell boundaries

YAML parses before Bash. Quote values containing colon, #, booleans, or special characters as needed. Use a YAML parser/linter and CodeBuild's supported schema. Block scalar syntax helps complex commands, but prefer versioned scripts under test over long inline shell.

Each phase should explicitly establish shell safety because environment behavior can differ. Quote variables, use arrays for commands where possible, avoid eval, validate paths, and preserve exit status. Do not pipe failures into successful commands without pipefail.

Variables and precedence

Variables can come from image, buildspec, project, start-build overrides, Parameter Store, Secrets Manager, CodePipeline namespaces, and reserved CODEBUILD_* values. Document precedence and forbid untrusted override of security-critical values such as artifact bucket, role, account, or test-skip flag.

Value classAppropriate sourceForbidden use
Public build constantVersioned buildspec/scriptSecret or environment-specific authority
Environment configurationProtected project/pipeline variableUnreviewed source override
Sensitive valueExact Parameter Store/Secrets Manager mappingPlaintext buildspec/project/log/export
AWS runtime metadataReserved CODEBUILD_* variableTreating it as human approval
Downstream release metadataExported non-secret variable or signed manifestCredential, token, or untrusted path

Exported variables pass small non-secret metadata to downstream pipeline actions. They are resolved by the end of post_build; do not export Parameter Store secrets, Secrets Manager values, or values whose names begin with AWS_. Prefer artifact manifests for richer immutable evidence.

Secrets and parameters

Parameter/secret mappings require the project role to read exact resources and KMS authorization where applicable. Retrieval success does not prevent build code from printing values. Protect source and buildspec changes, disable tracing, never echo environment, redact exceptions, and scan logs/artifacts with a fake canary.

Use a separate secret for each trust boundary, short lifetime where supported, rotation, and incident revocation. Do not place secret text into artifact names, reports, exported variables, cache keys, or command arguments.

Failure and retries

Buildspec supports phase failure behavior for eligible environments, including abort/continue and retry forms. Retrying a whole command block can repeat uploads, tests, mutations, or external actions. Restrict retry to classified transient/idempotent operations with bounded count and logs. Never continue after a mandatory test/security gate fails.

Use finally to gather redacted diagnostics and cleanup owned temporary state, then preserve failure. A post_build phase can run after some earlier outcomes depending on build behavior; it is not proof that build succeeded. Gate artifact publication on explicit required evidence.

Runtime versions and reproducibility

Specify supported runtime versions for the selected managed image. Record image digest, runtime/tool versions, lock digest, source revision, and locale/timezone. A runtime request unsupported by the image fails or behaves differently; verify in the actual image.

Dependency install must use a reviewed lock and trusted repository/origin controls. Network retrieval makes builds dependent on current upstream state unless packages are retained and hashes verified.

Cache design

Local cache can improve repeated builds on compatible hosts; S3 cache can persist across hosts. Neither is authoritative release input. Key cache by lock digest, OS/architecture, runtime/tool version, and trust class. Prevent untrusted branches from poisoning release caches.

Test cold, warm, stale, corrupted, and disabled cache. Compare output digests; a cache hit that changes artifact bytes is a reproducibility failure.

Reports and coverage

Reports can contain test, coverage, file paths, failure messages, and captured output. Choose supported format, base directory, files/globs, and report group. Enforce a quality threshold in tested commands; uploading a report alone does not fail a release for poor coverage.

Sanitize tests so reports contain no secrets/customer data. Define retention and access. Missing report should fail when it is required evidence.

Artifacts and secondary artifacts

base-directory changes the root for file matching. discard-paths can flatten names and cause collisions. Symlink behavior, hidden files, generated debug data, and broad **/* patterns require inspection.

Use secondary artifacts when distinct consumers need separate packages/reports/manifests, each with unique identifier and explicit files/base directory. Verify archive list and digest before upload. Bind source revision, build ID, lock, tests, image, and artifact hashes in a manifest.

Local validation

Run commands in the matching container/image where authorized, not merely the host:

python -m compileall -q src tests
pytest --junitxml=reports/junit.xml
python -m build
find dist -maxdepth 2 -type f -printf '%P\n' | sort
sha256sum dist/*

Use fake values for secret-canary tests. Never download production secrets to validate a buildspec locally.

Failure lab

Inject 12 cases: YAML parse error, wrong runtime, missing lock, test failure, command pipeline masks failure, secret printed by tracing, Parameter Store denial, KMS denial, stale cache changes output, report glob empty, artifact glob includes .env, and secondary artifact identifier mismatch.

For each record expected phase/status, logs safe to retain, whether artifact/report exists, retry decision, correction, and regression test.

Acceptance

Submit complete buildspec, versioned helper scripts, precedence/threat table, role/KMS requirements, cold/warm cache proof, report/coverage gate, archive file list, artifact/provenance digest, 12 failures, cleanup, and rollback.

Pass requires mandatory failures remain non-zero, no secret in any output, deterministic artifact from same inputs, exact reports/artifacts, bounded safe retry, and no security-critical untrusted override.

Official sources

Advertisement