Hugo Blog — Hardening the Pipeline

Securing the CI/CD Chain

In the previous post we built a full GitOps pipeline: push to Forgejo, build a Hugo image, ship it to k3s via Flux. It worked. But it was naive in a few ways — running nginx as root, no image scanning, no signing, and deploying by mutable tag.

This post covers the fixes. Three independent improvements, each worth doing on its own:

Unprivileged nginx — drop root from the container entirely
Trivy — scan the image for HIGH/CRITICAL CVEs before it ever touches the registry
Cosign — sign the image so the cluster can verify it came from CI

The full updated files are at the bottom. Here is the reasoning behind each change.

1 — Unprivileged nginx

The original Dockerfile used nginx:1.27-alpine. That image runs the master process as root (it needs to bind port 80). The worker processes drop to nginx, but root is still involved at startup.

The fix is nginxinc/nginx-unprivileged:1.27-alpine. This is the official upstream image maintained by the nginx team. It:

Listens on 8080 instead of 80 (no privileged port, no root needed)
Runs everything as the nginx user from the start
Ships with a conf that already expects unprivileged operation

The only catch: you still want to run apk upgrade to patch OS-level packages. Since the unprivileged image starts as nginx, you need to briefly switch to root to run the upgrade, then drop back:

# --- Builder Stage ---
FROM hugomods/hugo:0.157.0 AS builder
WORKDIR /site
COPY . .
RUN hugo --minify

# --- Final Stage ---
FROM nginxinc/nginx-unprivileged:1.27-alpine

USER root
RUN apk update && apk upgrade --no-cache
USER nginx

COPY --from=builder /site/public /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf

EXPOSE 8080
CMD ["nginx", "-g", "daemon off;"]

Nothing else in the k8s manifests needs to change — containerPort: 8080 was already there.

2 — Security Headers and nginx Hardening

While touching the nginx config, a few headers worth adding:

server_tokens off;  # don't leak nginx version in error pages / headers

add_header X-Content-Type-Options "nosniff" always;
add_header X-Frame-Options "DENY" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;

The always flag ensures headers are sent even on error responses, not just 200s. HSTS only makes sense because Cloudflare terminates TLS in front — the header tells browsers to enforce HTTPS for a year.

3 — Trivy: Scan Before Push

The workflow used to build, push, done. The new order is: build, scan, push only if clean.

Trivy runs against the locally-built image (via digest) before it ever reaches the registry. If it finds any unfixed HIGH or CRITICAL CVEs, the pipeline exits 1 and the image is never pushed.

- name: Build and push image
  run: |
    docker build -t ${{ env.IMAGE }}:${{ env.TAG }} .
    docker push ${{ env.IMAGE }}:${{ env.TAG }}
    DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' ${{ env.IMAGE }}:${{ env.TAG }})
    echo "IMAGE_WITH_DIGEST=${DIGEST}" >> $GITHUB_ENV

- name: Cache Trivy DB
  uses: actions/cache@v3
  with:
    path: ~/.cache/trivy
    key: trivy-db-${{ runner.os }}

- name: Scan image with Trivy
  run: |
    if ! command -v trivy &> /dev/null; then
      apt-get update && apt-get install -y wget apt-transport-https gnupg
      wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor > /usr/share/keyrings/trivy.gpg
      echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb generic main" > /etc/apt/sources.list.d/trivy.list
      apt-get update && apt-get install -y trivy
    fi
    trivy image \
      --exit-code 1 \
      --severity HIGH,CRITICAL \
      --format table \
      --ignore-unfixed \
      ${{ env.IMAGE_WITH_DIGEST }}

A few notes:

--ignore-unfixed skips CVEs where no fix exists yet — these are noise you can’t act on.
--exit-code 1 is what actually gates the pipeline. Without it, Trivy is just reporting.
The Trivy DB cache step matters on self-hosted runners where the DB has to be downloaded every run. On Forgejo runners it cuts scan time significantly.
Trivy is installed via the official apt repo rather than curl | bash, which is the right call in a CI environment.

4 — Cosign: Sign the Image

After scanning, the image gets signed with Cosign using a key stored in Forgejo secrets. This gives you a verifiable chain: this exact digest was produced by this CI run and nobody tampered with it between push and deploy.

- name: Install Cosign
  uses: sigstore/[email protected]

- name: Sign image with Cosign
  env:
    COSIGN_PASSWORD: ${{ secrets.COSIGN_PASSWORD }}
  run: |
    echo "${{ secrets.COSIGN_PRIVATE_KEY }}" > /tmp/cosign.key
    cosign sign --yes --key /tmp/cosign.key \
      --registry-referrers-mode=legacy \
      --registry-username ${{ secrets.REGISTRY_USER }} \
      --registry-password ${{ secrets.REGISTRY_TOKEN }} \
      ${{ env.IMAGE_WITH_DIGEST }}
    rm /tmp/cosign.key

--registry-referrers-mode=legacy is needed for Forgejo’s container registry, which doesn’t yet support the OCI 1.1 referrers API. Without this flag, Cosign tries to use the newer spec and fails.

The signature is stored as a separate tag in the same registry, referenced by the image digest.

Generating the key pair

If you haven’t done this yet:

cosign generate-key-pair
# outputs cosign.key (private) and cosign.pub (public)

Add to Forgejo secrets:

Secret	Value
`COSIGN_PRIVATE_KEY`	contents of `cosign.key`
`COSIGN_PASSWORD`	the passphrase you set

Keep cosign.pub in your repo or somewhere accessible — you’ll need it to verify.

Verifying a signature

cosign verify \
  --key cosign.pub \
  forgejo.uclab.dev/affragak/uclab@sha256:<digest>

5 — Deploy by Digest, Not Tag

The original sed in the workflow replaced the tag portion of the image ref:

# old
sed -i "s|image: ${{ env.IMAGE }}:.*|image: ${{ env.IMAGE }}:${{ env.TAG }}|" $DEPLOY_FILE

The new version replaces the entire image: line with the full digest ref:

# new
sed -i "s|image: .*|image: ${{ env.IMAGE_WITH_DIGEST }}|" $DEPLOY_FILE

A digest ref (image: registry/repo@sha256:abc123...) is immutable. A tag ref is not — someone could push a different image to the same tag. Deploying by digest means Kubernetes will always pull exactly the image that was scanned and signed, nothing else.

This also pairs naturally with Cosign verification: the digest in the manifest is the same one the signature is attached to.

Updated Files

Dockerfile

# --- Builder Stage ---
FROM hugomods/hugo:0.157.0 AS builder
WORKDIR /site
COPY . .
RUN hugo --minify

# --- Final Stage ---
FROM nginxinc/nginx-unprivileged:1.27-alpine

USER root
RUN apk update && apk upgrade --no-cache
USER nginx

COPY --from=builder /site/public /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf

EXPOSE 8080
CMD ["nginx", "-g", "daemon off;"]

nginx.conf

server {
    listen 8080 default_server;
    server_name _;

    root /usr/share/nginx/html;
    index index.html;

    server_tokens off;

    access_log /dev/stdout;
    error_log  /dev/stderr warn;

    gzip_static on;

    add_header X-Content-Type-Options "nosniff" always;
    add_header X-Frame-Options "DENY" always;
    add_header Referrer-Policy "strict-origin-when-cross-origin" always;
    add_header X-XSS-Protection "1; mode=block" always;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;

    location / {
        try_files $uri $uri/ $uri.html =404;
    }

    location ~* \.(css|js|png|jpg|jpeg|gif|ico|svg|woff|woff2)$ {
        expires 1y;
        add_header Cache-Control "public, immutable";
        access_log off;
    }

    location ~* \.html$ {
        add_header Cache-Control "no-cache";
    }

    location ~ /\.(?!well-known).* {
        deny all;
    }
}

.forgejo/workflows/build.yml

name: Build and Deploy
on:
  push:
    branches: [main]
env:
  REGISTRY: forgejo.uclab.dev
  IMAGE: forgejo.uclab.dev/affragak/uclab
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Hugo repo (with submodules)
        uses: actions/checkout@v4
        with:
          submodules: recursive
          fetch-depth: 0

      - name: Set image tag
        run: |
          SHORT_SHA=$(git rev-parse --short HEAD)
          echo "TAG=${SHORT_SHA}" >> $GITHUB_ENV

      - name: Login to Forgejo registry
        run: echo "${{ secrets.REGISTRY_TOKEN }}" | docker login ${{ env.REGISTRY }} -u ${{ secrets.REGISTRY_USER }} --password-stdin

      - name: Build and push image
        run: |
          docker build -t ${{ env.IMAGE }}:${{ env.TAG }} .
          docker push ${{ env.IMAGE }}:${{ env.TAG }}
          DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' ${{ env.IMAGE }}:${{ env.TAG }})
          echo "IMAGE_WITH_DIGEST=${DIGEST}" >> $GITHUB_ENV

      - name: Cache Trivy DB
        uses: actions/cache@v3
        with:
          path: ~/.cache/trivy
          key: trivy-db-${{ runner.os }}

      - name: Scan image with Trivy
        run: |
          if ! command -v trivy &> /dev/null; then
            apt-get update && apt-get install -y wget apt-transport-https gnupg
            wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor > /usr/share/keyrings/trivy.gpg
            echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb generic main" > /etc/apt/sources.list.d/trivy.list
            apt-get update && apt-get install -y trivy
          fi
          trivy image \
            --exit-code 1 \
            --severity HIGH,CRITICAL \
            --format table \
            --ignore-unfixed \
            ${{ env.IMAGE_WITH_DIGEST }}

      - name: Install Cosign
        uses: sigstore/[email protected]

      - name: Sign image with Cosign
        env:
          COSIGN_PASSWORD: ${{ secrets.COSIGN_PASSWORD }}
        run: |
          echo "${{ secrets.COSIGN_PRIVATE_KEY }}" > /tmp/cosign.key
          cosign sign --yes --key /tmp/cosign.key \
            --registry-referrers-mode=legacy \
            --registry-username ${{ secrets.REGISTRY_USER }} \
            --registry-password ${{ secrets.REGISTRY_TOKEN }} \
            ${{ env.IMAGE_WITH_DIGEST }}
          rm /tmp/cosign.key

      - name: Setup SSH for GitHub
        run: |
          mkdir -p ~/.ssh
          echo "${{ secrets.GH_DEPLOY_KEY }}" > ~/.ssh/id_ed25519
          chmod 600 ~/.ssh/id_ed25519
          ssh-keyscan github.com >> ~/.ssh/known_hosts

      - name: Clone k3s repo and update image tag
        run: |
          git clone [email protected]:affragak/pi5cluster.git /tmp/pi5cluster
          cd /tmp/pi5cluster
          git config user.name "forgejo-ci"
          git config user.email "[email protected]"
          DEPLOY_FILE="blog/base/uclab/deployment.yaml"
          sed -i "s|image: .*|image: ${{ env.IMAGE_WITH_DIGEST }}|" $DEPLOY_FILE
          git add $DEPLOY_FILE
          git diff --staged --quiet && echo "No changes to commit" && exit 0
          git commit -m "chore: deploy hugo ${{ env.TAG }}"
          git push

Summary of Changes

What	Before	After
Base image	`nginx:1.27-alpine` (runs as root)	`nginxinc/nginx-unprivileged:1.27-alpine`
CVE scanning	None	Trivy, gates on HIGH/CRITICAL unfixed
Image signing	None	Cosign with key stored in Forgejo secrets
Deployment ref	Mutable tag (`image:sha`)	Immutable digest (`image@sha256:...`)
nginx hardening	Minimal	`server_tokens off` + security headers

None of these are dramatic changes. Together they close the obvious gaps in the original setup: a root-running container, no verification that the image is clean, no way to prove the image wasn’t tampered with after it left CI, and a mutable tag that Kubernetes could silently re-pull to a different image.