refactor: consolidate apt source handling into install_deps.sh

- merge APT_MIRROR and EOL archive switching into install_deps.sh fix_apt_sources() (buster unconditional, bullseye probed via Release/InRelease with fallback to archive.debian.org) - make DEBMIRROR follow APT_MIRROR in version.env (nounset-safe) - remove switch_archive_sources.sh (no remaining callers) - simplify docker/Dockerfile.deps to single RUN and update CI to use install_deps.sh --fix-apt-only - update docs (AGENTS.md/README.md) to reflect new flow
2026-09-20 08:03:46 +08:00 · 2026-09-01 12:07:57 +08:00
parent 93ba9e6727
commit 82d0f3f0ab
8 changed files with 195 additions and 50 deletions
--- a/.github/workflows/build-deps-images.yml
+++ b/.github/workflows/build-deps-images.yml
@@ -6,7 +6,6 @@ on:
    branches: [ master ]
    paths:
      - 'install_deps.sh'
-      - 'switch_archive_sources.sh'
      - 'docker/Dockerfile.deps'

 env:
--- a/.github/workflows/create-release.yml
+++ b/.github/workflows/create-release.yml
@@ -57,7 +57,8 @@ jobs:
      - name: Install test with ${{ matrix.version }}
        run: |
          docker run --rm -v ${{ github.workspace }}:/work -w /work library/${{ matrix.version }} bash -c "
-            bash /work/switch_archive_sources.sh &&
+            # EOL archive fix via install_deps.sh --fix-apt-only
+            bash /work/install_deps.sh --fix-apt-only &&
            apt update -qq >/dev/null 2>&1 &&
            apt install -y --no-install-recommends ./output/*.deb && ssh -V"

--- a/AGENTS.md
+++ b/AGENTS.md
@@ -4,7 +4,7 @@ Shell scripts that backport OpenSSH from Debian sid to older Debian/Ubuntu distr

 ## Build order (must be sequential)

-1. `./install_deps.sh` — install build dependencies via apt
+1. `./install_deps.sh` — fix EOL/mirror sources then install build dependencies via apt
 2. `./pullsrc.sh` — download OpenSSH sources from Debian sid pool into `downloads/`
 3. `./compile.sh` — build .deb packages into `output/`

@@ -15,7 +15,7 @@ Shell scripts that backport OpenSSH from Debian sid to older Debian/Ubuntu distr
 ## Key env vars

 - `FORCESSL=1` — force static OpenSSL linking even on distros with libssl >= 3.0
- `APT_MIRROR` — substitute apt sources mirror (e.g. `mirrors.ustc.edu.cn`)
+- `APT_MIRROR` — substitute apt sources mirror (e.g. `mirrors.ustc.edu.cn`); also applied when switching EOL Debian sources to archive handling (archive.debian.org)
 - `DEB_BUILD_OPTIONS` and `DEB_BUILD_PROFILES` are set inside `compile.sh` to skip tests and udebs

 ## Package versioning
@@ -29,10 +29,7 @@ docker build --build-arg BASE_IMAGE=ubuntu:noble -f docker/Dockerfile.deps -t <t
 ```

 > **EOL distro sources**: For EOL Debian releases (e.g. buster) the default
-> `deb.debian.org` no longer serves the repository. Run
-> `switch_archive_sources.sh` inside the target image to switch sources to
-> `archive.debian.org` (adding the `-backports` pocket) before apt operations.
-> `docker/Dockerfile.deps` and the CI install-test step both invoke this script.
+> `deb.debian.org` no longer serves the repository. `install_deps.sh` now automatically switches EOL Debian sources to `archive.debian.org` (adding the `-backports` pocket) before apt operations; `switch_archive_sources.sh` has been removed; EOL handling (buster unconditional, bullseye probed via deb.debian.org Release check with fallback to archive.debian.org) is now fully inside `install_deps.sh`. `docker/Dockerfile.deps` and CI both invoke `install_deps.sh` directly.

 ## pullsrc on the host

--- a/README.md
+++ b/README.md
@@ -72,7 +72,7 @@ docker builder prune

 <summary>Using a APT mirror or proxy inside docker</summary>

-using `-e` to set environment variables inside docker.
+using `-e` to set environment variables inside docker. `APT_MIRROR` is automatically applied together with EOL archive handling (switch to `archive.debian.org` when needed) inside `install_deps.sh`.

 ```bash
    docker run --rm -v "$(pwd):/work" -w /work \
--- a/docker/Dockerfile.deps
+++ b/docker/Dockerfile.deps
@@ -4,12 +4,6 @@ ARG BASE_IMAGE
 FROM ${BASE_IMAGE}

 COPY --chmod=755 install_deps.sh /tmp/install_deps.sh
-COPY --chmod=755 switch_archive_sources.sh /tmp/switch_archive_sources.sh
 COPY builddep/ /tmp/builddep/
-
-# EOL Debian releases (e.g. buster) no longer exist on deb.debian.org;
-# switch to the official archive source before installing build deps.
-RUN /tmp/switch_archive_sources.sh && rm /tmp/switch_archive_sources.sh
-
-RUN cd /tmp && ./install_deps.sh \
-    && rm -rf /tmp/install_deps.sh /tmp/builddep
+# install_deps.sh handles EOL archive switching (buster unconditional, bullseye probed) and APT_MIRROR
+RUN cd /tmp && ./install_deps.sh && rm -rf /tmp/install_deps.sh /tmp/builddep
--- a/install_deps.sh
+++ b/install_deps.sh
@@ -17,20 +17,194 @@ __root="$(cd "$(dirname "${__dir}")" && pwd)" # <-- change this as it depends on

 export DEBIAN_FRONTEND=noninteractive

-if [[ -n "${APT_MIRROR:-}" ]]; then
-	if [[ -f /etc/apt/sources.list ]]; then
-		# Extract the hostname from the first 'deb' line and replace it
-		original_mirror=$(awk '/^deb/{print $2}' /etc/apt/sources.list | head -n1 | cut -d/ -f3)
-		sed -i "s|${original_mirror}|${APT_MIRROR}|" /etc/apt/sources.list
-		# Comment out the security update source to avoid potential issues from mixed mirror sources
-		sed -i "/security.ubuntu.com/s|^|#|" /etc/apt/sources.list
-	fi
+# ---------------------------------------------------------------------------
+# fix_apt_sources: unified APT source fixing
+# Order: fix EOL archives first, then apply APT_MIRROR.
+# If both are triggered, APT_MIRROR takes precedence but a warning is emitted.
+# ---------------------------------------------------------------------------
+# Global flag set by _fix_eol when EOL handling actually rewrote sources.
+_fix_apt_eol_fixed=0

-	if [[ -f /etc/apt/sources.list.d/debian.sources ]]; then
-		sed -i "s|deb.debian.org|${APT_MIRROR}|" /etc/apt/sources.list.d/debian.sources
-	fi
+fix_apt_sources() {
+    # --- EOL archive fixing (Debian only) ---
+    _fix_eol() {
+        # Only Debian needs archive switching; Ubuntu and others are unaffected.
+        if [[ ! -f /etc/os-release ]]; then
+            return 0
+        fi
+        # shellcheck disable=SC1091
+        . /etc/os-release
+        if [[ "${ID:-}" != "debian" ]]; then
+            return 0
+        fi
+
+        # EOL codenames that have been moved to archive.debian.org.
+        # buster is definitively EOL, bullseye just passed EOL (2026-09) and
+        # is in transitional state where archive may not yet be ready, so
+        # probing is required.
+        local EOL_CODENAMES=("buster" "bullseye")
+
+        local codename="${VERSION_CODENAME:-}"
+        local is_eol=0
+        local c
+        for c in "${EOL_CODENAMES[@]}"; do
+            if [[ "$c" == "$codename" ]]; then
+                is_eol=1
+                break
+            fi
+        done
+        if [[ "$is_eol" -ne 1 ]]; then
+            return 0
+        fi
+
+        echo "[fix-apt] EOL codename detected: ${codename} (ID=debian), switching to archive.debian.org..."
+
+        case "${codename}" in
+            buster)
+                # Legacy sources.list (buster default)
+                for f in /etc/apt/sources.list /etc/apt/sources.list.d/*.list; do
+                    [[ -f "$f" ]] || continue
+                    if grep -q "deb.debian.org" "$f" 2>/dev/null; then
+                        echo "[fix-apt]   $f: deb.debian.org -> archive.debian.org"
+                        sed -i 's|deb.debian.org|archive.debian.org|g' "$f"
+                    fi
+                    if grep -q "security.debian.org" "$f" 2>/dev/null; then
+                        echo "[fix-apt]   $f: security.debian.org -> archive.debian.org"
+                        sed -i 's|security.debian.org|archive.debian.org|g' "$f"
+                    fi
+                done
+                # DEB822 .sources (e.g. /etc/apt/sources.list.d/debian.sources)
+                for f in /etc/apt/sources.list.d/*.sources; do
+                    [[ -f "$f" ]] || continue
+                    if grep -q "deb.debian.org" "$f" 2>/dev/null; then
+                        echo "[fix-apt]   $f: deb.debian.org -> archive.debian.org (DEB822)"
+                        sed -i 's|deb.debian.org|archive.debian.org|g' "$f"
+                    fi
+                    if grep -q "security.debian.org" "$f" 2>/dev/null; then
+                        echo "[fix-apt]   $f: security.debian.org -> archive.debian.org (DEB822)"
+                        sed -i 's|security.debian.org|archive.debian.org|g' "$f"
+                    fi
+                done
+                # Append buster-backports if not already present (needed for dwz etc.)
+                if ! grep -q "${codename}-backports" /etc/apt/sources.list 2>/dev/null; then
+                    echo "[fix-apt]   adding ${codename}-backports to /etc/apt/sources.list"
+                    echo "deb http://archive.debian.org/debian ${codename}-backports main" >> /etc/apt/sources.list
+                else
+                    echo "[fix-apt]   ${codename}-backports already present, skipping"
+                fi
+                _fix_apt_eol_fixed=1
+                ;;
+            bullseye)
+                # Bullseye EOL transitional: probe official source first.
+                # If still reachable, keep official; if not reachable, switch to archive.
+                local _probe_ok=0
+                # Try both Release and InRelease with timeout 5, spider mode; suppress errexit
+                if wget -q --spider --timeout=5 "http://deb.debian.org/debian/dists/bullseye/Release" 2>/dev/null; then
+                    _probe_ok=1
+                elif wget -q --spider --timeout=5 "http://deb.debian.org/debian/dists/bullseye/InRelease" 2>/dev/null; then
+                    _probe_ok=1
+                fi
+                if [[ $_probe_ok -eq 1 ]]; then
+                    echo "[fix-apt] bullseye still served from deb.debian.org (probe succeeded), keeping official sources"
+                    # do NOT set _fix_apt_eol_fixed, remain on official
+                else
+                    echo "[fix-apt] bullseye official source not reachable (probe failed), switching to archive.debian.org..."
+                    # copy same sed logic as buster but for bullseye (legacy + DEB822 + backports)
+                    for f in /etc/apt/sources.list /etc/apt/sources.list.d/*.list; do
+                        [[ -f "$f" ]] || continue
+                        if grep -q "deb.debian.org" "$f" 2>/dev/null; then
+                            echo "[fix-apt]   $f: deb.debian.org -> archive.debian.org"
+                            sed -i 's|deb.debian.org|archive.debian.org|g' "$f"
+                        fi
+                        if grep -q "security.debian.org" "$f" 2>/dev/null; then
+                            echo "[fix-apt]   $f: security.debian.org -> archive.debian.org"
+                            sed -i 's|security.debian.org|archive.debian.org|g' "$f"
+                        fi
+                    done
+                    for f in /etc/apt/sources.list.d/*.sources; do
+                        [[ -f "$f" ]] || continue
+                        if grep -q "deb.debian.org" "$f" 2>/dev/null; then
+                            echo "[fix-apt]   $f: deb.debian.org -> archive.debian.org (DEB822)"
+                            sed -i 's|deb.debian.org|archive.debian.org|g' "$f"
+                        fi
+                        if grep -q "security.debian.org" "$f" 2>/dev/null; then
+                            echo "[fix-apt]   $f: security.debian.org -> archive.debian.org (DEB822)"
+                            sed -i 's|security.debian.org|archive.debian.org|g' "$f"
+                        fi
+                    done
+                    if ! grep -q "${codename}-backports" /etc/apt/sources.list 2>/dev/null; then
+                        echo "[fix-apt]   adding ${codename}-backports to /etc/apt/sources.list"
+                        echo "deb http://archive.debian.org/debian ${codename}-backports main" >> /etc/apt/sources.list
+                    else
+                        echo "[fix-apt]   ${codename}-backports already present, skipping"
+                    fi
+                    _fix_apt_eol_fixed=1
+                fi
+                ;;
+            *)
+                echo "[fix-apt] WARNING: EOL codename ${codename} matched but no handler defined"
+                ;;
+        esac
+    }
+
+    # --- APT_MIRROR handling ---
+    _apply_mirror() {
+        if [[ -z "${APT_MIRROR:-}" ]]; then
+            return 0
+        fi
+        echo "[fix-apt] applying APT_MIRROR=${APT_MIRROR} ..."
+
+        # Legacy list files: /etc/apt/sources.list and /etc/apt/sources.list.d/*.list
+        for f in /etc/apt/sources.list /etc/apt/sources.list.d/*.list; do
+            [[ -f "$f" ]] || continue
+            # Extract hostname from first deb line and replace it
+            local original_mirror
+            original_mirror=$(awk '/^deb /{print $2}' "$f" | head -n1 | cut -d/ -f3)
+            if [[ -n "${original_mirror:-}" ]]; then
+                echo "[fix-apt]   $f: replacing host ${original_mirror} -> ${APT_MIRROR}"
+                sed -i "s|${original_mirror}|${APT_MIRROR}|g" "$f"
+            fi
+            # Comment out security.ubuntu.com to avoid mixed-mirror issues (Ubuntu only)
+            if grep -q "security.ubuntu.com" "$f" 2>/dev/null; then
+                echo "[fix-apt]   $f: commenting security.ubuntu.com"
+                sed -i "/security.ubuntu.com/s|^|#|" "$f"
+            fi
+        done
+
+        # DEB822 .sources files: replace deb.debian.org hostname with mirror
+        for f in /etc/apt/sources.list.d/*.sources; do
+            [[ -f "$f" ]] || continue
+            if grep -q "deb.debian.org" "$f" 2>/dev/null; then
+                echo "[fix-apt]   $f: deb.debian.org -> ${APT_MIRROR} (DEB822)"
+                sed -i "s|deb.debian.org|${APT_MIRROR}|g" "$f"
+            fi
+        done
+    }
+
+    _fix_eol
+    _apply_mirror
+
+    if [[ "${_fix_apt_eol_fixed}" -eq 1 && -n "${APT_MIRROR:-}" ]]; then
+        echo "[fix-apt] WARNING: EOL archive fix was applied but APT_MIRROR=${APT_MIRROR} overrides archive.debian.org; you are responsible for ensuring the mirror provides the archive path."
+    fi
+}
+
+# Parse --fix-apt-only before any heavy work.
+# When set, only fix apt sources and exit 0 (lightweight CI install-test).
+FIX_APT_ONLY=0
+if [[ "${1:-}" == "--fix-apt-only" ]]; then
+    FIX_APT_ONLY=1
 fi

+if [[ "$FIX_APT_ONLY" -eq 1 ]]; then
+    fix_apt_sources
+    echo "[fix-apt] --fix-apt-only: done, exiting 0"
+    exit 0
+fi
+
+# Normal flow: fix sources then continue with dependency installation
+fix_apt_sources
+
 apt update
 apt upgrade -y
 apt install -y --no-install-recommends lsb-release wget sudo pkgconf build-essential fakeroot \
--- a/switch_archive_sources.sh
+++ b/switch_archive_sources.sh
@@ -1,21 +0,0 @@
-#!/bin/bash
-# Switch Debian package sources to archive.debian.org for EOL releases.
-# Called from docker/Dockerfile.deps and CI workflows before apt operations.
-# Non-Debian and non-EOL distros are not affected.
-set -e
-. /etc/os-release
-[ "${ID:-}" = "debian" ] || exit 0
-
-case "${VERSION_CODENAME:-}" in
-    buster)
-        # EOL Debian releases served from archive.debian.org.
-        # NOTE: do NOT add bullseye until deb.debian.org/security.debian.org
-        # stop serving it (its repos were still live as of 2026-09).
-        sed -i 's|deb.debian.org|archive.debian.org|g' /etc/apt/sources.list
-        # buster bundles security under deb.debian.org (path /debian-security)
-        sed -i 's|security.debian.org|archive.debian.org|g' /etc/apt/sources.list
-        # The backports source is not in the base image; add it so that
-        # install_deps.sh can pull packages like dwz from backports.
-        echo "deb http://archive.debian.org/debian ${VERSION_CODENAME}-backports main" >> /etc/apt/sources.list
-        ;;
-esac
--- a/version.env
+++ b/version.env
@@ -2,7 +2,8 @@ OPENSSLVER=3.5.7
 OPENSSLMIR=https://github.com/openssl/openssl/releases/download/openssl-${OPENSSLVER}/
 OPENSSLSRC=openssl-${OPENSSLVER}.tar.gz

-DEBMIRROR=http://deb.debian.org/debian/
+DEBMIRROR=${APT_MIRROR:+http://${APT_MIRROR}/debian/}
+DEBMIRROR=${DEBMIRROR:-http://deb.debian.org/debian/}
 OPENSSH_SIDPKG=10.4p1-5
 [[ -z $OPENSSH_SIDPKG ]] && \
 	OPENSSH_SIDPKG=$(wget -qO- http://deb.debian.org/debian/pool/main/o/openssh/ | grep -oP 'openssh_\K[0-9]+\.[0-9]+p[0-9]+-[0-9]+(?:~bpo[0-9]+(?:\+[0-9]+)?)?' | sort -V | tail -n 1)