refactor: download sid debhelper .debs on host via pullsrc.sh instead of bundling in git

- pullsrc.sh fetches debhelper/libdebhelper-perl 14.3 (pinned DEBHELPER_SIDPKG in version.env, empty = auto-detect latest sid) into gitignored builddep/ - install_deps.sh installs builddep/*.deb and adds compat sed hacks for sid debhelper >= 13.27: rewrite Dh_Lib.pm bucket 'cp --update=none' to '-n' on coreutils < 9.3 (buster/bionic/bookworm), downgrade use v5.28 pragmas in Dh_Lib.pm/dh_assistant for perl < 5.28, handle ${tmpdir} brace form in the non-merged-usr hack; drop the dead sid-apt-source _DEBIAN_DEBHELPER - Dockerfile.deps: BuildKit bind mounts instead of COPY (nothing in layers) - CI deps-image build runs ./pullsrc.sh debhelper first to populate context
2026-09-20 08:03:46 +08:00 · 2026-09-01 15:41:24 +08:00
parent 0ab53f15fa
commit 4ef6b27bec
10 changed files with 87 additions and 69 deletions
--- a/.github/workflows/build-deps-images.yml
+++ b/.github/workflows/build-deps-images.yml
@@ -30,6 +30,9 @@ jobs:
      - name: Checkout repository
        uses: actions/checkout@v5

+      - name: Fetch debhelper .debs into build context
+        run: ./pullsrc.sh debhelper
+
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

--- a/.gitignore
+++ b/.gitignore
@@ -1,5 +1,6 @@
 build/
 downloads/
+builddep/
 *.swp
 *.tar.gz
 *.deb
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -4,13 +4,13 @@ Shell scripts that backport OpenSSH from Debian sid to older Debian/Ubuntu distr

 ## Build order (must be sequential)

-1. `./install_deps.sh` — fix EOL/mirror sources then install build dependencies via apt
-2. `./pullsrc.sh` — download OpenSSH sources from Debian sid pool into `downloads/`
+1. `./pullsrc.sh` — download OpenSSH sources from Debian sid pool into `downloads/`, plus the sid debhelper .debs into `builddep/` (needed by `install_deps.sh` on old distros; `./pullsrc.sh debhelper` fetches only the .debs)
+2. `./install_deps.sh` — fix EOL/mirror sources then install build dependencies via apt; on distros with debhelper < 13.12 it installs `builddep/*.deb` and applies old-distro compat sed hacks
 3. `./compile.sh` — build .deb packages into `output/`

 ## Version source of truth

-`version.env` defines `OPENSSLVER` and auto-detects `OPENSSH_SIDPKG` by scraping `http://deb.debian.org/debian/pool/main/o/openssh/`. It is sourced (not executed) by `compile.sh` and `pullsrc.sh`. Do not run it directly.
+`version.env` defines `OPENSSLVER`, pins `DEBHELPER_SIDPKG` (sid debhelper .debs; empty = auto-detect latest from the pool), and auto-detects `OPENSSH_SIDPKG` by scraping `http://deb.debian.org/debian/pool/main/o/openssh/`. It is sourced (not executed) by `compile.sh` and `pullsrc.sh`. Do not run it directly.

 ## Key env vars

@@ -25,9 +25,12 @@ The distro codename is appended to the package version (`~${BUILD_CODENAME}`) du
 ## Docker build

 ```bash
+./pullsrc.sh debhelper   # host: populate builddep/ (required for old-distro dep images)
 docker build --build-arg BASE_IMAGE=ubuntu:noble -f docker/Dockerfile.deps -t <tag> .
 ```

+`docker/Dockerfile.deps` uses BuildKit `--mount=type=bind` for `install_deps.sh` and `builddep/`, so neither lands in an image layer (BuildKit is required; it is the default in modern docker).
+
 > **EOL distro sources**: For EOL Debian releases (e.g. buster) the default
 > `deb.debian.org` no longer serves the repository. `install_deps.sh` now automatically switches EOL Debian sources to `archive.debian.org` (adding the `-backports` pocket) before apt operations; `switch_archive_sources.sh` has been removed; EOL handling (buster unconditional, bullseye probed via deb.debian.org Release check with fallback to archive.debian.org) is now fully inside `install_deps.sh`. `docker/Dockerfile.deps` and CI both invoke `install_deps.sh` directly.

@@ -36,8 +39,11 @@ docker build --build-arg BASE_IMAGE=ubuntu:noble -f docker/Dockerfile.deps -t <t
 Old distros often ship old `ca-certificates` (or none in minimal images),
 which can no longer verify GitHub's TLS chain. The release CI avoids this
 by running `./pullsrc.sh` on the host runner (current CA store), then
-mounting the repository into the build container with `-v`. When building
-manually for an old distro, apply the same pattern:
+mounting the repository into the build container with `-v`. `pullsrc.sh`
+also downloads the sid debhelper .debs into `builddep/` (gitignored), which
+`install_deps.sh` installs on distros whose own debhelper is < 13.12 — so
+`pullsrc.sh` must run before `install_deps.sh`. When building manually for
+an old distro, apply the same pattern:

 ```bash
 ./pullsrc.sh
@@ -51,7 +57,7 @@ docker run --rm -v "$(pwd):/work" -w /work debian:buster bash -c "./install_deps
 | `downloads/` | Downloaded source tarballs (gitignored)  |
 | `build/`     | Temporary build tree (gitignored)        |
 | `output/`    | Final .deb packages                      |
-| `builddep/`  | Pre-built debhelper .debs for old distros |
+| `builddep/`  | Sid debhelper .debs downloaded by `pullsrc.sh` (gitignored) |

 ## Release workflow

--- a/README.md
+++ b/README.md
@@ -43,12 +43,12 @@ sudo bash -c "$(curl -L https://gh-proxy.com/github.com/boypt/openssh-deb/raw/ma
 ## Direct Build

 ```bash
+# pull source (also fetches sid debhelper .debs into builddep/ for old distros)
+./pullsrc.sh
+
 # Install Dependencies
 ./install_deps.sh

-# pull source
-./pullsrc.sh
-
 # direct build
 ./compile.sh
 ```
--- a/builddep/debhelper_13.14.1_all.deb
+++ b/builddep/debhelper_13.14.1_all.deb
--- a/builddep/libdebhelper-perl_13.14.1_all.deb
+++ b/builddep/libdebhelper-perl_13.14.1_all.deb
--- a/docker/Dockerfile.deps
+++ b/docker/Dockerfile.deps
@@ -1,9 +1,11 @@
 # Dep image
 #   docker build --build-arg BASE_IMAGE=ubuntu:noble -f docker/Dockerfile.deps -t xxx .
+# Requires `./pullsrc.sh debhelper` on the host first (populates builddep/).
+# BuildKit bind mounts keep install_deps.sh and the .debs out of image layers.
 ARG BASE_IMAGE
 FROM ${BASE_IMAGE}

-COPY --chmod=755 install_deps.sh /tmp/install_deps.sh
-COPY builddep/ /tmp/builddep/
 # install_deps.sh handles EOL archive switching (buster unconditional, bullseye probed) and APT_MIRROR
-RUN cd /tmp && ./install_deps.sh && rm -rf /tmp/install_deps.sh /tmp/builddep
+RUN --mount=type=bind,source=install_deps.sh,target=/tmp/install_deps.sh \
+	--mount=type=bind,source=builddep,target=/tmp/builddep \
+	cd /tmp && bash ./install_deps.sh
--- a/install_deps.sh
+++ b/install_deps.sh
@@ -226,43 +226,21 @@ if [[ $(apt-cache search --names-only 'libcrypt-dev' | wc -l) -gt 0 ]]; then
 fi


-# install the latest debhelper from debian sid by adding debian sources
-_DEBIAN_DEBHELPER() {
-
-    local __coreutils_ver="$(dpkg-query -f '${Version}' -W coreutils || true)"
-    [[ -z $__coreutils_ver ]] && __coreutils_ver="0.0.0"
-    echo "DEBUG: __coreutils_ver:$__coreutils_ver"
-
-    # Note: with coreutils < 9.5, `cp --update=none` is not supported.
-    # But the latest debhelper generate such commands.
-    # Using the latest debhelper would fail.
-    if dpkg --compare-versions "$__coreutils_ver" lt '9.5~'; then
-        sudo apt install -y --allow-downgrades "$__dir"/builddep/*.deb
-        return 0
-    fi
-
-    DEBIAN_SOURCE="http://deb.debian.org/debian/"
-    [[ -n "${APT_MIRROR:-}" ]] && \
-        DEBIAN_SOURCE="http://${APT_MIRROR}/debian/"
-
-    # Download Debian sid GPG key
-    wget -O /usr/share/keyrings/debian-sid.gpg https://deb.debian.org/debian/dists/sid/Release.gpg
-
-    # Add Debian sid source with the GPG key
-    echo "deb [signed-by=/usr/share/keyrings/debian-sid.gpg] $DEBIAN_SOURCE sid main" > /etc/apt/sources.list.d/debian-sid.list
-
-    apt update
-    apt install -y debhelper
-    rm /etc/apt/sources.list.d/debian-sid.list
-}
-
+# Old distros ship a debhelper too old to build the sid OpenSSH source
+# (dh-sequence-movetousr needs >= 13.11.7). The sid .debs are downloaded on
+# the host by pullsrc.sh into builddep/ (gitignored) and installed here.
+# No sid apt source is configured inside the container: expired GPG/CA and
+# dependency churn make that fragile on old distros.
 __debhelper_ver="$(dpkg-query -f '${Version}' -W debhelper || true)"
 [[ -z $__debhelper_ver ]] && __debhelper_ver="0.0.0"
 echo "DEBUG: __debhelper_ver:$__debhelper_ver"
 if dpkg --compare-versions "$__debhelper_ver" lt '13.12~'; then
-   # dh-sequence-movetousr was added to debhelper in 13.11.7
+   if ! ls "$__dir"/builddep/debhelper_*_all.deb >/dev/null 2>&1; then
+       echo "ERROR: builddep/debhelper_*.deb missing. Run ./pullsrc.sh on the host first (it downloads them into builddep/)." >&2
+       exit 1
+   fi

-   # debhelper 13.14 needs dwz >= 0.12.20190711, newer than some distros ship
+   # debhelper needs dwz >= 0.12.20190711, newer than some distros ship
   # (Ubuntu 18.04 has 0.12-2): pull it from the distro backports pocket.
   __dwz_ver="$(apt-cache policy dwz 2>/dev/null | awk '/Candidate:/{print $2; exit}')"
   [[ -z $__dwz_ver || $__dwz_ver == "(none)" ]] && __dwz_ver=0
@@ -270,14 +248,26 @@ if dpkg --compare-versions "$__debhelper_ver" lt '13.12~'; then
       apt install -y -t "$(lsb_release -sc)-backports" dwz
   fi

-   sudo apt install -y "$__dir"/builddep/*.deb
+   apt install -y --allow-downgrades "$__dir"/builddep/*.deb

-   # debhelper 13.14 uses Perl >= 5.30 syntax (state vars in list context) and
-   # dh_missing declares v5.28; downgrade both for older perls (Ubuntu 18.04: 5.26).
+   # debhelper >= 13.27 restores its bucket files with `cp --update=none`,
+   # which coreutils only learned in 9.3. On the older coreutils shipped by
+   # these distros `-n` has identical semantics (upstream itself used
+   # `cp -an` at these two Dh_Lib.pm call sites until 13.26), so rewrite it.
+   __coreutils_ver="$(dpkg-query -f '${Version}' -W coreutils || true)"
+   [[ -z $__coreutils_ver ]] && __coreutils_ver="0.0.0"
+   echo "DEBUG: __coreutils_ver:$__coreutils_ver"
+   if dpkg --compare-versions "$__coreutils_ver" lt '9.3~'; then
+       sed -i "s/'--update=none'/'-n'/g" /usr/share/perl5/Debian/Debhelper/Dh_Lib.pm
+   fi
+
+   # debhelper >= 14 declares `use v5.28` (Dh_Lib.pm, dh_assistant) and uses
+   # `state` with initializers (perl >= 5.28); downgrade both for older perls
+   # (Ubuntu 18.04: 5.26). Bare `state $x;` / `state %h;` work on 5.26.
   __perl_ver="$(perl -MConfig -e 'print $Config{version}')"
   if dpkg --compare-versions "$__perl_ver" lt '5.28'; then
       sed -i -E 's/^(\s*)state\s+([%@][^=]+=)/\1my \2/; s/^(\s*)state\s+\(([^)]+)\)/\1my (\2)/' /usr/share/perl5/Debian/Debhelper/Dh_Lib.pm
-       sed -i 's/^use v5\.28;/use v5.26;/' /usr/bin/dh_missing
+       sed -i 's/^use v5\.28;/use v5.26;/' /usr/share/perl5/Debian/Debhelper/Dh_Lib.pm /usr/bin/dh_assistant /usr/bin/dh_missing
   fi

   # debhelper hardcodes versioned deps on init-system-helpers that old distros
@@ -295,28 +285,12 @@ if dpkg --compare-versions "$__debhelper_ver" lt '13.12~'; then
   fi

   # On non-merged-usr distros (e.g. Ubuntu 18.04) deb-systemd-helper only
-   # searches /lib/systemd/system, but debhelper 13.14 installs units to
+   # searches /lib/systemd/system, but debhelper installs units to
   # /usr/lib: keep units in /lib so services actually get enabled.
+   # (debhelper >= 14 also uses the ${tmpdir} brace form; rewrite both.)
   if [ ! -L /lib ]; then
-       sed -i 's|\$tmpdir/usr/lib/systemd/system|$tmpdir/lib/systemd/system|g' /usr/bin/dh_installsystemd
+       sed -i 's|$tmpdir/usr/lib/systemd/system|$tmpdir/lib/systemd/system|g; s|${tmpdir}/usr/lib/systemd/system|${tmpdir}/lib/systemd/system|g' /usr/bin/dh_installsystemd
+   fi
 fi
-fi
-
-#CODE_NAME=$(lsb_release -sc)
-# if [ "${CODE_NAME}" != "focal" ]; then
-#     apt install -y dh-virtualenv
-# fi
-# case ${CODE_NAME} in
-#     # dists with coreutils >= 9.5 can use the latest debhelper from debian sid
-#     trixie)
-#         _DEBIAN_DEBHELPER
-#         ;;
-#     plucky|questing|resolute)
-#         _DEBIAN_DEBHELPER
-#         ;;
-#     *)
-#         echo "$CODE_NAME does NOT NEED to add Debian sources."
-#         ;;
-# esac

 exit 0
--- a/pullsrc.sh
+++ b/pullsrc.sh
@@ -18,14 +18,36 @@ arg1="${1:-}"

 source $__dir/version.env

+# debhelper .debs for old distros: downloaded here on the host (current CA and
+# network), installed by install_deps.sh from builddep/ (gitignored). Old
+# distros cannot build the sid OpenSSH source with their own debhelper
+# (dh-sequence-movetousr needs >= 13.11.7), and adding sid apt sources inside
+# old containers is fragile, so we just fetch the two arch-all .debs directly.
+DEBHELPER_LINKS=(
+	$DEBMIRROR/pool/main/d/debhelper/debhelper_${DEBHELPER_SIDPKG}_all.deb
+	$DEBMIRROR/pool/main/d/debhelper/libdebhelper-perl_${DEBHELPER_SIDPKG}_all.deb
+)
+
+# `./pullsrc.sh debhelper` fetches only the .debs (used by the deps-image CI
+# before docker build, where the openssh sources are not needed).
+if [[ "$arg1" == "debhelper" ]]; then
+	mkdir -p $__dir/builddep
+	echo "> INFO: downloading debhelper ${DEBHELPER_SIDPKG} .debs into builddep/."
+	wget --continue -P "$__dir/builddep" "${DEBHELPER_LINKS[@]}"
+	exit 0
+fi
+
 DOWNLOADLINKS=(
 	$DEBMIRROR/pool/main/o/openssh/openssh_${OPENSSH_SIDPKG}.{debian.tar.xz,dsc}
 	$DEBMIRROR/pool/main/o/openssh/openssh_${OPENSSHVER}.orig.tar.gz{,.asc}
 	${OPENSSLMIR}/${OPENSSLSRC}
 )

-mkdir -p $__dir/downloads && cd $__dir/downloads
+mkdir -p $__dir/downloads $__dir/builddep && cd $__dir/downloads
 echo "> INFO: downloading the following sources."
 echo "${DOWNLOADLINKS[@]}" | tr " " "\n"
 wget --continue "${DOWNLOADLINKS[@]}"

+echo "> INFO: downloading debhelper ${DEBHELPER_SIDPKG} .debs into builddep/."
+wget --continue -P "$__dir/builddep" "${DEBHELPER_LINKS[@]}"
+
--- a/version.env
+++ b/version.env
@@ -11,3 +11,13 @@ OPENSSH_SIDPKG=10.5p1-1

 OPENSSHVER=$(echo $OPENSSH_SIDPKG|cut -d- -f1)

+# debhelper .debs for old distros: pullsrc.sh downloads them on the host into
+# builddep/, install_deps.sh installs them there (old distros cannot build the
+# sid OpenSSH source with their own debhelper: dh-sequence-movetousr needs
+# >= 13.11.7). The sid pool only keeps the current version, so bump this pin
+# when it drifts; leave empty to auto-detect the latest from the pool.
+DEBHELPER_SIDPKG=14.3
+if [[ -z $DEBHELPER_SIDPKG ]]; then
+	DEBHELPER_SIDPKG=$(wget -qO- $DEBMIRROR/pool/main/d/debhelper/ | grep -oP 'debhelper_\K[0-9][0-9.]+(?=_all\.deb)' | sort -V | tail -n 1)
+fi
+