refactor: download sid debhelper .debs on host via pullsrc.sh instead of bundling in git

- pullsrc.sh fetches debhelper/libdebhelper-perl 14.3 (pinned DEBHELPER_SIDPKG
  in version.env, empty = auto-detect latest sid) into gitignored builddep/
- install_deps.sh installs builddep/*.deb and adds compat sed hacks for sid
  debhelper >= 13.27: rewrite Dh_Lib.pm bucket 'cp --update=none' to '-n' on
  coreutils < 9.3 (buster/bionic/bookworm), downgrade use v5.28 pragmas in
  Dh_Lib.pm/dh_assistant for perl < 5.28, handle ${tmpdir} brace form in the
  non-merged-usr hack; drop the dead sid-apt-source _DEBIAN_DEBHELPER
- Dockerfile.deps: BuildKit bind mounts instead of COPY (nothing in layers)
- CI deps-image build runs ./pullsrc.sh debhelper first to populate context
This commit is contained in:
boypt
2026-09-01 15:41:24 +08:00
parent 0ab53f15fa
commit 4ef6b27bec
10 changed files with 87 additions and 69 deletions

View File

@@ -30,6 +30,9 @@ jobs:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@v5 uses: actions/checkout@v5
- name: Fetch debhelper .debs into build context
run: ./pullsrc.sh debhelper
- name: Set up Docker Buildx - name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v3

1
.gitignore vendored
View File

@@ -1,5 +1,6 @@
build/ build/
downloads/ downloads/
builddep/
*.swp *.swp
*.tar.gz *.tar.gz
*.deb *.deb

View File

@@ -4,13 +4,13 @@ Shell scripts that backport OpenSSH from Debian sid to older Debian/Ubuntu distr
## Build order (must be sequential) ## Build order (must be sequential)
1. `./install_deps.sh` — fix EOL/mirror sources then install build dependencies via apt 1. `./pullsrc.sh` — download OpenSSH sources from Debian sid pool into `downloads/`, plus the sid debhelper .debs into `builddep/` (needed by `install_deps.sh` on old distros; `./pullsrc.sh debhelper` fetches only the .debs)
2. `./pullsrc.sh` — download OpenSSH sources from Debian sid pool into `downloads/` 2. `./install_deps.sh` — fix EOL/mirror sources then install build dependencies via apt; on distros with debhelper < 13.12 it installs `builddep/*.deb` and applies old-distro compat sed hacks
3. `./compile.sh` build .deb packages into `output/` 3. `./compile.sh` build .deb packages into `output/`
## Version source of truth ## Version source of truth
`version.env` defines `OPENSSLVER` and auto-detects `OPENSSH_SIDPKG` by scraping `http://deb.debian.org/debian/pool/main/o/openssh/`. It is sourced (not executed) by `compile.sh` and `pullsrc.sh`. Do not run it directly. `version.env` defines `OPENSSLVER`, pins `DEBHELPER_SIDPKG` (sid debhelper .debs; empty = auto-detect latest from the pool), and auto-detects `OPENSSH_SIDPKG` by scraping `http://deb.debian.org/debian/pool/main/o/openssh/`. It is sourced (not executed) by `compile.sh` and `pullsrc.sh`. Do not run it directly.
## Key env vars ## Key env vars
@@ -25,9 +25,12 @@ The distro codename is appended to the package version (`~${BUILD_CODENAME}`) du
## Docker build ## Docker build
```bash ```bash
./pullsrc.sh debhelper # host: populate builddep/ (required for old-distro dep images)
docker build --build-arg BASE_IMAGE=ubuntu:noble -f docker/Dockerfile.deps -t <tag> . docker build --build-arg BASE_IMAGE=ubuntu:noble -f docker/Dockerfile.deps -t <tag> .
``` ```
`docker/Dockerfile.deps` uses BuildKit `--mount=type=bind` for `install_deps.sh` and `builddep/`, so neither lands in an image layer (BuildKit is required; it is the default in modern docker).
> **EOL distro sources**: For EOL Debian releases (e.g. buster) the default > **EOL distro sources**: For EOL Debian releases (e.g. buster) the default
> `deb.debian.org` no longer serves the repository. `install_deps.sh` now automatically switches EOL Debian sources to `archive.debian.org` (adding the `-backports` pocket) before apt operations; `switch_archive_sources.sh` has been removed; EOL handling (buster unconditional, bullseye probed via deb.debian.org Release check with fallback to archive.debian.org) is now fully inside `install_deps.sh`. `docker/Dockerfile.deps` and CI both invoke `install_deps.sh` directly. > `deb.debian.org` no longer serves the repository. `install_deps.sh` now automatically switches EOL Debian sources to `archive.debian.org` (adding the `-backports` pocket) before apt operations; `switch_archive_sources.sh` has been removed; EOL handling (buster unconditional, bullseye probed via deb.debian.org Release check with fallback to archive.debian.org) is now fully inside `install_deps.sh`. `docker/Dockerfile.deps` and CI both invoke `install_deps.sh` directly.
@@ -36,8 +39,11 @@ docker build --build-arg BASE_IMAGE=ubuntu:noble -f docker/Dockerfile.deps -t <t
Old distros often ship old `ca-certificates` (or none in minimal images), Old distros often ship old `ca-certificates` (or none in minimal images),
which can no longer verify GitHub's TLS chain. The release CI avoids this which can no longer verify GitHub's TLS chain. The release CI avoids this
by running `./pullsrc.sh` on the host runner (current CA store), then by running `./pullsrc.sh` on the host runner (current CA store), then
mounting the repository into the build container with `-v`. When building mounting the repository into the build container with `-v`. `pullsrc.sh`
manually for an old distro, apply the same pattern: also downloads the sid debhelper .debs into `builddep/` (gitignored), which
`install_deps.sh` installs on distros whose own debhelper is < 13.12 so
`pullsrc.sh` must run before `install_deps.sh`. When building manually for
an old distro, apply the same pattern:
```bash ```bash
./pullsrc.sh ./pullsrc.sh
@@ -51,7 +57,7 @@ docker run --rm -v "$(pwd):/work" -w /work debian:buster bash -c "./install_deps
| `downloads/` | Downloaded source tarballs (gitignored) | | `downloads/` | Downloaded source tarballs (gitignored) |
| `build/` | Temporary build tree (gitignored) | | `build/` | Temporary build tree (gitignored) |
| `output/` | Final .deb packages | | `output/` | Final .deb packages |
| `builddep/` | Pre-built debhelper .debs for old distros | | `builddep/` | Sid debhelper .debs downloaded by `pullsrc.sh` (gitignored) |
## Release workflow ## Release workflow

View File

@@ -43,12 +43,12 @@ sudo bash -c "$(curl -L https://gh-proxy.com/github.com/boypt/openssh-deb/raw/ma
## Direct Build ## Direct Build
```bash ```bash
# pull source (also fetches sid debhelper .debs into builddep/ for old distros)
./pullsrc.sh
# Install Dependencies # Install Dependencies
./install_deps.sh ./install_deps.sh
# pull source
./pullsrc.sh
# direct build # direct build
./compile.sh ./compile.sh
``` ```

Binary file not shown.

View File

@@ -1,9 +1,11 @@
# Dep image # Dep image
# docker build --build-arg BASE_IMAGE=ubuntu:noble -f docker/Dockerfile.deps -t xxx . # docker build --build-arg BASE_IMAGE=ubuntu:noble -f docker/Dockerfile.deps -t xxx .
# Requires `./pullsrc.sh debhelper` on the host first (populates builddep/).
# BuildKit bind mounts keep install_deps.sh and the .debs out of image layers.
ARG BASE_IMAGE ARG BASE_IMAGE
FROM ${BASE_IMAGE} FROM ${BASE_IMAGE}
COPY --chmod=755 install_deps.sh /tmp/install_deps.sh
COPY builddep/ /tmp/builddep/
# install_deps.sh handles EOL archive switching (buster unconditional, bullseye probed) and APT_MIRROR # install_deps.sh handles EOL archive switching (buster unconditional, bullseye probed) and APT_MIRROR
RUN cd /tmp && ./install_deps.sh && rm -rf /tmp/install_deps.sh /tmp/builddep RUN --mount=type=bind,source=install_deps.sh,target=/tmp/install_deps.sh \
--mount=type=bind,source=builddep,target=/tmp/builddep \
cd /tmp && bash ./install_deps.sh

View File

@@ -226,43 +226,21 @@ if [[ $(apt-cache search --names-only 'libcrypt-dev' | wc -l) -gt 0 ]]; then
fi fi
# install the latest debhelper from debian sid by adding debian sources # Old distros ship a debhelper too old to build the sid OpenSSH source
_DEBIAN_DEBHELPER() { # (dh-sequence-movetousr needs >= 13.11.7). The sid .debs are downloaded on
# the host by pullsrc.sh into builddep/ (gitignored) and installed here.
local __coreutils_ver="$(dpkg-query -f '${Version}' -W coreutils || true)" # No sid apt source is configured inside the container: expired GPG/CA and
[[ -z $__coreutils_ver ]] && __coreutils_ver="0.0.0" # dependency churn make that fragile on old distros.
echo "DEBUG: __coreutils_ver:$__coreutils_ver"
# Note: with coreutils < 9.5, `cp --update=none` is not supported.
# But the latest debhelper generate such commands.
# Using the latest debhelper would fail.
if dpkg --compare-versions "$__coreutils_ver" lt '9.5~'; then
sudo apt install -y --allow-downgrades "$__dir"/builddep/*.deb
return 0
fi
DEBIAN_SOURCE="http://deb.debian.org/debian/"
[[ -n "${APT_MIRROR:-}" ]] && \
DEBIAN_SOURCE="http://${APT_MIRROR}/debian/"
# Download Debian sid GPG key
wget -O /usr/share/keyrings/debian-sid.gpg https://deb.debian.org/debian/dists/sid/Release.gpg
# Add Debian sid source with the GPG key
echo "deb [signed-by=/usr/share/keyrings/debian-sid.gpg] $DEBIAN_SOURCE sid main" > /etc/apt/sources.list.d/debian-sid.list
apt update
apt install -y debhelper
rm /etc/apt/sources.list.d/debian-sid.list
}
__debhelper_ver="$(dpkg-query -f '${Version}' -W debhelper || true)" __debhelper_ver="$(dpkg-query -f '${Version}' -W debhelper || true)"
[[ -z $__debhelper_ver ]] && __debhelper_ver="0.0.0" [[ -z $__debhelper_ver ]] && __debhelper_ver="0.0.0"
echo "DEBUG: __debhelper_ver:$__debhelper_ver" echo "DEBUG: __debhelper_ver:$__debhelper_ver"
if dpkg --compare-versions "$__debhelper_ver" lt '13.12~'; then if dpkg --compare-versions "$__debhelper_ver" lt '13.12~'; then
# dh-sequence-movetousr was added to debhelper in 13.11.7 if ! ls "$__dir"/builddep/debhelper_*_all.deb >/dev/null 2>&1; then
echo "ERROR: builddep/debhelper_*.deb missing. Run ./pullsrc.sh on the host first (it downloads them into builddep/)." >&2
exit 1
fi
# debhelper 13.14 needs dwz >= 0.12.20190711, newer than some distros ship # debhelper needs dwz >= 0.12.20190711, newer than some distros ship
# (Ubuntu 18.04 has 0.12-2): pull it from the distro backports pocket. # (Ubuntu 18.04 has 0.12-2): pull it from the distro backports pocket.
__dwz_ver="$(apt-cache policy dwz 2>/dev/null | awk '/Candidate:/{print $2; exit}')" __dwz_ver="$(apt-cache policy dwz 2>/dev/null | awk '/Candidate:/{print $2; exit}')"
[[ -z $__dwz_ver || $__dwz_ver == "(none)" ]] && __dwz_ver=0 [[ -z $__dwz_ver || $__dwz_ver == "(none)" ]] && __dwz_ver=0
@@ -270,14 +248,26 @@ if dpkg --compare-versions "$__debhelper_ver" lt '13.12~'; then
apt install -y -t "$(lsb_release -sc)-backports" dwz apt install -y -t "$(lsb_release -sc)-backports" dwz
fi fi
sudo apt install -y "$__dir"/builddep/*.deb apt install -y --allow-downgrades "$__dir"/builddep/*.deb
# debhelper 13.14 uses Perl >= 5.30 syntax (state vars in list context) and # debhelper >= 13.27 restores its bucket files with `cp --update=none`,
# dh_missing declares v5.28; downgrade both for older perls (Ubuntu 18.04: 5.26). # which coreutils only learned in 9.3. On the older coreutils shipped by
# these distros `-n` has identical semantics (upstream itself used
# `cp -an` at these two Dh_Lib.pm call sites until 13.26), so rewrite it.
__coreutils_ver="$(dpkg-query -f '${Version}' -W coreutils || true)"
[[ -z $__coreutils_ver ]] && __coreutils_ver="0.0.0"
echo "DEBUG: __coreutils_ver:$__coreutils_ver"
if dpkg --compare-versions "$__coreutils_ver" lt '9.3~'; then
sed -i "s/'--update=none'/'-n'/g" /usr/share/perl5/Debian/Debhelper/Dh_Lib.pm
fi
# debhelper >= 14 declares `use v5.28` (Dh_Lib.pm, dh_assistant) and uses
# `state` with initializers (perl >= 5.28); downgrade both for older perls
# (Ubuntu 18.04: 5.26). Bare `state $x;` / `state %h;` work on 5.26.
__perl_ver="$(perl -MConfig -e 'print $Config{version}')" __perl_ver="$(perl -MConfig -e 'print $Config{version}')"
if dpkg --compare-versions "$__perl_ver" lt '5.28'; then if dpkg --compare-versions "$__perl_ver" lt '5.28'; then
sed -i -E 's/^(\s*)state\s+([%@][^=]+=)/\1my \2/; s/^(\s*)state\s+\(([^)]+)\)/\1my (\2)/' /usr/share/perl5/Debian/Debhelper/Dh_Lib.pm sed -i -E 's/^(\s*)state\s+([%@][^=]+=)/\1my \2/; s/^(\s*)state\s+\(([^)]+)\)/\1my (\2)/' /usr/share/perl5/Debian/Debhelper/Dh_Lib.pm
sed -i 's/^use v5\.28;/use v5.26;/' /usr/bin/dh_missing sed -i 's/^use v5\.28;/use v5.26;/' /usr/share/perl5/Debian/Debhelper/Dh_Lib.pm /usr/bin/dh_assistant /usr/bin/dh_missing
fi fi
# debhelper hardcodes versioned deps on init-system-helpers that old distros # debhelper hardcodes versioned deps on init-system-helpers that old distros
@@ -295,28 +285,12 @@ if dpkg --compare-versions "$__debhelper_ver" lt '13.12~'; then
fi fi
# On non-merged-usr distros (e.g. Ubuntu 18.04) deb-systemd-helper only # On non-merged-usr distros (e.g. Ubuntu 18.04) deb-systemd-helper only
# searches /lib/systemd/system, but debhelper 13.14 installs units to # searches /lib/systemd/system, but debhelper installs units to
# /usr/lib: keep units in /lib so services actually get enabled. # /usr/lib: keep units in /lib so services actually get enabled.
# (debhelper >= 14 also uses the ${tmpdir} brace form; rewrite both.)
if [ ! -L /lib ]; then if [ ! -L /lib ]; then
sed -i 's|\$tmpdir/usr/lib/systemd/system|$tmpdir/lib/systemd/system|g' /usr/bin/dh_installsystemd sed -i 's|$tmpdir/usr/lib/systemd/system|$tmpdir/lib/systemd/system|g; s|${tmpdir}/usr/lib/systemd/system|${tmpdir}/lib/systemd/system|g' /usr/bin/dh_installsystemd
fi fi
fi fi
#CODE_NAME=$(lsb_release -sc)
# if [ "${CODE_NAME}" != "focal" ]; then
# apt install -y dh-virtualenv
# fi
# case ${CODE_NAME} in
# # dists with coreutils >= 9.5 can use the latest debhelper from debian sid
# trixie)
# _DEBIAN_DEBHELPER
# ;;
# plucky|questing|resolute)
# _DEBIAN_DEBHELPER
# ;;
# *)
# echo "$CODE_NAME does NOT NEED to add Debian sources."
# ;;
# esac
exit 0 exit 0

View File

@@ -18,14 +18,36 @@ arg1="${1:-}"
source $__dir/version.env source $__dir/version.env
# debhelper .debs for old distros: downloaded here on the host (current CA and
# network), installed by install_deps.sh from builddep/ (gitignored). Old
# distros cannot build the sid OpenSSH source with their own debhelper
# (dh-sequence-movetousr needs >= 13.11.7), and adding sid apt sources inside
# old containers is fragile, so we just fetch the two arch-all .debs directly.
DEBHELPER_LINKS=(
$DEBMIRROR/pool/main/d/debhelper/debhelper_${DEBHELPER_SIDPKG}_all.deb
$DEBMIRROR/pool/main/d/debhelper/libdebhelper-perl_${DEBHELPER_SIDPKG}_all.deb
)
# `./pullsrc.sh debhelper` fetches only the .debs (used by the deps-image CI
# before docker build, where the openssh sources are not needed).
if [[ "$arg1" == "debhelper" ]]; then
mkdir -p $__dir/builddep
echo "> INFO: downloading debhelper ${DEBHELPER_SIDPKG} .debs into builddep/."
wget --continue -P "$__dir/builddep" "${DEBHELPER_LINKS[@]}"
exit 0
fi
DOWNLOADLINKS=( DOWNLOADLINKS=(
$DEBMIRROR/pool/main/o/openssh/openssh_${OPENSSH_SIDPKG}.{debian.tar.xz,dsc} $DEBMIRROR/pool/main/o/openssh/openssh_${OPENSSH_SIDPKG}.{debian.tar.xz,dsc}
$DEBMIRROR/pool/main/o/openssh/openssh_${OPENSSHVER}.orig.tar.gz{,.asc} $DEBMIRROR/pool/main/o/openssh/openssh_${OPENSSHVER}.orig.tar.gz{,.asc}
${OPENSSLMIR}/${OPENSSLSRC} ${OPENSSLMIR}/${OPENSSLSRC}
) )
mkdir -p $__dir/downloads && cd $__dir/downloads mkdir -p $__dir/downloads $__dir/builddep && cd $__dir/downloads
echo "> INFO: downloading the following sources." echo "> INFO: downloading the following sources."
echo "${DOWNLOADLINKS[@]}" | tr " " "\n" echo "${DOWNLOADLINKS[@]}" | tr " " "\n"
wget --continue "${DOWNLOADLINKS[@]}" wget --continue "${DOWNLOADLINKS[@]}"
echo "> INFO: downloading debhelper ${DEBHELPER_SIDPKG} .debs into builddep/."
wget --continue -P "$__dir/builddep" "${DEBHELPER_LINKS[@]}"

View File

@@ -11,3 +11,13 @@ OPENSSH_SIDPKG=10.5p1-1
OPENSSHVER=$(echo $OPENSSH_SIDPKG|cut -d- -f1) OPENSSHVER=$(echo $OPENSSH_SIDPKG|cut -d- -f1)
# debhelper .debs for old distros: pullsrc.sh downloads them on the host into
# builddep/, install_deps.sh installs them there (old distros cannot build the
# sid OpenSSH source with their own debhelper: dh-sequence-movetousr needs
# >= 13.11.7). The sid pool only keeps the current version, so bump this pin
# when it drifts; leave empty to auto-detect the latest from the pool.
DEBHELPER_SIDPKG=14.3
if [[ -z $DEBHELPER_SIDPKG ]]; then
DEBHELPER_SIDPKG=$(wget -qO- $DEBMIRROR/pool/main/d/debhelper/ | grep -oP 'debhelper_\K[0-9][0-9.]+(?=_all\.deb)' | sort -V | tail -n 1)
fi