mirror of
https://hubproxy.babadafafafafa.cn/https://github.com/1Panel-dev/1Panel.git
synced 2026-09-20 08:03:55 +08:00
feat: 重构OpenResty模块部分以支持动态编译 (#13291)
* feat: support dynamic module build for OpenResty * feat: add dynamic module build page for OpenResty * refactor: drop auto fallback, gate dynamic build by version support - remove auto-to-static fallback; dynamic build failure now reports the error and hints switching to static build manually - gate dynamic builds on module support files (Dockerfile.modules + module.catalog.json) instead of version numbers, expose dynamicSupported in the modules API - collect repeated path/status/operate strings into constants - move nginx module regex patterns into utils/re with semantic helpers - reorganize nginx_module.go around the main build flows and inline single-use thin helpers * feat: limit nginx module build mode options by version support - build mode radio offers only dynamic and static (auto maps to dynamic for legacy data) - disable the dynamic option with a hint when the installed OpenResty version lacks dynamic build support * feat: complete i18n for nginx module pages Fill in the new nginx module keys for all eleven language files (translations other than zh/en are draft machine translations). * feat: probe dynamic module support on load and drop the auto build mode - probe each non-static module's configure params when loading the module list and report dynamicSupport=supported/unsupported up front - normalize the legacy auto build mode to dynamic * feat: clarify module build modes in the UI - build drawer lists dynamic modules (tagged, hot-reload) and static modules (tagged, full rebuild + container restart) separately - disable the dynamic option per module when its params do not support dynamic build, distinct from the version gate hint - drop the auto build mode wording everywhere and sync all eleven language files * feat: clarify purpose of the nginx module build drawer - add a purpose hint explaining dynamic (hot reload) vs static (full rebuild + container restart) - drop the per-module mode tags now that section headers carry the semantics - allow submitting with zero dynamic modules selected when static modules are present, so static-only users can trigger a build * feat: pass apt mirror through to dynamic module builds The mirror selected in the build dialog (or CONTAINER_PACKAGE_URL in the app env as fallback) is now forwarded as a build arg so the module builder uses the same apt source as the static build path. test-builder gains a --mirror option. * feat: add Lao translations for nginx module pages
This commit is contained in:
Snrat
171
scripts/openresty-modules/README.md
Normal file
171
scripts/openresty-modules/README.md
Normal file
@@ -0,0 +1,171 @@
|
||||
# OpenResty Dynamic Module Linux Tests
|
||||
|
||||
These scripts test the local dynamic-module build path and collect diagnostics
|
||||
from an installed 1Panel OpenResty instance. Run them on a disposable Linux
|
||||
host with Docker access before testing on a production installation.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Bash 4.3 or newer
|
||||
- Docker Engine with the Compose v2 plugin
|
||||
- `jq`, `python3`, `file`, `binutils`, `tar`, and GNU coreutils
|
||||
- Internet access for runtime images and Ubuntu build packages
|
||||
- Go, only when `--source-checks` is used
|
||||
|
||||
On Debian or Ubuntu:
|
||||
|
||||
```bash
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y jq python3 file binutils tar
|
||||
```
|
||||
|
||||
Make the scripts executable:
|
||||
|
||||
```bash
|
||||
chmod +x scripts/openresty-modules/*.sh
|
||||
```
|
||||
|
||||
## Builder Test
|
||||
|
||||
Start with one version and one small module:
|
||||
|
||||
```bash
|
||||
./scripts/openresty-modules/test-builder.sh \
|
||||
--appstore ../appstore \
|
||||
--versions 1.31.1.1-0-noble \
|
||||
--modules ngx_brotli \
|
||||
--source-checks
|
||||
```
|
||||
|
||||
Test every catalog module against all refactored OpenResty versions:
|
||||
|
||||
```bash
|
||||
./scripts/openresty-modules/test-builder.sh --appstore ../appstore
|
||||
```
|
||||
|
||||
Bypass Docker's module build cache when reproducing a compiler problem:
|
||||
|
||||
```bash
|
||||
./scripts/openresty-modules/test-builder.sh \
|
||||
--appstore ../appstore \
|
||||
--versions 1.31.1.1-0-noble \
|
||||
--modules geoip2 \
|
||||
--no-cache \
|
||||
--keep-context \
|
||||
--keep-docker
|
||||
```
|
||||
|
||||
The builder test performs these phases for every selected version:
|
||||
|
||||
1. Validate appstore JSON, shell scripts, Compose mounts, and Nginx include.
|
||||
2. Pull and identify the exact target runtime image.
|
||||
3. Convert catalog options to dynamic configure options.
|
||||
4. Build every module with `Dockerfile.modules` and copy `/out` locally.
|
||||
5. Record SHA-256, ELF metadata, compiler output, and runtime dependencies.
|
||||
6. Validate individual modules for debugging. Individual failures are warnings
|
||||
by default because modules may depend on an earlier module.
|
||||
7. Validate all modules together in catalog `loadOrder`.
|
||||
8. Start an isolated OpenResty master, add module configs, and hot reload.
|
||||
9. Inject a missing module, prove `nginx -t` rejects it, restore the config,
|
||||
and prove the running process remains healthy.
|
||||
|
||||
Use `--strict-individual` when every selected module is expected to load alone.
|
||||
|
||||
Use `--mirror URL` (environment variable `MIRROR`) to pass an apt mirror as
|
||||
`CONTAINER_PACKAGE_URL` to module builds, matching the 1Panel module build.
|
||||
|
||||
Results are written to:
|
||||
|
||||
```text
|
||||
openresty-module-test-results/<run-id>/
|
||||
```
|
||||
|
||||
Important files:
|
||||
|
||||
- `summary.tsv`: result per OpenResty version
|
||||
- `work/<version>/logs/build-*.log`: complete BuildKit output
|
||||
- `work/<version>/logs/load-combined.log`: authoritative ABI/load-order test
|
||||
- `work/<version>/artifacts.tsv`: module paths, checksums, and sizes
|
||||
- `work/<version>/image-inspect.json`: exact target image identity
|
||||
- `work/<version>/runtime/`: reload and rollback test logs
|
||||
- `<result-dir>.tar.gz`: automatically created when an unexpected failure occurs
|
||||
|
||||
## Installed Instance Diagnostics
|
||||
|
||||
Find the OpenResty installation directory first. A common path is similar to:
|
||||
|
||||
```text
|
||||
/opt/1panel/apps/openresty/openresty
|
||||
```
|
||||
|
||||
Run the diagnostic collector:
|
||||
|
||||
```bash
|
||||
./scripts/openresty-modules/diagnose-install.sh \
|
||||
/opt/1panel/apps/openresty/openresty
|
||||
```
|
||||
|
||||
Override container discovery when needed:
|
||||
|
||||
```bash
|
||||
./scripts/openresty-modules/diagnose-install.sh \
|
||||
/opt/1panel/apps/openresty/openresty \
|
||||
--container 1Panel-openresty
|
||||
```
|
||||
|
||||
The collector checks:
|
||||
|
||||
- `module.json` artifact paths and SHA-256 checksums
|
||||
- managed `load_module` files and host/container path mapping
|
||||
- read-only Compose mounts
|
||||
- current container image ID versus enabled module target image IDs
|
||||
- container state, Nginx build options, `nginx -t`, loaded module directives,
|
||||
module checksums inside the container, `ldd`, and recent logs
|
||||
|
||||
The default report does not retain full `nginx -T` output. Use
|
||||
`--full-config` only on a test host because the resulting archive may contain
|
||||
credentials or private site configuration.
|
||||
|
||||
Module scripts are redacted from the copied state files by default. Container
|
||||
logs and error strings can still contain site names, URLs, or command output;
|
||||
review an archive before sharing it outside your team.
|
||||
|
||||
## Final Manual Matrix
|
||||
|
||||
Run this matrix through the 1Panel UI on a disposable installation. Collect a
|
||||
diagnostic archive after each important transition.
|
||||
|
||||
1. Install the oldest selected OpenResty version with every module disabled.
|
||||
2. Switch one module to `auto`, enable it, and build it locally.
|
||||
3. Confirm `buildStatus=ready`, `compatibility=compatible`, and `nginx -t`.
|
||||
4. Force rebuild it. Confirm the artifact path changes and the old config is
|
||||
replaced only after the new artifact passes validation.
|
||||
5. Enable all catalog modules and verify catalog load order with the builder
|
||||
test and the installed-instance collector.
|
||||
6. On the test host, make one module script return a failure. Confirm the old
|
||||
managed config and old ready artifact remain active.
|
||||
7. Restore the module definition and rebuild successfully.
|
||||
8. Upgrade OpenResty. Confirm every enabled dynamic module has a ready build
|
||||
whose target image ID matches the new running container.
|
||||
9. Restart the container and host. Run the diagnostic collector again to prove
|
||||
the persisted mounts and configs remain valid.
|
||||
10. Switch a module to `static`, rebuild, then switch it back to `dynamic` and
|
||||
verify that all enabled dynamic modules are regenerated for the new image.
|
||||
|
||||
## Failure Triage
|
||||
|
||||
| Symptom | First evidence to inspect |
|
||||
| --- | --- |
|
||||
| Docker build fails | `logs/build-<module>.log`, `inputs/<module>/` |
|
||||
| `.so` missing | build log and the `module-output` stage `/out` checks |
|
||||
| Individual load fails, combined passes | module dependency and `loadOrder` |
|
||||
| Combined load fails | ABI mismatch, duplicate module, missing shared library |
|
||||
| `ldd` shows `not found` | bundled `lib/`, RPATH, or future runtime packages |
|
||||
| Checksum mismatch | interrupted copy, manual modification, stale state file |
|
||||
| Target image mismatch | module was not rebuilt after image upgrade/rebuild |
|
||||
| Builder passes, installed `nginx -t` fails | Compose mounts or managed config |
|
||||
| Reload fails but old process runs | inspect rollback logs and old config snapshot |
|
||||
|
||||
Do not edit generated module state or managed config files while a 1Panel app
|
||||
task is running. Preserve the result directory and archive before retrying a
|
||||
failed build.
|
||||
475
scripts/openresty-modules/diagnose-install.sh
Normal file
475
scripts/openresty-modules/diagnose-install.sh
Normal file
@@ -0,0 +1,475 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -Eeuo pipefail
|
||||
|
||||
RUN_ID="$(date -u +%Y%m%dT%H%M%SZ)-$$"
|
||||
INSTALL_DIR=""
|
||||
OUTPUT_DIR="${OUTPUT_DIR:-${PWD}/openresty-module-diagnostics/${RUN_ID}}"
|
||||
CONTAINER=""
|
||||
FULL_CONFIG=0
|
||||
CREATE_ARCHIVE=1
|
||||
NGINX_TEST=1
|
||||
UNEXPECTED_FAILURE=""
|
||||
declare -a FAILED_CHECKS=()
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: diagnose-install.sh INSTALL_DIR [options]
|
||||
|
||||
Collect a mostly read-only diagnostic report for an installed 1Panel OpenResty.
|
||||
The only container command with behavior is `nginx -t`; no reload is performed.
|
||||
|
||||
Options:
|
||||
--output PATH Result directory
|
||||
--container NAME Override the container discovered from Docker Compose
|
||||
--full-config Retain full `nginx -T` output (may contain sensitive data)
|
||||
--no-nginx-test Do not execute nginx -t/-T in the running container
|
||||
--no-archive Do not create a .tar.gz report
|
||||
-h, --help Show this help
|
||||
EOF
|
||||
}
|
||||
|
||||
log() {
|
||||
printf '[%s] %s\n' "$(date -u +%H:%M:%S)" "$*" | tee -a "${OUTPUT_DIR}/run.log"
|
||||
}
|
||||
|
||||
mark_failed() {
|
||||
FAILED_CHECKS+=("$1")
|
||||
log "CHECK FAILED: $1"
|
||||
}
|
||||
|
||||
mark_passed() {
|
||||
log "CHECK PASSED: $1"
|
||||
}
|
||||
|
||||
require_command() {
|
||||
if ! command -v "$1" >/dev/null 2>&1; then
|
||||
printf 'Required command not found: %s\n' "$1" >&2
|
||||
exit 2
|
||||
fi
|
||||
}
|
||||
|
||||
on_error() {
|
||||
local status="$1" line="$2" command="$3"
|
||||
UNEXPECTED_FAILURE="line ${line}: ${command} (exit ${status})"
|
||||
return "${status}"
|
||||
}
|
||||
|
||||
finalize() {
|
||||
local status=$?
|
||||
set +e
|
||||
if [[ -n "${UNEXPECTED_FAILURE}" ]]; then
|
||||
printf '%s\n' "${UNEXPECTED_FAILURE}" >"${OUTPUT_DIR}/unexpected-failure.txt"
|
||||
fi
|
||||
{
|
||||
printf 'install_dir=%s\n' "${INSTALL_DIR}"
|
||||
printf 'container=%s\n' "${CONTAINER}"
|
||||
printf 'failed_checks=%s\n' "${#FAILED_CHECKS[@]}"
|
||||
local check
|
||||
for check in "${FAILED_CHECKS[@]:-}"; do
|
||||
[[ -n "${check}" ]] && printf 'failure=%s\n' "${check}"
|
||||
done
|
||||
} >"${OUTPUT_DIR}/summary.txt"
|
||||
|
||||
if [[ "${CREATE_ARCHIVE}" -eq 1 ]]; then
|
||||
local archive="${OUTPUT_DIR%/}.tar.gz"
|
||||
tar -czf "${archive}" -C "$(dirname -- "${OUTPUT_DIR}")" "$(basename -- "${OUTPUT_DIR}")" 2>/dev/null || true
|
||||
printf 'Diagnostic archive: %s\n' "${archive}"
|
||||
fi
|
||||
printf 'Diagnostic directory: %s\n' "${OUTPUT_DIR}"
|
||||
|
||||
if [[ "${status}" -eq 0 && "${#FAILED_CHECKS[@]}" -gt 0 ]]; then
|
||||
status=1
|
||||
fi
|
||||
exit "${status}"
|
||||
}
|
||||
|
||||
if [[ $# -eq 0 ]]; then
|
||||
usage >&2
|
||||
exit 2
|
||||
fi
|
||||
if [[ "$1" == "-h" || "$1" == "--help" ]]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
INSTALL_DIR="$1"
|
||||
shift
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--output)
|
||||
OUTPUT_DIR="$2"
|
||||
shift 2
|
||||
;;
|
||||
--container)
|
||||
CONTAINER="$2"
|
||||
shift 2
|
||||
;;
|
||||
--full-config)
|
||||
FULL_CONFIG=1
|
||||
shift
|
||||
;;
|
||||
--no-nginx-test)
|
||||
NGINX_TEST=0
|
||||
shift
|
||||
;;
|
||||
--no-archive)
|
||||
CREATE_ARCHIVE=0
|
||||
shift
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
printf 'Unknown option: %s\n' "$1" >&2
|
||||
usage >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -d "${INSTALL_DIR}" ]] || {
|
||||
printf 'Install directory not found: %s\n' "${INSTALL_DIR}" >&2
|
||||
exit 2
|
||||
}
|
||||
INSTALL_DIR="$(cd -- "${INSTALL_DIR}" && pwd -P)"
|
||||
if [[ -d "${OUTPUT_DIR}" ]] && find "${OUTPUT_DIR}" -mindepth 1 -print -quit | grep -q .; then
|
||||
printf 'Output directory must be empty: %s\n' "${OUTPUT_DIR}" >&2
|
||||
exit 2
|
||||
fi
|
||||
mkdir -p "${OUTPUT_DIR}"
|
||||
OUTPUT_DIR="$(cd -- "${OUTPUT_DIR}" && pwd -P)"
|
||||
|
||||
trap 'on_error "$?" "$LINENO" "$BASH_COMMAND"' ERR
|
||||
trap finalize EXIT
|
||||
|
||||
preflight() {
|
||||
[[ "$(uname -s)" == "Linux" ]] || {
|
||||
printf 'This diagnostic script must run on Linux.\n' >&2
|
||||
exit 2
|
||||
}
|
||||
require_command docker
|
||||
require_command jq
|
||||
require_command python3
|
||||
require_command sha256sum
|
||||
require_command tar
|
||||
|
||||
docker version >"${OUTPUT_DIR}/docker-version.txt" 2>&1
|
||||
docker info >"${OUTPUT_DIR}/docker-info.txt" 2>&1
|
||||
uname -a >"${OUTPUT_DIR}/uname.txt"
|
||||
cp /etc/os-release "${OUTPUT_DIR}/os-release.txt" 2>/dev/null || true
|
||||
df -h >"${OUTPUT_DIR}/disk-free.txt"
|
||||
free -h >"${OUTPUT_DIR}/memory.txt" 2>&1 || true
|
||||
log "Inspecting ${INSTALL_DIR}"
|
||||
}
|
||||
|
||||
collect_filesystem_state() {
|
||||
[[ -d "${INSTALL_DIR}/modules" ]] || mark_failed "module artifact directory is missing"
|
||||
[[ -d "${INSTALL_DIR}/conf/modules-enabled" ]] || mark_failed "managed module config directory is missing"
|
||||
find "${INSTALL_DIR}/modules" -maxdepth 5 -printf '%M\t%u:%g\t%s\t%TY-%Tm-%TdT%TH:%TM:%TS\t%p\n' \
|
||||
>"${OUTPUT_DIR}/module-files.txt" 2>&1 || true
|
||||
find "${INSTALL_DIR}/conf/modules-enabled" -maxdepth 1 -type f -printf '%f\n' \
|
||||
>"${OUTPUT_DIR}/managed-config-files.txt" 2>&1 || true
|
||||
grep -RnsE '^[[:space:]]*load_module[[:space:]]+' "${INSTALL_DIR}/conf/modules-enabled" \
|
||||
>"${OUTPUT_DIR}/load-module-directives.txt" 2>&1 || true
|
||||
grep -E '^(RESTY_|CONTAINER_NAME=|PANEL_APP_PORT_HTTP=)' "${INSTALL_DIR}/.env" \
|
||||
>"${OUTPUT_DIR}/relevant-env.txt" 2>/dev/null || true
|
||||
|
||||
if [[ -f "${INSTALL_DIR}/build/module.json" ]]; then
|
||||
jq 'map(if has("script") then .script = "<redacted>" else . end)' \
|
||||
"${INSTALL_DIR}/build/module.json" >"${OUTPUT_DIR}/module-state.json"
|
||||
else
|
||||
mark_failed "module state file is missing"
|
||||
fi
|
||||
if [[ -f "${INSTALL_DIR}/build/module.catalog.json" ]]; then
|
||||
jq 'map(if has("script") then .script = "<redacted>" else . end)' \
|
||||
"${INSTALL_DIR}/build/module.catalog.json" >"${OUTPUT_DIR}/module-catalog.json"
|
||||
fi
|
||||
}
|
||||
|
||||
validate_artifacts() {
|
||||
local state="${INSTALL_DIR}/build/module.json"
|
||||
[[ -f "${state}" ]] || return 0
|
||||
if python3 - "${state}" "${INSTALL_DIR}/modules" >"${OUTPUT_DIR}/artifact-validation.tsv" <<'PY'
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
state_path = pathlib.Path(sys.argv[1])
|
||||
modules_root = pathlib.Path(sys.argv[2]).resolve()
|
||||
modules = json.loads(state_path.read_text(encoding="utf-8"))
|
||||
failed = False
|
||||
print("module\tbuild_status\ttarget_key\tartifact\texpected\tactual\tresult")
|
||||
for module in modules:
|
||||
for build in module.get("builds") or []:
|
||||
target_key = (build.get("target") or {}).get("key", "")
|
||||
for artifact in build.get("artifacts") or []:
|
||||
relative = artifact.get("path", "")
|
||||
expected = artifact.get("checksum", "")
|
||||
result = "OK"
|
||||
actual = ""
|
||||
try:
|
||||
pure = pathlib.PurePosixPath(relative)
|
||||
if not relative or pure.is_absolute() or ".." in pure.parts or "\\" in relative:
|
||||
raise ValueError("unsafe-path")
|
||||
candidate = modules_root / pathlib.Path(*pure.parts)
|
||||
if candidate.is_symlink():
|
||||
raise ValueError("symlink-not-allowed")
|
||||
full_path = candidate.resolve(strict=True)
|
||||
if modules_root not in full_path.parents:
|
||||
raise ValueError("outside-module-root")
|
||||
if not full_path.is_file():
|
||||
raise ValueError("not-regular-file")
|
||||
digest = hashlib.sha256()
|
||||
with full_path.open("rb") as handle:
|
||||
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
|
||||
digest.update(chunk)
|
||||
actual = digest.hexdigest()
|
||||
if actual.lower() != expected.lower():
|
||||
raise ValueError("checksum-mismatch")
|
||||
except Exception as error:
|
||||
result = str(error)
|
||||
failed = True
|
||||
print("\t".join([
|
||||
module.get("name", ""), build.get("status", ""), target_key,
|
||||
relative, expected, actual, result,
|
||||
]))
|
||||
sys.exit(1 if failed else 0)
|
||||
PY
|
||||
then
|
||||
mark_passed "artifact paths and checksums"
|
||||
else
|
||||
mark_failed "artifact paths or checksums"
|
||||
fi
|
||||
}
|
||||
|
||||
validate_managed_configs() {
|
||||
if python3 - "${INSTALL_DIR}/conf/modules-enabled" "${INSTALL_DIR}/modules" \
|
||||
>"${OUTPUT_DIR}/managed-config-validation.tsv" <<'PY'
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
|
||||
config_root = pathlib.Path(sys.argv[1])
|
||||
modules_root = pathlib.Path(sys.argv[2]).resolve()
|
||||
container_prefix = "/usr/local/openresty/nginx/modules/1panel/"
|
||||
pattern = re.compile(r"^\s*load_module\s+([^;]+);", re.MULTILINE)
|
||||
failed = False
|
||||
print("config\tcontainer_path\thost_path\tresult")
|
||||
if config_root.exists():
|
||||
for config in sorted(config_root.glob("1panel-module-*.conf")):
|
||||
content = config.read_text(encoding="utf-8")
|
||||
for value in pattern.findall(content):
|
||||
container_path = value.strip().strip('"\'')
|
||||
result = "OK"
|
||||
host_path = ""
|
||||
try:
|
||||
if not container_path.startswith(container_prefix):
|
||||
raise ValueError("unexpected-container-path")
|
||||
relative = pathlib.PurePosixPath(container_path[len(container_prefix):])
|
||||
if ".." in relative.parts:
|
||||
raise ValueError("unsafe-path")
|
||||
resolved = (modules_root / pathlib.Path(*relative.parts)).resolve(strict=True)
|
||||
if modules_root not in resolved.parents or not resolved.is_file():
|
||||
raise ValueError("missing-artifact")
|
||||
host_path = str(resolved)
|
||||
except Exception as error:
|
||||
result = str(error)
|
||||
failed = True
|
||||
print("\t".join([config.name, container_path, host_path, result]))
|
||||
sys.exit(1 if failed else 0)
|
||||
PY
|
||||
then
|
||||
mark_passed "managed load_module configs"
|
||||
else
|
||||
mark_failed "managed load_module configs"
|
||||
fi
|
||||
}
|
||||
|
||||
collect_compose_state() {
|
||||
local compose_file="${INSTALL_DIR}/docker-compose.yml"
|
||||
if [[ ! -f "${compose_file}" ]]; then
|
||||
mark_failed "docker-compose.yml is missing"
|
||||
return 0
|
||||
fi
|
||||
|
||||
if (cd "${INSTALL_DIR}" && docker compose config --format json) >"${OUTPUT_DIR}/compose.json" 2>"${OUTPUT_DIR}/compose-config.log"; then
|
||||
mark_passed "docker compose config"
|
||||
else
|
||||
mark_failed "docker compose config"
|
||||
return 0
|
||||
fi
|
||||
|
||||
if jq -e '
|
||||
[.services[].volumes[]?]
|
||||
| (any(.[]; (.target | rtrimstr("/")) == "/usr/local/openresty/nginx/modules/1panel" and .read_only == true))
|
||||
and (any(.[]; (.target | rtrimstr("/")) == "/usr/local/openresty/nginx/conf/modules-enabled" and .read_only == true))
|
||||
' "${OUTPUT_DIR}/compose.json" >/dev/null; then
|
||||
mark_passed "read-only module mounts"
|
||||
else
|
||||
mark_failed "read-only module mounts"
|
||||
fi
|
||||
|
||||
(cd "${INSTALL_DIR}" && docker compose ps -a --format json) >"${OUTPUT_DIR}/compose-ps.json" 2>&1 || true
|
||||
if [[ -z "${CONTAINER}" ]]; then
|
||||
local cid
|
||||
cid="$(cd "${INSTALL_DIR}" && docker compose ps -q 2>/dev/null | head -n 1)"
|
||||
if [[ -n "${cid}" ]]; then
|
||||
CONTAINER="$(docker inspect --format '{{.Name}}' "${cid}" | sed 's#^/##')"
|
||||
else
|
||||
CONTAINER="$(jq -r '[.services[] | select((.image // "") | test("openresty"; "i")) | .container_name][0] // empty' \
|
||||
"${OUTPUT_DIR}/compose.json")"
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
compare_target_identity() {
|
||||
local state="${INSTALL_DIR}/build/module.json"
|
||||
local current_image_id="$1"
|
||||
[[ -f "${state}" ]] || return 0
|
||||
if python3 - "${state}" "${current_image_id}" "${INSTALL_DIR}/conf/modules-enabled" \
|
||||
>"${OUTPUT_DIR}/target-identity.tsv" <<'PY'
|
||||
import json
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
|
||||
modules = json.load(open(sys.argv[1], encoding="utf-8"))
|
||||
current = sys.argv[2]
|
||||
config_root = pathlib.Path(sys.argv[3])
|
||||
pattern = re.compile(r"^\s*load_module\s+([^;]+);", re.MULTILINE)
|
||||
prefix = "/usr/local/openresty/nginx/modules/1panel/"
|
||||
loaded = set()
|
||||
if config_root.exists():
|
||||
for config in config_root.glob("1panel-module-*.conf"):
|
||||
for value in pattern.findall(config.read_text(encoding="utf-8")):
|
||||
container_path = value.strip().strip('"\'')
|
||||
if container_path.startswith(prefix):
|
||||
loaded.add(container_path[len(prefix):])
|
||||
failed = False
|
||||
print("module\tenabled\tmode\tready_image_ids\tmanaged_artifacts\tresult")
|
||||
for module in modules:
|
||||
mode = module.get("buildMode") or "static"
|
||||
if module.get("deleted") or not module.get("enable") or mode == "static":
|
||||
continue
|
||||
ready = [build for build in module.get("builds") or [] if build.get("status") == "ready"]
|
||||
digests = sorted({(build.get("target") or {}).get("imageDigest", "") for build in ready})
|
||||
candidates = []
|
||||
if not ready:
|
||||
result = "NO-READY-BUILD"
|
||||
failed = True
|
||||
elif current in digests:
|
||||
result = "MATCH"
|
||||
candidates = [build for build in ready if (build.get("target") or {}).get("imageDigest") == current]
|
||||
elif not any(digests):
|
||||
result = "UNKNOWN-NO-IMAGE-DIGEST"
|
||||
candidates = ready
|
||||
else:
|
||||
result = "MISMATCH"
|
||||
failed = True
|
||||
|
||||
managed = []
|
||||
if candidates:
|
||||
for build in candidates:
|
||||
paths = [artifact.get("path", "") for artifact in build.get("artifacts") or []]
|
||||
if paths and all(path in loaded for path in paths):
|
||||
managed = paths
|
||||
break
|
||||
if not managed:
|
||||
result += "+NOT-IN-MANAGED-CONFIG"
|
||||
failed = True
|
||||
print("\t".join([
|
||||
module.get("name", ""), str(module.get("enable", False)),
|
||||
mode, ",".join(digests), ",".join(managed), result,
|
||||
]))
|
||||
sys.exit(1 if failed else 0)
|
||||
PY
|
||||
then
|
||||
mark_passed "enabled module target image identity"
|
||||
else
|
||||
mark_failed "enabled module target image identity"
|
||||
fi
|
||||
}
|
||||
|
||||
collect_container_state() {
|
||||
if [[ -z "${CONTAINER}" ]]; then
|
||||
mark_failed "OpenResty container could not be discovered"
|
||||
return 0
|
||||
fi
|
||||
if ! docker inspect "${CONTAINER}" >/dev/null 2>&1; then
|
||||
mark_failed "container ${CONTAINER} does not exist"
|
||||
return 0
|
||||
fi
|
||||
|
||||
docker inspect "${CONTAINER}" | jq '.[0] | {
|
||||
Id, Name, Image, State,
|
||||
Config: {Image: .Config.Image},
|
||||
Mounts: [.Mounts[] | {Type, Source, Destination, RW}]
|
||||
}' >"${OUTPUT_DIR}/container.json"
|
||||
docker logs --tail 1000 --timestamps "${CONTAINER}" >"${OUTPUT_DIR}/container.log" 2>&1 || true
|
||||
|
||||
local running image_id image_name
|
||||
running="$(docker inspect --format '{{.State.Running}}' "${CONTAINER}")"
|
||||
image_id="$(docker inspect --format '{{.Image}}' "${CONTAINER}")"
|
||||
image_name="$(docker inspect --format '{{.Config.Image}}' "${CONTAINER}")"
|
||||
docker image inspect "${image_id}" | jq '.[0] | {Id, RepoTags, RepoDigests, Architecture, Os, Created}' \
|
||||
>"${OUTPUT_DIR}/runtime-image.json" 2>&1 || true
|
||||
printf 'container=%s\nrunning=%s\nimage_name=%s\nimage_id=%s\n' \
|
||||
"${CONTAINER}" "${running}" "${image_name}" "${image_id}" >"${OUTPUT_DIR}/runtime.txt"
|
||||
compare_target_identity "${image_id}"
|
||||
|
||||
if [[ "${running}" != "true" ]]; then
|
||||
mark_failed "container ${CONTAINER} is not running"
|
||||
return 0
|
||||
fi
|
||||
mark_passed "container is running"
|
||||
|
||||
docker exec "${CONTAINER}" /usr/local/openresty/nginx/sbin/nginx -V \
|
||||
>"${OUTPUT_DIR}/nginx-version.txt" 2>&1 || mark_failed "nginx -V"
|
||||
docker exec "${CONTAINER}" /bin/sh -c \
|
||||
'find /usr/local/openresty/nginx/modules/1panel -type f -name "*.so" -exec sha256sum {} \; | sort' \
|
||||
>"${OUTPUT_DIR}/container-artifact-checksums.txt" 2>&1 || true
|
||||
docker exec "${CONTAINER}" /bin/sh -c \
|
||||
'for f in $(find /usr/local/openresty/nginx/modules/1panel -type f -name "*.so" | sort); do echo "### $f"; ldd "$f" || true; done' \
|
||||
>"${OUTPUT_DIR}/container-artifact-ldd.txt" 2>&1 || true
|
||||
|
||||
if [[ "${NGINX_TEST}" -eq 1 ]]; then
|
||||
if docker exec "${CONTAINER}" /usr/local/openresty/nginx/sbin/nginx -t \
|
||||
>"${OUTPUT_DIR}/nginx-test.txt" 2>&1; then
|
||||
mark_passed "running container nginx -t"
|
||||
else
|
||||
mark_failed "running container nginx -t"
|
||||
fi
|
||||
|
||||
local full_output="${OUTPUT_DIR}/nginx-T.full.tmp"
|
||||
docker exec "${CONTAINER}" /usr/local/openresty/nginx/sbin/nginx -T >"${full_output}" 2>&1 || true
|
||||
if [[ "${FULL_CONFIG}" -eq 1 ]]; then
|
||||
mv "${full_output}" "${OUTPUT_DIR}/nginx-T.full.txt"
|
||||
log "WARNING: nginx-T.full.txt may contain credentials or private configuration"
|
||||
else
|
||||
grep -nE 'load_module|modules-enabled|nginx version:|configure arguments:' "${full_output}" \
|
||||
>"${OUTPUT_DIR}/nginx-T-modules.txt" 2>/dev/null || true
|
||||
rm -f "${full_output}"
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
main() {
|
||||
preflight
|
||||
collect_filesystem_state
|
||||
validate_artifacts
|
||||
validate_managed_configs
|
||||
collect_compose_state
|
||||
collect_container_state
|
||||
|
||||
if [[ "${#FAILED_CHECKS[@]}" -gt 0 ]]; then
|
||||
log "Diagnostics completed with ${#FAILED_CHECKS[@]} failed checks"
|
||||
return 0
|
||||
fi
|
||||
log "Diagnostics completed without failed checks"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
542
scripts/openresty-modules/test-builder.sh
Normal file
542
scripts/openresty-modules/test-builder.sh
Normal file
@@ -0,0 +1,542 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -Eeuo pipefail
|
||||
export DOCKER_BUILDKIT="${DOCKER_BUILDKIT:-1}"
|
||||
|
||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
|
||||
REPO_ROOT="$(cd -- "${SCRIPT_DIR}/../.." && pwd -P)"
|
||||
DEFAULT_APPSTORE_ROOT="$(cd -- "${REPO_ROOT}/.." && pwd -P)/appstore"
|
||||
RUN_ID="$(date -u +%Y%m%dT%H%M%SZ)-$$"
|
||||
|
||||
APPSTORE_ROOT="${APPSTORE_ROOT:-${DEFAULT_APPSTORE_ROOT}}"
|
||||
VERSIONS_CSV="1.27.1.2-5-1-focal,1.29.2.5-0-noble,1.31.1.1-0-noble"
|
||||
MODULES_CSV=""
|
||||
OUTPUT_DIR="${OUTPUT_DIR:-${PWD}/openresty-module-test-results/${RUN_ID}}"
|
||||
MIRROR="${MIRROR:-}"
|
||||
SKIP_PULL=0
|
||||
NO_CACHE=0
|
||||
KEEP_DOCKER=0
|
||||
KEEP_CONTEXT=0
|
||||
RUN_SOURCE_CHECKS=0
|
||||
STRICT_INDIVIDUAL=0
|
||||
CLEANUP_READY=0
|
||||
|
||||
declare -a CREATED_CONTAINERS=()
|
||||
declare -a CREATED_IMAGES=()
|
||||
declare -a VERSIONS=()
|
||||
declare -a REQUESTED_MODULES=()
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: test-builder.sh [options]
|
||||
|
||||
Build and load-test OpenResty dynamic modules directly from the appstore tree.
|
||||
|
||||
Options:
|
||||
--appstore PATH Appstore repository root (default: sibling appstore repo)
|
||||
--versions CSV App versions to test
|
||||
--modules CSV Module names to test (default: every catalog module)
|
||||
--output PATH Persistent result directory
|
||||
--mirror URL apt mirror for module build packages (CONTAINER_PACKAGE_URL)
|
||||
--skip-pull Use local runtime images without pulling
|
||||
--no-cache Pass --no-cache to every module Docker build
|
||||
--strict-individual Fail when a module cannot load by itself
|
||||
--source-checks Run Go module tests and go vet before Docker tests
|
||||
--keep-docker Keep temporary images and containers
|
||||
--keep-context Keep copied Docker build contexts
|
||||
-h, --help Show this help
|
||||
|
||||
Environment equivalents: APPSTORE_ROOT, OUTPUT_DIR, MIRROR, DOCKER_BUILDKIT.
|
||||
EOF
|
||||
}
|
||||
|
||||
log() {
|
||||
printf '[%s] %s\n' "$(date -u +%H:%M:%S)" "$*" | tee -a "${OUTPUT_DIR}/run.log"
|
||||
}
|
||||
|
||||
die() {
|
||||
log "ERROR: $*"
|
||||
return 1
|
||||
}
|
||||
|
||||
require_command() {
|
||||
command -v "$1" >/dev/null 2>&1 || die "required command not found: $1"
|
||||
}
|
||||
|
||||
split_csv() {
|
||||
local value="$1"
|
||||
local -n destination="$2"
|
||||
IFS=',' read -r -a destination <<<"${value}"
|
||||
}
|
||||
|
||||
safe_name() {
|
||||
local value="$1"
|
||||
local base digest
|
||||
base="$(printf '%s' "${value}" | sed -E 's/[^a-zA-Z0-9._-]+/-/g; s/^-+//; s/-+$//' | cut -c1-48)"
|
||||
[[ -n "${base}" ]] || base="module"
|
||||
digest="$(printf '%s' "${value}" | sha256sum | awk '{print substr($1,1,8)}')"
|
||||
printf '%s-%s' "${base}" "${digest}"
|
||||
}
|
||||
|
||||
run_logged() {
|
||||
local log_file="$1"
|
||||
shift
|
||||
mkdir -p "$(dirname -- "${log_file}")"
|
||||
set +e
|
||||
"$@" > >(tee "${log_file}") 2>&1
|
||||
local status=$?
|
||||
set -e
|
||||
return "${status}"
|
||||
}
|
||||
|
||||
docker_rm_container() {
|
||||
local name="$1"
|
||||
docker rm -f "${name}" >/dev/null 2>&1 || true
|
||||
}
|
||||
|
||||
cleanup() {
|
||||
local status=$?
|
||||
if [[ "${CLEANUP_READY}" -eq 0 ]]; then
|
||||
exit "${status}"
|
||||
fi
|
||||
if [[ "${KEEP_DOCKER}" -eq 0 ]]; then
|
||||
local item
|
||||
for item in "${CREATED_CONTAINERS[@]:-}"; do
|
||||
[[ -n "${item}" ]] && docker_rm_container "${item}"
|
||||
done
|
||||
for item in "${CREATED_IMAGES[@]:-}"; do
|
||||
[[ -n "${item}" ]] && docker image rm -f "${item}" >/dev/null 2>&1 || true
|
||||
done
|
||||
fi
|
||||
if [[ "${KEEP_CONTEXT}" -eq 0 && -d "${OUTPUT_DIR}/work" ]]; then
|
||||
find "${OUTPUT_DIR}/work" -mindepth 2 -maxdepth 2 -type d -name context -prune -exec rm -rf -- {} + 2>/dev/null || true
|
||||
fi
|
||||
exit "${status}"
|
||||
}
|
||||
|
||||
write_debug_bundle() {
|
||||
local status="$1" line="$2" command="$3"
|
||||
{
|
||||
printf 'exit_status=%s\n' "${status}"
|
||||
printf 'line=%s\n' "${line}"
|
||||
printf 'command=%s\n' "${command}"
|
||||
printf 'run_id=%s\n' "${RUN_ID}"
|
||||
} >"${OUTPUT_DIR}/failure.txt"
|
||||
|
||||
docker ps -a --no-trunc >"${OUTPUT_DIR}/docker-ps.txt" 2>&1 || true
|
||||
docker image ls --digests --no-trunc >"${OUTPUT_DIR}/docker-images.txt" 2>&1 || true
|
||||
docker system df >"${OUTPUT_DIR}/docker-system-df.txt" 2>&1 || true
|
||||
|
||||
local item
|
||||
for item in "${CREATED_CONTAINERS[@]:-}"; do
|
||||
[[ -n "${item}" ]] || continue
|
||||
docker inspect "${item}" >"${OUTPUT_DIR}/container-${item}.json" 2>&1 || true
|
||||
docker logs "${item}" >"${OUTPUT_DIR}/container-${item}.log" 2>&1 || true
|
||||
done
|
||||
|
||||
local archive="${OUTPUT_DIR%/}.tar.gz"
|
||||
tar --exclude='*/context' -czf "${archive}" -C "$(dirname -- "${OUTPUT_DIR}")" "$(basename -- "${OUTPUT_DIR}")" 2>/dev/null || true
|
||||
printf 'Debug bundle: %s\n' "${archive}" >&2
|
||||
}
|
||||
|
||||
on_error() {
|
||||
local status="$1" line="$2" command="$3"
|
||||
set +e
|
||||
log "FAILED at line ${line}: ${command} (exit ${status})"
|
||||
write_debug_bundle "${status}" "${line}" "${command}"
|
||||
return "${status}"
|
||||
}
|
||||
|
||||
trap cleanup EXIT
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--appstore)
|
||||
APPSTORE_ROOT="$2"
|
||||
shift 2
|
||||
;;
|
||||
--versions)
|
||||
VERSIONS_CSV="$2"
|
||||
shift 2
|
||||
;;
|
||||
--modules)
|
||||
MODULES_CSV="$2"
|
||||
shift 2
|
||||
;;
|
||||
--output)
|
||||
OUTPUT_DIR="$2"
|
||||
shift 2
|
||||
;;
|
||||
--mirror)
|
||||
MIRROR="$2"
|
||||
shift 2
|
||||
;;
|
||||
--skip-pull)
|
||||
SKIP_PULL=1
|
||||
shift
|
||||
;;
|
||||
--no-cache)
|
||||
NO_CACHE=1
|
||||
shift
|
||||
;;
|
||||
--strict-individual)
|
||||
STRICT_INDIVIDUAL=1
|
||||
shift
|
||||
;;
|
||||
--source-checks)
|
||||
RUN_SOURCE_CHECKS=1
|
||||
shift
|
||||
;;
|
||||
--keep-docker)
|
||||
KEEP_DOCKER=1
|
||||
shift
|
||||
;;
|
||||
--keep-context)
|
||||
KEEP_CONTEXT=1
|
||||
shift
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
printf 'Unknown option: %s\n' "$1" >&2
|
||||
usage >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ -d "${OUTPUT_DIR}" ]] && find "${OUTPUT_DIR}" -mindepth 1 -print -quit | grep -q .; then
|
||||
printf 'Output directory must be empty: %s\n' "${OUTPUT_DIR}" >&2
|
||||
exit 2
|
||||
fi
|
||||
mkdir -p "${OUTPUT_DIR}/work"
|
||||
OUTPUT_DIR="$(cd -- "${OUTPUT_DIR}" && pwd -P)"
|
||||
CLEANUP_READY=1
|
||||
trap 'on_error "$?" "$LINENO" "$BASH_COMMAND"' ERR
|
||||
APPSTORE_ROOT="$(cd -- "${APPSTORE_ROOT}" && pwd -P)"
|
||||
split_csv "${VERSIONS_CSV}" VERSIONS
|
||||
if [[ "${#VERSIONS[@]}" -eq 0 || -z "${VERSIONS[0]}" ]]; then
|
||||
die "at least one OpenResty version is required"
|
||||
fi
|
||||
if [[ -n "${MODULES_CSV}" ]]; then
|
||||
split_csv "${MODULES_CSV}" REQUESTED_MODULES
|
||||
fi
|
||||
|
||||
preflight() {
|
||||
[[ "$(uname -s)" == "Linux" ]] || die "this integration test must run on Linux"
|
||||
require_command docker
|
||||
require_command jq
|
||||
require_command python3
|
||||
require_command sha256sum
|
||||
require_command sed
|
||||
require_command awk
|
||||
require_command tar
|
||||
require_command file
|
||||
require_command readelf
|
||||
|
||||
docker version >"${OUTPUT_DIR}/docker-version.txt" 2>&1
|
||||
docker info >"${OUTPUT_DIR}/docker-info.txt" 2>&1
|
||||
docker compose version >"${OUTPUT_DIR}/docker-compose-version.txt" 2>&1
|
||||
uname -a >"${OUTPUT_DIR}/uname.txt"
|
||||
cp /etc/os-release "${OUTPUT_DIR}/os-release.txt" 2>/dev/null || true
|
||||
df -h >"${OUTPUT_DIR}/disk-free.txt"
|
||||
free -h >"${OUTPUT_DIR}/memory.txt" 2>&1 || true
|
||||
|
||||
[[ -d "${APPSTORE_ROOT}/apps/openresty" ]] || die "invalid appstore root: ${APPSTORE_ROOT}"
|
||||
log "Results: ${OUTPUT_DIR}"
|
||||
log "Appstore: ${APPSTORE_ROOT}"
|
||||
log "Docker architecture: $(docker info --format '{{.Architecture}}')"
|
||||
}
|
||||
|
||||
run_source_checks() {
|
||||
[[ "${RUN_SOURCE_CHECKS}" -eq 1 ]] || return 0
|
||||
require_command go
|
||||
log "Running Go dynamic-module tests"
|
||||
run_logged "${OUTPUT_DIR}/go-test.log" bash -c \
|
||||
"cd '${REPO_ROOT}/agent' && go test ./app/service -run 'NginxModule|DynamicModule' -count=1 -v"
|
||||
log "Running go vet"
|
||||
run_logged "${OUTPUT_DIR}/go-vet.log" bash -c "cd '${REPO_ROOT}/agent' && go vet ./..."
|
||||
}
|
||||
|
||||
validate_template() {
|
||||
local version="$1"
|
||||
local app_dir="${APPSTORE_ROOT}/apps/openresty/${version}"
|
||||
local catalog="${app_dir}/build/module.catalog.json"
|
||||
|
||||
[[ "${version}" =~ ^[a-zA-Z0-9._-]+$ ]] || die "unsafe version value: ${version}"
|
||||
[[ -f "${app_dir}/build/Dockerfile.modules" ]] || die "missing Dockerfile.modules for ${version}"
|
||||
[[ -f "${catalog}" ]] || die "missing module catalog for ${version}"
|
||||
jq -e 'type == "array" and length > 0 and all(.[]; .name and .params and .provider == "local")' \
|
||||
"${catalog}" >/dev/null
|
||||
bash -n "${app_dir}/scripts/init.sh"
|
||||
bash -n "${app_dir}/scripts/upgrade.sh"
|
||||
grep -Fq 'conf/modules-enabled:/usr/local/openresty/nginx/conf/modules-enabled/:ro' "${app_dir}/docker-compose.yml"
|
||||
grep -Fq './modules:/usr/local/openresty/nginx/modules/1panel/:ro' "${app_dir}/docker-compose.yml"
|
||||
grep -Fq 'include /usr/local/openresty/nginx/conf/modules-enabled/*.conf;' "${app_dir}/conf/nginx.conf"
|
||||
|
||||
mkdir -p "${OUTPUT_DIR}/work/${version}/website/conf.d" "${OUTPUT_DIR}/work/${version}/website/stream.d"
|
||||
(
|
||||
cd "${app_dir}"
|
||||
CONTAINER_NAME="openresty-template-check" \
|
||||
WEBSITE_DIR="${OUTPUT_DIR}/work/${version}/website" \
|
||||
PANEL_APP_PORT_HTTP=18080 \
|
||||
docker compose config -q
|
||||
) >"${OUTPUT_DIR}/work/${version}/compose-config.log" 2>&1
|
||||
}
|
||||
|
||||
write_module_inputs() {
|
||||
local catalog="$1" module="$2" context="$3" input_dir="$4"
|
||||
local module_script params dynamic_params packages
|
||||
|
||||
module_script="$(jq -er --arg name "${module}" '.[] | select(.name == $name) | .script' "${catalog}")"
|
||||
params="$(jq -er --arg name "${module}" '.[] | select(.name == $name) | .params' "${catalog}")"
|
||||
packages="$(jq -er --arg name "${module}" '.[] | select(.name == $name) | (.packages // []) | join(" ")' "${catalog}")"
|
||||
dynamic_params="${params//--add-module=/--add-dynamic-module=}"
|
||||
|
||||
mkdir -p "${input_dir}"
|
||||
printf '%s\n' "${module_script}" >"${input_dir}/script.txt"
|
||||
printf '%s\n' "${params}" >"${input_dir}/params.original.txt"
|
||||
printf '%s\n' "${dynamic_params}" >"${input_dir}/params.dynamic.txt"
|
||||
printf '%s\n' "${packages}" >"${input_dir}/packages.txt"
|
||||
printf '#!/bin/bash\nset -e\n%s\n' "${module_script}" >"${context}/tmp/module-pre.sh"
|
||||
|
||||
python3 - "${dynamic_params}" >"${context}/tmp/module-config.args" <<'PY'
|
||||
import shlex
|
||||
import sys
|
||||
|
||||
params = sys.argv[1]
|
||||
args = shlex.split(params, posix=True)
|
||||
if not args:
|
||||
raise SystemExit("dynamic module parameters are empty")
|
||||
if not any(arg.startswith("--add-dynamic-module=") or "=dynamic" in arg for arg in args):
|
||||
raise SystemExit("module does not declare a dynamic configure option")
|
||||
for arg in args:
|
||||
if not arg.startswith("--"):
|
||||
raise SystemExit(f"unsupported configure argument: {arg!r}")
|
||||
if any(char in arg for char in "\x00\r\n;&|<>"):
|
||||
raise SystemExit(f"unsafe configure argument: {arg!r}")
|
||||
print(arg)
|
||||
PY
|
||||
}
|
||||
|
||||
validate_load_directives() {
|
||||
local image="$1" modules_root="$2" directives_file="$3" config_path="$4" log_file="$5"
|
||||
{
|
||||
cat "${directives_file}"
|
||||
printf 'error_log stderr notice;\npid /tmp/nginx.pid;\nevents {}\nhttp {}\n'
|
||||
} >"${config_path}"
|
||||
|
||||
run_logged "${log_file}" docker run --rm --network none \
|
||||
-v "${modules_root}:/usr/local/openresty/nginx/modules/1panel:ro" \
|
||||
-v "${config_path}:/tmp/1panel-module-test.conf:ro" \
|
||||
--entrypoint /usr/local/openresty/nginx/sbin/nginx \
|
||||
"${image}" -t -c /tmp/1panel-module-test.conf
|
||||
}
|
||||
|
||||
build_module() {
|
||||
local version="$1" module="$2" image="$3" app_dir="$4" version_dir="$5" context="$6" sequence="$7"
|
||||
local catalog="${app_dir}/build/module.catalog.json"
|
||||
local module_key module_dir input_dir tag build_log cid packages artifact relative checksum
|
||||
local -a artifacts=()
|
||||
module_key="$(safe_name "${module}")"
|
||||
module_dir="${version_dir}/modules/${module_key}/${RUN_ID}"
|
||||
input_dir="${version_dir}/inputs/${module_key}"
|
||||
tag="1panel/openresty-module-test:${module_key}-$(safe_name "${version}")-${RUN_ID}"
|
||||
tag="${tag:0:127}"
|
||||
build_log="${version_dir}/logs/build-${module_key}.log"
|
||||
|
||||
log "[${version}] building ${module}"
|
||||
write_module_inputs "${catalog}" "${module}" "${context}" "${input_dir}"
|
||||
packages="$(cat "${input_dir}/packages.txt")"
|
||||
|
||||
local -a build_args=(
|
||||
build --progress=plain --target module-output
|
||||
-f "${context}/Dockerfile.modules"
|
||||
-t "${tag}"
|
||||
--build-arg "PANEL_OPENRESTY_VERSION=${version}"
|
||||
--build-arg "RESTY_ADD_PACKAGE_BUILDDEPS=${packages}"
|
||||
)
|
||||
[[ "${NO_CACHE}" -eq 0 ]] || build_args+=(--no-cache)
|
||||
[[ -z "${MIRROR}" ]] || build_args+=(--build-arg "CONTAINER_PACKAGE_URL=${MIRROR}")
|
||||
build_args+=("${context}")
|
||||
|
||||
run_logged "${build_log}" docker "${build_args[@]}"
|
||||
CREATED_IMAGES+=("${tag}")
|
||||
|
||||
cid="1panel-module-copy-${module_key}-${RUN_ID}"
|
||||
cid="${cid:0:63}"
|
||||
docker create --name "${cid}" "${tag}" /bin/true >"${input_dir}/container-id.txt"
|
||||
CREATED_CONTAINERS+=("${cid}")
|
||||
mkdir -p "${module_dir}"
|
||||
docker cp "${cid}:/out/." "${module_dir}"
|
||||
docker_rm_container "${cid}"
|
||||
|
||||
mapfile -t artifacts < <(find "${module_dir}" -maxdepth 1 -type f -name '*.so' -print | sort)
|
||||
[[ "${#artifacts[@]}" -gt 0 ]] || die "${module} produced no top-level .so files"
|
||||
|
||||
: >"${input_dir}/load-directives.conf"
|
||||
for artifact in "${artifacts[@]}"; do
|
||||
relative="${artifact#${version_dir}/modules/}"
|
||||
[[ "${relative}" =~ ^[a-zA-Z0-9_./+-]+\.so$ ]] || \
|
||||
die "unsafe module artifact path: ${relative}"
|
||||
checksum="$(sha256sum "${artifact}" | awk '{print $1}')"
|
||||
printf '%s\t%s\t%s\t%s\n' "${module}" "${relative}" "${checksum}" "$(stat -c '%s' "${artifact}")" \
|
||||
>>"${version_dir}/artifacts.tsv"
|
||||
printf 'load_module /usr/local/openresty/nginx/modules/1panel/%s;\n' "${relative}" \
|
||||
>>"${input_dir}/load-directives.conf"
|
||||
file "${artifact}" >>"${input_dir}/file.txt"
|
||||
readelf -d "${artifact}" >>"${input_dir}/readelf-dynamic.txt" 2>&1 || true
|
||||
done
|
||||
|
||||
if ! validate_load_directives "${image}" "${version_dir}/modules" \
|
||||
"${input_dir}/load-directives.conf" "${input_dir}/individual-nginx.conf" \
|
||||
"${version_dir}/logs/load-${module_key}.log"; then
|
||||
printf '%s\tindividual-load-failed\n' "${module}" >>"${version_dir}/status.tsv"
|
||||
if [[ "${STRICT_INDIVIDUAL}" -eq 1 ]]; then
|
||||
die "${module} failed individual load validation"
|
||||
fi
|
||||
log "[${version}] ${module} cannot load alone; combined validation will decide"
|
||||
else
|
||||
printf '%s\tindividual-load-ok\n' "${module}" >>"${version_dir}/status.tsv"
|
||||
fi
|
||||
|
||||
cat "${input_dir}/load-directives.conf" >>"${version_dir}/combined-load-directives.conf"
|
||||
cp "${input_dir}/load-directives.conf" \
|
||||
"${version_dir}/ordered-configs/$(printf '%04d' "${sequence}")-${module_key}.conf"
|
||||
}
|
||||
|
||||
runtime_reload_test() {
|
||||
local version="$1" image="$2" version_dir="$3"
|
||||
local runtime_dir="${version_dir}/runtime" container="1panel-module-runtime-$(safe_name "${version}")-${RUN_ID}"
|
||||
local -a module_configs=()
|
||||
container="${container:0:63}"
|
||||
mkdir -p "${runtime_dir}/modules-enabled"
|
||||
|
||||
cat >"${runtime_dir}/nginx.conf" <<'EOF'
|
||||
error_log stderr notice;
|
||||
pid /tmp/nginx.pid;
|
||||
include /tmp/modules-enabled/*.conf;
|
||||
events {}
|
||||
http {}
|
||||
EOF
|
||||
printf '# empty initial module set\n' >"${runtime_dir}/modules-enabled/0000-empty.conf"
|
||||
|
||||
mapfile -t module_configs < <(find "${version_dir}/ordered-configs" -type f -name '*.conf' -print | sort)
|
||||
[[ "${#module_configs[@]}" -gt 0 ]] || die "no module configs available for runtime test"
|
||||
|
||||
log "[${version}] starting runtime reload test"
|
||||
docker run -d --name "${container}" --network none \
|
||||
-v "${version_dir}/modules:/usr/local/openresty/nginx/modules/1panel:ro" \
|
||||
-v "${runtime_dir}/modules-enabled:/tmp/modules-enabled:ro" \
|
||||
-v "${runtime_dir}/nginx.conf:/tmp/1panel-runtime-nginx.conf:ro" \
|
||||
--entrypoint /usr/local/openresty/nginx/sbin/nginx \
|
||||
"${image}" -c /tmp/1panel-runtime-nginx.conf -g 'daemon off;' \
|
||||
>"${runtime_dir}/container-id.txt"
|
||||
CREATED_CONTAINERS+=("${container}")
|
||||
|
||||
local attempt
|
||||
for ((attempt = 1; attempt <= 20; attempt++)); do
|
||||
if [[ "$(docker inspect --format '{{.State.Running}}' "${container}" 2>/dev/null || true)" == "true" ]]; then
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
if [[ "$(docker inspect --format '{{.State.Running}}' "${container}" 2>/dev/null || true)" != "true" ]]; then
|
||||
docker logs "${container}" >"${runtime_dir}/startup-failure.log" 2>&1 || true
|
||||
die "runtime container failed to start"
|
||||
fi
|
||||
|
||||
local config
|
||||
for config in "${module_configs[@]}"; do
|
||||
cp "${config}" "${runtime_dir}/modules-enabled/$(basename -- "${config}")"
|
||||
done
|
||||
run_logged "${runtime_dir}/nginx-test.log" docker exec "${container}" \
|
||||
/usr/local/openresty/nginx/sbin/nginx -t -c /tmp/1panel-runtime-nginx.conf
|
||||
run_logged "${runtime_dir}/nginx-reload.log" docker exec "${container}" \
|
||||
/usr/local/openresty/nginx/sbin/nginx -s reload -c /tmp/1panel-runtime-nginx.conf
|
||||
|
||||
printf 'load_module /usr/local/openresty/nginx/modules/1panel/not-found.so;\n' \
|
||||
>"${runtime_dir}/modules-enabled/9999-invalid.conf"
|
||||
if docker exec "${container}" /usr/local/openresty/nginx/sbin/nginx \
|
||||
-t -c /tmp/1panel-runtime-nginx.conf >"${runtime_dir}/expected-invalid.log" 2>&1; then
|
||||
die "nginx -t unexpectedly accepted a missing module"
|
||||
fi
|
||||
rm -f "${runtime_dir}/modules-enabled/9999-invalid.conf"
|
||||
run_logged "${runtime_dir}/rollback-nginx-test.log" docker exec "${container}" \
|
||||
/usr/local/openresty/nginx/sbin/nginx -t -c /tmp/1panel-runtime-nginx.conf
|
||||
[[ "$(docker inspect --format '{{.State.Running}}' "${container}")" == "true" ]] || \
|
||||
die "runtime container stopped during rollback test"
|
||||
|
||||
docker logs "${container}" >"${runtime_dir}/container.log" 2>&1 || true
|
||||
docker exec "${container}" /bin/sh -c \
|
||||
'for f in /usr/local/openresty/nginx/modules/1panel/*/*/*.so; do echo "### $f"; ldd "$f" || true; done' \
|
||||
>"${runtime_dir}/ldd.txt" 2>&1 || true
|
||||
docker_rm_container "${container}"
|
||||
}
|
||||
|
||||
test_version() {
|
||||
local version="$1"
|
||||
local app_dir="${APPSTORE_ROOT}/apps/openresty/${version}"
|
||||
local version_dir="${OUTPUT_DIR}/work/${version}"
|
||||
local context="${version_dir}/context"
|
||||
local catalog="${app_dir}/build/module.catalog.json"
|
||||
local image="1panel/openresty:${version}"
|
||||
local -a modules=()
|
||||
|
||||
validate_template "${version}"
|
||||
mkdir -p "${version_dir}/logs" "${version_dir}/inputs" "${version_dir}/modules" \
|
||||
"${version_dir}/ordered-configs"
|
||||
cp -a "${app_dir}/build" "${context}"
|
||||
: >"${version_dir}/artifacts.tsv"
|
||||
: >"${version_dir}/status.tsv"
|
||||
: >"${version_dir}/combined-load-directives.conf"
|
||||
|
||||
if [[ "${#REQUESTED_MODULES[@]}" -gt 0 ]]; then
|
||||
modules=("${REQUESTED_MODULES[@]}")
|
||||
else
|
||||
mapfile -t modules < <(jq -r 'sort_by([.loadOrder // 50, .name])[] | .name' "${catalog}")
|
||||
fi
|
||||
|
||||
if [[ "${SKIP_PULL}" -eq 0 ]]; then
|
||||
log "[${version}] pulling ${image}"
|
||||
run_logged "${version_dir}/logs/image-pull.log" docker pull "${image}"
|
||||
fi
|
||||
docker image inspect "${image}" >"${version_dir}/image-inspect.json"
|
||||
run_logged "${version_dir}/logs/nginx-version.log" docker run --rm \
|
||||
--entrypoint /usr/local/openresty/nginx/sbin/nginx "${image}" -V
|
||||
sha256sum "${app_dir}/build/Dockerfile.modules" >"${version_dir}/builder.sha256"
|
||||
find "${app_dir}/build/tmp" -maxdepth 1 -type f -print0 | sort -z | xargs -0 sha256sum \
|
||||
>"${version_dir}/build-inputs.sha256"
|
||||
|
||||
local module sequence=0
|
||||
for module in "${modules[@]}"; do
|
||||
sequence=$((sequence + 1))
|
||||
jq -e --arg name "${module}" 'any(.[]; .name == $name)' "${catalog}" >/dev/null || \
|
||||
die "module ${module} is not present in ${version} catalog"
|
||||
build_module "${version}" "${module}" "${image}" "${app_dir}" "${version_dir}" "${context}" "${sequence}"
|
||||
done
|
||||
|
||||
log "[${version}] validating the combined load order"
|
||||
validate_load_directives "${image}" "${version_dir}/modules" \
|
||||
"${version_dir}/combined-load-directives.conf" "${version_dir}/combined-nginx.conf" \
|
||||
"${version_dir}/logs/load-combined.log"
|
||||
runtime_reload_test "${version}" "${image}" "${version_dir}"
|
||||
printf '%s\tPASS\n' "${version}" >>"${OUTPUT_DIR}/summary.tsv"
|
||||
log "[${version}] PASS"
|
||||
}
|
||||
|
||||
main() {
|
||||
preflight
|
||||
run_source_checks
|
||||
printf 'version\tresult\n' >"${OUTPUT_DIR}/summary.tsv"
|
||||
local version
|
||||
for version in "${VERSIONS[@]}"; do
|
||||
test_version "${version}"
|
||||
done
|
||||
log "All requested OpenResty module tests passed"
|
||||
log "Summary: ${OUTPUT_DIR}/summary.tsv"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Reference in New Issue
Block a user