Compare commits

...

2 Commits

10 changed files with 98 additions and 32 deletions

View File

@@ -11,7 +11,7 @@ metadata:
APIs fail differently from web UIs: there is no rendered surface to crawl, the interesting bugs are authorization-shaped rather than injection-shaped, and the same endpoint behaves differently per token. This workflow targets those specifics with Strix's autonomous agents, using the current [OWASP API Security Top 10 (2023)](https://owasp.org/API-Security/editions/2023/en/0x11-t10/) as the coverage checklist. For the web-app equivalent, the current edition is the OWASP Top 10:2025 — see **owasp-top-10-testing**.
Install, LLM setup, full CLI flags, and the managed-cloud path are in the **penetration-testing-with-strix** skill. Read it if `strix --version` fails or the target is not an API.
Install, LLM setup, full CLI flags, and the managed-cloud path are in the **penetration-testing-with-strix** skill. Read it if `strix --version` fails or the target is not an API. For a run with no Docker and no LLM key, the same binary drives the managed platform: `strix cloud login`, then `strix cloud scans start ...` (details in **managed-pentesting-with-strix**).
## 1. Gather what the agents need

View File

@@ -11,7 +11,7 @@ metadata:
Entry point for "make my application secure" requests, where the target is not yet a single URL or repo. The job here is to pick the right test per asset, run it, and produce one ranked plan — not to run everything at maximum depth.
Install, LLM setup, all CLI flags, and the managed-cloud path live in the **penetration-testing-with-strix** skill. Read it first if `strix --version` fails.
Install, LLM setup, all CLI flags, and the managed-cloud path live in the **penetration-testing-with-strix** skill. Read it first if `strix --version` fails. For a run with no Docker and no LLM key, the same binary drives the managed platform: `strix cloud login`, then `strix cloud scans start ...` (details in **managed-pentesting-with-strix**).
Only test assets the user owns or is authorized to test. Confirm authorization before the first run, and prefer staging over production, because the agents send real exploit payloads and can change data.

View File

@@ -113,11 +113,11 @@ Gate the pipeline on the exit code (see the budget/fail-open caveat above — gi
# Option B — Managed platform (no runner infra)
No workflow file, no Docker, no LLM key. Two ways to use it:
No workflow file, no Docker, no LLM key. Three ways to use it:
1. **PR-review app (zero code):** the user installs the Strix GitHub/GitLab/Bitbucket app and enables PR reviews for the repo in the app.strix.ai dashboard. Every PR is then reviewed automatically, with findings posted as PR comments. Nothing to add to the repo. This is the lowest-effort path — recommend it first when the user just wants PR gating.
2. **API-triggered from any pipeline:** if you want to trigger from an existing pipeline (or a system without the SCM app), call the API with a token that has `pr_reviews:write` (or `scans:write`). Store the token as a CI secret; ask the user to create it at **Settings → API Access**. Example GitHub Actions step:
2. **CLI-triggered from any pipeline:** if you want to trigger from an existing pipeline (or a system without the SCM app), use the same `strix` binary with a token that has `pr_reviews:write`. Store the token as a CI secret and ask the user to create it at **Settings → API Access**. Read the repository's `provider` and `installation_id` once with `strix cloud repos list`. Example GitHub Actions step:
```yaml
- name: Strix PR review (managed)
@@ -125,12 +125,25 @@ No workflow file, no Docker, no LLM key. Two ways to use it:
env:
STRIX_API_TOKEN: ${{ secrets.STRIX_API_TOKEN }}
run: |
curl -sS --fail https://app.strix.ai/api/v1/pr-reviews/start \
-H "Authorization: Bearer $STRIX_API_TOKEN" \
-H "Content-Type: application/json" \
-d "{\"repository_full_name\":\"${{ github.repository }}\",\"pr_number\":${{ github.event.pull_request.number }}}"
curl -sSL https://strix.ai/install | bash
strix cloud pr-reviews start \
--provider github \
--installation-id "${{ vars.STRIX_INSTALLATION_ID }}" \
--repository-full-name "${{ github.repository }}" \
--pr-number "${{ github.event.pull_request.number }}"
```
To gate the build on results, poll the PR review / scan status and fail on unresolved criticals/highs. Full endpoints (PR reviews, scans, SARIF export, schedules for scheduled deep scans) are in the **managed-pentesting-with-strix** skill.
Output is JSON when stdout is not a terminal, and there are no prompts without a TTY. To gate the build on results, poll `strix cloud pr-reviews get <id> --json` and fail on unresolved criticals or highs. The raw REST endpoint (`POST /api/v1/pr-reviews/start`) works too when the pipeline cannot install the CLI.
3. **Source upload from a pipeline without an SCM app:** upload the checked-out tree as a cloud code review (`scans:write` and `uploads:write`). The two-step digest handoff keeps a human in control of what leaves the runner:
```bash
strix cloud scans start --source . --dry-run --show-files --json # review, capture source.archive_sha256
strix cloud scans start --source . --approve-sha256 "$SOURCE_SHA256" --wait
```
Exit codes: `0` success, `4` auth or plan limit, `5` payment required. Non-Enterprise scans consume credits.
Full CLI coverage (PR reviews, scans, SARIF export, schedules) is in the **managed-pentesting-with-strix** skill.
Recommend Option B for most teams (no maintenance, central dashboard); use Option A when scans must stay entirely within your own infrastructure.

View File

@@ -11,7 +11,7 @@ metadata:
White-box security review with Strix: the agents read the source to build a model of routes, sinks, and authorization checks, then attempt real exploitation. Findings come with a proof-of-concept, so the output is a short list of proven issues rather than the hundreds of "potential" hits a pattern-matching scanner produces.
Install, LLM setup, all flags, and the managed-cloud path are in the **penetration-testing-with-strix** skill.
Install, LLM setup, all flags, and the managed-cloud path are in the **penetration-testing-with-strix** skill. For a run with no Docker and no LLM key, the same binary drives the managed platform: `strix cloud login`, then `strix cloud scans start ...` (details in **managed-pentesting-with-strix**).
## Run it

View File

@@ -18,7 +18,7 @@ Get the findings from wherever the scan ran:
- **OSS CLI** — artifacts in `strix_runs/<run-name>/`:
- `vulnerabilities/*.md` — one finding per file: description, severity, PoC steps or script, affected code locations, remediation guidance.
- `vulnerabilities.json` — the same findings as JSON (ids, severity, CWE/CVE, `code_locations` with `fix_before`/`fix_after` suggestions when available).
- **Cloud (app.strix.ai)** — fetch the scan's `vulnerabilities[]` via `GET /api/v1/scans/{scanId}` (or `GET /api/v1/vulnerabilities` org-wide). Each carries `severity, cwe, endpoint, method, impact, technical_analysis, poc_description, poc_script_code` and, for code findings, `code_file`/`code_diff`/`code_before`/`code_after`. See the **managed-pentesting-with-strix** skill for auth.
- **Cloud (app.strix.ai)** — pull findings with the CLI: `strix cloud vulns list --scan-id <scan-id> --json` (or `strix cloud scans get <scan-id> --json | jq '.vulnerabilities'`, or `strix cloud vulns list --severity critical` org-wide). Each finding carries `severity, cwe, endpoint, method, impact, technical_analysis, poc_description, poc_script_code` and, for code findings, `code_file`/`code_diff`/`code_before`/`code_after`. After a fix is verified, mark it with `strix cloud vulns update <id> --status fixed`. See the **managed-pentesting-with-strix** skill for `strix cloud login` and scopes.
Order work by severity: critical → high → medium → low. Every Strix finding was validated with a working proof-of-concept, so do not dismiss findings as false positives without re-testing the PoC yourself.

View File

@@ -13,7 +13,7 @@ The OWASP Top 10 is a taxonomy of risk categories, not a test suite — "OWASP T
**Use the current edition: [OWASP Top 10:2025](https://owasp.org/Top10/)** (8th installment, superseding 2021). Ask the user before targeting an older edition — some compliance checklists still reference 2021, and a report labelled with the wrong edition is misleading. Key differences from 2021: **SSRF is folded into A01**, **A03 Software Supply Chain Failures** expands the old "Vulnerable and Outdated Components", and **A10 Mishandling of Exceptional Conditions** is new; A02 Security Misconfiguration moved 5→2.
Install, LLM setup, and the managed-cloud alternative: **penetration-testing-with-strix**.
Install, LLM setup, and the managed-cloud alternative: **penetration-testing-with-strix**. For a run with no Docker and no LLM key, the same binary drives the managed platform: `strix cloud login`, then `strix cloud scans start ...` (details in **managed-pentesting-with-strix**).
## What is and is not testable by an agent

View File

@@ -11,7 +11,7 @@ metadata:
Black-box (and optionally source-assisted) penetration testing of a running web app with Strix's autonomous agents. Every reported finding is validated with a working exploit, so there are no signature-based false positives to triage.
Install, LLM setup, all CLI flags, and the managed-cloud alternative are covered in the **penetration-testing-with-strix** skill — read it if the target is not a running web app, or if `strix --version` fails. This skill is the web-app-specific workflow.
Install, LLM setup, all CLI flags, and the managed-cloud alternative are covered in the **penetration-testing-with-strix** skill — read it if the target is not a running web app, or if `strix --version` fails. For a run with no Docker and no LLM key, the same binary drives the managed platform: `strix cloud login`, then `strix cloud scans start ...` (details in **managed-pentesting-with-strix**). This skill is the web-app-specific workflow.
## 1. Confirm authorization and scope

View File

@@ -50,13 +50,6 @@ def _unknown_connection(connection: str, registry: McpRegistry) -> str:
return f"Unknown MCP connection {connection!r}. Available connections: {available}."
def _unavailable_connection(connection: str) -> str:
return (
f"MCP connection {connection!r} is unavailable: its live session failed and "
"could not be reconnected, so it is unavailable for the rest of this run."
)
def _format_tool(tool: MCPTool) -> str:
schema = json.dumps(tool.inputSchema or {"type": "object"}, indent=2, ensure_ascii=False)
description = (tool.description or "").strip() or "(no description)"
@@ -114,8 +107,8 @@ async def describe_mcp(ctx: RunContextWrapper, connection: str) -> str:
return _unknown_connection(connection, registry)
try:
tools = await entry.session.list_tools()
except McpConnectionUnavailableError:
return _unavailable_connection(connection)
except McpConnectionUnavailableError as exc:
return str(exc)
if not tools:
return f"MCP connection {connection!r} offers no tools."
header = f"MCP connection {connection!r} offers {len(tools)} tool(s):"
@@ -170,8 +163,8 @@ async def call_mcp(
return invalid_arguments
try:
available = await entry.session.list_tools()
except McpConnectionUnavailableError:
return _errored_tool_output(_unavailable_connection(connection))
except McpConnectionUnavailableError as exc:
return _errored_tool_output(str(exc))
valid_names = {mcp_tool.name for mcp_tool in available}
if tool not in valid_names:
offered = ", ".join(sorted(valid_names)) or "(none)"

View File

@@ -109,10 +109,12 @@ def _call_semaphore(name: str, limit: int) -> asyncio.Semaphore:
class McpConnectionUnavailableError(RuntimeError):
"""A dead MCP connection could not be reached and did not come back.
"""The MCP connection cannot take requests right now.
Raised by :meth:`SupervisedMcpSession.list_tools` when the connection is dead
so the read-only dispatch tools (``describe_mcp``) can report it cleanly.
or in a quarantine cooldown. Its message is the session's own status text, so
the dispatch tools (``describe_mcp``, ``call_mcp``) can pass it to the agent
as-is: a cooldown reads as temporary, a dead connection as final.
:meth:`SupervisedMcpSession.dispatch` does not raise it: a call to a dead
connection returns the standard failed-tool output instead.
"""
@@ -607,12 +609,7 @@ class SupervisedMcpSession:
return _Outcome(call_failure=failure)
self._mark_dead(failure, attempt=attempt)
return _Outcome(dead=True)
if (
phase == "call"
and failure.kind == "protocol"
and failure.status is not None
and 400 <= failure.status <= 499
):
if phase == "call" and failure.kind == "protocol":
return _Outcome(call_failure=failure)
if attempt == _MAX_ATTEMPTS:
await self._quarantine(failure, attempt=attempt)
@@ -729,6 +726,13 @@ class SupervisedMcpSession:
"that resource, then retry."
)
if failure.kind == "protocol":
if failure.status is None:
return (
f"MCP connection {self._name!r} rejected this call: the provider "
"returned an error for this request, not the connection. The connection "
"is still available. The resource may not exist or the arguments may be "
"wrong. Check them with describe_mcp, then retry or move on."
)
return (
f"MCP connection {self._name!r} rejected this call as invalid "
f"(status={failure.status}): the request itself was malformed, not the "

View File

@@ -301,6 +301,62 @@ async def test_call_http_rejection_preserves_session(
await session.aclose()
@pytest.mark.asyncio
async def test_call_jsonrpc_error_preserves_session(
monkeypatch: pytest.MonkeyPatch,
) -> None:
# A JSON-RPC error is a well-formed reply to this request, so the session stays
# up: no reconnect, no retry, no quarantine. The streamable-HTTP client also
# synthesizes one (status-less "Session terminated") for an HTTP 404, which some
# providers return for a missing resource.
error = McpError(ErrorData(code=32600, message="Session terminated"))
builds = 0
def build(_config: Any) -> Any:
nonlocal builds
builds += 1
return _built_server(_sequence_server("rpc-error", error))
monkeypatch.setattr(mcp_client, "_build_server", build)
monkeypatch.setattr(mcp_session, "_retry_delay", _zero_delay)
session = mcp_session.SupervisedMcpSession(_config("rpc-error"))
assert await session.start()
result = await session.dispatch("read", {}, label="rpc_error_read")
assert result["success"] is False
assert "not the connection" in result["content"]
assert "still available" in result["content"]
assert session.is_dead is False
assert session.is_unavailable is False
assert session._quarantine_count == 0
assert builds == 1
await session.aclose()
@pytest.mark.asyncio
async def test_list_tools_during_quarantine_reports_temporary_state(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(mcp_session, "_retry_delay", _zero_delay)
monkeypatch.setattr(asyncio, "sleep", _no_sleep)
clock = [100.0]
monkeypatch.setattr("strix.tools.mcp.session.time.monotonic", lambda: clock[0])
builds = iter([_sequence_server("cooldown", _http_error(500)) for _ in range(3)])
monkeypatch.setattr(mcp_client, "_build_server", lambda _config: _built_server(next(builds)))
session = mcp_session.SupervisedMcpSession(_config("cooldown"))
assert await session.start()
await session.dispatch("read", {}, label="cooldown_read")
assert session.is_unavailable is True
with pytest.raises(mcp_session.McpConnectionUnavailableError) as excinfo:
await session.list_tools()
message = str(excinfo.value)
assert "temporarily unavailable" in message
assert "retrying in about 30 seconds" in message
assert "rest of this run" not in message
await session.aclose()
@pytest.mark.asyncio
async def test_call_http_403_during_list_tools_dies() -> None:
server = _list_tools_error_server("connect-403", _http_error(403))