1 line
29 KiB
JSON
1 line
29 KiB
JSON
{"repo": "blueif16/piflow", "files": {"docs/design/credential-architecture.md": {"791c2ecd7d8724cf619cf3afde9e0e3891ca73ca": {"t": 1785747188.0644767, "n": 9928, "h": "ca373696e672b38d", "c": "# Credential Architecture — one `SecretResolver`, two homes, reference-by-`$VAR`\n\n> **Status:** DESIGN (2026-06-26). The mechanism (`$VAR` reference + `SecretResolver` seam + allowlist\n> forwarding) is ALREADY SHIPPED in core; this doc records the credential-ORGANIZATION layer on top of it\n> (store, typed schema, DAG-surfacing, GUI input, cloud injection) and pins the M1 cloud wiring. Provenance:\n> the pi-credential placement was read from pi's OWN docs (not assumed); the orchestrator patterns from n8n +\n> Windmill docs/source (Exa, 2026-06-26). Cross-ref: `docs/design/node-action-protocol.md` §4 (the secret\n> allowlist + provider-credential parity check), `docs/design/daytona-cloud-integration.md` (M0–M4),\n> `docs/design/cloud-tool-gateway-architecture.md` (gateway topology). Memory: `daytona-cloud-path`.\n\n---\n\n## 1. Where credentials live TODAY (verified, not assumed)\n\n**pi's provider credential** (read from pi `docs/providers.md` + `docs/containerization.md`, pi `0.80.2`).\nResolution priority: **`auth.json` → env var**.\n- **`~/.pi/agent/auth.json`** (mode `0600`), per-provider record: `{\"anthropic\":{\"type\":\"api_key\",\"key\":\"sk-...\"}}`.\n Subscriptions (Claude Pro/Max, Codex, Copilot) store auto-refreshed OAuth tokens here too. *(On this host the\n file is empty `{}` — auth is via env vars.)*\n- **Provider env var** — `ANTHROPIC_API_KEY`, `DEEPSEEK_API_KEY`, `NEBIUS_API_KEY` (present in this host's env),\n etc. Full map: pi's `packages/ai/src/env-api-keys.ts`.\n- **Custom gateways** (the `--provider tokenrouter/nebius/cp` piflow passes) are defined in **`~/.pi/agent/models.json`**\n with `apiKey:\"$PROXY_API_KEY\"`-style `$VAR` refs + a provider-scoped `env` block.\n- **Container (= the Daytona VM)** — pi's doc prescribes exactly two ways: `docker run -e ANTHROPIC_API_KEY`\n (forward the provider env var) **or** mount `~/.pi/agent` (the auth file). The \"keys never enter the sandbox,\n a gateway injects upstream\" variant is OpenShell/`inference.local`.\n\n**piflow's tool/MCP credential** — `node.json.mcp.servers` carries `$VAR`/`${VAR}` REFERENCES only (literal\nsecrets are rejected by `checkMcpSecrets`, `workflow/template/checks.ts:266`); the runner resolves the\nreferenced set through the `SecretResolver` and forwards ONLY that allowlist (`mcpEnvAdditions` `runner.ts:481`,\n`referencedEnvVars` `:444`, cloud-allowlist delete `:495-499`).\n\n**The existing seam (the load-bearing primitive — DO NOT reinvent).** `SecretResolver` (`types.ts:626`,\ndefault `process.env` `:632`) is the single host seam: core owns the `$VAR` vocabulary + the allowlist\ncontract; the host owns the binding. The resolver receives `{nodeId, isCloud}` (`:472,490`) so it can mint a\nper-node, cloud-only SCOPED token. `CLOUD_KINDS={daytona,e2b}` (`runner.ts:430`) drives the cloud allowlist.\n\n---\n\n## 2. How the orchestrators do it (n8n · Windmill — the convergent model)\n\n| Principle | n8n | Windmill |\n|---|---|---|\n| Creds are SEPARATE records, never inlined in the workflow | `credentials_entity`; workflow refs by id | Variables / Resources; flow refs by path |\n| Encrypted at rest, ONE instance master key | AES-256, `N8N_ENCRYPTION_KEY` (env/config, auto-gen on first boot); EE adds a 2-layer ROTATABLE data key | workspace-specific symmetric key |\n| Reference-by-token, resolved at exec with the CALLER's perms | (refs by id) | `$var:<path>` / `$res:<path>` / `$WM_*`; worker fetches via an EPHEMERAL token, fails if caller lacks read |\n| Typed credential schemas (drive the UI form + \"what's needed\") | credential types | Resource Types (JSON Schema per integration) |\n| Scoping / sharing | personal-by-default; share-to-project; `credential:shareGlobally` scope; RBAC | workspace + folder perms; global vs project vaults |\n| External vault escape hatch | External Secrets (Vault…), global/project | EE delegates storage to an external secret manager |\n| Never echo key material; mask in logs | API returns only metadata | masks secret values in job stdout/stderr |\n\n**The single shared idea:** one encrypted store of TYPED credential records, referenced by a `$token`,\nresolved at job time with the runner's permissions. That is *already* piflow's `$VAR` + `SecretResolver`. The\ngap is purely the **store + typed schema + GUI input + DAG-surfacing** layers above the seam.\n\n---\n\n## 3. The piflow design — two homes, ONE resolver\n\n**Decision (2026-06-26): two homes, one resolver.**\n- **Provider keys → pi's native `~/.pi/agent/auth.json`** (or the provider env var). pi already owns these,\n `0600`, with subscription auto-refresh — do not duplicate them into a piflow store.\n- **Tool/MCP `$VAR`s → a new `~/.piflow/credentials.json`** — encrypted at rest, `0600`, master key\n `PIFLOW_ENCRYPTION_KEY` (env, else an auto-generated key file in `~/.piflow/`, mirroring n8n's first-boot\n key). Lives in the home global dir per the SDK data-boundary rule (`CLAUDE.md`: global secrets/index live in\n `~/.piflow/`, NEVER in `packages/`, the repo, or `gui/`).\n- **`SecretResolver` is the single seam** that reads from whichever home (or mints a scoped cloud token) and\n returns the value for a referenced `$VAR`. The SDK stays product-agnostic: it sees only the resolver; the\n store + master key live in the host/CLI/GUI layer.\n\n**Reference-only in the DAG.** node.json / the template carry `$VAR` references ONLY — already enforced\n(`checkMcpSecrets`). The DAG never stores a secret; the run-view never serializes one.\n\n**DAG surfaces the required credentials.** Derive the per-node required-cred SET statically:\n`referencedEnvVars(node.mcp)` ∪ the node's provider key (provider → env-var via pi's env-map, resolved by\n`model-routing.ts`). The GUI renders a pre-run preflight — each cred present / MISSING (probed through the\nresolver; the value is NEVER revealed) — an extension of today's dry-run plan, mirroring n8n's \"credential\nneeded\" gate. *(V1: the provider→env-var requirement may be declared explicitly per run; V2: parse the\nselected provider's `models.json` entry for its `$VAR` refs, the same deep-walk `referencedEnvVars` already\ndoes for `mcp.servers`.)*\n\n**GUI input → global store.** The GUI collects a typed credential (Windmill resource-type style: provider key,\nMCP header token, …), writes it encrypted to the right home (`auth.json` for a provider key, else\n`~/.piflow/credentials.json`), masked in the UI, never committed, never echoed back. Read happens only through\nthe resolver at run time.\n\n**Cloud injection = mint-scoped-token / forward-one-var, NEVER the raw long-lived key.** The cloud\n`SecretResolver` returns a short-lived scoped token (or forwards exactly the one provider env var into the VM\nexec env via the existing allowlist) — matching Windmill's ephemeral `WM_TOKEN` and pi's OpenShell\n\"keys stay outside the sandbox\" pattern. A cloud VM never inherits host `process.env` (`daytona.ts:261` merges\nonly `{...this.env,...opts.env}`; VM env `{PI_RUN}` `:518`), so the allowlist additions are the ONLY path in —\nand must be exactly the referenced set.\n\n**Deferred (not built now):** RBAC / credential sharing / project-vs-global vaults (single-user local needs\nnone — adopt n8n's scope model when multi-user/cloud lands); an external-vault backend (the resolver seam\nalready admits one — a host can plug Vault/AWS-SM later without a core change).\n\n---\n\n## 4. M1 — the cloud wiring this design pins\n\nThe pi gateway credential is NOT yet on the resolver seam for cloud (the `node-action-protocol.md` §4\nprovider-credential parity gap). M1 closes it, matching §3:\n1. **`--sandbox daytona` CLI branch** — `packages/cli/src/run.ts:355-394` handles `local`/`danger-full-access`/\n `inmemory` only; add the `daytona` case constructing `createDaytonaProvider({ image: process.env.DAYTONA_IMAGE,\n apiKey: process.env.DAYTONA_API_KEY })` (factory in `sandbox/daytona-sdk.ts`).\n2. **Thread a cloud `SecretResolver`** from the CLI into `runFromTemplate` (`RunOptions.secretResolver`\n `runner.ts:242`, already spread through `entry.ts`), defaulting to `process.env` read host-side.\n3. **Forward the provider gateway credential into the VM** — extend the cloud env additions so the selected\n provider's required env var(s) join the same allowlist `mcpEnvAdditions` forwards. **SHIPPED:** a declared\n `cloudSecrets: string[]` (`RunOptions`/`RunContext`) is resolved by `cloudCredEnvAdditions` (`runner.ts`)\n through the SAME `SecretResolver` and folded into the cloud `CreateOpts.env`; the CLI derives the var from\n `--provider` (or `--cloud-secret NAME`).\n4. **Custom-gateway config staging — SHIPPED (M1b).** A BUILT-IN provider reads `<PROVIDER>_API_KEY` directly,\n so M1's key forwarding alone suffices. A CUSTOM gateway (`nebius`/`mmgw`/`tokenrouter`) is defined ONLY in\n the host's `~/.pi/agent/models.json` (the official `docs/models.md` shape: `{providers:{<name>:{baseUrl,api,\n apiKey:\"$VAR\",models[]}}}`) which the M0 image bakes none of — so pi in the VM can't resolve `--provider`\n without it. The CLI's `parsePiProvider` (pure) scopes that file to the selected provider + extracts its\n `$VAR` cred refs (the authoritative allowlist); `DaytonaSandboxProvider.stageHome` writes the scoped\n `{providers:{[name]:entry}}` into the VM at `<home>/.pi/agent/models.json` BEFORE any node. The staged config\n carries `$VAR` REFERENCES only — never a resolved secret (the key still crosses via the cred allowlist).\n5. **Gates:** unit (no creds) — the `daytona` branch builds the provider, the cloud additions forward exactly\n the declared var (allowlist invariant), `parsePiProvider` scopes + extracts `$VAR`s, and `stageHome` writes\n the config into the VM (each must FAIL when its code is broken — mutation-checked); a credential-gated e2e\n (`skipIf`) running ONE node in a real VM asserting a real model call. **OPEN:** a usable `DAYTONA_IMAGE` ref\n must be supplied (M0 built via the declarative builder; promote to a named snapshot); the e2e is unrun.\n"}, "826eca170bd3372ae774e977ed5a7a91a5a8506a": {"t": 1785771155.6742892, "n": 9928, "h": "ca373696e672b38d", "c": "# Credential Architecture — one `SecretResolver`, two homes, reference-by-`$VAR`\n\n> **Status:** DESIGN (2026-06-26). The mechanism (`$VAR` reference + `SecretResolver` seam + allowlist\n> forwarding) is ALREADY SHIPPED in core; this doc records the credential-ORGANIZATION layer on top of it\n> (store, typed schema, DAG-surfacing, GUI input, cloud injection) and pins the M1 cloud wiring. Provenance:\n> the pi-credential placement was read from pi's OWN docs (not assumed); the orchestrator patterns from n8n +\n> Windmill docs/source (Exa, 2026-06-26). Cross-ref: `docs/design/node-action-protocol.md` §4 (the secret\n> allowlist + provider-credential parity check), `docs/design/daytona-cloud-integration.md` (M0–M4),\n> `docs/design/cloud-tool-gateway-architecture.md` (gateway topology). Memory: `daytona-cloud-path`.\n\n---\n\n## 1. Where credentials live TODAY (verified, not assumed)\n\n**pi's provider credential** (read from pi `docs/providers.md` + `docs/containerization.md`, pi `0.80.2`).\nResolution priority: **`auth.json` → env var**.\n- **`~/.pi/agent/auth.json`** (mode `0600`), per-provider record: `{\"anthropic\":{\"type\":\"api_key\",\"key\":\"sk-...\"}}`.\n Subscriptions (Claude Pro/Max, Codex, Copilot) store auto-refreshed OAuth tokens here too. *(On this host the\n file is empty `{}` — auth is via env vars.)*\n- **Provider env var** — `ANTHROPIC_API_KEY`, `DEEPSEEK_API_KEY`, `NEBIUS_API_KEY` (present in this host's env),\n etc. Full map: pi's `packages/ai/src/env-api-keys.ts`.\n- **Custom gateways** (the `--provider tokenrouter/nebius/cp` piflow passes) are defined in **`~/.pi/agent/models.json`**\n with `apiKey:\"$PROXY_API_KEY\"`-style `$VAR` refs + a provider-scoped `env` block.\n- **Container (= the Daytona VM)** — pi's doc prescribes exactly two ways: `docker run -e ANTHROPIC_API_KEY`\n (forward the provider env var) **or** mount `~/.pi/agent` (the auth file). The \"keys never enter the sandbox,\n a gateway injects upstream\" variant is OpenShell/`inference.local`.\n\n**piflow's tool/MCP credential** — `node.json.mcp.servers` carries `$VAR`/`${VAR}` REFERENCES only (literal\nsecrets are rejected by `checkMcpSecrets`, `workflow/template/checks.ts:266`); the runner resolves the\nreferenced set through the `SecretResolver` and forwards ONLY that allowlist (`mcpEnvAdditions` `runner.ts:481`,\n`referencedEnvVars` `:444`, cloud-allowlist delete `:495-499`).\n\n**The existing seam (the load-bearing primitive — DO NOT reinvent).** `SecretResolver` (`types.ts:626`,\ndefault `process.env` `:632`) is the single host seam: core owns the `$VAR` vocabulary + the allowlist\ncontract; the host owns the binding. The resolver receives `{nodeId, isCloud}` (`:472,490`) so it can mint a\nper-node, cloud-only SCOPED token. `CLOUD_KINDS={daytona,e2b}` (`runner.ts:430`) drives the cloud allowlist.\n\n---\n\n## 2. How the orchestrators do it (n8n · Windmill — the convergent model)\n\n| Principle | n8n | Windmill |\n|---|---|---|\n| Creds are SEPARATE records, never inlined in the workflow | `credentials_entity`; workflow refs by id | Variables / Resources; flow refs by path |\n| Encrypted at rest, ONE instance master key | AES-256, `N8N_ENCRYPTION_KEY` (env/config, auto-gen on first boot); EE adds a 2-layer ROTATABLE data key | workspace-specific symmetric key |\n| Reference-by-token, resolved at exec with the CALLER's perms | (refs by id) | `$var:<path>` / `$res:<path>` / `$WM_*`; worker fetches via an EPHEMERAL token, fails if caller lacks read |\n| Typed credential schemas (drive the UI form + \"what's needed\") | credential types | Resource Types (JSON Schema per integration) |\n| Scoping / sharing | personal-by-default; share-to-project; `credential:shareGlobally` scope; RBAC | workspace + folder perms; global vs project vaults |\n| External vault escape hatch | External Secrets (Vault…), global/project | EE delegates storage to an external secret manager |\n| Never echo key material; mask in logs | API returns only metadata | masks secret values in job stdout/stderr |\n\n**The single shared idea:** one encrypted store of TYPED credential records, referenced by a `$token`,\nresolved at job time with the runner's permissions. That is *already* piflow's `$VAR` + `SecretResolver`. The\ngap is purely the **store + typed schema + GUI input + DAG-surfacing** layers above the seam.\n\n---\n\n## 3. The piflow design — two homes, ONE resolver\n\n**Decision (2026-06-26): two homes, one resolver.**\n- **Provider keys → pi's native `~/.pi/agent/auth.json`** (or the provider env var). pi already owns these,\n `0600`, with subscription auto-refresh — do not duplicate them into a piflow store.\n- **Tool/MCP `$VAR`s → a new `~/.piflow/credentials.json`** — encrypted at rest, `0600`, master key\n `PIFLOW_ENCRYPTION_KEY` (env, else an auto-generated key file in `~/.piflow/`, mirroring n8n's first-boot\n key). Lives in the home global dir per the SDK data-boundary rule (`CLAUDE.md`: global secrets/index live in\n `~/.piflow/`, NEVER in `packages/`, the repo, or `gui/`).\n- **`SecretResolver` is the single seam** that reads from whichever home (or mints a scoped cloud token) and\n returns the value for a referenced `$VAR`. The SDK stays product-agnostic: it sees only the resolver; the\n store + master key live in the host/CLI/GUI layer.\n\n**Reference-only in the DAG.** node.json / the template carry `$VAR` references ONLY — already enforced\n(`checkMcpSecrets`). The DAG never stores a secret; the run-view never serializes one.\n\n**DAG surfaces the required credentials.** Derive the per-node required-cred SET statically:\n`referencedEnvVars(node.mcp)` ∪ the node's provider key (provider → env-var via pi's env-map, resolved by\n`model-routing.ts`). The GUI renders a pre-run preflight — each cred present / MISSING (probed through the\nresolver; the value is NEVER revealed) — an extension of today's dry-run plan, mirroring n8n's \"credential\nneeded\" gate. *(V1: the provider→env-var requirement may be declared explicitly per run; V2: parse the\nselected provider's `models.json` entry for its `$VAR` refs, the same deep-walk `referencedEnvVars` already\ndoes for `mcp.servers`.)*\n\n**GUI input → global store.** The GUI collects a typed credential (Windmill resource-type style: provider key,\nMCP header token, …), writes it encrypted to the right home (`auth.json` for a provider key, else\n`~/.piflow/credentials.json`), masked in the UI, never committed, never echoed back. Read happens only through\nthe resolver at run time.\n\n**Cloud injection = mint-scoped-token / forward-one-var, NEVER the raw long-lived key.** The cloud\n`SecretResolver` returns a short-lived scoped token (or forwards exactly the one provider env var into the VM\nexec env via the existing allowlist) — matching Windmill's ephemeral `WM_TOKEN` and pi's OpenShell\n\"keys stay outside the sandbox\" pattern. A cloud VM never inherits host `process.env` (`daytona.ts:261` merges\nonly `{...this.env,...opts.env}`; VM env `{PI_RUN}` `:518`), so the allowlist additions are the ONLY path in —\nand must be exactly the referenced set.\n\n**Deferred (not built now):** RBAC / credential sharing / project-vs-global vaults (single-user local needs\nnone — adopt n8n's scope model when multi-user/cloud lands); an external-vault backend (the resolver seam\nalready admits one — a host can plug Vault/AWS-SM later without a core change).\n\n---\n\n## 4. M1 — the cloud wiring this design pins\n\nThe pi gateway credential is NOT yet on the resolver seam for cloud (the `node-action-protocol.md` §4\nprovider-credential parity gap). M1 closes it, matching §3:\n1. **`--sandbox daytona` CLI branch** — `packages/cli/src/run.ts:355-394` handles `local`/`danger-full-access`/\n `inmemory` only; add the `daytona` case constructing `createDaytonaProvider({ image: process.env.DAYTONA_IMAGE,\n apiKey: process.env.DAYTONA_API_KEY })` (factory in `sandbox/daytona-sdk.ts`).\n2. **Thread a cloud `SecretResolver`** from the CLI into `runFromTemplate` (`RunOptions.secretResolver`\n `runner.ts:242`, already spread through `entry.ts`), defaulting to `process.env` read host-side.\n3. **Forward the provider gateway credential into the VM** — extend the cloud env additions so the selected\n provider's required env var(s) join the same allowlist `mcpEnvAdditions` forwards. **SHIPPED:** a declared\n `cloudSecrets: string[]` (`RunOptions`/`RunContext`) is resolved by `cloudCredEnvAdditions` (`runner.ts`)\n through the SAME `SecretResolver` and folded into the cloud `CreateOpts.env`; the CLI derives the var from\n `--provider` (or `--cloud-secret NAME`).\n4. **Custom-gateway config staging — SHIPPED (M1b).** A BUILT-IN provider reads `<PROVIDER>_API_KEY` directly,\n so M1's key forwarding alone suffices. A CUSTOM gateway (`nebius`/`mmgw`/`tokenrouter`) is defined ONLY in\n the host's `~/.pi/agent/models.json` (the official `docs/models.md` shape: `{providers:{<name>:{baseUrl,api,\n apiKey:\"$VAR\",models[]}}}`) which the M0 image bakes none of — so pi in the VM can't resolve `--provider`\n without it. The CLI's `parsePiProvider` (pure) scopes that file to the selected provider + extracts its\n `$VAR` cred refs (the authoritative allowlist); `DaytonaSandboxProvider.stageHome` writes the scoped\n `{providers:{[name]:entry}}` into the VM at `<home>/.pi/agent/models.json` BEFORE any node. The staged config\n carries `$VAR` REFERENCES only — never a resolved secret (the key still crosses via the cred allowlist).\n5. **Gates:** unit (no creds) — the `daytona` branch builds the provider, the cloud additions forward exactly\n the declared var (allowlist invariant), `parsePiProvider` scopes + extracts `$VAR`s, and `stageHome` writes\n the config into the VM (each must FAIL when its code is broken — mutation-checked); a credential-gated e2e\n (`skipIf`) running ONE node in a real VM asserting a real model call. **OPEN:** a usable `DAYTONA_IMAGE` ref\n must be supplied (M0 built via the declarative builder; promote to a named snapshot); the e2e is unrun.\n"}, "fa6bb3d6c8c8340dd1d31c31c20e24265537e149": {"t": 1785771158.317608, "n": 8877, "h": "995e549862c32241", "c": "# Credential Architecture — one `SecretResolver`, two homes, reference-by-`$VAR`\n\n> **Status:** DESIGN (2026-06-26). The mechanism (`$VAR` reference + `SecretResolver` seam + allowlist\n> forwarding) is ALREADY SHIPPED in core; this doc records the credential-ORGANIZATION layer on top of it\n> (store, typed schema, DAG-surfacing, GUI input, cloud injection) and pins the M1 cloud wiring. Provenance:\n> the pi-credential placement was read from pi's OWN docs (not assumed); the orchestrator patterns from n8n +\n> Windmill docs/source (Exa, 2026-06-26). Cross-ref: `docs/design/node-action-protocol.md` §4 (the secret\n> allowlist + provider-credential parity check), `docs/design/daytona-cloud-integration.md` (M0–M4),\n> `docs/design/cloud-tool-gateway-architecture.md` (gateway topology). Memory: `daytona-cloud-path`.\n\n---\n\n## 1. Where credentials live TODAY (verified, not assumed)\n\n**pi's provider credential** (read from pi `docs/providers.md` + `docs/containerization.md`, pi `0.80.2`).\nResolution priority: **`auth.json` → env var**.\n- **`~/.pi/agent/auth.json`** (mode `0600`), per-provider record: `{\"anthropic\":{\"type\":\"api_key\",\"key\":\"sk-...\"}}`.\n Subscriptions (Claude Pro/Max, Codex, Copilot) store auto-refreshed OAuth tokens here too. *(On this host the\n file is empty `{}` — auth is via env vars.)*\n- **Provider env var** — `ANTHROPIC_API_KEY`, `DEEPSEEK_API_KEY`, `NEBIUS_API_KEY` (present in this host's env),\n etc. Full map: pi's `packages/ai/src/env-api-keys.ts`.\n- **Custom gateways** (the `--provider tokenrouter/nebius/cp` piflow passes) are defined in **`~/.pi/agent/models.json`**\n with `apiKey:\"$PROXY_API_KEY\"`-style `$VAR` refs + a provider-scoped `env` block.\n- **Container (= the Daytona VM)** — pi's doc prescribes exactly two ways: `docker run -e ANTHROPIC_API_KEY`\n (forward the provider env var) **or** mount `~/.pi/agent` (the auth file). The \"keys never enter the sandbox,\n a gateway injects upstream\" variant is OpenShell/`inference.local`.\n\n**piflow's tool/MCP credential** — `node.json.mcp.servers` carries `$VAR`/`${VAR}` REFERENCES only (literal\nsecrets are rejected by `checkMcpSecrets`, `workflow/template/checks.ts:266`); the runner resolves the\nreferenced set through the `SecretResolver` and forwards ONLY that allowlist (`mcpEnvAdditions` `runner.ts:481`,\n`referencedEnvVars` `:444`, cloud-allowlist delete `:495-499`).\n\n**The existing seam (the load-bearing primitive — DO NOT reinvent).** `SecretResolver` (`types.ts:626`,\ndefault `process.env` `:632`) is the single host seam: core owns the `$VAR` vocabulary + the allowlist\ncontract; the host owns the binding. The resolver receives `{nodeId, isCloud}` (`:472,490`) so it can mint a\nper-node, cloud-only SCOPED token. `CLOUD_KINDS={daytona,e2b}` (`runner.ts:430`) drives the cloud allowlist.\n\n---\n\n## 2. How the orchestrators do it (n8n · Windmill — the convergent model)\n\n| Principle | n8n | Windmill |\n|---|---|---|\n| Creds are SEPARATE records, never inlined in the workflow | `credentials_entity`; workflow refs by id | Variables / Resources; flow refs by path |\n| Encrypted at rest, ONE instance master key | AES-256, `N8N_ENCRYPTION_KEY` (env/config, auto-gen on first boot); EE adds a 2-layer ROTATABLE data key | workspace-specific symmetric key |\n| Reference-by-token, resolved at exec with the CALLER's perms | (refs by id) | `$var:<path>` / `$res:<path>` / `$WM_*`; worker fetches via an EPHEMERAL token, fails if caller lacks read |\n| Typed credential schemas (drive the UI form + \"what's needed\") | credential types | Resource Types (JSON Schema per integration) |\n| Scoping / sharing | personal-by-default; share-to-project; `credential:shareGlobally` scope; RBAC | workspace + folder perms; global vs project vaults |\n| External vault escape hatch | External Secrets (Vault…), global/project | EE delegates storage to an external secret manager |\n| Never echo key material; mask in logs | API returns only metadata | masks secret values in job stdout/stderr |\n\n**The single shared idea:** one encrypted store of TYPED credential records, referenced by a `$token`,\nresolved at job time with the runner's permissions. That is *already* piflow's `$VAR` + `SecretResolver`. The\ngap is purely the **store + typed schema + GUI input + DAG-surfacing** layers above the seam.\n\n---\n\n## 3. The piflow design — two homes, ONE resolver\n\n**Decision (2026-06-26): two homes, one resolver.**\n- **Provider keys → pi's native `~/.pi/agent/auth.json`** (or the provider env var). pi already owns these,\n `0600`, with subscription auto-refresh — do not duplicate them into a piflow store.\n- **Tool/MCP `$VAR`s → a new `~/.piflow/credentials.json`** — encrypted at rest, `0600`, master key\n `PIFLOW_ENCRYPTION_KEY` (env, else an auto-generated key file in `~/.piflow/`, mirroring n8n's first-boot\n key). Lives in the home global dir per the SDK data-boundary rule (`CLAUDE.md`: global secrets/index live in\n `~/.piflow/`, NEVER in `packages/`, the repo, or `gui/`).\n- **`SecretResolver` is the single seam** that reads from whichever home (or mints a scoped cloud token) and\n returns the value for a referenced `$VAR`. The SDK stays product-agnostic: it sees only the resolver; the\n store + master key live in the host/CLI/GUI layer.\n\n**Reference-only in the DAG.** node.json / the template carry `$VAR` references ONLY — already enforced\n(`checkMcpSecrets`). The DAG never stores a secret; the run-view never serializes one.\n\n**DAG surfaces the required credentials.** Derive the per-node required-cred SET statically:\n`referencedEnvVars(node.mcp)` ∪ the node's provider key (provider → env-var via pi's env-map, resolved by\n`model-routing.ts`). The GUI renders a pre-run preflight — each cred present / MISSING (probed through the\nresolver; the value is NEVER revealed) — an extension of today's dry-run plan, mirroring n8n's \"credential\nneeded\" gate. *(V1: the provider→env-var requirement may be declared explicitly per run; V2: parse the\nselected provider's `models.json` entry for its `$VAR` refs, the same deep-walk `referencedEnvVars` already\ndoes for `mcp.servers`.)*\n\n**GUI input → global store.** The GUI collects a typed credential (Windmill resource-type style: provider key,\nMCP header token, …), writes it encrypted to the right home (`auth.json` for a provider key, else\n`~/.piflow/credentials.json`), masked in the UI, never committed, never echoed back. Read happens only through\nthe resolver at run time.\n\n**Cloud injection = mint-scoped-token / forward-one-var, NEVER the raw long-lived key.** The cloud\n`SecretResolver` returns a short-lived scoped token (or forwards exactly the one provider env var into the VM\nexec env via the existing allowlist) — matching Windmill's ephemeral `WM_TOKEN` and pi's OpenShell\n\"keys stay outside the sandbox\" pattern. A cloud VM never inherits host `process.env` (`daytona.ts:261` merges\nonly `{...this.env,...opts.env}`; VM env `{PI_RUN}` `:518`), so the allowlist additions are the ONLY path in —\nand must be exactly the referenced set.\n\n**Deferred (not built now):** RBAC / credential sharing / project-vs-global vaults (single-user local needs\nnone — adopt n8n's scope model when multi-user/cloud lands); an external-vault backend (the resolver seam\nalready admits one — a host can plug Vault/AWS-SM later without a core change).\n\n---\n\n## 4. M1 — the cloud wiring this design pins\n\nThe pi gateway credential is NOT yet on the resolver seam for cloud (the `node-action-protocol.md` §4\nprovider-credential parity gap). M1 closes it, matching §3:\n1. **`--sandbox daytona` CLI branch** — `packages/cli/src/run.ts:355-394` handles `local`/`danger-full-access`/\n `inmemory` only; add the `daytona` case constructing `createDaytonaProvider({ image: process.env.DAYTONA_IMAGE,\n apiKey: process.env.DAYTONA_API_KEY })` (factory in `sandbox/daytona-sdk.ts`).\n2. **Thread a cloud `SecretResolver`** from the CLI into `runFromTemplate` (`RunOptions.secretResolver`\n `runner.ts:242`, already spread through `entry.ts`), defaulting to `process.env` read host-side.\n3. **Forward the provider gateway credential into the VM** — extend the cloud env additions so the selected\n provider's required env var(s) join the same allowlist `mcpEnvAdditions` forwards (V1: a declared\n provider-cred var name; the resolver mints/reads it host-side). Determine whether the VM's pi also needs the\n matching `models.json` provider entry staged (custom gateways are defined there) — if a full in-VM\n model-config staging proves to need its own design, HALT and report rather than over-build.\n4. **Gates:** a unit test (no creds) proving the `daytona` branch builds the provider AND the cloud additions\n include the forwarded provider var (must FAIL if either is absent); a credential-gated e2e\n (`skipIf(!DAYTONA_API_KEY)`) running ONE node in a real VM that asserts the node produced its artifact via a\n real model call.\n"}}}} |