Widget runner — insightworker --ui and declarative app forms

For apps that have a widget.json, InsightWorker can render a real form-and-results UI instead of dumping output to the terminal. Two ways to launch it:

# Standalone: pop a single app open in a browser.
insightworker /app run <name> --ui

# Inside web mode: click the app in the sidebar.
insightworker --web

Both serve the same React-based renderer over an internal HTTP server. SSE drives the live panels; cancellation works the same way as in chat.

When to use this

You have an app with structured inputs (a folder path, a few dropdowns, a date range) and structured outputs (a JSON summary, a dataframe, an editable email draft, generated files). Rather than asking the user to remember the prompt shape and parsing free-form text back, you describe the form and the output panels declaratively, and the renderer takes care of the rest.

Good fits:

• Underwriter apps: pick a submission folder, run intake → comparison → loss-run → draft email, edit and send.
• Pipeline triage: paste a DAG ID and a time window, get back a structured report with quick-action buttons.
• Document operations: drop in a PDF, watch the OCR/extraction panels light up, download the artifacts.

Bad fits:

• Open-ended Q&A → stay in chat.
• One-line shell-style commands → stay in the REPL.

Widget schema (v1.0)

A widget.json sits in the app folder:

<workspace>/.insightworker/workflows/<name>/
├── widget.json
├── workflow.yaml     (optional — the spec the agent executes)
├── tasks.md          (optional — free-form spec)
└── dictionaries/     (optional — JSON enums referenced by select fields)

Minimal example:

{
  "version": "1.0",
  "title": "Process broker submission",
  "description": "Pick the submission folder and parameters. Agent runs intake → policy comparison → loss-run → drafts the underwriter email.",
  "inputs": [
    {
      "key": "submission_folder",
      "type": "folder",
      "label": "Submission folder",
      "placeholder": "/path/to/submission/folder",
      "required": true
    },
    {
      "key": "carrier",
      "type": "select",
      "label": "Target carrier",
      "options": { "type": "dictionary", "source": "carriers" }
    },
    {
      "key": "effective_date",
      "type": "date",
      "label": "Effective date"
    }
  ],
  "submit": {
    "label": "Run intake",
    "steps": [
      { "id": "extract_policy",     "name": "Extract policy" },
      { "id": "analyze_loss_run",   "name": "Analyze loss run" },
      { "id": "risk_briefing",      "name": "Build risk briefing" },
      { "id": "draft_email",        "name": "Draft underwriter email" },
      { "id": "artifacts",          "name": "Generate artifacts" }
    ]
  },
  "outputs": [
    { "type": "json_summary",  "key": "extract_policy",   "title": "Policy summary" },
    { "type": "dataframe",     "key": "analyze_loss_run", "title": "Loss-run line items", "page_size": 10 },
    { "type": "markdown",      "key": "risk_briefing",    "title": "Risk briefing" },
    { "type": "editable_text", "key": "draft_email",      "title": "Draft underwriter email",
      "format": "email",
      "actions": ["copy", "download", "send_email"] },
    { "type": "files",         "key": "artifacts",        "title": "Generated artifacts" }
  ]
}

Field types

Options can be a literal list [{value, label}, …] or pulled from a dictionary file via {type: "dictionary", source: "<name>"} (reads <app>/dictionaries/<name>.json).

Most fields support depends_on for cascading lookups — e.g. show "Plan" only after "Carrier" is chosen.

Output panels

Each panel binds to a step id (or key); when the SSE stream emits step_completed for that id, the panel populates.

How the run actually happens

1. The renderer POSTs { widget, inputs } to /api/run.
2. The server (in-process inside the CLI) hands the widget + inputs to CliWorkflowRunner, which iterates the declared steps.
3. For each step, the runner picks one of two execution paths based on the matching workflow.yaml entry:
   • action: shell (deterministic) — runs the command: directly via child_process, no LLM in the loop. Files dropped into {{runDir}} become the step's output for files panels; stdout flows to markdown / json_summary / dataframe panels.
   • prompt: (LLM-driven) — calls runAgentLoop with a step-specific prompt that includes prior step outputs as materialized files (real JSON/CSV/MD/EML on disk under <app>/runs/<run_id>/).
4. SSE events stream back: step_started, step_completed, step_error, completed.
5. Output panels render incrementally as their step completes.

Artifacts in the files panel are real files on disk — not mocks. They survive after the run; you can grep them, re-open them, attach them to a ticket.

Shell-action steps — for I/O and deterministic conversion

When a step's job is a script (file conversion, ETL, compilation, image processing), use action: shell in workflow.yaml instead of an LLM prompt. The runner execs the command, captures files dropped into the per-run directory, and skips the model entirely.

# workflow.yaml — Word → PDF converter
steps:
  - id: convert            # MUST match a widget.outputs[].key
    name: Convert DOCX to PDF
    action: shell
    command: "soffice --headless --convert-to pdf {{input.docx_file}} --outdir {{runDir}}"
    timeout_sec: 120

Template variables in command::

Interpolated values are auto single-quoted to neutralise shell metacharacters in user input. A non-zero exit code fails the step with the last 10 lines of stderr in the SSE event.

You can mix step types in the same workflow — e.g. an LLM prompt: step does the planning, then a deterministic action: shell step does the actual file work, then another LLM step writes a human-readable summary.

Running on a remote box

Same shape as web mode — the renderer is served by the same HTTP server on the same port. Tunnel with ssh -L 8765:127.0.0.1:8765 user@host and browse to localhost:8765. The folder picker shows the server's filesystem; uploads go to the server. That's almost always what you want.

Cancellation

Once a run is in flight, the Run button becomes Cancel. Cancellation:

1. Sends POST /api/cancel with the active run_id.
2. The server aborts the AbortController for that run.
3. The provider SDK stops streaming (Anthropic/OpenAI/Bedrock/Gemini/Vertex all honor signal).
4. The runner emits step_error: cancelled and failed: cancelled.
5. The UI flips back to idle. Any artifacts produced before the cancel are kept.

Closing the browser tab counts as a cancel — the server detects the dropped TCP connection and aborts.

Authoring a widget

Two paths:

1. By hand — write widget.json next to your app's tasks.md / workflow.yaml. The schema is small enough to learn in 10 minutes; copy the example above and tweak.
2. Have the agent draft one — in chat, ask: Create a widget.json for my "<name>" app with inputs <X, Y, Z> and outputs <P, Q>. Then open it in --ui to iterate.

The renderer treats widget.json as the contract. If a step emits an output key that no panel declares, it's still saved to disk under runs/<run_id>/; it just doesn't render. That's intentional — it lets you ship a polished form-and-output view while the underlying agent steps stay rich.

See also

• Web mode — the full chat + apps browser UI
• Apps & scheduling — overview — how YAML apps compose
• Skills — when to bundle a multi-step procedure as a skill instead

Source: docs/interactive-ui/widget-runner.md in the public repo. Open a PR with corrections.