InsightWorker Logo
  • contact@verticalserve.com
Docs / Sharing & InsightStudio / Join a Studio worker farm

Join an InsightStudio worker farm — insightworker --worker

Your InsightWorker box can be more than a personal CLI. With insightworker --worker, it becomes a worker daemon in an InsightStudio coordinator's farm: it polls Studio for queued runs, executes them on your local agent, and streams events back so business users see live progress in their browser.

This is the deployment model that lets a small team of authors publish apps once and have multiple workers execute them in parallel — on whichever boxes you've installed credentials, GPUs, or skills the apps require.

How it works

  1. You launch insightworker --worker --studio <url> --token <bearer>.
  2. The daemon registers with Studio: a stable worker id, hostname, installed skill names, capability vector (credentials present, GPU availability).
  3. Two loops run in parallel: a heartbeat every 10s to stay alive in the registry, and a poll every ~2s to claim queued runs.
  4. On claim, the daemon pulls the app bundle from S3, runs it through the standard CliWorkflowRunner, and POSTs each step event back to Studio's /runs/<id>/events/ingest endpoint.
  5. Studio's SSE relay forwards those events to the user's browser in near-realtime.

Launch the daemon

# Minimum
insightworker --worker --studio https://studio.your-org.com --token YOUR_BEARER_TOKEN

# Common knobs
insightworker --worker \
  --studio https://studio.your-org.com \
  --token YOUR_BEARER_TOKEN \
  --hostname runner-1.prod \
  --role runner \
  --creds sharepoint,salesforce \
  --poll-interval-ms 2000 \
  --heartbeat-interval-ms 10000

Flags

FlagRequiredDefaultWhat it does
--studio <url>Yes—Base URL of the Studio coordinator's API.
--token <bearer>Yes—Bearer token Studio accepts. Can also come from INSIGHTWORKER_STUDIO_TOKEN.
--worker-id <id>NoAuto-generated, persisted to ~/.insightworker/worker_idOverride the worker identity. Re-registers under the same id across restarts so Studio's history stays joined.
--hostname <name>Noos.hostname()Display name in Studio's worker list.
--role <runner|author|both>NorunnerWhat the worker is willing to do. Only runner and both claim queued runs.
--creds c1,c2,...No(empty)Comma-separated credential names this box has provisioned (e.g. sharepoint). Used by Studio's capability matcher to route jobs to qualified workers.
--gpuNooffAdvertise GPU availability. Apps that require GPU will prefer GPU-equipped workers.
--poll-interval-ms NNo2000How often (ms) the daemon polls for queued runs while idle.
--heartbeat-interval-ms NNo10000How often (ms) the daemon heartbeats. Studio's lazy reaper marks workers offline after 30s of silence.

What gets reported to Studio

On register and every heartbeat, the daemon sends:

  • id — stable across restarts (persisted to ~/.insightworker/worker_id)
  • hostname, role
  • installed_skills — auto-discovered from ~/.insightworker/skills/; sent as a list of skill names. Studio's matcher compares this against each app's requires.skills.
  • capabilities — { creds: [...], gpu: bool }
  • status — idle | busy | draining | offline

What stays local

Your secrets (~/.insightworker/.env, AWS profile, OAuth tokens) never leave the box. Studio only sees what you advertise. The daemon makes outbound HTTPS calls; it never opens an inbound port.

Graceful shutdown

Send SIGINT or SIGTERM — the daemon flips its loop guard, finishes any in-flight ingest, and calls DELETE /workers/<id> so Studio removes the row from its list immediately.

# Ctrl-C in foreground
^C
[worker] received SIGINT, draining…
[worker] unregistered (HTTP 204)

Cancellation

When a user clicks Cancel in Studio, the daemon learns about it on its very next ingest ack (within one step boundary) and aborts the running agent. No more wasted Bedrock tokens after a cancel.

Observability

Studio's /admin/workers page (or GET /api/v1/workers) shows the live fleet:

id                       status  last_heartbeat        skills  creds
iw-f550f108df50608b3313  idle    2s ago                15      sharepoint,salesforce
iw-a3b2c1...             busy    1s ago                12      sharepoint
runner-build-host        offline 1240s ago             8       (none)

Background daemonization

The CLI runs in the foreground. To leave it up across reboots use your platform's standard tooling — systemd on Linux, launchd on macOS. Example unit file:

[Unit]
Description=InsightWorker farm daemon
After=network.target

[Service]
Type=simple
User=insightworker
EnvironmentFile=/etc/insightworker.env
ExecStart=/usr/local/bin/insightworker --worker --studio ${IW_STUDIO_URL} --token ${IW_STUDIO_TOKEN}
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Related: Publish apps to S3 · InsightStudio overview