AI Model Experiments

liveAIDeveloper Tools

Model Lab — run the same prompts across many AI models simultaneously and compare outputs, latency, and cost, with an AI-written summary of the differences. Prepaid (provider cost × 1.5, $0.10 minimum), topped up via x402 USDC.

8tools
0msauth
free tier50 calls/day

Tools

experiment_models

List AI models available for experiments (about 300 across Anthropic, OpenAI, Google, Meta, Mistral, DeepSeek, Qwen and more), with context window and OUR per-token prices (provider cost × 1.5 — what

No parameters required.

▶Try it
experiment_estimate

Dry-run cost estimate for an experiment BEFORE creating it — cell count and estimated billed cost range (at our 1.5× pricing) for prompts × models × reps. Free to call, no side effects, does not need

No parameters required.

▶Try it
experiment_create

Create and start an experiment: run each prompt against each model (× reps), collecting output, tokens, latency, and billed cost per cell. PREPAID: requires Pipeworx credit balance ≥ max_spend_usd (to

No parameters required.

▶Try it
experiment_status

Progress of an experiment: cell counts by state (pending/running/ok/error/skipped), spend so far vs cap, and whether it is complete. Poll this after experiment_create (every few seconds). Example: exp

No parameters required.

▶Try it
experiment_results

Results of an experiment: per-model aggregates (mean latency, tokens, total billed cost, error rate), per-cell outputs, and the AI-written summary comparing how the models differed (if enabled). Use i

No parameters required.

▶Try it
experiment_list

List your experiments, newest first, with status and spend. Example: experiment_list({ limit: 10 })

No parameters required.

▶Try it
experiment_cancel

Cancel a running experiment: pending cells are skipped (not billed); in-flight cells finish and bill. Example: experiment_cancel({ experiment_id: "..." })

No parameters required.

▶Try it
experiment_topup

How to add prepaid credits for experiments (and your current balance). Payment is x402 — USDC on Base, paid in-band by any wallet-equipped agent: POST https://gateway.pipeworx.io/credits/topup?amount_

No parameters required.

▶Try it

Test with curl

The gateway speaks JSON-RPC 2.0 over HTTP POST. You can test any pack directly from the terminal.

List available tools
bash
curl -X POST https://gateway.pipeworx.io/ai-model-experiments/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'
Call a tool
bash
curl -X POST https://gateway.pipeworx.io/ai-model-experiments/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"experiment_models","arguments":{}}}'

Use with the SDK

Install @pipeworx/sdk to call tools from any TypeScript/Node project.

TypeScript
import { Pipeworx } from '@pipeworx/sdk';
const px = new Pipeworx();
const result = await px.call("experiment_models", {});
ask_pipeworx
// Or ask in plain English:
const answer = await px.ask("model lab — run the same prompts across many ai models simultaneously and compare outputs, latency, and cost, with an ai-written summary of the differences");