- name
- tool-calling
- description
- Deep workflow for LLM tool/function calling—schema design, validation, permissions, errors, idempotency, testing, and safe orchestration with agents. Use when wiring models to APIs, databases, or internal tools.
Tool Calling (Deep Workflow)
Tool calling is contract design between a probabilistic planner (the model) and deterministic systems. Failures are usually schema, permissions, or ambiguity—not the LLM “being dumb.”
When to Offer This Workflow
Trigger conditions:
- Designing OpenAI/Anthropic-style functions, MCP tools, or internal JSON tool protocols
- Debugging wrong arguments, hallucinated parameters, or unsafe side effects
- Building agents with many tools—selection and routing problems
Initial offer:
Use six stages: (1) define tool surface, (2) schema & validation, (3) authz & safety, (4) execution semantics, (5) errors & observability, (6) evaluation & regression. Confirm side-effect class (read-only vs write).
Stage 1: Define Tool Surface
Goal: Minimize tools; maximize clarity per tool.
Principles
- One action per tool when possible—avoid mega-tools with mode flags unless necessary
- Names descriptive:
search_ordersnotdo_stuff - Prefer idempotent operations where writes exist; separate read vs write clearly
Anti-patterns
- Exposing raw SQL or shell to the model
- Too many overlapping tools → routing errors
Exit condition: Tool list with purpose, inputs, outputs, side effects table.
Stage 2: Schema & Validation
Goal: Arguments are typed, constrained, and machine-validated before execution.
Practices
- JSON Schema: enums, min/max, patterns, required fields
- Normalize dates, IDs, currencies server-side—never trust model formatting alone
- Default behaviors explicit in description + schema
Descriptions
- Tool and parameter docstrings seen by model—precise language; examples of valid args
Exit condition: Validator rejects invalid args with actionable errors back to model or orchestrator.
Stage 3: Authorization & Safety
Goal: Every tool call runs as some principal with least privilege.
Patterns
- User-scoped credentials carried from session; tool implementation re-checks ownership (e.g., order_id belongs to user)
- Admin tools behind explicit allowlists and human approval when needed
- Rate limits per user + global circuit breakers
Data exfiltration
- Tools that read sensitive data need output filtering and logging policies
Exit condition: Threat brief: “What if model is tricked into calling tool X?” answered.
Stage 4: Execution Semantics
Goal: Clear transactionality, retries, and idempotency.
Design
- Idempotency keys for writes; dedupe window
- Timeouts and cancellation propagation
- Ordering: parallel safe vs must be serial
Long operations
- Async jobs with poll tool vs blocking calls—prefer non-blocking for UX and cost
Exit condition: Semantics documented for retry behavior (at-least-once delivery common).
Stage 5: Errors & Observability
Goal: Model (or orchestrator) can recover from failures without leaking internals.
Error messages
- Structured error codes:
ORDER_NOT_FOUND,PERMISSION_DENIED - Hints for model on how to fix—without stack traces to end users
Observability
- Trace IDs across tool calls; audit log for write tools (who/when/args hash)
Exit condition: Dashboards/alerts on tool error rate, latency, denials.
Stage 6: Evaluation & Regression
Goal: Tool changes are tested like APIs.
Harness
- Golden conversations with expected tool calls (args normalized)
- Adversarial prompts attempting privilege escalation
- Version tools; deprecate with compatibility window
Exit condition: CI or manual eval suite before deploying new tools/schemas.
Final Review Checklist
- [ ] Minimal orthogonal tool set
- [ ] Strict schema validation on server
- [ ] AuthZ enforced per call; sensitive reads controlled
- [ ] Idempotency and timeouts defined for writes
- [ ] Structured errors + observability + eval harness
Tips for Effective Guidance
- Treat tool descriptions as API docs the model reads—iterate wording like UX copy.
- Recommend two-step patterns for dangerous ops: propose → confirm (human or policy).
- When using MCP, same discipline—server must validate everything.
Handling Deviations
- Read-only RAG: fewer semantic risks—still validate query args and injection into search backends.
- Local tools (filesystem): sandbox, path allowlists, size limits.