- name
- langfuse-continuous-optimizer
- description
- Continuous LangFuse-driven optimization loop for OpenClaw/OpenRouter model routing and prompt usage controls with persistent local memory. Use when Codex needs to ingest LangFuse observations and evaluator scores, generate task-level routing policy JSON, and run scheduled safe promotion cycles that tune cost-quality-latency tradeoffs automatically.
- metadata
- {"openclaw":{"emoji":"🧠","requires":{"env":["LANGFUSE_PUBLIC_KEY","LANGFUSE_SECRET_KEY"],"anyBins":["python","python3"]},"primaryEnv":"LANGFUSE_SECRET_KEY"}}
Langfuse Continuous Optimizer
Overview
Run an automated observe -> evaluate -> adapt loop backed by LangFuse data.
This skill is independent and self-contained: it includes both policy builder and continuous optimizer scripts.
Quick Start
# Single optimization cycle (LangFuse API -> staged policy -> promoted live policy if gate passes)
python scripts/langfuse_openclaw_optimizer.py run-once \
--langfuse-host https://us.cloud.langfuse.com \
--window-hours 24 \
--out-dir ~/.openclaw/optimizer \
--live-policy-path ~/.openclaw/llm_routing_policy.json \
--promote-live-policy \
--write-memory \
--save-config
# Continuous daemon
python scripts/langfuse_openclaw_optimizer.py daemon \
--interval-min 30 \
--save-config
# Toggle settings later (persisted)
python scripts/langfuse_openclaw_optimizer.py configure --disable-promote-live-policy --show
python scripts/langfuse_openclaw_optimizer.py configure --promote-live-policy --write-memory --showCredentials:
LANGFUSE_PUBLIC_KEYLANGFUSE_SECRET_KEY
Workflow
- Pull LangFuse observations and scores from the configured time window.
- Normalize telemetry and build staged routing policy artifacts.
- Compare staged policy against current live policy with switch guardrails.
- Promote only when gain and quality constraints are met and promotion is explicitly enabled.
- Persist cycle memory to reduce policy churn and enable rollback reasoning.
Safety
- Network egress: calls LangFuse Public API.
- Local writes: writes raw snapshots, staged artifacts, and optional memory state under
--out-dir. - Live policy overwrite is opt-in via
--promote-live-policy. - Without
--promote-live-policy, cycles are non-destructive (stage/evaluate only). - Save persisted defaults with
--save-config; edit/toggle withconfigure.
Runtime Integration
Use the generated live policy in OpenClaw/LLM runtime via:
--llm-routing-policy-file ~/.openclaw/llm_routing_policy.json
--llm-policy-reload-sec 300Tag requests with stable task keys (planning, tool-selection, retrieval, summarization, generation, etc.) so per-task routing converges quickly.
Resources (optional)
scripts/
scripts/langfuse_openclaw_optimizer.py: API pull + cycle orchestration + promotion gating + persistent memory.scripts/closed_loop_prompt_ops.py: normalization and policy generation engine used by the optimizer.
references/
references/data-contracts.md: input/output schemas and artifacts.references/closed-loop-playbook.md: guardrails, mutation policy, memory strategy, runtime integration notes.