Dcp Wrap

dcp包装

TypeScript实现 数据成本协议（DCP） --LLM的紧凑结构数据编码，以及高通量AI驱动数据流的完整流水线控制层。

一个包裹里有两样东西

1.DCP编码器 --将JSON转换为位置数组格式。向LLM提供结构化数据时，令牌减少40-60%。

2.管道控制 --一个完整的流式管道，LLM永远不会进入数据路径。架构验证($V)，路由($R)，统计($ST)，以及Brain AI决策——所有这些都在不暂停流的情况下应用。

看 dcp-docs.pages.dev 对于完整的协议设计，以及 Minecraft管道演示 对于一个具有测量延迟数的端到端工作示例。

______________________________________________________________________

DCP编码

而不是发送 {"endpoint":"/v1/users","method":"GET","status":200} 对于每条记录，DCP声明一次模式并按位置写入值：

["$S","api-response:v1","endpoint","method","status","latency_ms"]
["/v1/users","GET",200,42]
["/v1/orders","POST",201,187]

代币减少40-60%。零精度成本。看 基准.

安装

npm install dcp-wrap

命令行界面

从JSON推断模式

cat api-response.json | npx dcp-wrap init api-response

输出：

Schema: api-response:v1
Fields: 4

  endpoint: string (source: endpoint, unique: 4/4)
  method: string (source: method, unique: 2/4)  [enum(2)]
  status: number (source: status, unique: 2/4)
  latency_ms: number (source: latency_ms, unique: 4/4)

Saved: dcp-schemas/api-response.v1.json
Saved: dcp-schemas/api-response.v1.mapping.json

将JSON编码为DCP

cat data.json | npx dcp-wrap encode --schema dcp-schemas/api-response.v1.json

输出：

["$S","api-response:v1","endpoint","method","status","latency_ms"]
["/v1/users","GET",200,42]
["/v1/orders","POST",201,187]
["/v1/auth","POST",200,95]

检查架构

npx dcp-wrap inspect dcp-schemas/api-response.v1.json

程序化API

快速——一个功能，无文件

import { dcpEncode } from "dcp-wrap";

const dcp = dcpEncode(results, {
  id: "engram-recall:v1",
  fields: ["id", "relevance", "summary", "tags", "hitCount", "weight", "status"],
});
// ["$S","engram-recall:v1","id","relevance","summary","tags","hitCount","weight","status"]
// ["abc123",0.95,"port conflict fix","docker,gotcha",12,3.2,"fixed"]

数组字段用逗号自动连接。使用 transform 对于自定义处理：

const dcp = dcpEncode(records, schema, {
  transform: { relevance: (v) => +(v as number).toFixed(3) },
});

完整--模式生成+编码

import { SchemaGenerator, DcpEncoder, DcpSchema, FieldMapping } from "dcp-wrap";

const gen = new SchemaGenerator();
const draft = gen.fromSamples(jsonRecords, { domain: "github-pr" });

const schema = new DcpSchema(draft.schema);
const mapping = new FieldMapping(draft.mapping);
const encoder = new DcpEncoder(schema, mapping);

const batch = encoder.encode(jsonRecords);
console.log(DcpEncoder.toString(batch));

嵌套JSON

嵌套对象通过点符号自动展平：

{"id": "pr-1", "metadata": {"author": "alice", "state": "open"}}

生成器映射 metadata.author → author, metadata.state → state.

嵌套DCP（对象数组）

对象数组使用以下方式编码 $N 参考文献:

["$S","user:v1","id","name","teams"]
["u001","Alice",["$N","user.teams:v1",["t01","Infra","lead"],["t02","Security","member"]]]
["u002","Bob",["$N","user.teams:v1",["t03","Frontend","member"]]]
["u003","Charlie",["$N","user.teams:v1"]]

______________________________________________________________________

流水线控制

dcp-wrap包括一个完整的流管道控制层。核心思想： AI从外部观察并重新配置管道——它从不进入数据路径。

IngestionBus
  → Preprocessor     ← structural validation → Quarantine
  → Gate ($V)        ← schema constraint validation → pass/fail
  → StCollector      ← rolling statistics → $ST-v (pass_rate, throughput)
  → Bot              ← anomaly detection → $I packets
  → Brain (2s tick)  ← BrainAdapter.evaluate() → BrainDecision
  → PostBox          ← routing_update / throttle / quarantine_approve
  → PipelineControl  ← applies to next row, zero downtime

关键属性

• 数据路径延迟：p50=45μs（摄入→ 闸门），已测量
• 控制路径：大脑每2秒发出一次滴答声——观察 $ST 指标、问题决策
• 延迟切换：路由和验证更改应用于下一行，无暂停，无重启（p50=63μs）
• 自我修复：当异常清除时，大脑会恢复之前的路由和 $V 自动地

脑AI接口

import { Brain, RuleBasedBrain, ClaudeBrain } from "dcp-wrap";
import type { BrainAdapter, BrainInput, BrainDecision } from "dcp-wrap";

// Rule-based (no LLM)
class MyBrain implements BrainAdapter {
  async evaluate(input: BrainInput): Promise
 {
    if (input.packets.some(p => p.severity === "high")) {
      return { rerouteSchema: { schemaId: "events:v1", toPipelineId: "audit-pipeline" } };
    }
    return {};
  }
}

// Claude (Haiku) — drop-in replacement, same interface
const brain = new ClaudeBrain({ model: "claude-haiku-4-5-20251001" });

在基于规则和LLM之间切换 BRAIN_MODE=claude.管路接线相同。

阴影层

有关完整的协议规范和设计原理，请参阅 dcp-docs.pages.dev/dcp/pipeline.

工作演示

这 Minecraft管道演示 显示了所有三个层协同工作：异常检测、大脑重新路由、， $V 动态更新、隔离审批——在三种情况下进行验证，数据路径不需要LLM。

______________________________________________________________________

处理杂乱的数据

需要时覆盖：

const draft = gen.fromSamples(samples, {
  domain: "some-api",
  maxDepth: 2,
  maxFields: 10,
  minPresence: 0.5,
});

设计

• 零运行时依赖关系
• 模式生成遵循DCP字段排序约定（标识符→ 分类器→ 数值→ text)
• 支持JSON数组和NDJSON作为输入
• 模式+映射文件是纯JSON——查看、版本和编辑它们

许可证

阿帕奇-2.0