01 Agentic Sre Platform

🤖 SRE代理平台

    

生产级多智能体SRE系统 该工具使用LangGraph、MCP工具服务器、RAG支持的runbook检索和A2A代理协调自动诊断和补救Kubernetes事件，部署在AWS EKS和Azure AKS上。

作者 Balakrishna V |首席架构师/代理人工智能工程师| CareFirst BCBS github： @巴拉克里希纳·瓦卢里·巴拉

______________________________________________________________________

📐 建筑

BigPanda / Datadog Alert
         │
         ▼
  ┌─────────────┐
  │  FastAPI     │  POST /alert
  │  (port 8000) │
  └──────┬──────┘
         │
         ▼
  ┌─────────────────────────────────────────────────────┐
  │          LangGraph Orchestrator Agent                │
  │                                                      │
  │  receive_alert → rag_lookup → classify_alert         │
  │         │                          │                 │
  │         ▼                          ▼                 │
  │   ChromaDB RAG           [CrashLoop|CPU|Metrics|DB]  │
  │   (runbook lookup)                 │                 │
  │                                    ▼                 │
  │              ┌─────────────────────────────────┐    │
  │              │    Agent Router (A2A)            │    │
  │              └──┬──────────┬──────────┬─────────┘    │
  │                 │          │          │              │
  │                 ▼          ▼          ▼              │
  │         ┌──────────┐ ┌─────────┐ ┌──────────┐      │
  │         │K8s Agent │ │Metrics  │ │Incident  │      │
  │         │          │ │Agent    │ │Agent     │      │
  │         └────┬─────┘ └────┬────┘ └────┬─────┘      │
  │              │             │           │             │
  └──────────────┼─────────────┼───────────┼─────────────┘
                 │             │           │
        ┌────────▼──┐  ┌───────▼───┐  ┌───▼──────────┐
        │  K8s MCP  │  │  AWS MCP  │  │ ServiceNow   │
        │  Server   │  │  Azure    │  │ MCP Server   │
        │ (kubectl) │  │  Monitor  │  │              │
        └───────────┘  └───────────┘  └──────────────┘

美人鱼流程图：

flowchart TD
    A[BigPanda Alert] --> B[FastAPI /alert]
    B --> C[LangGraph Orchestrator]
    C --> D{RAG Lookup}
    D --> E[ChromaDB Runbooks]
    C --> F{Classify Alert}
    F -->|CrashLoop/OOM| G[Kubernetes Agent]
    F -->|High CPU/Memory| H[Metrics Agent]
    F -->|P1/P2 Incident| I[Incident Agent]
    G --> J[K8s MCP Server]
    H --> K[AWS/Azure MCP Server]
    I --> L[ServiceNow MCP Server]
    J --> M[kubectl: restart/scale]
    K --> N[CloudWatch/Azure Monitor]
    L --> O[ServiceNow Ticket Created]
    C --> P[Summarize & Notify]

______________________________________________________________________

✨ 特性

关键指标（来自生产CareFirst BCBS）：

• ⬇️ MTTD：10-15分钟→ 2-5分钟
• ⬇️ MTTR:45-90分钟→ 15-30分钟
• 🤖 自动修复： 40%的P3/P4事件

______________________________________________________________________

🚀 快速开始

先决条件

• Python 3.11+
• Docker+Docker组合
• kubectl（用于K8s功能）
• Terraform 1.5+（用于云部署）

本地开发

# 1. Clone the repo
git clone https://github.com/balakrishna-valluri-bala/agentic-sre-platform.git
cd agentic-sre-platform

# 2. Set up environment
cp .env.example .env
# Edit .env — add your OPENAI_API_KEY at minimum

# 3. Start all services
docker-compose up -d

# 4. Ingest runbooks into ChromaDB
python rag/ingest.py

# 5. Test the API
curl -X POST http://localhost:8000/alert \
  -H "Content-Type: application/json" \
  -d '{
    "alert_type": "CrashLoopBackOff",
    "severity": "high",
    "namespace": "production",
    "pod_name": "api-server-7d9b8f-xyz",
    "message": "Pod restarted 8 times in last 10 minutes",
    "source": "bigpanda"
  }'

预期响应

{
  "alert_id": "alert-abc123",
  "status": "remediated",
  "assigned_agent": "kubernetes-agent",
  "remediation_steps": [
    "Retrieved runbook: crashloopbackoff.md",
    "Fetched pod logs: OOMKilled — memory limit exceeded",
    "Restarted pod: api-server-7d9b8f-xyz",
    "Updated memory limit from 512Mi to 1Gi"
  ],
  "incident_number": null,
  "summary": "CrashLoopBackOff resolved via pod restart. Root cause: OOMKill due to memory limit. Recommended: increase memory limit."
}

______________________________________________________________________

📡 API 参考

POST/alert--请求正文

{
  "alert_type": "CrashLoopBackOff | HighCPU | OOMKilled | NodeNotReady | ...",
  "severity": "critical | high | medium | low",
  "namespace": "production",
  "pod_name": "my-pod-xyz",
  "message": "Human-readable alert description",
  "source": "bigpanda | datadog | pagerduty"
}

______________________________________________________________________

🧠 代理架构

编排器(agents/orchestrator.py)

具有7个节点的LangGraph状态图。在多回合互动中保持完整的对话状态。使用MemorySaver进行持久化。

Kubernetes代理(agents/kubernetes_agent.py)

通过K8s MCP服务器使用6个工具重新激活代理。诊断pod故障、读取日志、重启pod并扩展部署。

度量代理(agents/metrics_agent.py)

查询Datadog API和AWS CloudWatch。使用pandas执行异常检测。生成自然语言见解。

事件代理(agents/incident_agent.py)

创建、更新和关闭ServiceNow事件。包括重复数据删除功能--不会为同一警报创建重复的票证。

RAG代理(agents/rag_agent.py)

ChromaDB runbook集合上的语义+关键字混合搜索。返回具有相似性得分的排名Runbook。

______________________________________________________________________

🔧 MCP服务器

______________________________________________________________________

📚 RAG系统

10本生产运行手册，涵盖： crashloopbackoff · high-cpu-usage · pod-oom-killed · deployment-failed · node-not-ready · pvc-pending · service-unavailable · etcd-backup · alert-fatigue · database-connection

摄入： python rag/ingest.py Chunking： 递归字符TextSplitter（chunk_size=500，重叠=50） 嵌入： OpenAI文本嵌入3-small 矢量存储： ChromaDB

______________________________________________________________________

☁️ 基础设施部署

AWS-EKS

cd infrastructure/aws
terraform init
terraform plan
terraform apply
aws eks update-kubeconfig --region us-east-1 --name sre-platform-eks
helm upgrade --install agentic-sre ./helm/agentic-sre -n sre-platform

Azure AKS

cd infrastructure/azure
terraform init
terraform plan
terraform apply
az aks get-credentials --resource-group sre-platform-rg --name sre-platform-aks
helm upgrade --install agentic-sre ./helm/agentic-sre -n sre-platform

______________________________________________________________________

🏗️ 项目结构

agentic-sre-platform/
├── agents/          # LangGraph + LangChain agents
├── mcp_servers/     # FastAPI MCP tool servers
├── a2a/             # Agent-to-Agent registry and routing
├── rag/             # ChromaDB RAG pipeline + runbooks
├── api/             # FastAPI main application
├── infrastructure/  # Terraform: AWS + Azure
├── helm/            # Helm chart for K8s deployment
├── tests/           # pytest unit tests
└── .github/         # CI/CD workflows

______________________________________________________________________

🤝 贡献

1. 分叉回购
2. 创建要素分支： git checkout -b feature/my-feature
3. 运行测试： pytest tests/ -v
4. 格式代码： black .
5. 提交PR

______________________________________________________________________

📜 许可证

MIT许可证——见 许可证

______________________________________________________________________

*建造于 巴拉克里希纳五世 --首席架构师/代理人工智能工程师*