cognite（cognite 搜索）

Cognite Data Fusion (CDF) skill

Product summary

Cognite Data Fusion is an industrial data platform that ingests, models, and exposes data from operational technology (OT) and information technology (IT) systems. Agents use CDF to build data pipelines (extract → transform → contextualize), define industrial knowledge graphs via data modeling, query resources via REST API or SDKs, and deploy configurations using the Cognite Toolkit. Key entry points: REST API at https://{cluster}.cognitedata.com/api/v1, Python SDK (cognite-sdk), JavaScript SDK (@cognite/sdk), and the Cognite Toolkit CLI (cdf). Primary docs: docs.cognite.com

When to use

Reach for this skill when:

• Integrating data: Setting up extractors (OPC UA, PI, SAP, databases) to stream data into CDF, configuring extraction pipelines, or monitoring data ingestion
• Modeling data: Designing data models with spaces, containers, views, and instances; querying graphs with GraphQL or REST; managing access via spaces
• Working with Assets in CDM: Creating or updating CogniteAsset hierarchies, populating assets via transformations, querying assets in CDF Search (shown as "Asset")
• Working with Assets (legacy): Maintaining existing asset-centric applications using the /assets API
• Querying resources: Retrieving assets, time series, events, files, or sequences; using external IDs for lookups; filtering with advanced query language (AQL)
• Deploying infrastructure: Using the Cognite Toolkit to manage CDF projects as code, setting up CI/CD pipelines, or deploying modules
• Building applications: Authenticating with OAuth 2.0/OIDC, using Python/JavaScript SDKs to read/write data, or calling REST endpoints
• Managing access: Configuring OIDC providers, creating groups, assigning capabilities, or controlling access to spaces and data sets
• Automating workflows: Setting up data workflows with tasks and triggers, transforming data, or orchestrating multi-step processes

Core data modeling vs. legacy

The Cognite Data Fusion (CDF) platform supports two modes:

For new development, prefer core data modeling. Use the legacy APIs only when maintaining existing applications.

Quick reference

Core resource types (core data modeling)

Legacy resource types

Other resource types

Common developer goals

Authentication patterns

Cognite Toolkit workflow

cdf modules init                    # Initialize project structure
cdf build --env=dev                 # Build artifacts from YAML configs
cdf deploy --dry-run --env=dev      # Validate before deploying
cdf deploy --env=dev                # Deploy to CDF project

Configuration files

• cdf.toml — Global CLI settings (organization, environment, plugins)
• config.[env].yaml — Per-environment project settings and module selection
• modules/ — Resource directories (data_modeling/, raw/, access/, etc.)
• *.yaml — Resource configs (e.g., my.space.yaml, my.container.yaml)

Common API endpoints

Decision guidance

When to use REST API vs SDK

When to use RAW vs data modeling

When to use external ID vs internal ID

Query approach: CRUD vs advanced query

Workflow

1. Set up authentication and project access

• Identify the authentication method (API key, OAuth 2.0, service principal)
• For Cognite Toolkit: configure cdf.toml with organization and environment
• For SDKs: set up OIDC provider (Entra ID or Amazon Cognito) and obtain credentials
• Test connectivity: cdf status or client = CogniteClient() in Python

2. Understand your data model

• Identify source systems and their data types (time series, events, assets)
• Map external IDs from source systems to CDF resources
• Decide: will you use RAW for staging, or ingest directly into data models?
• If using data modeling: design spaces, containers, and views for your domain

3. Extract and stage data

• Choose an extractor (OPC UA, PI, SAP, database, REST, file, etc.)
• Configure the extractor with source credentials and CDF connection
• Set up an extraction pipeline to monitor ingestion
• Verify data arrives in RAW or target data model

4. Transform and contextualize

• Write SQL transformations to reshape RAW data into CDF resource types
• Use entity matching to link entities across source systems
• Build relationships between assets, time series, and events
• Assign external IDs consistently across all resources

5. Query and expose data

• Use REST API or SDK to retrieve resources by external ID or filter
• For complex queries: use GraphQL on data models or advanced query language (AQL)
• Implement pagination for large result sets (max 10,000 per request)
• Cache results where appropriate to reduce API calls

6. Deploy with infrastructure-as-code

• Define all resources (spaces, containers, views, access groups) in YAML
• Organize configs in modules under modules/ directory
• Use cdf build to validate and generate artifacts
• Use cdf deploy --dry-run to preview changes before applying
• Commit configs to version control and integrate with CI/CD (GitHub Actions, Azure DevOps, GitLab)

Common gotchas

• Asset search discovery: Searching for "Asset" in the API reference surfaces legacy /assets endpoints. For Core Data Modeling, use the Instances API with CogniteAsset—see Core data model and Building an asset hierarchy.
• External ID uniqueness: External IDs are unique per resource type, not globally. An asset and time series can both have externalId=123. Enforce this in your source system mapping.
• Search vs CRUD consistency: Advanced query (AQL) is eventually consistent and slower than CRUD endpoints. Don't use it for large-scale batch synchronization; use /byids or /list instead.
• Pagination limits: Max 10,000 items per page. For data modeling queries, pagination only works at the top level; nested results cannot be paginated.
• Data modeling access control: Access is scoped to spaces, not data sets. Users need dataModelsAcl.READ to the space and dataModelInstancesAcl.READ/WRITE to instances.
• Extraction pipeline monitoring: Create extraction pipelines to track ingestion health. Without them, you won't see run history or failure notifications.
• Toolkit module ordering: Spaces must be created before containers, containers before views, and views before data models. Use numbered prefixes (e.g., 01_space.yaml, 02_container.yaml) if dependencies exist within the same type.
• Token expiration: OAuth 2.0 tokens expire. SDKs handle refresh automatically, but custom HTTP clients must implement token refresh logic.
• Rate limiting: CDF enforces throttling on parallel requests. Default: 20 parallel time series operations, 10 parallel data point operations. Respect these limits or requests will be queued.
• Deprecated API versions: API v0.5, v0.6 are removed. Always use the latest API version (v1).
• Missing capabilities: Users need specific capabilities (e.g., timeseries:read, assets:write) to perform operations. Check access errors against the capabilities reference.

Verification checklist

Before submitting work:

• Authentication works: Test with cdf status (Toolkit) or client.data_modeling.instances.list() (SDK, CDM) or client.assets.list() (SDK, legacy)
• External IDs are set: All resources have unique, consistent external IDs from source systems
• Data flows end-to-end: Verify data appears in CDF (check extraction pipeline runs, query a sample asset/time series)
• Access is configured: Test that intended users/groups can read/write resources; check space and capability assignments
• Queries are efficient: For data modeling, run with profile: true to check debug notices; avoid full table scans
• Pagination is handled: If retrieving >10,000 items, implement cursor-based pagination
• Dry-run passes: Run cdf deploy --dry-run and review changes before applying
• CI/CD is integrated: Toolkit configs are in version control and CI/CD pipeline validates/deploys on merge
• Monitoring is in place: Extraction pipelines, data workflows, and functions have alerts configured
• Documentation is updated: Record external ID mappings, data model schema, and access policies

Resources

Core data modeling

• Core data model reference: CogniteAsset, CogniteTimeSeries, CogniteFile
• Build asset hierarchy: dm_cdm_build_asset_hierarchy
• Integrate time series: dm_integrate_with_time_series
• Integrate files: dm_integrate_files
• Instances API: Overview

General

• Comprehensive navigation: docs.cognite.com/llms.txt — Full page-by-page listing of all documentation
• REST API reference: /dev/use_the_API
• Python SDK: docs.cognite.com/dev/sdks/python
• JavaScript SDK: docs.cognite.com/dev/sdks/js
• Cognite Toolkit: docs.cognite.com/cdf/deploy/cdf_toolkit
• Data modeling: docs.cognite.com/cdf/dm
• Data integration: docs.cognite.com/cdf/integration
• Access control: docs.cognite.com/cdf/access

For additional documentation and navigation, see: docs.cognite.com/llms.txt