Setting the file. One moment.
Subchapter 2.171
references/integrations/INDEX.mdMarkdown10 KBView on GitHub
Dagster provides integration libraries for a range of tools and technologies. This reference directory contains detailed information about specific integrations.
Integration libraries are typically named dagster-<technology>, where <technology> is the name of the tool or technology being integrated. Integrations marked as are maintained by the community rather than the Dagster team.
All integration reference files contain a link to the official documentation for the integration library, which can be referenced in cases where the local documentation does not provide sufficient information.
Before writing ANY code, classify every external system in the request against the Reference Files Index below. Then for each system, use the FIRST option that applies:
dg scaffold defs.get_asset_spec(), execute(), or build_defs() (build_defs_from_state() for StateBackedComponent). See Subclassing Components.Component — no library exists, no external state needed. See Creating Components.StateBackedComponent — no library exists, definitions require external state. See Creating State-Backed Components. Parent class may need to be inspected to determine the specific state format for write_state_to_path() / build_defs_from_state().NEVER build a custom component when a library integration exists, even for mocks or testing usecases.
IMPORTANT: If an integration library offers a Component, ALWAYS prefer this over the Pythonic integration unless there is an explicit reason to prefer the Pythonic integration. Components have a simpler, more deterministic interface and are easier to understand and manage.
# Install the integration library
uv add dagster-<technology>
# List the available components
uv run dg list components --json
# [Optional] If a component exists, inspect its scaffolding or defs yaml schema
uv run dg utils inspect-component <ComponentType> --scaffold-params-schema
uv run dg utils inspect-component <ComponentType> --defs-yaml-schema
# Scaffold an instance of the component
uv run dg scaffold defs <ComponentType> <instance_name>NEVER skip this workflow. Do not write Pythonic integration code (e.g. FivetranWorkspace, @dbt_assets, CensusResource) when a component is available. The dg list components command is the authoritative source for what components exist — do not guess or assume from memory.
If the desired external tool has no published dagster-* library (e.g. custom REST API), do NOT fall back to raw @dg.asset or @dg.sensor definitions. Instead, create a custom component:
# Scaffold a new component type
uv run dg scaffold component MyToolComponent
# Verify registration and get the full type path
uv run dg list components
# Scaffold an instance of your new component
uv run dg scaffold defs my_project.components.my_tool_component.MyToolComponent my_instanceSee Designing Component Integrations for patterns on structuring the component (definition-only, observing via sensors, or orchestrating via execution). Components are the standard unit of integration in Dagster — even for tools without a published library.