Apache Airflow
by Apache Software Foundation
Open-source workflow orchestrator; its stable REST API returns DAGs, DAG runs, task instances, logs and datasets.
- Category
- Cloud & Data Platforms
- Type
- Orchestration
- Interface
- REST API
- Built with
- Connect Any API
- Credentials
- Access token, Username and password
02Data
What you can sync.
- Dags
- Dag runs
- Task instances
- Task logs
- Assets
Load Apache Airflow dags into your Lake and model it in the Warehouse with the rest of your stack.
Need more? Name any other Apache Airflow data the API returns, and the AI adds it to the connector.
03Credentials
Credentials you will need.
- An access token
- A username and password
Apache Airflow uses an access token or a username and password. You add the credentials once, when the AI asks for them, and it tells you where to create them in Apache Software Foundation if you need help.
04How it works
How it works in DataMind.
-
Pick Apache Airflow.
Open Lake, pick Apache Airflow in the source picker and name the connection. Connect Any API opens with a ready prompt that points the AI at the Apache Airflow API docs.
-
Answer a few questions.
The AI reads the Apache Airflow documentation, then asks which credentials you will use, which data to sync (for example dags and dag runs), whether you want a one-time load or a recurring sync, and which accounts or filters apply.
-
Your data lands in Lake.
It builds the connector, tests it and shows you sample rows. Then Apache Airflow data loads into your Lake on your schedule, ready for the Warehouse, DataPrism BI and AI Minds.
