AWS Glue
by Amazon Web Services
AWS Glue keeps a Data Catalog of databases and tables, crawlers that keep schemas current, and ETL jobs with their run history. Platform teams query it to know what data exists and whether last night's jobs worked. With catalog and run data in the Lake, schema changes and failed runs can be traced to the reports they affect.
- Category
- Cloud & Data Platforms
- Type
- Data integration
- Interface
- REST API
- Built with
- Connect Anything
- Credentials
- AWS access keys
02Data
What you can sync.
- Catalog tables
- Databases
- Jobs
- Job runs
- Crawlers
Load AWS Glue catalog tables into your Lake and model it in the Warehouse with the rest of your stack.
Need more? Name any other AWS Glue data the API returns, and the AI adds it to the connector.
03Credentials
Credentials you will need.
- AWS access keys
AWS Glue uses AWS access keys. You add the credentials once, when the AI asks for them, and it tells you where to create them in Amazon Web Services if you need help.
04Questions
Questions you can answer.
Once AWS Glue data is in your Lake, ask an AI Mind in plain words or build the answer in DataPrism BI.
- Which Glue job runs failed or ran over their usual duration this week?
- Which catalog tables changed schema after the last crawler run?
- How much DPU time did each job consume last month?
- Which databases contain tables no crawler has updated in 90 days?
05How it works
How it works in DataMind.
-
Pick AWS Glue.
Open Lake, pick AWS Glue in the source picker and name the connection. Connect Anything opens with a ready prompt that points the AI at the AWS Glue API docs.
-
Answer a few questions.
The AI reads the AWS Glue documentation, then asks which credentials you will use, which data to sync (for example catalog tables and databases), whether you want a one-time load or a recurring sync, and which accounts or filters apply.
-
Your data lands in Lake.
The connector agent writes a Python connector for AWS Glue, tests it against the live API and shows you sample rows. Then the data loads into your Lake on your schedule, ready for the Warehouse, DataPrism BI and AI Minds.
06Pairs
Works well with AWS Glue.
Sources teams join with AWS Glue in the Warehouse, each one connected the same way.
