Connectors
The ydata-sdk provides built-in connectors to interact with various data sources, allowing seamless integration with cloud storage, databases, and local files.
Available Connectors
- Local Files – Work with datasets stored on local disk.
- AWS S3 – Retrieve and store datasets in AWS S3.
- Azure Blob Storage – Connect to Azure Blob Storage and interact with datasets.
- Google Cloud Storage – Access and manage datasets stored in Google Cloud Storage.
- BigQuery – Query and analyze datasets directly from Google BigQuery.
- Microsoft SQL Server – Connect to an MSSQL database.
- MySQL – Use MySQL as a data source.
- Snowflake – Access and query Snowflake data warehouses.
- Databricks Lakehouse – Integrate with Databricks for data processing and analysis.
- Databricks Unity Catalog – Manage and query datasets in Databricks Unity Catalog.
Installation
Each connector's client libraries are installed with its own flavour, so you only install what you use:
| Connector | Flavour |
|---|---|
| Local files | base install |
| AWS S3 | s3 |
| Azure Blob Storage | azure |
| Google Cloud Storage, BigQuery | google |
| Microsoft SQL Server / Azure SQL | azuresql |
| MySQL | mysql |
| PostgreSQL | postgresql |
| Snowflake | snowflake |
| Databricks Lakehouse, Unity Catalog | databricks |
pip install "ydata-sdk[snowflake]" # one connector
pip install "ydata-sdk[s3,postgresql]" # several
pip install "ydata-sdk[all-connectors]" # everything
See Installation for details.
Click on any connector to learn more about its implementation.