# Spark Python Data Source

> Build custom Python data sources for Apache Spark using the PySpark DataSource API — batch and st…

Build custom Python data sources for Apache Spark using the PySpark DataSource API — batch and streaming readers/writers for external systems. Use this skill whenever someone wants to connect Spark to an external system (database, API, message queue, custom protocol), build a Spark connector or plugin in Python, implement a DataSourceReader or DataSourceWriter, pull data from or push data to a system via Spark, or work with the PySpark DataSource API in any way. Even if they just say "read from X in Spark" or "write DataFrame to Y" and there's no native connector, this skill applies.

- Page: https://ss33.ai/skills/databricks/spark-python-data-source-52770293559
- Publisher: databricks
- Category: [Data & Analytics](https://ss33.ai/skills/category/data-and-analytics-m1r3i9) › [Data pipelines & ETL](https://ss33.ai/skills/category/data-and-analytics-m1r3i9/data-pipelines-and-etl-c8rdes)
- Source: https://github.com/databricks/databricks-agent-skills/blob/c570f88de43af3575bd33de6fef6186671afa9ba/experimental/spark-python-data-source/SKILL.md

## Skill

This skill's licence does not allow redistribution, so the store does not serve its text. Get it from the author: https://github.com/databricks/databricks-agent-skills/blob/c570f88de43af3575bd33de6fef6186671afa9ba/experimental/spark-python-data-source/SKILL.md