What is Databricks Connect v2?

It is a client library that allows you to connect your local IDE (like VS Code or PyCharm) to a Databricks cluster, allowing you to run Spark code on the cluster while coding locally.

How does this skill handle Databricks Asset Bundles?

It generates a standard databricks.yml configuration file that defines development, staging, and production targets for seamless deployment.

Does this support VS Code integration?

Absolutely. It provides specific JSON configurations for .vscode/settings.json and launch.json to ensure your environment is fully integrated with Databricks.

Can I run tests locally without a cluster?

Yes, the skill sets up a local SparkSession fixture for unit tests that run entirely on your machine, saving cluster costs during initial development.

Databricks Local Development Loop

Name: Databricks Local Development Loop
Author: jeremylongshore

byjeremylongshore

•

1,243

•

Data Science & ML

Configures a high-velocity local development environment for Databricks using Databricks Connect and Asset Bundles.

This skill automates the setup of a professional Databricks development workflow on your local machine, bridging the gap between local IDEs and cloud-based clusters. It orchestrates the installation of the Databricks SDK, CLI, and Databricks Connect v2, while establishing a standardized project structure that supports version control and CI/CD. By configuring integrated testing frameworks with pytest and providing ready-to-use IDE settings, it enables data engineers to iterate faster, debug efficiently, and maintain higher code quality than traditional notebook-only development.

Key Features

01Standardized Databricks project structure implementation

02Pre-configured VS Code and PyCharm settings for Databricks

031,243 GitHub stars

04Integrated testing patterns for local and remote validation

05Automated setup for Databricks Asset Bundles (DABs)

06Databricks Connect v2 configuration for local Spark execution

Use Cases

01Setting up CI/CD-ready Databricks projects using modern Asset Bundles

02Migrating from notebook-based development to a robust local IDE workflow

03Debugging complex Spark pipelines locally against remote Databricks clusters

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add jeremylongshore/claude-code-plugins-plus-skills databricks-local-dev-loop

For use in Claude.ai and ChatGPT

Download Skill

Key Features

01Standardized Databricks project structure implementation

02Pre-configured VS Code and PyCharm settings for Databricks

031,243 GitHub stars

04Integrated testing patterns for local and remote validation

05Automated setup for Databricks Asset Bundles (DABs)

06Databricks Connect v2 configuration for local Spark execution

Use Cases

01Setting up CI/CD-ready Databricks projects using modern Asset Bundles

02Migrating from notebook-based development to a robust local IDE workflow

03Debugging complex Spark pipelines locally against remote Databricks clusters

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add jeremylongshore/claude-code-plugins-plus-skills databricks-local-dev-loop

For use in Claude.ai and ChatGPT

Download Skill