Can DuckDB query files without importing them first?

Yes, DuckDB can query Parquet, CSV, and JSON files directly using SQL 'SELECT' statements on the file path, which is significantly more memory-efficient than traditional loading.

When should I use DuckDB instead of Pandas?

You should prefer DuckDB for complex joins, large-scale aggregations, window functions, or when working with datasets that are near or larger than your available RAM.

Is DuckDB suitable for high-frequency transactional writes?

No, DuckDB is an OLAP engine designed for analysis. For high-frequency row-level inserts (OLTP), SQLite or PostgreSQL are better choices.

What is the primary benefit of using DuckDB with Claude Code?

DuckDB allows Claude to perform professional-grade SQL analysis on local files and Python variables with extreme speed, avoiding the memory overhead of loading massive datasets into standard DataFrames.

Does DuckDB require a database server like MySQL or PostgreSQL?

No, DuckDB is an in-process database. It runs entirely within your Python script, requiring no external server management or configuration.

DuckDB for Scientific Data

Name: DuckDB for Scientific Data
Author: tondevrel

bytondevrel

•

Data Science & ML

Integrates high-performance analytical SQL capabilities into Python workflows for efficient data processing and large-scale querying.

This skill enables Claude to leverage DuckDB, an in-process SQL database management system optimized for Online Analytical Processing (OLAP). It provides specialized guidance for performing complex SQL queries—including window functions and joins—directly on Pandas DataFrames, Polars objects, or external files like Parquet and CSV without requiring a separate database server. It is particularly useful for managing datasets that exceed memory limits, building analytical pipelines, and performing advanced feature engineering for machine learning while maintaining seamless interoperability with the Python data ecosystem.

Key Features

01In-process SQL execution with no server management or installation overhead

02Columnar vectorized engine for high-speed aggregations on large datasets

03Advanced SQL support including complex joins, window functions, and sampling

041 GitHub stars

05Seamless conversion between SQL result sets and Pandas, Polars, or Arrow formats

06Direct querying of Parquet, CSV, and JSON files without an ingestion step

Use Cases

01Performing memory-efficient feature engineering and data cleaning for ML models

02Analyzing local datasets that are too large for Excel but don't require PostgreSQL

03Building high-performance data pipelines that join multiple Parquet files with Python variables

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add tondevrel/scientific-agent-skills duckdb

For use in Claude.ai and ChatGPT

Download Skill