Search Shortcut cmd + k | ctrl + k

- Installation
- Documentation

- Getting Started
- Connect

- Overview
- Concurrency

- Data Import

- Overview
- Data Sources
- CSV Files

- Overview
- Auto Detection
- Reading Faulty CSV Files
- Tips

- JSON Files

- Overview
- Creating JSON
- Loading JSON
- Writing JSON
- JSON Type
- JSON Functions
- Format Settings
- Installing and Loading
- SQL to / from JSON
- Caveats

- Multiple Files

- Overview
- Combining Schemas

- Parquet Files

- Overview
- Metadata
- Encryption
- Tips

- Partitioning

- Hive Partitioning
- Partitioned Writes

- Appender
- INSERT Statements

- Client APIs

- Overview
- C

- Overview
- Startup
- Configuration
- Query
- Data Chunks
- Vectors
- Values
- Types
- Prepared Statements
- Appender
- Table Functions
- Replacement Scans
- API Reference

- C++
- CLI

- Overview
- Arguments
- Dot Commands
- Output Formats
- Editing
- Autocomplete
- Syntax Highlighting

- Dart
- Go
- Java
- Julia
- Node.js (Neo)

- Overview

- Node.js

- Overview
- API Reference

- Python

- Overview
- Data Ingestion
- Conversion between DuckDB and Python
- DB API
- Relational API
- Function API
- Types API
- Expression API
- Spark API
- API Reference
- Known Python Issues

- R
- Rust
- Swift
- Wasm

- Overview
- Instantiation
- Data Ingestion
- Query
- Extensions

- ADBC
- ODBC

- Overview
- Linux Setup
- Windows Setup
- macOS Setup
- Configuration

- SQL

- Introduction
- Statements

- Overview
- ANALYZE
- ALTER TABLE
- ALTER VIEW
- ATTACH and DETACH
- CALL
- CHECKPOINT
- COMMENT ON
- COPY
- CREATE INDEX
- CREATE MACRO
- CREATE SCHEMA
- CREATE SECRET
- CREATE SEQUENCE
- CREATE TABLE
- CREATE VIEW
- CREATE TYPE
- DELETE
- DESCRIBE
- DROP
- EXPORT and IMPORT DATABASE
- INSERT
- LOAD / INSTALL
- PIVOT
- Profiling
- SELECT
- SET / RESET
- SET VARIABLE
- SUMMARIZE
- Transaction Management
- UNPIVOT
- UPDATE
- USE
- VACUUM

- Query Syntax

- SELECT
- FROM and JOIN
- WHERE
- GROUP BY
- GROUPING SETS
- HAVING
- ORDER BY
- LIMIT and OFFSET
- SAMPLE
- Unnesting
- WITH
- WINDOW
- QUALIFY
- VALUES
- FILTER
- Set Operations
- Prepared Statements

- Data Types

- Overview
- Array
- Bitstring
- Blob
- Boolean
- Date
- Enum
- Interval
- List
- Literal Types
- Map
- NULL Values
- Numeric
- Struct
- Text
- Time
- Timestamp
- Time Zones
- Union
- Typecasting

- Expressions

- Overview
- CASE Statement
- Casting
- Collations
- Comparisons
- IN Operator
- Logical Operators
- Star Expression
- Subqueries

- Functions

- Overview
- Aggregate Functions
- Array Functions
- Bitstring Functions
- Blob Functions
- Date Format Functions
- Date Functions
- Date Part Functions
- Enum Functions
- Interval Functions
- Lambda Functions
- List Functions
- Map Functions
- Nested Functions
- Numeric Functions
- Pattern Matching
- Regular Expressions
- Struct Functions
- Text Functions
- Time Functions
- Timestamp Functions
- Timestamp with Time Zone Functions
- Union Functions
- Utility Functions
- Window Functions

- Constraints
- Indexes
- Meta Queries

- Information Schema
- Metadata Functions

- DuckDB's SQL Dialect

- Overview
- Indexing
- Friendly SQL
- Keywords and Identifiers
- Order Preservation
- PostgreSQL Compatibility

- Samples

- Configuration

- Overview
- Pragmas
- Secrets Manager

- Extensions

- Overview
- Core Extensions
- Community Extensions
- Working with Extensions
- Versioning of Extensions
- Arrow
- AutoComplete
- AWS
- Azure
- Delta
- Excel
- Full Text Search
- httpfs (HTTP and S3)

- Overview
- HTTP(S) Support
- Hugging Face
- S3 API Support
- Legacy Authentication Scheme for S3 API

- Iceberg
- ICU
- inet
- jemalloc
- MySQL
- PostgreSQL
- Spatial

- Overview
- Function Reference
- R-Tree Indexes
- GDAL Integration

- SQLite
- Substrait
- TPC-DS
- TPC-H
- VSS

- Guides

- Overview
- Data Viewers

- Tableau
- CLI Charting with YouPlot

- Database Integration

- Overview
- MySQL Import
- PostgreSQL Import
- SQLite Import

- File Formats

- Overview
- CSV Import
- CSV Export
- Directly Reading Files
- Excel Import
- Excel Export
- JSON Import
- JSON Export
- Parquet Import
- Parquet Export
- Querying Parquet Files

- Network and Cloud Storage

- Overview
- HTTP Parquet Import
- S3 Parquet Import
- S3 Parquet Export
- S3 Iceberg Import
- S3 Express One
- GCS Import
- Cloudflare R2 Import
- DuckDB over HTTPS / S3

- Meta Queries

- Describe Table
- EXPLAIN: Inspect Query Plans
- EXPLAIN ANALYZE: Profile Queries
- List Tables
- Summarize
- DuckDB Environment

- ODBC

- ODBC Guide

- Performance

- Overview
- Environment
- Import
- Schema
- Indexing
- Join Operations
- File Formats
- How to Tune Workloads
- My Workload Is Slow
- Benchmarks

- Python

- Installation
- Executing SQL
- Jupyter Notebooks
- SQL on Pandas
- Import from Pandas
- Export to Pandas
- Import from Numpy
- Export to Numpy
- SQL on Arrow
- Import from Arrow
- Export to Arrow
- Relational API on Pandas
- Multiple Python Threads
- Integration with Ibis
- Integration with Polars
- Using fsspec Filesystems

- SQL Editors

- DBeaver SQL IDE

- SQL Features

- AsOf Join
- Full-Text Search
- query and query_table Functions

- Snippets

- Create Synthetic Data
- Sharing Macros

- Glossary of Terms
- Browse Offline

- Operations Manual

- Overview
- Limits
- Non-Deterministic Behavior
- Embedding DuckDB
- DuckDB's Footprint

- Files Created by DuckDB
- Gitignore for DuckDB
- Reclaiming Space

- Securing DuckDB

- Overview
- Securing Extensions

- Development

- DuckDB Repositories
- Testing

- Overview
- sqllogictest Introduction
- Writing Tests
- Debugging
- Result Verification
- Persistent Testing
- Loops
- Multiple Connections
- Catch

- Profiling
- Release Calendar
- Building

- Overview
- Build Instructions
- Build Configuration
- Building Extensions
- Platforms
- Troubleshooting

- Benchmark Suite

- Internals

- Overview
- Storage Versions and Format
- Execution Format
- Pivot

- Sitemap
- Why DuckDB
- Media
- FAQ
- Code of Conduct
- Live Demo

Friendly SQL

DuckDB offers several advanced SQL features and syntactic sugar to make SQL queries more concise. We refer to these colloquially as “friendly SQL”.

> Several of these features are also supported in other systems while some are (currently) exclusive to DuckDB.

## Clauses

- Creating tables and inserting data:
  - `CREATE OR REPLACE TABLE`: avoid `DROP TABLE IF EXISTS` statements in scripts.
  - `CREATE TABLE ... AS SELECT` (CTAS): create a new table from the output of a table without manually defining a schema.
  - `INSERT INTO ... BY NAME`: this variant of the `INSERT` statement allows using column names instead of positions.
  - `INSERT OR IGNORE INTO ...`: insert the rows that do not result in a conflict due to `UNIQUE` or `PRIMARY KEY` constraints.
  - `INSERT OR REPLACE INTO ...`: insert the rows that do not result in a conflict due to `UNIQUE` or `PRIMARY KEY` constraints. For those that result in a conflict, replace the columns of the existing row to the new values of the to-be-inserted row.
- Describing tables and computing statistics:
  - `DESCRIBE`: provides a succinct summary of the schema of a table or query.
  - `SUMMARIZE`: returns summary statistics for a table or query.
- Making SQL clauses more compact:
  - `FROM`-first syntax with an optional `SELECT` clause: DuckDB allows queries in the form of `FROM tbl` which selects all columns (performing a `SELECT *` statement).
  - `GROUP BY ALL`: omit the group-by columns by inferring them from the list of attributes in the `SELECT` clause.
  - `ORDER BY ALL`: shorthand to order on all columns (e.g., to ensure deterministic results).
  - `SELECT * EXCLUDE`: the `EXCLUDE` option allows excluding specific columns from the `*` expression.
  - `SELECT * REPLACE`: the `REPLACE` option allows replacing specific columns with different expressions in a `*` expression.
  - `UNION BY NAME`: perform the `UNION` operation along the names of columns (instead of relying on positions).
- Transforming tables:
  - `PIVOT` to turn long tables to wide tables.
  - `UNPIVOT` to turn wide tables to long tables.
- Defining SQL-level variables:
  - `SET VARIABLE`
  - `RESET VARIABLE`

## Query Features

- Column aliases in `WHERE`, `GROUP BY`, and `HAVING`. (Note that column aliases cannot be used in the `ON` clause of `JOIN` clauses.)
- `COLUMNS()` expression can be used to execute the same expression on multiple columns:
  - with regular expressions
  - with `EXCLUDE` and `REPLACE`
  - with lambda functions
- Reusable column aliases, e.g.: `SELECT i + 1 AS j, j + 2 AS k FROM range(0, 3) t(i)`
- Advanced aggregation features for analytical (OLAP) queries:
  - `FILTER` clause
  - `GROUPING SETS`, `GROUP BY CUBE`, `GROUP BY ROLLUP` clauses
- `count()` shorthand for `count(*)`

## Literals and Identifiers

- Case-insensitivity while maintaining case of entities in the catalog
- Deduplicating identifiers
- Underscores as digit separators in numeric literals

## Data Types

- `MAP` data type
- `UNION` data type

## Data Import

- Auto-detecting the headers and schema of CSV files
- Directly querying CSV files and Parquet files
- Loading from files using the syntax `FROM 'my.csv'`, `FROM 'my.csv.gz'`, `FROM 'my.parquet'`, etc.
- Filename expansion (globbing), e.g.: `FROM 'my-data/part-*.parquet'`

## Functions and Expressions

- Dot operator for function chaining: `SELECT ('hello').upper()`
- String formatters: the `format()` function with the `fmt` syntax and the `printf() function`
- List comprehensions
- List slicing
- String slicing
- `STRUCT.*` notation
- Simple `LIST` and `STRUCT` creation

## Join Types

- `ASOF` joins
- `LATERAL` joins
- `POSITIONAL` joins

## Trailing Commas

DuckDB allows trailing commas, both when listing entities (e.g., column and table names) and when constructing `LIST` items. For example, the following query works:

    SELECT
        42 AS x,
        ['a', 'b', 'c',] AS y,
        'hello world' AS z,
    ;

## "Top-N in Group" Queries

Computing the "top-N rows in a group" ordered by some criteria is a common task in SQL that unfortunately often requires a complex query involving window functions and/or subqueries.

To aid in this, DuckDB provides the aggregate functions `max(arg, n)`, `min(arg, n)`, `arg_max(arg, val, n)`, `arg_min(arg, val, n)`, `max_by(arg, val, n)` and `min_by(arg, val, n)` to efficiently return the "top" `n` rows in a group based on a specific column in either ascending or descending order.

For example, let's use the following table:

    SELECT * FROM t1;

    ┌─────────┬───────┐
    │   grp   │  val  │
    │ varchar │ int32 │
    ├─────────┼───────┤
    │ a       │     2 │
    │ a       │     1 │
    │ b       │     5 │
    │ b       │     4 │
    │ a       │     3 │
    │ b       │     6 │
    └─────────┴───────┘

We want to get a list of the top-3 `val` values in each group `grp`. The conventional way to do this is to use a window function in a subquery:

    SELECT array_agg(rs.val), rs.grp
    FROM
        (SELECT val, grp, row_number() OVER (PARTITION BY grp ORDER BY val DESC) AS rid
        FROM t1 ORDER BY val DESC) AS rs
    WHERE rid < 4
    GROUP BY rs.grp;

    ┌───────────────────┬─────────┐
    │ array_agg(rs.val) │   grp   │
    │      int32[]      │ varchar │
    ├───────────────────┼─────────┤
    │ [3, 2, 1]         │ a       │
    │ [6, 5, 4]         │ b       │
    └───────────────────┴─────────┘

But in DuckDB, we can do this much more concisely (and efficiently!):

    SELECT max(val, 3) FROM t1 GROUP BY grp;

    ┌─────────────┐
    │ max(val, 3) │
    │   int32[]   │
    ├─────────────┤
    │ [3, 2, 1]   │
    │ [6, 5, 4]   │
    └─────────────┘

## Related Blog Posts

- “Friendlier SQL with DuckDB” blog post
- “Even Friendlier SQL with DuckDB” blog post
- “SQL Gymnastics: Bending SQL into Flexible New Shapes” blog post

##### About this page

- Report content issue
- Edit this page on GitHub