Export ClickHouse to AWS S3 in BSON

Fast, parallel data export with zero intermediate storage

Terminal

.\FastBCP.exe `
--connectiontype "clickhouse" `
--server "host.domain | host.domain,port | host.domain,port/service" `
--database "tpch" `
--trusted `
--sourceschema "tpch10" `
--sourcetable "orders" `
--query "SELECT * FROM tpch10.orders" `
--directory "s3://rootbucket/fastbcpexport/raw/{sourcedatabase}/{sourceschema}" `
--fileoutput "{sourcetable}.bson" `
--method "Ntile" `
--distributekeycolumn "o_orderkey" `
--paralleldegree -2 `
--merge false `
--runid "runidfromcaller"

Get FastBCP

Source - ClickHouse

ClickHouse is an ultra-fast column-oriented analytics database. FastBCP leverages ClickHouse's architecture for high-performance exports.

Features:

•Optimized for analytical queries
•Native columnar format support
•Exceptional performance on large volumes

Parallel Method - Ntile

Divides data into N equal partitions based on a numeric column.

Requirement: Requires a numeric distribution column

Available parallel methods with ClickHouse:

Output Format - BSON (Binary JSON)

BSON is a binary-encoded serialization format, primarily used by MongoDB. FastBCP exports to BSON format for efficient integration with MongoDB ecosystems.

Features:

•Binary encoding for compact storage
•MongoDB native format
•Data type preservation
•Efficient for document databases

Destination - Amazon S3

Amazon S3 is the industry-leading object storage service. FastBCP uploads exported files directly to S3 buckets with parallel multipart uploads for optimal performance.