Export MariaDB to OneLake in JSON

Fast, parallel data export with zero intermediate storage

Terminal

.\FastBCP.exe `
--connectiontype "mysql" `
--server "host.domain | host.domain,port | host.domain,port/service" `
--database "tpch" `
--trusted `
--sourceschema "tpch10" `
--sourcetable "orders" `
--query "SELECT * FROM tpch10.orders" `
--directory "onelake://workspace/lakehouse/fastbcpexports/raw/{sourcedatabase}/{sourceschema}" `
--fileoutput "{sourcetable}.json" `
--decimalseparator "." `
--dateformat "yyyy-MM-dd HH:mm:ss" `
--encoding "UTF-8" `
--method "Ntile" `
--distributekeycolumn "o_orderkey" `
--paralleldegree -2 `
--merge false `
--runid "runidfromcaller"

Get FastBCP

Source - MariaDB

MariaDB is a MySQL fork offering advanced features and better performance. FastBCP ensures full compatibility with MariaDB.

Features:

•Full MySQL compatibility
•Support for advanced MariaDB features
•Optimized performance

Parallel Method - Ntile

Divides data into N equal partitions based on a numeric column.

Requirement: Requires a numeric distribution column

Available parallel methods with MariaDB:

Output Format - JSON (JavaScript Object Notation)

JSON is the most widely used data exchange format on the web. FastBCP exports data to JSON Lines (NDJSON) format, with one record per line for efficient streaming.

Features:

•JSON Lines (NDJSON) format
•One record per line for streaming
•UTF-8 encoding
•Compatible with all JSON parsers

Destination - Microsoft OneLake

OneLake is Microsoft Fabric's unified data lake. FastBCP exports data directly to OneLake workspaces, enabling seamless integration with Fabric analytics services.