AppsAutessaDB

Data lake and analytics · AutessaDB

Bulk loading and partitioning

Fill the AutessaDB data lake in bulk. Upload CSV or Parquet files, have the server fetch a large file itself so it never passes through your machine, or copy every row of an existing AutessaDB table, then choose Append or Replace. Rows that cannot be parsed are reported rather than silently dropped, and a partition spec groups data by date or key so filtered queries read only what they need.

A database essential, built in
Load data · app_eventsAPPENDSAMPLE
sourcerowsrejectedstatus
events-2026-10.parquet412M0loaded
crm_export.csv9.4M31 reportedloaded
table · support_tickets1.2M0loading
partitioned by MONTH(event_date)estimateSAMPLE
JFMAMJJASOND
WHERE event_date >= DATE '2026-10-01' reads 3 of 12 partitions

-- the problem

Getting big data in is usually its own project

Exports from other systems arrive as large CSV and Parquet files, and the usual path is a pipeline, a staging area and a script nobody wants to own. In AutessaDB a load is a button, a CLI command or an API call that runs as one job, with progress, a row count and the rejected rows.

What you get

  • Start a table from a sample CSV or Parquet file: columns and types are inferred, sample rows are shown, and problems such as mixed types or unparseable dates are listed as warnings.
  • Upload files from your computer (large files upload in parts), or let the server fetch a file itself.
  • Copy every row of an ordinary AutessaDB table into a data lake table, mapping its columns.
  • Append adds rows; Replace swaps the table's contents for the new rows.
  • Progress shows in the Loads card, and rows that cannot be parsed are reported with the load.
  • Scripts and apps append small batches of up to 10,000 rows or 5 MB per call, committed together every few seconds.
  • Partition by YEAR, MONTH, DAY or HOUR of a date, by IDENTITY, by BUCKET(n) or by TRUNCATE(w), with up to two fields.
  • The editor suggests a partition, such as month of order_date, and an estimate shows how much a query will scan before it runs.

-- how it works

How does bulk loading and partitioning work in AutessaDB?

  1. Drop a sample file

    Columns and types are inferred; adjust them and read the warnings.

  2. Choose a partition

    Usually the month or day of the main date column.

  3. Load

    Leave Load this file after creating checked, or use Load data later from a file or a table.

  4. Query cheaply

    Filter on the partition column and the estimate drops to a fraction of a full scan.

-- every option

Load and partition options

optionwhat it does
Upload fileA CSV or Parquet file from your computer, uploaded in parts when large.
Server-fetched fileOne file the server fetches itself, so large sources never pass through your machine.
From AutessaDB tableCopy every row of a relational table, mapping its columns.
ModeAPPEND (default) adds rows; REPLACE swaps the contents.
Small appendsUp to 10,000 rows or 5 MB per call from scripts and apps.
YEAR / MONTH / DAY / HOURGroup by a date or timestamp truncated to that unit.
IDENTITYGroup by the exact value, for low-cardinality columns such as region.
BUCKET(n) / TRUNCATE(w)Group by a hash into n buckets, or by a text prefix or number range of width w.
Deleting rowsDelete the rows that match one condition, as a job.

-- example

Load files and check a query's costbash
autessa autessadb analytics load --tableId V1StGXR8Z5 --localFile ./orders-2026-09.parquet
autessa autessadb analytics load --tableId V1StGXR8Z5 --localFile ./orders.csv --mode REPLACE

autessa autessadb analytics estimate --tableId V1StGXR8Z5 \
  --sql "SELECT COUNT(*) FROM order_lines WHERE order_date >= DATE '2026-01-01'"

Full reference in the AutessaDB CLI docs and the API reference.

-- permissions

Who can do what?

accesscan
WRITELoad and append data.
ADMINCreate tables, choose partitions and change the schema.

-- relations

How it connects

autessadb feature

Columnar data lake tables

Where the data lands.

autessadb feature

Bulk and batch writes

The row-table way to write many records.

autessadb feature

Retention and archive tiers

What happens to old days.

-- faq

Questions about bulk loading and partitioning

What files can I load into the data lake?

CSV and Parquet files, from your computer or fetched by the server, plus every row of an existing AutessaDB table.

What happens to rows that cannot be parsed?

They are reported with the load rather than silently dropped, so you can fix the source and load again.

Why partition a table?

A query that filters on the partition column reads only the matching groups, so it runs faster and scans a fraction of the data.

Can apps write to the data lake as events happen?

Yes. Scripts and apps append small batches of rows, and appends that arrive close together are committed together every few seconds.

$ autessa autessadb tables create

Try bulk loading and partitioning in AutessaDB

Load CSV and Parquet files or whole AutessaDB tables, append or replace, see every bad row reported, and partition by date so queries read only what they need.

Read the AutessaDB guide or install the CLI.

100% customizable, code and all

Make AutessaDB fully your own.

Use it out of the box, then change anything. Ask Autobot, Autessa’s AI builder, for a new field, screen, report, workflow or agent and it builds it into your copy of the app. Or open the code and edit it yourself. Nothing is locked: the app, its data and its code are yours.

Start free and make it yours