Getting Started

GreatEmbeddo generates a complete, self-contained C99 database for your embedded system from a .gedb schema file. There is no runtime library to link — the generated code is the database.

Follow these four steps to integrate GreatEmbeddo into a project:

1. Write a .gedb schema describing your tables and indexes.
2. Run the greatembeddo CLI to generate C99 source files.
3. Add the generated .c and .h files to your build.
4. Implement the three I/O callbacks (read block, erase block, write block) for your storage.

Schema Example

A typical myproject.gedb file for an automotive ECU storing fault codes:

-- myproject.gedb
-- Schema for ECU fault log

CREATE TABLE ecu_config (
    id          uint32 PRIMARY KEY NOT NULL,
    key         char[32] NOT NULL,
    value_i32   int32,
    value_f     float
) EXPECT 64 REPLICAS 1;

CREATE TABLE fault_log (
    id          uint64 PRIMARY KEY NOT NULL,
    ts_ms       uint64 NOT NULL,
    dtc_code    uint32 NOT NULL,
    severity    uint8  NOT NULL DEFAULT 0,
    freeze_data uint8[16]
) EXPECT 4096 REPLICAS 2;

CREATE TABLE calibration (
    param_id    uint16 PRIMARY KEY NOT NULL,
    table_id    uint8  NOT NULL,
    value       float  NOT NULL,
    min_val     float  NOT NULL,
    max_val     float  NOT NULL
) EXPECT 512 REPLICAS 1;

CREATE INDEX idx_fault_dtc  ON fault_log (dtc_code);
CREATE INDEX idx_cal_table  ON calibration (table_id);

QUERY get_faults_by_dtc
    SELECT * FROM fault_log WHERE dtc_code = @dtc uint32 ORDER BY ts_ms DESC;

CLI Usage

> greatembeddo <schema.gedb> [options]

Options:
  -o, --output <dir>          Output directory (default: current directory)
  --block-size <bytes>        Flash block size, power of 2 (required)
  --program-page-size <bytes> NAND program-page size, power of 2 (default: 0 =
                              byte-granular writes for RAM/FRAM/MRAM)
  --cache-size <bytes>        RAM cache size in bytes (required)
  --block-count <n>           Total number of blocks on the flash device (required)
  --metadata-replicas <n>     Metadata replication factor (default: 3, max 4)
  --lock <strategy>           Writer-lock strategy: mtx (default), pthread, spin
  --max-stack <bytes>         Stack budget for the full call chain from any public
                              API entry point (default: 65536, min 256); each
                              query's sort buffer is derived to fit this limit
  --stats                     Print storage statistics only, do not generate code
  --memory                    Print static RAM breakdown only, do not generate code
  --caveats                   Print performance caveats only, do not generate code
  --yes / -y                  Auto-accept all performance caveats
  --strict                    Abort (exit 1) if any caveats exist
  --skip-lock-check           Skip the .gedb.lock append-only check (last resort)
  --rust                      Also make the output directory a Rust crate (see Rust Crate)
  --quiet                     Suppress all output except errors
  -h, --help                  Show this help message
  -v, --version               Print version and exit

Migration generation is available in the Pro version.

Example — generate code for a 4 MiB NOR flash with 4 KiB blocks:

> greatembeddo myproject.gedb \
    --output src/generated/ \
    --block-size 4096 \
    --block-count 1024 \
    --cache-size 65536

Check storage requirements before generating:

> greatembeddo myproject.gedb --stats --block-count 1024 --block-size 4096 --cache-size 65536

Schema version : 5
Tables         : 3
Block size     : 4096 bytes
Metadata blocks: 16

Table                Rows  Row bytes  Blocks (est)  Replicas
-----------------------------------------------------------------
ecu_config             64         52             2         1
fault_log            4096         40           165         2
calibration           512         22             5         1
-----------------------------------------------------------------
Total (estimated)                               207

Schema Reference

The .gedb schema language is a subset of SQL DDL extended with embedded-specific annotations. All identifiers are case-insensitive. Keywords are uppercase by convention.

Column Types

CREATE TABLE

CREATE TABLE <name> (
    <col> <type> [PRIMARY KEY] [NOT NULL] [DEFAULT <value>]
                 [RANGE (<min>, <max>)]
                 [REFERENCES <table>(<col>) [ON DELETE RESTRICT|CASCADE|SET NULL]],
    ...
) [EXPECT <rows>] [REPLICAS <n>];

• EXPECT n — hint for storage sizing; has no effect on correctness
• REPLICAS n — number of redundant copies stored on flash (default 2 for tables, 1 for indexes; max 4)
• RANGE(min, max) — inserts/updates with an out-of-range value are rejected with GEDB_ERR_PARAM
• Exactly one PRIMARY KEY column is required per table

CREATE INDEX / CREATE UNIQUE INDEX

CREATE INDEX <name> ON <table> (<col> [, <col> ...]);
CREATE UNIQUE INDEX <name> ON <table> (<col> [, <col> ...]);

-- When adding a UNIQUE index to an existing table (migration), ON CONFLICT is required:
CREATE UNIQUE INDEX <name> ON <table> (<col> [, ...]) ON CONFLICT REMOVE_DUPLICATES_REASSIGN_PRIMARY;
CREATE UNIQUE INDEX <name> ON <table> (<col> [, ...]) ON CONFLICT REMOVE_DUPLICATES_CASCADE_DELETE;
CREATE UNIQUE INDEX <name> ON <table> (<col> [, ...]) ON CONFLICT REMOVE_DUPLICATES_SET_NULL;
CREATE UNIQUE INDEX <name> ON <table> (<col> [, ...]) ON CONFLICT REMOVE_ALL_DUPLICATES_CASCADE_DELETE;
CREATE UNIQUE INDEX <name> ON <table> (<col> [, ...]) ON CONFLICT REMOVE_ALL_DUPLICATES_SET_NULL;

• Composite indexes are supported
• UNIQUE indexes enforce a uniqueness constraint at insert/update time
• A column declared UNIQUE gets its own unique index, IX_<table>_<col>, unless a unique index on that column alone exists; a non-unique or composite index over the column does not stand in for it
• REMOVE_DUPLICATES_* — keep the row with the lowest primary key per duplicate group; delete the rest
• REMOVE_ALL_DUPLICATES_* — delete every row in each duplicate group
• The suffix says what happens to the rows that reference a deleted row through a foreign key, whatever their ON DELETE: _REASSIGN_PRIMARY points them at the row that was kept, _SET_NULL sets the reference to NULL, _CASCADE_DELETE deletes them (their own children then follow their own ON DELETE). A reference never dangles.
• Refused at compile time: _REASSIGN_PRIMARY or _SET_NULL when a UNIQUE constraint covers a referencing column, _SET_NULL when a referencing column is NOT NULL, and _CASCADE_DELETE when the deletion reaches an ON DELETE RESTRICT

Schema Changes

Schema-changing commands (ALTER TABLE, DROP TABLE, DROP COLUMN, DROP INDEX) are always permitted in the .gedb file and validated at code-generation time. Applying them to an existing database on a deployed device requires the Pro edition, which generates a migration function that upgrades the on-flash data in place.

-- Add a column (default value required for existing rows)
ALTER TABLE <name> ADD <col> <type> [DEFAULT <value>] [NOT NULL] ...;

-- Change a column's type (ON NARROW required for narrowing conversions)
ALTER TABLE <name> ALTER <col> <new-type>;
ALTER TABLE <name> ALTER <col> <new-type> ON NARROW CLAMP;
ALTER TABLE <name> ALTER <col> <new-type> ON NARROW WRAP;
ALTER TABLE <name> ALTER <col> <new-type> ON NARROW NULL;

-- ... and ON CONFLICT when the change can merge values under a unique index
ALTER TABLE <name> ALTER <col> <new-type> ON NARROW CLAMP ON CONFLICT REMOVE_DUPLICATES_REASSIGN_PRIMARY;
ALTER TABLE <name> ALTER <col> <new-type> ON CONFLICT REMOVE_DUPLICATES_CASCADE_DELETE;

-- Update expected row count hint
ALTER TABLE <name> EXPECT <n>;

-- Change replication factor
ALTER TABLE <name> REPLICAS <n>;

DROP TABLE <name>;
ALTER TABLE <name> DROP COLUMN <column>;
DROP INDEX <name>;

Narrowing policies apply when the new type cannot represent all values of the old type (e.g. int32 → int16, float → int32, and a signed type to an unsigned one of any width, such as int8 → uint16, which cannot hold a negative value):

• ON NARROW CLAMP — clamp values to the min/max of the new type
• ON NARROW WRAP — truncate via C cast (modular arithmetic)
• ON NARROW NULL — set out-of-range values to null (column must be nullable)

Widening conversions (e.g. int16 → int32, float → double) do not require ON NARROW. Shortening a non-char array (e.g. uint32[4] → uint32[2], or uint16[4] → uint32[2]) drops its trailing elements and does not require it either; an array whose element type narrows (uint32[2] → uint16[2]) does, and each element follows the policy. Primary key columns cannot be altered. A single value and an array or char buffer convert through element 0: the value becomes element 0 and the other elements are zero (int8 → int8[2]), and an array becomes its element 0 (int8[2] → int8). ON NARROW is needed when element 0's conversion needs it (int32 → char[4]), a char counting as a byte. A varbinary and a single value convert through byte 0 the same way; an empty varbinary becomes NULL, or 0 in a NOT NULL column.

Under a unique index, a change that can turn two distinct values into equal ones makes duplicates the migration has to remove, so it also needs an ON CONFLICT policy (the same REMOVE_DUPLICATES_* / REMOVE_ALL_DUPLICATES_* policies as above, with the same compile-time refusals), unless the UNIQUE declares its own: every narrowing, whatever its ON NARROW; a char buffer or array that shrinks; and float → double, which turns a signaling NaN into the matching quiet NaN. The ALTER's policy applies to that step only, to every unique index over the column, composite ones included. A change that keeps values apart (an integer to one that holds all its values, a buffer grown) takes no policy.

varbinary columns support capacity-only changes (varbinary(M) → varbinary(N)); changing to or from varbinary is not supported.

Data migrations (seeding & transformation)

Between schema statements you can interleave INSERT, UPDATE and DELETE statements to seed or transform data as part of a migration. They run in file order when a device upgrades, each operating on the schema defined by all preceding statements — so a backfill UPDATE can target a column added by an ALTER TABLE directly above it. Values must be literals (no @parameters — those are only for QUERY definitions).

-- Backfill a newly added column on existing rows
ALTER TABLE Device ADD Region uint8 DEFAULT 0;
UPDATE Device SET Region = 1 WHERE Id < 1000;

-- Seed a row, prune rows
INSERT INTO Device (Id, Name) VALUES (9001, "gateway");
DELETE FROM Device WHERE Region = 0;

-- ON MISSING controls FK references whose parent row does not exist:
--   FAIL (abort the migration) | SKIP (drop this row) | NULL (null the FK column)
INSERT INTO Reading (Id, DeviceId, Value) VALUES (1, 9001, 0.0) ON MISSING SKIP;

Like other schema changes, applying data migrations to a deployed device requires the Pro edition.

QUERY

QUERY <name>
    SELECT <cols> FROM <table> [WHERE ...] [ORDER BY ...] [LIMIT ...];

Named queries are compiled into typed C99 iterator functions. Parameters are declared with @name type syntax (e.g. @id uint32) and become typed function arguments.

Integer overflow wraps in two's complement and a division by zero is NULL; neither is an error. A query's _next returns GEDB_WARN_ARITHMETIC instead of GEDB_OK for a row one of whose values wrapped or divided by zero, and returns the row as usual, so a row loop continues on it: while ((st = gedb_query_q_next(txn, &it, &row)) == GEDB_OK || st == GEDB_WARN_ARITHMETIC). An overflow in the WHERE, ON, HAVING or ORDER BY key that kept a row warns on that row; a row a clause dropped carries nothing. A CAST to an integer type that wraps or saturates warns too. A compiled DML statement returns the warning after its writes succeeded, including when it stores a value its integer column does not hold.

UNION and UNION ALL

A QUERY may combine two or more SELECT branches with UNION ALL (keep every row) or UNION (drop duplicates). The branches compile into a single iterator — there is no temporary table and no allocation; whatever the shape needs is part of sizeof(gedb_query_..._iter_t) and fixed at code-generation time. A trailing ORDER BY, LIMIT or OFFSET binds to the whole compound, as in SQL, and all three are accepted on every UNION shape.

QUERY recent_or_critical
    SELECT * FROM fault_log WHERE dtc_code = @dtc uint32
    UNION ALL
    SELECT * FROM fault_log WHERE severity >= @sev uint8
    ORDER BY ts_ms DESC LIMIT 20;

When every branch reads the same table, the result row is the projection its branches select — the columns named, in order, with a <field>_is_null flag for a nullable one, exactly as a plain SELECT of those columns yields. Two spellings keep that table's own gedb_<table>_row_t instead: a lone SELECT *, which selects every column and so already is the table row, and branch lists that differ from one another — the projection is applied once, from the first branch's list, so a later branch's row has no labelling it could be given. A * standing beside other items is written out as the columns it stands for, in every branch at once, so such a compound projects like any other. Branches that read different tables project a result row of their own; see below.

What a same-table UNION deduplicates on is a separate question from what it returns. When every branch selects the table's primary key, the branches merge in primary-key order and a repeated key is dropped: within one table equal keys are the same row and distinct keys differ in the tuple, so keying on the primary key is SQL's deduplication on the selected tuple. When any branch omits the key, that equivalence breaks — two rows can agree on every selected column and still have different keys, and SELECT Cat FROM T UNION SELECT Cat FROM T must return one row per distinct Cat, not one per id — so such a compound runs the same reconciled cursor the cross-table shapes use: it sorts on the projected tuple and drops a repeat of it. UNION ALL deduplicates nothing, and is unaffected either way.

UNION and UNION ALL have equal precedence and associate left to right, so a compound that mixes them means what SQL says it means: A UNION B UNION ALL C is (A UNION B) UNION ALL C — A and B are deduplicated, then every row of C is appended, including rows that duplicate one already emitted. A UNION ALL B UNION C is (A UNION ALL B) UNION C, where the outermost junction deduplicates and the result has no duplicates at all. In general the compound splits at its last deduplicating junction: everything left of it is one deduplicated group, everything after it is appended untouched. Mixing junctions costs a compound nothing else: it runs whichever cursor the same compound would run without the mix — the same-table one, the reconciled one when the deduplicated group omits the primary key, or the cross-table one — with each buffered row tagged by the branch that produced it.

How each shape runs

Only the streaming append is single-pass and O(1). The other two fill a bounded buffer and re-scan every branch on each pass, which the CLI reports as a performance caveat:

Query 'recent_or_critical' uses two-phase multi-pass UNION ALL with ORDER BY (buffer 2526 rows
per phase). Every branch is re-scanned on each outer pass. Consider an index whose leading
columns match the ORDER BY to reduce passes.

Branches over different tables

Branches may read different tables. Each branch scans its own table and projects into one common result row — a struct of its own rather than any table's row:

-- given: CREATE TABLE service_log (
--            id uint64 PRIMARY KEY NOT NULL, ts_ms uint64 NOT NULL, action_code uint32 NOT NULL);
QUERY event_timeline
    SELECT ts_ms AS at_ms, dtc_code AS code FROM fault_log
    UNION ALL
    SELECT ts_ms, action_code FROM service_log;

/* generated result row */
typedef struct {
    uint64_t at_ms;
    uint32_t code;
} gedb_query_event_timeline_row_t;

• Explicit lists of equal length. * is rejected across tables — it has no shape the branches can share.
• Types match by position — type and array length, so int32[4] and int32[2] do not match, and neither do int32 and uint32.
• Nullability may differ. A position nullable in any branch is nullable in the result and gains a <field>_is_null flag; a branch whose own column is NOT NULL stores false into it.
• Names come from the first branch — its AS alias, else the column name.
• Row order is the concatenation's own: branch by branch, and within a branch in primary-key order. Rows that tie on an ORDER BY key keep that order.
• ORDER BY must name a selected column. What gets sorted is the projection, and no table row survives to sort by — SQL requires the same of a compound query.

A position where any branch selects a computed item projects a synthetic scalar slot instead — int64_t for general expressions, double for float-valued ones, uint64_t for a bare uint64 column — and every branch's item at that position, plain columns included, is evaluated into that slot. The branches must agree on the slot's value domain, and aggregates, window functions and whole array / char / varbinary values cannot enter one.

Cross-table branches that neither deduplicate nor sort stream in O(1), as event_timeline above does. With either, the query runs a multi-pass bounded sort over the projected rows — and since the sort key is that projected row, no index can serve it. Appending ORDER BY at_ms DESC to the query above changes its caveat to:

Query 'event_timeline' uses multi-pass bounded UNION over the projected rows (buffer 7893 rows). Every branch
is re-scanned on each pass, and the sort key is the projected row — no index can serve it.
Consider UNION ALL without ORDER BY to stream instead.

A branch — the leading SELECT included — may also carry its own WHERE, DISTINCT, GROUP BY with aggregates and its own HAVING, and its own JOIN. A joined branch runs as an internal sub-cursor that the UNION drives, and the whole sub-cursor counts against --max-stack: such a query needs a budget several scan iterators larger than the equivalent scan-only UNION.

A branch may also carry a window function, which likewise runs as an internal sub-cursor. The window is over the branch, not the UNION — SQL composes a compound query from independent SELECTs — so each branch's ROW_NUMBER() counts its own rows from 1, and since a branch carries no ORDER BY of its own, every row is a peer: ROW_NUMBER() numbers arrival order and RANK() is 1 throughout.

A branch may carry a window function and its own JOIN — one sub-cursor does both, and what it numbers is the joined pairs: an outer row matching three inner rows contributes three separately numbered rows. The leading SELECT is no exception: its join, WHERE and window call move into branch 0's plan, so it streams and numbers its pairs exactly as a later branch does.

A branch may likewise carry SELECT DISTINCT over its own JOIN, in any position. The dedup is the UNION's, not the branch's: under UNION the set semantics already remove everything it could, and under UNION ALL the multi-pass cursor clusters that branch's equal tuples — so the duplicates the join manufactured within the branch go, while the same tuple in another branch survives.

Shapes that are reported instead

Every combination the bounded-memory generator cannot serve is reported at code-generation time, naming the branch and the clause responsible. Among them:

• Same-table branches with a computed select item must use identical select lists — the projection is applied once, from the first branch's list. A deduplicating compound that does not select the primary key is the exception: it runs the reconciled cursor, which evaluates each branch's own item into the slot, so its lists need only reconcile (equal length, agreeing value domain per position).
• A branch carrying both a JOIN and GROUP BY runs under UNION ALL without ORDER BY; deduplicating or sorting that shape is reported.
• ROW_NUMBER / RANK over a deduplicating UNION of grouped queries — under UNION ALL each branch numbers its own groups, but the duplicate test cannot know the number an earlier branch gave a group.
• A branch other than the last carrying its own ORDER BY. A trailing ORDER BY is not this case: it binds to the whole compound and is supported.

Limits & Capacity

All limits are compile-time constants derived from --block-size, --block-count, and the schema. There is no dynamic memory allocation; every number below is fixed at code-generation time.

Flash Device

Metadata commits are appended within the current metadata replica group rather than overwriting it. The group is only erased when its slots fill up and the ring advances, reducing metadata erase pressure by roughly the slot count.

RAM (static, inside gedb_db_t)

There is no dynamic allocation — every component above is fixed at code-generation time and lives inside gedb_db_t. Two knobs dominate the total: the cache and the per-level traversal scratch grow with the logical block size, while the bitmaps grow with the logical block count — so a large device divided into many small blocks spends more on bitmaps, and few large blocks spend more on cache and scratch. Run --memory to print the exact byte breakdown for a given configuration, or read sizeof(gedb_db_t) at compile time.

With a 1 MiB --cache-size, gedb_db_t exceeds 1 MiB — too large for the stack. Allocate it as a file-scope static or on the heap:

static gedb_db_t g_db;          /* file-scope static — zero-initialised, safe on any platform */

gedb_db_t *db = malloc(sizeof(gedb_db_t));  /* heap — requires free() later */

Schema Limits

I/O Layer

The generated runtime never touches your storage medium directly. Every storage access goes through three callbacks you provide to gedb_init: read, erase, and write. What follows is the contract those callbacks must honour, with the rationale for each clause — so you know which corners you can safely cut.

Callback contract

typedef gedb_status_t (*gedb_read_block_fn)(uint64_t block_id, uint32_t offset,
                                            void *buf, uint32_t len, void *ctx);
typedef gedb_status_t (*gedb_erase_block_fn)(uint64_t block_id, void *ctx);
typedef gedb_status_t (*gedb_write_block_fn)(uint64_t block_id, uint32_t offset,
                                             const void *data, uint32_t len, void *ctx);

• block_id: 0-indexed block number, in [0, GEDB_BLOCK_COUNT).
• offset: byte offset within the block, in [0, GEDB_BLOCK_SIZE).
• len: byte count for the access. offset + len ≤ GEDB_BLOCK_SIZE always holds.
• ctx: the io_ctx pointer you passed to gedb_init. Use it to thread your driver state — the runtime treats it as opaque.
• Return GEDB_OK (zero) on success. Any non-zero return is treated as an I/O failure immediately — the runtime does not retry. If you want to absorb a transient glitch, retry inside your own write_fn/erase_fn; that is where retry policy belongs. A failed write or erase retires the physical block on the first failure: it lands on the persistent bad-block list and the allocator skips it for the rest of the device's life.
• Per-replica failures are tolerated. As long as at least one replica in a block group accepts the write or erase, the operation succeeds and the transaction commits. Surviving replicas serve future reads; the lagging replica is queued for repair and restored to full redundancy by the next gedb_repair_step call. A transaction only fails when every replica of a block fails simultaneously — a real device-wide hardware fault, not a single bad block.

Under page-granular mode (GEDB_PROGRAM_PAGE_SIZE > 0), every write_fn call is page-aligned and page-sized: offset % PAGE_SIZE == 0 and len % PAGE_SIZE == 0. Under byte-granular mode (the default for FRAM, MRAM, or battery-backed SRAM), writes can be any size from 1 byte up to one full block.

Erase

The runtime calls erase_fn before reusing a block. What that needs to do depends on your storage medium:

• File-backed, FRAM, MRAM, battery-backed SRAM — direct overwrite is supported, so erase_fn can just return GEDB_OK.
• NOR / NAND flash — must physically erase the block so the subsequent write_fn sees all 0xFF cells. Return GEDB_OK only after the erase has completed.

Write may verify with read-back

The runtime trusts write_fn's return value: GEDB_OK means the data is durable enough that the next read_fn for the same (block_id, offset, len) returns the same bytes. If your hardware needs read-back verification to achieve that guarantee — for example, programming a NAND page and reading it back to confirm bit settling — do it inside write_fn. The runtime doesn't care how you achieve durability, only that the contract holds.

Conversely, fire-and-forget writes are fine if your medium reports failures synchronously. The runtime's per-record CRC catches silent corruption on subsequent reads and queues the block for repair (healed by the next gedb_repair_step call) regardless of which strategy your write_fn uses.

Custom ECC and integrity

GreatEmbeddo already CRC-protects every record and replicates blocks per each table's REPLICAS setting. If your medium needs additional protection — NAND with mandatory parity bytes, raw flash with read-disturb at scale, exotic radiation-hard memory — layer that protection underneath the callbacks. Store ECC bytes in a separate spare area, fix correctable errors during read_fn, and surface uncorrectable errors by returning a non-zero status.

The runtime treats your callback's view of the medium as the ground truth. As long as write_fn(b, 0, data, BLOCK_SIZE) followed by read_fn(b, 0, buf, BLOCK_SIZE) returns the same bytes, the runtime doesn't care what mechanism you use to achieve that round-trip. ECC, wear-aware retry, partial-page caching, write-leveling — all invisible above the callback boundary.

Concurrency is not your problem

The runtime serialises every callback invocation via an internal I/O lock. Under every lock strategy (--lock mtx, --lock pthread, --lock spin), the three callbacks are never invoked concurrently — your driver does not need to be thread-safe. A single, plain, single-threaded driver is sufficient even when the database is shared across many threads. --lock spin (a CAS busy-wait spinlock) provides this without any OS thread library, for bare-metal or RTOS targets.

The same guarantee covers concurrent gedb_transaction_read callers: every reader serialises with the writer — and with every other reader — through the I/O lock at the callback boundary, even though readers do not acquire the writer lock.

API Reference

All functions return gedb_status_t. Always check the return value.

Initialisation

gedb_db_t is a plain struct whose size is known at compile time, but it is too large for the stack (typically >1 MiB with a 1 MiB cache). Allocate it as a file-scope static or on the heap, then pass a pointer to gedb_init. gedb_init takes I/O callbacks and optional notification callbacks as individual parameters. It never formats or erases the device on its own: when it finds no valid database (blank flash, or a device formatted by a different database) it returns GEDB_NEEDS_FORMATTING, and the application decides whether to call gedb_format. It returns GEDB_ERR_SCHEMA if the flash was formatted by different generated code.

static gedb_db_t db;   /* file-scope static — do not declare gedb_db_t on the stack */
gedb_status_t st = gedb_init(&db,
    my_flash_read, my_flash_erase, my_flash_write,
    &flash_ctx,   /* io_ctx — passed to every I/O callback */
    NULL);         /* corruption_fn — optional: announces detected damage, repairs and data loss */
if (st == GEDB_NEEDS_FORMATTING) {
    /* No valid database found. Format with a caller-chosen non-zero 64-bit
       database id (mixed into every record's CRC scope, so a later reformat
       with a new id instantly invalidates old data — no erase pass needed). */
    st = gedb_format(&db, my_flash_read, my_flash_erase, my_flash_write,
                     &flash_ctx, NULL, my_database_id);
}
if (st != GEDB_OK) { /* handle error */ }

With the Pro version, gedb_init automatically detects an older on-disk schema version and applies generated migrations before returning. In the Free version, GEDB_ERR_SCHEMA indicates a hard mismatch between the on-disk data and the generated code.

Transactions

GreatEmbeddo provides two transaction functions: gedb_transaction_write for mutations and gedb_transaction_read for read-only queries. Both take a callback; return GEDB_OK from the callback to commit (or complete), any other value to roll back. The one exception is GEDB_WARN_ARITHMETIC, a compiled statement's warning that a value it wrote or stored wrapped: gedb_transaction_write commits on it and returns it.

typedef gedb_status_t (*gedb_txn_fn)(gedb_txn_t *txn, void *user_ctx);

gedb_status_t gedb_transaction_write(gedb_db_t *db, gedb_txn_fn fn, void *user_ctx);
gedb_status_t gedb_transaction_read(gedb_db_t *db, gedb_txn_fn fn, void *user_ctx);

Write transactions

All writes must occur inside gedb_transaction_write. At most one writer may be active at a time. With --lock mtx or --lock pthread, callers block until the writer lock is available using the OS thread library. With --lock spin, callers block by busy-waiting (CAS spinlock) — suitable for bare-metal or RTOS environments where no OS thread library is available.

Nested write transactions are not supported. Calling gedb_transaction_write from inside another gedb_transaction_write callback on the same thread returns GEDB_ERR_REENTRANT under --lock mtx and --lock pthread. Under --lock spin there is no thread identity, so the same mistake deadlocks on the writer-lock reacquire. Perform all further writes through the gedb_txn_t * passed to your callback; do not open a new transaction from within one.

/* Example: write transaction */
static gedb_status_t do_insert(gedb_txn_t *txn, void *ctx) {
    gedb_fault_log_row_t *row = (gedb_fault_log_row_t *)ctx;
    return gedb_fault_log_insert(txn, row);  /* non-GEDB_OK triggers rollback */
}

gedb_fault_log_row_t row = { .id = 1, .dtc_code = 0xC0DE, .ts_ms = now_ms() };
gedb_status_t r = gedb_transaction_write(&db, do_insert, &row);
if (r != GEDB_OK) { /* handle error — rollback already happened */ }

Batch inserts

To insert multiple rows atomically, call _insert in a loop inside a single gedb_transaction_write callback. All rows are committed together; if any insert fails, the entire batch is rolled back. This is also faster than inserting one-by-one because the writer lock is acquired once and only one superblock-update record is committed.

/* Example: insert three rows atomically — all-or-nothing */
static gedb_status_t do_batch_insert(gedb_txn_t *txn, void *ctx) {
    (void)ctx;
    gedb_fault_log_row_t row;
    memset(&row, 0, sizeof(row));
    gedb_status_t st;

    row.dtc_code = 0xC0DE; row.ts_ms = 1000;
    st = gedb_fault_log_insert(txn, &row);
    if (st != GEDB_OK) return st;

    row.dtc_code = 0xBEEF; row.ts_ms = 2000;
    st = gedb_fault_log_insert(txn, &row);
    if (st != GEDB_OK) return st;

    row.dtc_code = 0xCAFE; row.ts_ms = 3000;
    st = gedb_fault_log_insert(txn, &row);
    if (st != GEDB_OK) return st;

    return GEDB_OK;  // commit all three rows atomically
}

gedb_status_t r = gedb_transaction_write(&db, do_batch_insert, NULL);
if (r != GEDB_OK) { /* handle error — all three rows were rolled back */ }

Read transactions

gedb_transaction_read is lock-free with respect to the writer lock: it never acquires the writer lock and never blocks other readers or writers. It captures a consistent snapshot of the database at the moment the callback begins. All reads within the callback — including multi-step scans — see this frozen snapshot. Concurrent writes are invisible for the entire duration of the callback.

Flash access is protected by a separate internal IO lock. Under every lock strategy (--lock mtx, --lock pthread, or --lock spin), the IO lock is held for the duration of each callback, so the three I/O functions (read_fn, erase_fn, write_fn) are never called concurrently — callers may use a single non-thread-safe driver.

/* Example: read transaction with snapshot isolation */
static gedb_status_t do_scan(gedb_txn_t *txn, void *ctx) {
    gedb_fault_log_iter_t iter;
    gedb_fault_log_scan_open(txn, &iter);
    gedb_fault_log_row_t row;
    gedb_status_t st;
    while ((st = gedb_fault_log_scan_next(txn, &iter, &row)) == GEDB_OK) {
        /* process row — guaranteed consistent snapshot */
    }
    return st == GEDB_DONE ? GEDB_OK : st;
}

gedb_status_t r = gedb_transaction_read(&db, do_scan, NULL);
if (r != GEDB_OK) { /* handle error */ }

Concurrency & Isolation

GreatEmbeddo is designed for concurrent access. Writers are serialised by an exclusive lock; readers are fully lock-free. The isolation guarantees depend on how you access the data:

How it works

On every successful commit, the writer first flushes dirty block deltas or rare replacement blocks, then appends one committed metadata snapshot and publishes that snapshot via a seqlock. gedb_transaction_read copies the committed snapshot at the start of the callback and uses it for the entire duration. Most small writes append directly into live leaf/internal blocks; when a block fills, the writer falls back to a replacement-block path. An RCU-style epoch mechanism ensures retired blocks are not reclaimed while a reader could still observe the older committed snapshot.

Scan iterators (scan_open / scan_next) are always used inside one of the two transaction functions — they take the gedb_txn_t * that gedb_transaction_read or gedb_transaction_write passes to your callback, so the whole scan sees that transaction's single consistent snapshot.

CRUD Operations (generated per table)

All CRUD functions take the gedb_txn_t * passed to your transaction callback; call them inside gedb_transaction_write for writes or gedb_transaction_read for _get.

/* Insert a row. Returns GEDB_ERR_DUPLICATE if PK already exists. */
gedb_status_t gedb_fault_log_insert(gedb_txn_t *txn, const gedb_fault_log_row_t *row);

/* Get row by primary key. Returns GEDB_ERR_NOT_FOUND if absent. */
gedb_status_t gedb_fault_log_get(gedb_txn_t *txn, uint64_t id, gedb_fault_log_row_t *out_row);

/* Full replace of an existing row. Returns GEDB_ERR_NOT_FOUND if absent. */
gedb_status_t gedb_fault_log_update(gedb_txn_t *txn, const gedb_fault_log_row_t *row);

/* Delete a row by primary key. Returns GEDB_ERR_NOT_FOUND if absent. */
gedb_status_t gedb_fault_log_delete(gedb_txn_t *txn, uint64_t id);

/* Insert or replace (upsert). Always succeeds unless I/O error. */
gedb_status_t gedb_fault_log_upsert(gedb_txn_t *txn, const gedb_fault_log_row_t *row);

Scanning

Full-table scans use an iterator pattern: open a cursor, call next in a loop. No explicit close is needed (iterators are stack-allocated). Scans run inside gedb_transaction_read for snapshot isolation, or inside gedb_transaction_write when scanning as part of a mutation; both pass a gedb_txn_t * to your callback. See Concurrency & Isolation for details.

/* Inside a gedb_txn_fn callback that received gedb_txn_t *txn */
gedb_fault_log_iter_t iter;
gedb_status_t st = gedb_fault_log_scan_open(txn, &iter);
if (st != GEDB_OK) { /* handle error */ }

gedb_fault_log_row_t row;
while ((st = gedb_fault_log_scan_next(txn, &iter, &row)) == GEDB_OK) {
    /* process row */
}
if (st != GEDB_DONE) { /* handle error */ }

Corruption & Repair

Reads and writes never perform repair work — their latency stays predictable. When an operation encounters a damaged block, a lagging replica, or a corrupt secondary-index tree, it detects and queues the damage, announces it through corruption_fn, and degrades gracefully: reads return what is recoverable, writes that would need the damaged structure fail with GEDB_ERR_CORRUPT_NEEDS_REPAIR.

A read announces its loss once, however many rows it had to skip: a scan or query raises one GEDB_CORRUPTION_PARTIAL_RESULT (or GEDB_CORRUPTION_DATA_LOSS) for its whole run. That holds for a query over several tables too: a join, a subquery or a multi-pass sort that crosses damage in two tables reports once and names the first damaged table it met. The other damage is not lost: every damaged tree the query touched is queued for repair, and repair announces each table's loss when it removes it.

All healing happens inside gedb_repair_step — your firmware must call it, typically from a low-priority idle task. Each call first drains queued damage (known-corrupt blocks, then one flagged index/FK rebuild), then advances a bounded background sweep: block scrub, primary-tree walk, index verification. The work per call is bounded, so it never stalls the idle loop. gedb_repair_full loops the step to quiescence — use it after corruption_fn fires or an operation returns GEDB_ERR_CORRUPT_NEEDS_REPAIR, then retry the failed operation.

/* Idle task: one bounded unit of repair work per iteration. */
void idle_task(void) {
    gedb_repair_step(&db);
}

/* After a corruption callback or GEDB_ERR_CORRUPT_NEEDS_REPAIR:
   restore full redundancy, then retry the failed operation. */
gedb_status_t st = gedb_repair_full(&db);

Both functions run concurrently with readers and block on the writer lock like any other write. They return GEDB_OK even when irrecoverable blocks had to be freed — each loss is announced through corruption_fn as GEDB_CORRUPTION_DATA_LOSS, never hidden in a return code. If the flash keeps failing repair's own writes, gedb_repair_full ends early and announces each damaged block it could not remove the same way, then returns GEDB_OK. A later call removes those blocks once the flash works again, without announcing them a second time. A database that never calls gedb_repair_step keeps working, but detected damage stays degraded: lagging replicas are not restored to full redundancy, and writes touching a corrupt index keep failing until repair runs.

Rust Crate

--rust makes the output directory a Cargo crate as well: next to the C files it writes Cargo.toml, build.rs and lib.rs. Depend on it by path — gedb = { path = "generated" } — and use it from hosted or bare-metal Rust (1.81 or later).

let mut storage = DbStorage::boxed();   // gedb_db_t, zeroed, never moved
let mut flash = RamFlash::new();        // or your own impl gedb::Flash
let id = NonZeroU64::new(0x5EED).unwrap();
let db = Database::open_or_format(&mut storage, &mut flash, id)?;

db.write(|txn| {
    let mut row = gedb::tables::SensorRow::default(); // schema DEFAULTs, else NULL / zero
    row.set_id(1);
    row.set_name(b"pump")?;
    row.set_value(21.5);
    txn.sensor_insert(&row)                           // Err rolls the transaction back
})?;

let ids = db.read(|txn| {
    let mut iter = IterStorage::new();                // the C _iter_t, on the stack
    txn.query_get_by_name(&mut iter, b"pump")?
        .map(|row| row.map(|row| row.id()))
        .collect::<Result<Vec<_>, Error>>()
})?;

• No bindgen. build.rs compiles the generated C with the cc crate, the only dependency — the Pro build's migration files included, so Database::open migrates just as gedb_init does.
• Layout from the target's C compiler. build.rs writes gedb_rust_layout.c, a table of every size, alignment and member offset the Rust side relies on, into OUT_DIR (never beside the C), compiles it to an object file and reads it back — cross-compiling included. lib.rs sizes gedb_db_t and the iterators from it and checks every row struct against it at compile time.
• The API. gedb::sys declares the C functions as generated. The safe layer has a Flash trait for the I/O callbacks (and a defaulted corruption callback), read and write transactions as closures, <table>_get, _scan, _insert, _update, _upsert, _delete, and a query_<name> per query: an Iterator of rows for a SELECT, a statement for DML. _get_fingered speeds up a run of nearby keys through a PkFinger, and Database has the _auto functions, each in a transaction of its own.
• Statuses. Negative ones are Err(Error). A warning such as GEDB_WARN_ARITHMETIC is a Warning, never an error: a statement returns it, a cursor reports it for its row through warning().
• Rows keep the C layout, with an accessor per column: Option<T> for a nullable one, &[u8] for char[N] and varbinary, &[T; N] for an array. A table row's Default holds the schema's DEFAULTs, NULL or zero elsewhere.
• no_std without the default std feature, with no allocation: the DbStorage goes in a static, each cursor's IterStorage on the stack. Generate with --lock spin for a target without <threads.h>.
• Threads. Read transactions run lock-free beside each other and a writer, so read, write and the _auto functions take &self and Database is Send and Sync when its device is Send — with std, or in a --lock spin build. A device callback that calls back into its database gets Err(Error::Reentrant).

Migration Recovery

With the Pro version, gedb_init() transparently upgrades an older on-disk schema before returning. When a migration cannot complete, the function returns an error and the device stays on its previous schema version.

Handling migration results in C

static gedb_db_t db;   /* file-scope static — do not declare gedb_db_t on the stack */
gedb_status_t st = gedb_init(&db, my_read, my_erase, my_write, NULL, NULL);
switch (st) {
    case GEDB_OK:
        /* Database is open and at the latest schema version. */
        break;
    case GEDB_NEEDS_FORMATTING:
        /* No valid database on the device (e.g. blank flash). Format it,
           then it is ready at the latest schema version. */
        st = gedb_format(&db, my_read, my_erase, my_write, NULL, NULL, my_database_id);
        break;
    case GEDB_ERR_SCHEMA:
        /* On-disk schema is from a future or unrecognised version.
           This device needs firmware that matches its schema. */
        log_error("schema mismatch — cannot migrate");
        break;
    default:
        /* I/O error during migration. The transaction was rolled back;
           the database is still at the old schema version and intact.
           Old firmware can still boot this device. */
        log_error("migration failed: %d", st);
        break;
}

Correcting a schema mistake

The .gedb file is append-only: once a line has been generated and the .gedb.lock file is updated, that line cannot be edited. To undo a mistake, append corrective statements below the problem line:

-- Mistake (already locked)
ALTER TABLE sensors ADD bad_col float;

-- Fix — append this
ALTER TABLE sensors DROP COLUMN bad_col;

The CLI generates a migration path that walks every intermediate version in order. Devices that already ran the broken migration drop the column; devices on older schema versions also pass through the bad intermediate step — adding then dropping the column — on their way to the latest version. Either way the device ends up at the correct final schema.

--skip-lock-check

As a last resort, you can bypass the append-only checksum chain with --skip-lock-check. This lets you edit or remove locked lines and regenerate, but all deployed devices must receive updated firmware — the old checksum chain is no longer valid. The CLI prints a warning when this flag is active.

After using --skip-lock-check, delete the .gedb.lock file and regenerate normally to establish a fresh checksum chain.