Getting Started

GreatEmbeddo generates a complete, self-contained C99 database for your embedded system from a .gedb schema file. There is no runtime library to link — the generated code is the database.

Follow these four steps to integrate GreatEmbeddo into a project:

1. Write a .gedb schema describing your tables and indexes.
2. Run the greatembeddo CLI to generate C99 source files.
3. Add the generated .c and .h files to your build.
4. Implement the three I/O callbacks (read block, erase block, write block) for your storage.

Schema Example

A typical myproject.gedb file for an automotive ECU storing fault codes:

-- myproject.gedb
-- Schema for ECU fault log

CREATE TABLE ecu_config (
    id          uint32 PRIMARY KEY NOT NULL,
    key         char[32] NOT NULL,
    value_i32   int32,
    value_f     float
) EXPECT 64;

CREATE TABLE fault_log (
    id          uint64 PRIMARY KEY NOT NULL,
    ts_ms       uint64 NOT NULL,
    dtc_code    uint32 NOT NULL,
    severity    uint8  NOT NULL DEFAULT 0,
    freeze_data uint8[16]
) EXPECT 4096 REPLICAS 2;

CREATE TABLE calibration (
    param_id    uint16 PRIMARY KEY NOT NULL,
    table_id    uint8  NOT NULL,
    value       float  NOT NULL,
    min_val     float  NOT NULL,
    max_val     float  NOT NULL
) EXPECT 512;

CREATE INDEX idx_fault_dtc  ON fault_log (dtc_code);
CREATE INDEX idx_cal_table  ON calibration (table_id);

QUERY get_faults_by_dtc
    SELECT * FROM fault_log WHERE dtc_code = @dtc uint32 ORDER BY ts_ms DESC;

CLI Usage

> greatembeddo <schema.gedb> [options]

Options:
  -o, --output <dir>       Output directory (default: current directory)
  --block-size <bytes>     Flash block size, power of 2 (default: 16384)
  --cache-size <bytes>     RAM cache size in bytes (default: 1048576)
  --block-count <n>        Total number of blocks on the flash device (required)
  --metadata-replicas <n>  Metadata replication factor (default: 3, max 4)
  --lock <strategy>        Writer-lock strategy: mtx (default), pthread, spin, none
  --stats                  Print storage statistics only, do not generate code
  --memory                 Print static RAM breakdown only, do not generate code
  --caveats                Print performance caveats only, do not generate code
  --yes / -y               Auto-accept all performance caveats
  --strict                 Abort (exit 1) if any caveats exist
  --skip-lock-check        Skip the .gedb.lock append-only check (last resort)
  --quiet                  Suppress all output except errors
  -h, --help               Show this help message

Migration generation is available in the Pro version.

Example — generate code for a 4 MiB NOR flash with 4 KiB blocks:

> greatembeddo myproject.gedb \
    --output src/generated/ \
    --block-size 4096 \
    --block-count 1024 \
    --cache-size 65536

Check storage requirements before generating:

> greatembeddo myproject.gedb --stats --block-count 1024 --block-size 4096

Schema version : 5
Tables         : 3
Block size     : 4096 bytes
Metadata blocks: 35

Table                Rows  Row bytes  Blocks (est)  Replicas
-----------------------------------------------------------------
ecu_config             64         52             2         1
fault_log            4096         40           165         2
calibration           512         22             5         1
-----------------------------------------------------------------
Total (estimated)                               207

Schema Reference

The .gedb schema language is a subset of SQL DDL extended with embedded-specific annotations. All identifiers are case-insensitive. Keywords are uppercase by convention.

Column Types

CREATE TABLE

CREATE TABLE <name> (
    <col> <type> [PRIMARY KEY] [NOT NULL] [DEFAULT <value>]
                 [RANGE (<min>, <max>)]
                 [REFERENCES <table>(<col>) [ON DELETE RESTRICT|CASCADE|SET NULL]],
    ...
) [EXPECT <rows>] [REPLICAS <n>];

• EXPECT n — hint for storage sizing; has no effect on correctness
• REPLICAS n — number of redundant copies stored on flash (default 2 for tables, 1 for indexes; max 4)
• RANGE(min, max) — inserts/updates with an out-of-range value are rejected with GEDB_ERR_PARAM
• Exactly one PRIMARY KEY column is required per table

CREATE INDEX / CREATE UNIQUE INDEX

CREATE INDEX <name> ON <table> (<col> [, <col> ...]);
CREATE UNIQUE INDEX <name> ON <table> (<col> [, <col> ...]);

-- When adding a UNIQUE index to an existing table (migration), ON CONFLICT is required:
CREATE UNIQUE INDEX <name> ON <table> (<col> [, ...]) ON CONFLICT REMOVE_DUPLICATES;
CREATE UNIQUE INDEX <name> ON <table> (<col> [, ...]) ON CONFLICT REMOVE_ALL_DUPLICATES;

• Composite indexes are supported
• UNIQUE indexes enforce a uniqueness constraint at insert/update time
• ON CONFLICT REMOVE_DUPLICATES — keep the row with the lowest primary key per duplicate group; delete the rest
• ON CONFLICT REMOVE_ALL_DUPLICATES — delete every row in each duplicate group

Schema Changes

Schema-changing commands (ALTER TABLE, DROP TABLE, DROP COLUMN, DROP INDEX) are always permitted in the .gedb file and validated at code-generation time. Applying them to an existing database on a deployed device requires the Pro edition, which generates a migration function that upgrades the on-flash data in place.

-- Add a column (default value required for existing rows)
ALTER TABLE <name> ADD <col> <type> [DEFAULT <value>] [NOT NULL] ...;

-- Change a column's type (ON NARROW required for narrowing conversions)
ALTER TABLE <name> ALTER <col> <new-type>;
ALTER TABLE <name> ALTER <col> <new-type> ON NARROW CLAMP;
ALTER TABLE <name> ALTER <col> <new-type> ON NARROW WRAP;
ALTER TABLE <name> ALTER <col> <new-type> ON NARROW NULL;

-- Update expected row count hint
ALTER TABLE <name> EXPECT <n>;

-- Change replication factor
ALTER TABLE <name> REPLICAS <n>;

DROP TABLE <name>;
ALTER TABLE <name> DROP COLUMN <column>;
DROP INDEX <name>;

Narrowing policies apply when the new type cannot represent all values of the old type (e.g. int32 → int16, float → int32):

• ON NARROW CLAMP — clamp values to the min/max of the new type
• ON NARROW WRAP — truncate via C cast (modular arithmetic)
• ON NARROW NULL — set out-of-range values to null (column must be nullable)

Widening conversions (e.g. int16 → int32, float → double) do not require ON NARROW. Primary key columns cannot be altered. varbinary columns support capacity-only changes (varbinary(M) → varbinary(N)); changing to or from varbinary is not supported.

Data migrations (seeding & transformation)

Between schema statements you can interleave INSERT, UPDATE and DELETE statements to seed or transform data as part of a migration. They run in file order when a device upgrades, each operating on the schema defined by all preceding statements — so a backfill UPDATE can target a column added by an ALTER TABLE directly above it. Values must be literals (no @parameters — those are only for QUERY definitions).

-- Backfill a newly added column on existing rows
ALTER TABLE Device ADD Region uint8 DEFAULT 0;
UPDATE Device SET Region = 1 WHERE Id < 1000;

-- Seed a row, prune rows
INSERT INTO Device (Id, Name) VALUES (9001, "gateway");
DELETE FROM Device WHERE Region = 0;

-- ON MISSING controls FK references whose parent row does not exist:
--   FAIL (abort the migration) | SKIP (drop this row) | NULL (null the FK column)
INSERT INTO Reading (Id, DeviceId, Value) VALUES (1, 9001, 0.0) ON MISSING SKIP;

Like other schema changes, applying data migrations to a deployed device requires the Pro edition.

QUERY

QUERY <name>
    SELECT <cols> FROM <table> [WHERE ...] [ORDER BY ...] [LIMIT ...];

Named queries are compiled into typed C99 iterator functions. Parameters are declared with @name type syntax (e.g. @id uint32) and become typed function arguments.

Limits & Capacity

All limits are compile-time constants derived from --block-size, --block-count, and the schema. There is no dynamic memory allocation; every number below is fixed at code-generation time.

Flash Device

Metadata commits are appended within the current metadata replica group rather than overwriting it. The group is only erased when its slots fill up and the ring advances, reducing metadata erase pressure by roughly the slot count.

RAM (static, inside gedb_db_t)

There is no dynamic allocation — every component above is fixed at code-generation time and lives inside gedb_db_t. Two knobs dominate the total: the cache and the per-level traversal scratch grow with the logical block size, while the bitmaps grow with the logical block count — so a large device divided into many small blocks spends more on bitmaps, and few large blocks spend more on cache and scratch. Run --memory to print the exact byte breakdown for a given configuration, or read sizeof(gedb_db_t) at compile time.

At default settings gedb_db_t exceeds 1 MiB — too large for the stack. Allocate it as a file-scope static or on the heap:

static gedb_db_t g_db;          /* file-scope static — zero-initialised, safe on any platform */

gedb_db_t *db = malloc(sizeof(gedb_db_t));  /* heap — requires free() later */

Schema Limits

I/O Layer

The generated runtime never touches your storage medium directly. Every storage access goes through three callbacks you provide to gedb_init: read, erase, and write. What follows is the contract those callbacks must honour, with the rationale for each clause — so you know which corners you can safely cut.

Callback contract

typedef gedb_status_t (*gedb_read_block_fn)(uint64_t block_id, uint32_t offset,
                                            void *buf, uint32_t len, void *ctx);
typedef gedb_status_t (*gedb_erase_block_fn)(uint64_t block_id, void *ctx);
typedef gedb_status_t (*gedb_write_block_fn)(uint64_t block_id, uint32_t offset,
                                             const void *data, uint32_t len, void *ctx);

• block_id: 0-indexed block number, in [0, GEDB_BLOCK_COUNT).
• offset: byte offset within the block, in [0, GEDB_BLOCK_SIZE).
• len: byte count for the access. offset + len ≤ GEDB_BLOCK_SIZE always holds.
• ctx: the io_ctx pointer you passed to gedb_init. Use it to thread your driver state — the runtime treats it as opaque.
• Return GEDB_OK (zero) on success. Any non-zero return is treated as an I/O failure immediately — the runtime does not retry. If you want to absorb a transient glitch, retry inside your own write_fn/erase_fn; that is where retry policy belongs. A failed write or erase retires the physical block on the first failure: it lands on the persistent bad-block list and the allocator skips it for the rest of the device's life.
• Per-replica failures are tolerated. As long as at least one replica in a block group accepts the write or erase, the operation succeeds and the transaction commits. Surviving replicas serve future reads; the bad replica is repaired or replaced on a later access. A transaction only fails when every replica of a block fails simultaneously — a real device-wide hardware fault, not a single bad block.

Under page-granular mode (GEDB_PROGRAM_PAGE_SIZE > 0), every write_fn call is page-aligned and page-sized: offset % PAGE_SIZE == 0 and len % PAGE_SIZE == 0. Under byte-granular mode (the default for FRAM, MRAM, or battery-backed SRAM), writes can be any size from 1 byte up to one full block.

Erase

The runtime calls erase_fn before reusing a block. What that needs to do depends on your storage medium:

• File-backed, FRAM, MRAM, battery-backed SRAM — direct overwrite is supported, so erase_fn can just return GEDB_OK.
• NOR / NAND flash — must physically erase the block so the subsequent write_fn sees all 0xFF cells. Return GEDB_OK only after the erase has completed.

Write may verify with read-back

The runtime trusts write_fn's return value: GEDB_OK means the data is durable enough that the next read_fn for the same (block_id, offset, len) returns the same bytes. If your hardware needs read-back verification to achieve that guarantee — for example, programming a NAND page and reading it back to confirm bit settling — do it inside write_fn. The runtime doesn't care how you achieve durability, only that the contract holds.

Conversely, fire-and-forget writes are fine if your medium reports failures synchronously. The runtime's per-record CRC catches silent corruption on subsequent reads and triggers replica repair regardless of which strategy your write_fn uses.

Custom ECC and integrity

GreatEmbeddo already CRC-protects every record and replicates blocks per each table's REPLICAS setting. If your medium needs additional protection — NAND with mandatory parity bytes, raw flash with read-disturb at scale, exotic radiation-hard memory — layer that protection underneath the callbacks. Store ECC bytes in a separate spare area, fix correctable errors during read_fn, and surface uncorrectable errors by returning a non-zero status.

The runtime treats your callback's view of the medium as the ground truth. As long as write_fn(b, 0, data, BLOCK_SIZE) followed by read_fn(b, 0, buf, BLOCK_SIZE) returns the same bytes, the runtime doesn't care what mechanism you use to achieve that round-trip. ECC, wear-aware retry, partial-page caching, write-leveling — all invisible above the callback boundary.

Concurrency is not your problem

The runtime serialises every callback invocation via an internal I/O lock. With any of the OS-aware lock strategies (--lock mtx, --lock pthread, --lock spin), the three callbacks are never invoked concurrently — your driver does not need to be thread-safe. A single, plain, single-threaded driver is sufficient even when the database is shared across many threads.

With --lock none (intended for bare-metal single-threaded use), the runtime's internal CAS-based I/O guard provides the same guarantee: if a second caller would race with an in-flight callback, the second caller receives GEDB_ERR_BUSY immediately without entering the callback at all.

The same guarantee covers concurrent gedb_transaction_read callers: every reader serialises with the writer — and with every other reader — through the I/O lock at the callback boundary, even though readers do not acquire the writer lock.

API Reference

All functions return gedb_status_t. Always check the return value.

Initialisation

gedb_db_t is a plain struct whose size is known at compile time, but it is too large for the stack (typically >1 MiB at default settings). Allocate it as a file-scope static or on the heap, then pass a pointer to gedb_init. gedb_init takes I/O callbacks and optional notification callbacks as individual parameters. It automatically formats blank flash, refuses to silently erase an existing but corrupt device, and returns GEDB_ERR_SCHEMA if the flash was formatted by different generated code.

static gedb_db_t db;   /* file-scope static — do not declare gedb_db_t on the stack */
gedb_status_t st = gedb_init(&db,
    my_flash_read, my_flash_erase, my_flash_write,
    &flash_ctx,   /* io_ctx — passed to every I/O callback */
    NULL);         /* corruption_fn — optional: called when a replica repairs a block */
if (st != GEDB_OK) { /* handle error */ }

With the Pro version, gedb_init automatically detects an older on-disk schema version and applies generated migrations before returning. In the Free version, GEDB_ERR_SCHEMA indicates a hard mismatch between the on-disk data and the generated code.

Transactions

GreatEmbeddo provides two transaction functions: gedb_transaction_write for mutations and gedb_transaction_read for read-only queries. Both take a callback; return GEDB_OK from the callback to commit (or complete), any other value to roll back.

typedef gedb_status_t (*gedb_txn_fn)(gedb_db_t *db, void *ctx);

gedb_status_t gedb_transaction_write(gedb_db_t *db, gedb_txn_fn fn, void *user_ctx);
gedb_status_t gedb_transaction_read(gedb_db_t *db, gedb_txn_fn fn, void *user_ctx);

Write transactions

All writes must occur inside gedb_transaction_write. At most one writer may be active at a time. With --lock mtx or --lock pthread, callers block until the writer lock is available using the OS thread library. With --lock spin, callers block by busy-waiting (CAS spinlock) — suitable for bare-metal or RTOS environments where no OS thread library is available. With --lock none, the entire runtime is non-blocking: gedb_transaction_write returns GEDB_ERR_BUSY immediately if another writer is active, and any flash access that encounters a concurrent IO call also returns GEDB_ERR_BUSY without spinning.

Nested write transactions are not supported. Calling gedb_transaction_write from inside another gedb_transaction_write callback on the same thread will deadlock on the writer-lock reacquire — by design. Perform all further writes through the gedb_txn_t * passed to your callback; do not open a new transaction from within one.

/* Example: write transaction */
static gedb_status_t do_insert(gedb_db_t *db, void *ctx) {
    gedb_fault_log_row_t *row = (gedb_fault_log_row_t *)ctx;
    return gedb_fault_log_insert(db, row);  /* non-GEDB_OK triggers rollback */
}

gedb_fault_log_row_t row = { .id = 1, .dtc_code = 0xC0DE, .ts_ms = now_ms() };
gedb_status_t r = gedb_transaction_write(&db, do_insert, &row);
if (r != GEDB_OK) { /* handle error — rollback already happened */ }

Batch inserts

To insert multiple rows atomically, call _insert in a loop inside a single gedb_transaction_write callback. All rows are committed together; if any insert fails, the entire batch is rolled back. This is also faster than inserting one-by-one because the writer lock is acquired once and only one superblock-update record is committed.

/* Example: insert three rows atomically — all-or-nothing */
static gedb_status_t do_batch_insert(gedb_db_t *db, void *ctx) {
    (void)ctx;
    gedb_fault_log_row_t row;
    memset(&row, 0, sizeof(row));
    gedb_status_t st;

    row.dtc_code = 0xC0DE; row.ts_ms = 1000;
    st = gedb_fault_log_insert(db, &row);
    if (st != GEDB_OK) return st;

    row.dtc_code = 0xBEEF; row.ts_ms = 2000;
    st = gedb_fault_log_insert(db, &row);
    if (st != GEDB_OK) return st;

    row.dtc_code = 0xCAFE; row.ts_ms = 3000;
    st = gedb_fault_log_insert(db, &row);
    if (st != GEDB_OK) return st;

    return GEDB_OK;  // commit all three rows atomically
}

gedb_status_t r = gedb_transaction_write(&db, do_batch_insert, NULL);
if (r != GEDB_OK) { /* handle error — all three rows were rolled back */ }

Read transactions

gedb_transaction_read is lock-free with respect to the writer lock: it never acquires the writer lock and never blocks other readers or writers. It captures a consistent snapshot of the database at the moment the callback begins. All reads within the callback — including multi-step scans — see this frozen snapshot. Concurrent writes are invisible for the entire duration of the callback.

Flash access is protected by a separate internal IO lock that follows the same non-blocking contract as the writer lock strategy. With --lock mtx, --lock pthread, or --lock spin, the IO lock is always held for the duration of each callback, so the three I/O functions (read_fn, erase_fn, write_fn) are never called concurrently — callers may use a single non-thread-safe driver. With --lock none, the IO lock is a non-blocking CAS: if another IO call is already in progress the function returns GEDB_ERR_BUSY immediately. Callers using --lock none must either ensure single-threaded IO access or handle GEDB_ERR_BUSY from read and write transactions.

/* Example: read transaction with snapshot isolation */
static gedb_status_t do_scan(gedb_db_t *db, void *ctx) {
    gedb_fault_log_iter_t iter;
    gedb_fault_log_scan_open(db, &iter);
    gedb_fault_log_row_t row;
    gedb_status_t st;
    while ((st = gedb_fault_log_scan_next(db, &iter, &row)) == GEDB_OK) {
        /* process row — guaranteed consistent snapshot */
    }
    return st == GEDB_DONE ? GEDB_OK : st;
}

gedb_status_t r = gedb_transaction_read(&db, do_scan, NULL);
if (r != GEDB_OK) { /* handle error */ }

Concurrency & Isolation

GreatEmbeddo is designed for concurrent access. Writers are serialised by an exclusive lock; readers are fully lock-free. The isolation guarantees depend on how you access the data:

How it works

On every successful commit, the writer first flushes dirty block deltas or rare replacement blocks, then appends one committed metadata snapshot and publishes that snapshot via a seqlock. gedb_transaction_read copies the committed snapshot at the start of the callback and uses it for the entire duration. Most small writes append directly into live leaf/internal blocks; when a block fills, the writer falls back to a replacement-block path. An RCU-style epoch mechanism ensures retired blocks are not reclaimed while a reader could still observe the older committed snapshot.

Scan iterators (scan_open / scan_next) should always be used inside one of the two transaction functions. While bare calls outside a transaction are technically safe (the iterator falls back to per-call seqlock protection), each scan_next call would see an independently acquired snapshot, giving read-committed-per-row semantics — which is almost never what you want.

CRUD Operations (generated per table)

All CRUD functions take a gedb_db_t * directly; wrap calls in gedb_transaction_write for write operations.

/* Insert a row. Returns GEDB_ERR_DUPLICATE if PK already exists. */
gedb_status_t gedb_fault_log_insert(gedb_db_t *db, const gedb_fault_log_row_t *row);

/* Get row by primary key. Returns GEDB_ERR_NOT_FOUND if absent. */
gedb_status_t gedb_fault_log_get(gedb_db_t *db, uint64_t id, gedb_fault_log_row_t *out_row);

/* Full replace of an existing row. Returns GEDB_ERR_NOT_FOUND if absent. */
gedb_status_t gedb_fault_log_update(gedb_db_t *db, const gedb_fault_log_row_t *row);

/* Delete a row by primary key. Returns GEDB_ERR_NOT_FOUND if absent. */
gedb_status_t gedb_fault_log_delete(gedb_db_t *db, uint64_t id);

/* Insert or replace (upsert). Always succeeds unless I/O error. */
gedb_status_t gedb_fault_log_upsert(gedb_db_t *db, const gedb_fault_log_row_t *row);

Scanning

Full-table scans use an iterator pattern: open a cursor, call next in a loop. No explicit close is needed (iterators are stack-allocated). Wrap scans in gedb_transaction_read for snapshot isolation, or in gedb_transaction_write when scanning as part of a mutation. See Concurrency & Isolation for details.

gedb_fault_log_iter_t iter;
gedb_status_t st = gedb_fault_log_scan_open(&db, &iter);
if (st != GEDB_OK) { /* handle error */ }

gedb_fault_log_row_t row;
while ((st = gedb_fault_log_scan_next(&db, &iter, &row)) == GEDB_OK) {
    /* process row */
}
if (st != GEDB_DONE) { /* handle error */ }

Migration Recovery

With the Pro version, gedb_init() transparently upgrades an older on-disk schema before returning. When a migration cannot complete, the function returns an error and the device stays on its previous schema version.

Handling migration results in C

static gedb_db_t db;   /* file-scope static — do not declare gedb_db_t on the stack */
gedb_status_t st = gedb_init(&db, my_read, my_erase, my_write, NULL, NULL);
switch (st) {
    case GEDB_OK:
        /* Database is open and at the latest schema version. */
        break;
    case GEDB_ERR_SCHEMA:
        /* On-disk schema is from a future or unrecognised version.
           This device needs firmware that matches its schema. */
        log_error("schema mismatch — cannot migrate");
        break;
    default:
        /* I/O error during migration. The transaction was rolled back;
           the database is still at the old schema version and intact.
           Old firmware can still boot this device. */
        log_error("migration failed: %d", st);
        break;
}

Correcting a schema mistake

The .gedb file is append-only: once a line has been generated and the .gedb.lock file is updated, that line cannot be edited. To undo a mistake, append corrective statements below the problem line:

-- Mistake (already locked)
ALTER TABLE sensors ADD bad_col float;

-- Fix — append this
ALTER TABLE sensors DROP COLUMN bad_col;

The CLI generates a migration path that walks every intermediate version in order. Devices that already ran the broken migration drop the column; devices on older schema versions also pass through the bad intermediate step — adding then dropping the column — on their way to the latest version. Either way the device ends up at the correct final schema.

--skip-lock-check

As a last resort, you can bypass the append-only checksum chain with --skip-lock-check. This lets you edit or remove locked lines and regenerate, but all deployed devices must receive updated firmware — the old checksum chain is no longer valid. The CLI prints a warning when this flag is active.

After using --skip-lock-check, delete the .gedb.lock file and regenerate normally to establish a fresh checksum chain.