Developer Guide

This guide walks you through integrating GreatEmbeddo into a firmware project, from writing your first .gedb schema to handling transactions and schema evolution.

Prerequisites

• A C99-compatible compiler (GCC, Clang, or MSVC)
• The greatembeddo CLI tool installed on your development machine

Download the CLI

Writing a Schema

A .gedb file describes your tables, indexes, and named queries. Schema statements come first; query definitions go at the end. The file is append-only — you can only add new statements, never edit existing ones.

-- myproject.gedb

CREATE TABLE sensors (
    id       uint32 PRIMARY KEY NOT NULL,
    node_id  uint16 NOT NULL,
    temp_c   float  NOT NULL DEFAULT 0.0 RANGE(-40.0, 125.0),
    humidity float,
    flags    uint8  DEFAULT 0
) EXPECT 1000 REPLICAS 2;

CREATE TABLE events (
    id        uint64 PRIMARY KEY NOT NULL,
    sensor_id uint32 NOT NULL REFERENCES sensors(id) ON DELETE CASCADE,
    ts        uint64 NOT NULL,
    code      uint16 NOT NULL
) EXPECT 50000;

CREATE INDEX idx_events_sensor ON events (sensor_id);

QUERY get_sensor
    SELECT * FROM sensors WHERE id = @id uint32;

QUERY get_events_for_sensor
    SELECT * FROM events WHERE sensor_id = @sid uint32 ORDER BY ts DESC LIMIT @n uint32;

Key rules:

• Every table needs exactly one PRIMARY KEY column (integral type)
• EXPECT n is a hint for storage estimation — it has no effect on correctness
• REPLICAS n stores n redundant copies (1–4); tables default to 2, indexes to 1
• Query parameters use @name type syntax (e.g. @id uint32)
• Never edit lines you have already committed to the lock file

Generating Code

Run the CLI to generate your C99 source files. You need to know your flash device's block size and total block count:

greatembeddo myproject.gedb \
    --output src/generated/ \
    --block-size 4096 \
    --block-count 1024 \
    --cache-size 65536

Check storage requirements before committing to a block layout:

greatembeddo myproject.gedb --stats --block-size 4096 --block-count 1024 --cache-size 65536

Print the static RAM breakdown:

greatembeddo myproject.gedb --memory --block-size 4096 --block-count 1024 --cache-size 65536

The three required options are --block-size, --block-count, and --cache-size. The CLI also supports flags for metadata replication, writer-lock strategy, and CI integration.

See full CLI reference →

Generated File Structure

For a schema with two tables, one index, and one named query, the generator produces:

gedb.h                ← public API (include only this)
gedb_config.h         ← generated layout/sizing constants (pulled in by gedb.h)
gedb_internal_types.h ← implementation structs embedded in gedb_db_t (pulled in by gedb.h)
gedb_internal.h       ← implementation-only declarations (used by the .c files)
gedb_records.c        ← types, config, CRC, on-disk record format
gedb_block.c          ← block cache and superblock ring
gedb_runtime.c        ← storage engine
gedb_tables.c         ← all tables' insert/get/update/delete/scan
gedb_indexes.c        ← all secondary-index APIs
gedb_queries.c        ← all named-query iterators

Add all .c files to your firmware build system. Include only gedb.h in your application code.

#include "gedb.h"   /* pulls in everything */

I/O Callbacks

GreatEmbeddo never calls the filesystem or any hardware directly. You provide three function pointers that it calls for all flash I/O. This keeps the generated code portable to any hardware abstraction layer.

/* Read len bytes from block at offset into buf */
typedef gedb_status_t (*gedb_read_block_fn)(uint64_t block_id, uint32_t offset,
                                            void *buf, uint32_t len, void *ctx);

/* Erase a block (must happen before first write after a read) */
typedef gedb_status_t (*gedb_erase_block_fn)(uint64_t block_id, void *ctx);

/* Write len bytes into block at offset (block must be erased first) */
typedef gedb_status_t (*gedb_write_block_fn)(uint64_t block_id, uint32_t offset,
                                             const void *data, uint32_t len, void *ctx);

/* Example implementations for a NOR flash driver */
static gedb_status_t my_read(uint64_t block_id, uint32_t off,
                             void *buf, uint32_t len, void *ctx) {
    if (flash_read((uint32_t)block_id * FLASH_BLOCK_SIZE + off, buf, len) != 0)
        return GEDB_ERR_IO;
    return GEDB_OK;
}

static gedb_status_t my_erase(uint64_t block_id, void *ctx) {
    if (flash_erase_sector((uint32_t)block_id) != 0)
        return GEDB_ERR_IO;
    return GEDB_OK;
}

static gedb_status_t my_write(uint64_t block_id, uint32_t off,
                              const void *data, uint32_t len, void *ctx) {
    if (flash_program((uint32_t)block_id * FLASH_BLOCK_SIZE + off, data, len) != 0)
        return GEDB_ERR_IO;
    return GEDB_OK;
}

Initialisation

Allocate gedb_db_t as a file-scope static or on the heap — its size is known at compile time but typically exceeds 1 MiB (dominated by the embedded block cache), so it must not be placed on the stack. Call gedb_init with your I/O callbacks. gedb_init never formats the device itself: if it finds no valid database (e.g. blank flash) it returns GEDB_NEEDS_FORMATTING, and your application decides whether to call gedb_format. On an already-formatted device, the schema checksum is verified.

static gedb_db_t g_db;   /* file-scope static — do not declare gedb_db_t on the stack */

static void on_corruption(gedb_corruption_event_t event, gedb_table_id_t table_id,
                          uint64_t block_id, uint8_t replica_idx, void *ctx) {
    log_warn("GEDB: corruption event %d on block %llu replica %u",
             (int)event, (unsigned long long)block_id, replica_idx);
}

gedb_status_t st = gedb_init(&g_db,
    my_read, my_erase, my_write,
    NULL,            /* io_ctx */
    on_corruption);  /* optional but recommended */
if (st == GEDB_NEEDS_FORMATTING) {
    /* No valid database on the device. Pick a non-zero 64-bit database id and
       format. The id is mixed into every record's CRC scope, so a later reformat
       with a different id makes the old data invisible — no erase pass needed. */
    st = gedb_format(&g_db, my_read, my_erase, my_write,
                     NULL, on_corruption, my_database_id);
}
if (st == GEDB_ERR_SCHEMA) {
    /* Flash was formatted with a different schema version */
    handle_schema_mismatch();
} else if (st != GEDB_OK) {
    /* I/O error or unsupported device geometry */
    halt_system(st);
}

gedb_init is not thread-safe. Call it once at boot before starting any tasks that use the database. There is no gedb_close — the database is crash-safe without a clean shutdown.

Transactions

GreatEmbeddo provides two transaction wrappers: gedb_transaction_write for mutations and gedb_transaction_read for read-only queries. Both take a callback; return GEDB_OK to commit/complete, or any other value to roll back.

gedb_transaction_write acquires an exclusive writer lock — concurrent writers block until it is free; the lock strategy (--lock) selects the primitive (C11 mtx_t, pthread mutex, or a CAS spinlock). Calling gedb_transaction_write from inside another write callback on the same thread returns GEDB_ERR_REENTRANT (except under --lock spin, which has no thread identity and deadlocks — never nest write transactions). gedb_transaction_read is completely lock-free and provides snapshot isolation: all reads within the callback see a consistent, frozen point-in-time view of the database, unaffected by concurrent writes.

For the complete API signatures, example code, and isolation details, see Transaction API Reference and Concurrency & Isolation.

CRUD Operations

Each table gets strongly-typed insert, get, update, delete, upsert, and scan functions. Write operations must be wrapped in gedb_transaction_write; reads should be wrapped in gedb_transaction_read for snapshot isolation.

For the full function signatures and usage examples, see CRUD Operations and Scanning in the API Reference.

Named Queries

Named QUERY definitions compile into typed C99 iterator functions. SELECT queries use an iterator; INSERT/UPDATE/DELETE/UPSERT queries are single calls.

/* SELECT query with ORDER BY — result via iterator, inside a read transaction */
static gedb_status_t list_recent_events(gedb_txn_t *txn, void *ctx) {
    gedb_query_get_events_for_sensor_iter_t it;
    gedb_status_t st = gedb_query_get_events_for_sensor_open(txn, /*sid=*/42, /*n=*/10, &it);
    if (st != GEDB_OK) return st;

    gedb_query_get_events_for_sensor_row_t row;
    while ((st = gedb_query_get_events_for_sensor_next(txn, &it, &row)) == GEDB_OK
           || st == GEDB_WARN_ARITHMETIC) {   /* a row too: one whose value wrapped */
        printf("ts=%llu code=%u\n", (unsigned long long)row.ts, row.code);
    }
    return st == GEDB_DONE ? GEDB_OK : st;   /* GEDB_DONE = end of results */
}

gedb_transaction_read(&g_db, list_recent_events, NULL);

The iterator type is stack-allocatable; its exact size is sizeof(gedb_query_..._iter_t), known at compile time. ORDER BY results are materialised inside the iterator — no heap required.

A query may also combine several SELECT branches with UNION ALL or UNION, over one table or over different ones, still through a single iterator. UNION ALL without ORDER BY streams one branch after another in O(1); the deduplicating and sorted shapes use the same bounded buffer as ORDER BY and re-scan every branch per pass. See UNION and UNION ALL for the shape rules and the result-row layout.

Error Handling

Most functions return gedb_status_t. Always check the return value.

Repair: detection is automatic, healing is scheduled by you

Reads and writes never perform repair work themselves — their latency stays predictable. When any operation encounters a damaged block, a lagging replica, or a corrupt secondary-index tree, it detects and queues the damage (and announces it through corruption_fn), then degrades gracefully: reads return what is recoverable, writes that would need the damaged structure fail with GEDB_ERR_CORRUPT_NEEDS_REPAIR.

All healing happens inside gedb_repair_step. Each call first drains queued damage — known-corrupt blocks, then one flagged secondary-index/FK rebuild — and only then advances a bounded background sweep (block scrub, tree walk, index verification). Because the work per call is bounded, it is safe to call from a low-priority idle task. It runs concurrently with readers and blocks on the writer lock like any other write.

/* Schedule from a low-priority idle task. Damage queued by reads/writes is
   healed here — if you never call it, degraded blocks stay degraded. */
gedb_repair_step(&g_db);

Rules of thumb: call gedb_repair_step periodically from your idle loop; after corruption_fn fires, or when an operation returns GEDB_ERR_CORRUPT_NEEDS_REPAIR, run gedb_repair_full (or keep calling gedb_repair_step) to bring the database back to full redundancy, then retry the failed operation. Irrecoverable losses are announced through corruption_fn as GEDB_CORRUPTION_DATA_LOSS, never hidden in return codes.

Schema Evolution

Extend your schema by appending new statements to the .gedb file. Regenerate and rebuild. The .gedb.lock file tracks the append-only checksum chain — commit it to version control alongside your schema.

-- Original schema (lines 1–12) must not be changed

-- Append new statements below:
ALTER TABLE sensors ADD batch_id uint16 DEFAULT 0;
ALTER TABLE sensors EXPECT 2000;
ALTER TABLE sensors REPLICAS 2;

CREATE INDEX idx_sensors_batch ON sensors (batch_id);

QUERY get_batch
    SELECT * FROM sensors WHERE batch_id = @bid uint16;

Without the Pro version, existing devices with an older schema version will get GEDB_ERR_SCHEMA from gedb_init. This is intentional — the generated code must match the on-disk schema exactly.

Automatic Migration (Pro)

The Pro version generates C99 migration functions that automatically upgrade an older on-disk schema to the current version — row by row, in a single gedb_init() call.

/* The Pro version also generates: */
gedb_migration.h / gedb_migration.c       ← migration logic for gedb_init
gedb_migration_1_2.h / .c                ← schema v1 → v2
gedb_migration_2_3.h / .c                ← schema v2 → v3

/* gedb_init automatically applies migrations — same signature, no API change: */
gedb_status_t st = gedb_init(&g_db, my_read, my_erase, my_write,
                             NULL, on_corruption);

Migration safety guarantees:

• Migrations are generated only if the CLI can prove they cannot fail at runtime
• Narrowing type changes require an explicit ON NARROW CLAMP|WRAP|NULL policy
• Adding a unique index to an existing table requires an ON CONFLICT REMOVE_DUPLICATES_*|REMOVE_ALL_DUPLICATES_* policy, which also says what happens to the rows referencing a removed row
• So does a type change that can merge two values into one under a unique index (any narrowing, a shrinking char or array, float → double): ALTER TABLE … ALTER … ON NARROW CLAMP ON CONFLICT REMOVE_DUPLICATES_REASSIGN_PRIMARY, unless the UNIQUE declares its own policy
• Each v → v+1 step runs in its own write transaction and commits atomically — a power failure mid-migration leaves the device on the last intermediate version it successfully committed, and the next boot resumes from there
• Transient tables, columns, indexes, and FKs — those created in one step and dropped in a later step — are fully preserved in between, so intermediate data migrations can rely on them
• Reserved blocks ensure migration can always delete rows to resolve conflicts, even on a full database
• When a step adds, removes, or changes columns, both the old and new table data coexist in flash until the step commits — ensure devices have enough free space for approximately 2× the table’s block footprint before OTA-upgrading. The CLI warns when this may be tight.

Troubleshooting Migrations

Schema evolution is append-only, so mistakes in .gedb files need corrective action. The right fix depends on when the problem is caught.

The CLI rejected my schema change

If the CLI reports a validation failure, your new lines are not locked yet — you can freely edit or remove them and re-run the CLI. No special flags are needed.

A migration fails at runtime on the device

Each v → v+1 step runs in its own write transaction. If a step fails, only that step rolls back; previously-committed steps stay committed. The device remains on the last successfully-committed intermediate schema version and the next boot resumes migration from there. Old firmware can still boot devices on any intermediate version that firmware understands.

Fixing a locked mistake

The safest fix is to append corrective statements below the mistake. As a last resort, you can bypass the checksum chain with --skip-lock-check.

See full migration recovery guide →