ESP32 NVS Config Management: Schema, Migration & Recovery

Fielded ESP32 fleets rarely die the way the flash datasheet warns. They die when an OTA rollback drops a v1.2 image onto a device whose NVS was forward-migrated by v1.4, nvs_flash_init() returns ESP_ERR_NVS_NEW_VERSION_FOUND, and the getting-started error handler does the one thing that guarantees a truck roll: erases the partition. Wi-Fi credentials, cloud certificate and Modbus address gone together, on a device now invisible on the bus.

That is a design failure, and it is preventable. Below is the model we hold GizanTech firmware to when a device goes into a cabinet on someone else's site.

What NVS actually is

Definition: NVS is a log-structured key-value store in a flash partition — appended within a page, compacted when a page fills, with per-entry CRC32, no transactions, and no wear-levelling module.

Not a filesystem, not a database. It lives in a partition of type data, subtype nvs; anything beyond the default is registered by name with nvs_flash_init_partition(). The layout drives every decision that follows:

Two consequences people miss. Writes append: updating a key writes a new entry and marks the old one erased in the bitmap, and space returns only at compaction, which needs a free page — hence the reserved one. And nvs_commit() is an API contract obligation, not a transaction boundary; the implementation writes eagerly, with no rollback and no multi-key atomicity.

There is no wear-levelling module either — that is the wear_levelling component behind FATFS. Log-structuring spreads writes within the partition and nowhere else. Thread safety is a similar trap: the NVS API is internally locked, but your read-modify-write of a config blob is not. Serialize config writes through one task.

Before proposing "just add a partition": the table sits at 0x8000, is 0xC00 (3 KiB), uses 32 bytes per entry and appends an MD5 checksum entry — 95 usable entries. App partitions align to 64 KiB (0x10000), data partitions to 4 KiB.

The sizing math nobody does

Entries consumed by a blob of L bytes: 1 (index) + 1 (chunk header) + ceil(L/32).

Work a real case — a 1.5 KB config blob in the default 24 KiB nvs partition (6 sectors, 5 usable):

1. Entries per write ≈ 1 + 1 + 48 = 50.
2. Capacity per fill cycle = 126 × 5 = 630 entries.
3. Written hourly: 8760 × 50 = 438,000 entries/year → 438,000 / 630 ≈ 695 sector-erase equivalents/year.
4. Against a NOR sector rated on the order of 10⁵ erase cycles, that device is effectively immortal.

Now change one assumption. Write a 4 KB blob (~129 entries) every minute: 525,600 × 129 ≈ 67.8M entries per year → about 108,000 erase-cycle equivalents per year. Dead inside a year, same code, same hardware.

The rule of thumb:

years ≈ (rated_cycles × 126 × usable_pages) / (writes_per_year × entries_per_write)

Run it before agreeing to a "log the setpoint every minute" requirement. Then size against this:

• Two config slots resident at once doubles the steady-state footprint.
• Add headroom for the largest blob plus a full second copy during migration.
• Reserve room for the Wi-Fi/BT keys IDF puts in the default nvs partition — or better, give app config its own partition so IDF can never starve you.
• NVS caches page and entry metadata in RAM, and that RAM grows with page count. Do not provision 512 KiB "just in case".

Schema versioning: steal ext4's feature flags

"Old code meets new on-disk state" is solved, and the settled answer is ext4's three-class feature model. Copy it.

Store schema_version and min_compat_version — the oldest firmware permitted to write this representation. SQLite's user_version and Android Room migrations are the same idea.

Migrations are a ladder: pure, idempotent N → N+1 functions applied in sequence, never a "detect the shape and patch it" heuristic. Version bump and migrated data land in the same commit — the same slot flip — or you have invented the partially-migrated slot.

Encode by field number, not struct offset: CBOR, protobuf/nanopb, or hand-rolled TLV. Unknown fields should be preserved and round-tripped, so old firmware does not silently drop a key it cannot name. If you insist on a packed struct, fixed-width types, __attribute__((packed)), explicit little-endian and static_assert(sizeof(cfg_t) == N) are the minimum bar.

ESP_ERR_NVS_TYPE_MISMATCH is the canonical drift symptom — nvs_get_u32() on a key someone quietly re-typed to i32 in v1.4.

A fixed 16-byte blob header, never itself versioned, carries the rest: magic u32, schema_version u16, min_compat u16, payload_len u32, crc32 u32. Use CRC-32/IEEE-802.3 (poly 0x04C11DB7, reflected 0xEDB88320) over the payload; esp_crc32_le() is in ROM and free.

A/B config slots and commit-on-successful-boot

Namespaces cfg_a and cfg_b, plus a tiny cfg_ptr namespace holding {active_slot u8, seq u32}. This is otadata's design reused for config — which is the point: it is already proven against power failure.

The write protocol, in order:

1. Write the full config blob to the inactive slot.
2. Read it back and verify the CRC32.
3. Write the pointer — one entry, made atomic by NVS's own entry CRC and state bitmap.
4. nvs_commit().

Store the whole config as one blob. You then inherit NVS's blob-index behaviour: the index entry is written last, so a brownout mid-write leaves the old blob intact. Forty individual keys give you forty independent commits, and a brownout that leaves key 19 old and key 20 new — a state no migration ladder can reason about.

Then add commit-on-successful-boot. Load the active slot but mark it pending, and write the confirmation only after the application reaches a real health checkpoint: network up, sensor bus enumerated, cloud handshake complete. This mirrors ESP_OTA_IMG_PENDING_VERIFY and esp_ota_mark_app_valid_cancel_rollback() — a config change that kills connectivity self-reverts on the next boot instead of stranding the device.

The rollback property is the payoff: migration writes the new representation into the inactive slot and leaves the old one untouched, so downgraded firmware still has a slot it can parse. That single choice turns an OTA rollback from a site visit into a reboot.

Identity is not configuration

Identity belongs in its own partition — call it nvs_fact — generated at manufacture with nvs_partition_gen.py from a CSV, flashed once. It holds serial number, model, hardware revision, RS-485 address and baud, calibration constants, and the device or claim certificate.

It is opened only as NVS_READONLY through nvs_open_from_partition(), never registered for writes anywhere in the codebase. Newer IDF exposes a read-only flag for data partitions in the partition CSV; older setups enforce it by discipline and code review. Hardware write-protect bits are too coarse to help at runtime.

That split is the whole recovery story. If the Modbus address lives in the config blob you just lost, the device is unreachable on the bus and someone is driving out to it. If it lives in the factory partition, a wiped config is a nuisance — the device still answers, still reports, still accepts re-provisioning.

Better still, on ESP32-S2/S3/C3 and later, do not store a private key at all: put it behind the HMAC and Digital Signature peripherals with an eFuse-held key, so it never exists in flash in usable form. That is our default on new industrial IoT designs, alongside the firmware engineering review of the partition table itself.

Encrypted NVS and the one-way doors

NVS encryption is XTS-AES. Key material lives in a dedicated partition — type data, subtype nvs_keys, 4096 bytes — holding a 64-byte key blob (data key plus tweak key) and a CRC. That partition must itself be protected by flash encryption, or you have stored the safe key next to the safe. Newer chips can instead derive the NVS key through the HMAC peripheral from an eFuse key, removing that prerequisite.

Three properties that change designs:

• XTS is length-preserving, so encrypted NVS costs no extra entries — only the 4 KiB key sector and per-access AES time.
• Entry payloads are encrypted; page headers and entry-state bitmaps are not, so partition occupancy stays observable.
• Flash encryption is address-tweaked: ciphertext depends on flash offset. No relocating a partition, no cloning an image between devices, no restoring a dumped NVS to a different address.

And the door that does not reopen: Release mode is irreversible, and it removes your out-of-band recovery. No plaintext UART re-flash, no serial NVS dump. Design and test the in-band recovery path before burning those eFuses. Development mode keeps the escape hatch, which is the right choice for pilot units.

Anti-rollback (CONFIG_BOOTLOADER_APP_ANTI_ROLLBACK plus the eFuse secure_version) is the same shape of decision. Once burned, downgrade is impossible — forward-only config migration becomes safe by construction, at the cost of the rollback that would otherwise have saved you.

Config in otadata is the truck-roll mistake

otadata is type data, subtype ota, 0x2000 (8 KiB) — two sectors. Each holds a small select structure (ota_seq, label, ota_state, CRC32); two copies exist purely for power-fail safety, and the bootloader picks the higher valid sequence.

The application erases and rewrites a whole sector on esp_ota_set_boot_partition() and again on rollback confirmation. Anything you appended inside that sector is gone — and gone exactly at rollback, the moment you most need config intact. One bad update becomes a dead device.

Variants of the same mistake:

• Config appended after the app image. OTA erases the whole target partition, and secure boot's signature covers that partition, so the image stops verifying.
• Config inside the app partition at all. Slots A and B hold different configs: every OTA is a config reset, every rollback a different one. Split brain.
• Reusing nvs as OTA staging scratch.

Correct separation is five partitions doing five jobs: otadata for boot selection, ota_0/ota_1 for signed images, nvs (or a dedicated cfg partition) for mutable A/B config, nvs_fact for immutable identity, nvs_keys for encryption keys.

The recovery ladder

Document it, ship it, test it:

1. Active slot valid — magic present, version accepted, CRC32 correct — use it.
2. Active bad → try the other slot. If valid and version-acceptable, promote it and log the demotion.
3. Both bad → factory defaults plus identity. Device stays addressable, reports CONFIG_LOST, accepts re-provisioning.
4. Factory partition unreadable → compiled-in safe defaults, UNPROVISIONED state, provisioning channel forced up (BLE, SoftAP, serial console).

Never boot silently into defaults. Set a sticky diagnostic record — last_nvs_error, an nvs_get_stats() snapshot, a page-state histogram — and surface it in telemetry. A fleet quietly running factory defaults looks exactly like a healthy fleet until the first Modbus poll fails.

Handle these explicitly, and know what each means: ESP_ERR_NVS_NO_FREE_PAGES (partition full, compaction impossible) · ESP_ERR_NVS_NEW_VERSION_FOUND (downgraded firmware met a newer NVS format — the literal rollback scenario) · ESP_ERR_NVS_PART_NOT_FOUND · ESP_ERR_NVS_INVALID_STATE · ESP_ERR_NVS_NOT_ENOUGH_SPACE · ESP_ERR_NVS_VALUE_TOO_LONG · ESP_ERR_NVS_TYPE_MISMATCH · ESP_ERR_NVS_NOT_FOUND.

Ship the support tooling that prevents dispatches: config export / config import over the API you already have, CRC-checked and versioned in the same blob format, plus an nvs_entry_find() / nvs_entry_next() dump for diagnostics.

The highest-yield hour on a fielded fleet

Audit for one call. nvs_flash_erase() in the init error path appears in nearly every getting-started example: init returns NO_FREE_PAGES or NEW_VERSION_FOUND, so erase and re-init. On a dev board, fine. In the field, NEW_VERSION_FOUND is precisely the OTA-rollback signature, and the "fix" destroys credentials, certificates and the Modbus map in one call. That branch must log, fall back to the alternate slot or factory identity, and report — never erase.

Then audit the downgrade direction, which almost nobody tests because CI only ever installs newer firmware. Two structural defences and one process one: adopt the three-class feature model, so an old build meeting unknown state degrades to read-only instead of rewriting a schema it does not understand; migrate into the inactive slot, so the pre-migration representation survives; and make CI run the real matrix — v1.2 → v1.4 → v1.2, on physical hardware, with a power cut injected mid-migration.

Key takeaways

• NVS is atomic at one granularity — the 32-byte entry. One blob is the unit of consistency; one u32 pointer is the commit.
• Size from entries, not bytes: 1 + 1 + ceil(L/32) per write against (sectors − 1) × 126 usable.
• Version with schema_version + min_compat_version and ext4's compat/ro_compat/incompat classes; migrate into the inactive slot.
• Identity belongs in a read-only factory partition, so config loss never costs addressability.
• nvs_flash_erase() on init failure is the boilerplate that ships the brick. Audit for it first.
• Release-mode flash encryption and anti-rollback eFuses are one-way doors.

None of this is exotic — a couple of days of design and a week of test infrastructure, spent once, against a failure mode whose alternative is a vehicle and a site induction. Do it before the first production run.