V3 Standard: every flash write crawls, but the chip dumps perfectly. SPIFFS with zero free blocks?

Hi Ben. I’ve got a V3 Standard that went bad on 2026-07-20. Before touching anything I pulled a full flash dump, and I think I’ve ruled out the flash chip itself. I’d love a sanity check before I erase the filesystem partition, mostly because I can’t tell what an erased FS boots into.

Quick disclosure since this is my first post here: I leaned on Claude pretty heavily for both the diagnosis and this writeup. The bench work and the numbers are mine and real, but if this reads suspiciously organized for a newb, that’s why.

The symptom was odd. The board renders patterns fine and joins WiFi fine, but anything that touches flash crawls: pattern loads, config writes, POST /wifisave, even small HTTP responses get retried a few times and then finish agonizingly slowly, or time out. Every kind of write was affected about equally, and plenty of hard power cycles changed nothing. In AP mode HTTP would eventually answer after about 90 seconds; in station mode I never got an answer inside three minutes.

So I read the whole 4 MB out twice over a CP2102 on the expansion header, at 115200, back to back:

dump 1 dump 2
bytes 4,194,304 4,194,304
time 385.4 s 385.2 s
rate 87.1 kbit/s 87.1 kbit/s
sha256 b8819e24…ade3 b8819e24…ade3

Byte-identical, 0.2 s apart, and about 94% of the theoretical ceiling for 115200 8N1, so the serial link was the limit, not the flash. No retries or stalls during either read. A few supporting reads:

  • Chip: ESP32-D0WD-V3 rev 3.0. Flash mfr 0x20, device 0x4016, 4 MB, 3.3 V.
  • Flash status register 0x0200: QE set, no block-protection bits, no lock bits, no stuck WEL.
  • FLASH_CRYPT_CNT = 0.

All of which looks like a healthy chip, and is why I stopped believing my own “bad flash chip” theory.

Here’s what the image shows instead. Partition table:

nvs      data 0x02  0x009000   20K
otadata  data 0x00  0x00e000    8K
app0     app  0x10  0x010000 1280K
app1     app  0x11  0x150000 1280K
spiffs   data 0x82  0x290000 1472K

Measuring the SPIFFS region out of the dump:

used 74.0%  /  free (0xFF) 26.0%
4K blocks: 0 fully-used, 368 partial, 0 fully-erased  (of 368)

Every one of the 368 blocks is partially written and not one is fully erased. The free space is real, but it’s smeared across every block instead of concentrated anywhere usable. My read: SPIFFS has to garbage-collect on every single write (copy live pages out, erase the block, write back) with no clean block to work in, which would explain why reads stayed fine while writes fell off a cliff, and why bigger writes hurt more than small ones. There are also around 30 stale config.json copies scattered through the image that were never reclaimed. I should say the GC story is me reasoning backwards from the block layout, not something I watched happen; the block distribution is the measurement, the rest is my theory about it.

Some possibly relevant history: a few days before this started, I had an ESP8266 button box hanging off the Pixelblaze’s 5 V rail on alligator clips, a temporary lash-up so my kid could turn the lights off from bed. I now think that connection was intermittent and the board ate unclean power events for days before it gave up. I’ve seen you describe brownouts as a source of flash corruption, and the timing fits better than anything else I’ve got. That connection is gone for good; the bedside control is a battery-powered remote now.

One more thing I’m curious about: this board has the dual-app OTA scheme above, leaving 1472K for the filesystem. The replacement V3 Standard I’m running now reports 2,884,241 bytes of storage, nearly twice as much, and that number was identical before and after I updated it from v3.51 to v3.67, so it looks like a partition-scheme difference rather than a firmware-version thing. I couldn’t pull the dead board’s firmware version out of app0. If it was on an older scheme with half the filesystem, that would explain it filling up and fragmenting a lot faster.

The actual questions:

  1. If I erase just the SPIFFS partition, does the firmware reformat and rebuild it on next boot? The thing I’m worried about is that the web app lives on that FS and gets installed by the updater rather than regenerated, in which case I’d be trading “slow FS” for “no web UI.”
  2. Do /recovery and POST /update still work with an empty filesystem? If they do, this is low risk. If they don’t, I’d want the .stfu in hand first.
  3. Does this profile match what you’d expect from repeated brownouts? Perfect raw flash reads, zero free SPIFFS blocks, uniform write slowness. And is a board in this state worth reviving, or is a wipe and reflash the realistic path?

No rush on any of this. The wall is already back up on a replacement board, so nothing’s blocked. I’ve got the dumps, partition table and block statistics saved and can share any of the numbers, and I’m happy to run whatever else you’d like while the old board is still on the bench.

Thanks for building something that survived being diagnosed this thoroughly.

cheers, Tarek.

Yeah, sounds like the filesystem is getting too full to be functional. SPIFFS can do that when pushed near capacity with fragmentation. Basically it ends up thrashing trying to compact all that half used space by copying it around to make fully erased blocks, which are then immediately used. And the compaction process itself can kind of chain-react, and it grinds and can take tens of seconds to write tiny files. I would remove some patterns (get a backup first!) to free up space.

One of the things that has bugged me about SPIFFS and filesystems for microcontrollers in general! Something I’ve been working on. However this will be a Pixelblaze V4 thing, or at the least an XL-only thing, since those have some extra space for a second filesystem.

If you have a V3 XL and a regular V3, they have different sized flash and different partitions. Technically a non-XL could be flashed onto an XL, but yeah, you lose space. If you put an XL image on a non-XL, then I don’t imagine the filesystem that is half off the end of the flash would work out very well.

/recovery does not work without a filesystem, since it loads from recovery.html.gz. /update will though, and uploading an .stfu file can put the main web app (and recovery) back in place. HOWEVER, there’s no easy way to configure wifi without the main or recovery app, and with no filesystem it’s going to default to setup mode. You’d have to connect in setup mode, move the capture portal window out of the way, and use /upload in a browser.