Pattern runs, but has lost its code

Troubleshooting
#1

Has anybody else seen this? It has happened to me a couple of times on the PB3 (never on the 2+), but this is the first time I’ve caught it in action and couldn’t write it off to me just forgetting to press “Save”.

Pixelblaze 3 standard, firmware 3.17, 591K available.

I was editing a pattern – call it “A” – then saved it and switched to another pattern to grab a chunk of code that I needed. When I switched back, pattern “A” loaded and ran with my latest edits and all slider controls functional. But the code area of the IDE was empty.

I then switched away from “A” again and looked at several other patterns in the IDE – all fine. When I switched back, “A” still runs, and still has no source code.

Tried exporting “A” – absolutely nothing happens when I press the export button. Other patterns still export fine.

What’d I do? Is there a way to recover the source (it’s actually no big deal if not – maybe 10 minutes of work. I save frequently to external .js files.)

These are computationally intensive patterns – sometimes the framerates go as low as 7 or 8 fps. The UI can be a little slow to respond when running one of these things. Could that cause problems with file I/O? Is there anything diagnostic I could look at to provide more information?

Sequencer Page issues
#2

I feel like I’ve had something similar happen once, but I can’t remember which version I was on. My theory at the time was that I had somehow saved a corrupt version from a WiFi issue and it wasn’t able to parse the binary version back into code.

Can you try dumping the binary version and send it to me? I want to see what happens if I install it manually via curl.

The easiest way IMO is to run Firestorm, type “/”, and then use the “dump” button next to the board with the pattern.

Pixelblaze Firestorm 2021-07-16 10-07-04
Pixelblaze Firestorm 2021-07-16 10-07-041498×510 48.3 KB

Also, from a browser dev console on that Pixelblaze, inspect programList to figure out the program ID.

2021-07-16 10-06-30
2021-07-16 10-06-30816×400 41.8 KB

3 Likes
#3

@zranger1 ,
Uh no :frowning: Yeah this sounds like a partially written or corrupt file. The files combine name, bytecode, preview, and compressed source in that order.

It sounds like something is failing near the end of the file where the source is stored. This could have happened if wifi couldn’t send all the data, or if there were problems writing to the file (like the kind of issues updates had on your PB).

I can do a better job of handling these, detectable failure should at least leave the original intact. I also (and @jeff has recommended this as well) want to provide some feedback in the UI when a save is complete/successful.

Another idea is to use storage in the browser to snapshot source code so that you could recover the source. Ideally you’d be able to get source in the event of a failed save or accidentally reloading the page while editing, etc. Perhaps a listing of the last n copies. That relies on the browser’s page/origin identity so a change in IP address would break it, so not good for long term stuff like version control but would be a nice way to prevent a few issues.

1 Like
#4

That’s so clever. I wonder if Pixelblazers could make it a little more durable too if we configured local DNS with a hosts file mapping the board name with a .local TLD to its current IP.

1 Like
#5

Oh right! That does work better with mdns too!

#6

@Jeff, I’ll retrieve the binary and upload it this evening. I’m kind of curious about what’s in there.

I was contemplating writing a browser addon to periodically snapshot changes to the edit window, especially since I periodically save pattern source to a directory anyway. Having some backup/autosave capability built in to the IDE would be even better.

Edit: After firing up that Pixelblaze today, the pattern is still running, complete with preview and UI controls, but it shows source to a different unrelated pattern. Definitely not pressing “Save” on that one yet.

Wonder if the save hadn’t completed before I switched patterns, so it saved the correct binary, and grabbed the source from the newly loaded pattern. Possibly a threading thing? Here’s a screenshot of the weird mashup.
“Copy of Rock Concert” is the running pattern.

My editing window seems to be stuck on “Editing Matrix Green…” today, no matter what I load, although the source code shown actually is for the green matrix waterfall pattern, which is way wrong. Should I maybe contemplate erasing everything and restoring from scratch at this point?

image
image1342×936 47.1 KB

#7

Results: Same as yours! Name and preview loaded, I could run the pattern (oooh, ahhh! nice) but the editor was blank.

Sounds like the LZString-encoded source at the end of the binary got corrupted or truncated somehow. Sorry you lost some work.

1 Like
#8

Thanks for taking a look! That’s pretty much what I’d figured. Only a few minutes of constant fine tuning work lost, and fortunately for once, I remember what I was thinking while I was doing it.

#9

This disappearance of code thing happened a couple of times yesterday while running one of my v3s. Found I was running on weak WiFi @ -62 dBM. Fixed it, now running fine at -32. No problems on either of my v3s.
I, of course, will let greater minds sort the details. And thanks for doing so!

#10

When you hit save it queues up everything atomically on the UI side. It shouldn’t cross wires with a pattern just being loaded.

If it can’t load the source, it might leave the old source in the editor though. If you clear it out or hit New Pattern or something before you edit another pattern, does it still load this green pattern source?

Otherwise I wonder if your flash is dying, or there is severe filesystem corruption that is causing it to read/write the wrong files/data. If you want to do some advanced troubleshooting I can dig deeper. Maybe image the flash and try a few things.

#11

I’ve still got the backup of the bad pattern, but I deleted it from the Pixelblaze and cautiously went on working. I’ll keep my eyes open, but everything seems ok now. As far as I can tell, no other files on that Pixelblaze were affected.

I thought about the flash memory. This is my “dev” Pixelblaze, so I’m saving to flash quite often, and might someday actually wear it out. After poking around with a voltmeter though, I’m leaning towards the possibility of a power problem.

When this died, I was inadvertently running my matrix at a 100% brightness, doing things that often resulted in bright white pixels. The way I had it wired (for convenience), the voltage at the Pixelblaze could drop considerably depending on what the LEDs were doing. I’ve since rewired (for sanity), using heavier wire and feeding power to the Pixelblaze first. It has been stable so far. If it happens again, I’ll let you know.

1 Like
#12

FWIW, I just had this happen last night. I’ve been on a wide variety of access points; last night was a very strong SNR and great bandwidth (but I don’t know the model - it was a commercial radio with sectored antennas).

On the next power up, trying to edit that pattern resulted in a blank editor and “Copy of” pretended to the pattern name, interestingly, though that wasn’t shown in the main list.

For the record, it was a ridiculously large pattern (~45K).

#13

My wifi connection is very solid too. I’ve not had a recurrence, but since it happened, I’ve been extremely careful to be sure that my Pixelblaze has finished whatever it was doing – generating the preview, saving, refreshing the web page – and has returned to its stable framerate before I change patterns or save again.

Might be time to go back to being less careful to see if I can make it happen again. (45k! That really is large!)

#14

I just had this happen… in the midst of editing. Pattern continued to run, but things crashed. Reloaded browser window, Pattern continued, but editor was empty. Opening up the saved version of the pattern lost the new changes.

Clearly, some v3 bug still to be narrowed down.

#15

Just happened to me as well, I saved it, switched to another pattern and now the original is empty
but runs fine. Ill learn to keep text file backups now, that was 3 days work down the tubes on beat analysis. Perhaps on save in future have the web page offer a local save of the text file or something.

#16

I have fixed this in a yet to be released version. Not quite done yet with the rest. Anyone want to beta test?

I’m sorry for the lost code and time :pleading_face:

1 Like
#17

I should point out for archival purposes that it wasn’t the PB’s fault.
I tested many times to figure it out, I was sending too much array
data. I had several multidimensional arrays going on and it was too
much traffic that garbled the data.
I do suggest though if I may, that the export of the frequencydata
shouldnt have to be necessary. If I dont export the data set is empty
to the program as well, so to work on FFT data you have to have the debugger
sending that 32 array all the time. It be nice if instead it was a global
switch FFT = ON perhaps as the raw export is useless to debugging.

SO far, Im having a lot of fun really. Dont sweat the lost time, its all
an education… :slight_smile:

1 Like
#18

I’m happy to give the beta a try.

What does “the rest” include? Two or three times I’ve had a pattern go bad on my development v3 (I do regular backups, so the first symptom was that the BIN file downloaded from http://pixelblaze-IP.local/p/patternID was malformed and the EPE/JS extraction failed). When I looked at the pattern on the PB itself, I couldn’t even edit, select or export the pattern. The most recent time it happened, I imaged the EEPROM and tried to extract the pattern from the SPIFFS segment, but I couldn’t figure out the right parameters to get MKSPIFFS to mount the volume (I tried -b 4096 -p 256 as they’re the Arduino defaults for ESP32 flash) so I gave up. Haven’t seen it on other v3s, though that may be because they’re used only for display purposes so they have no editing activity and thus less flash wear.

What did you decide about persisting the editor contents into browser local storage? It looks like it would be pretty easy to stash the current pattern code and a compilation ID into local storage after each compilation, and to populate an empty editor from local storage if the PB engine’s compilation ID matches the one in local storage. (And once you’ve got the pattern in local storage, there’s a browser-based GIT client that would only take a handful of lines to integrate into the editor.)

#19

Now saves are written to a temp file, sanity checked to make sure the header and number of bytes match, then the original is replaced. An invalid or partial save no longer corrupts the existing pattern file.

The UI shows a small progress bar/line, and the save button changes to indicate save status. On error, a modal error message is shown. You can try again.

Also in the update are new UI controls. Toggles, trigger buttons, number inputs, and output controls: number, meter.

The mapper preview can now be manually controlled.

Live preview pixels limit is now 1000. If you have more pixels some are skipped but the whole length is captured. Saved previews are also expanded, and are more heavily compressed if over 10k, to a minimum of 50 jpg quality (can be 50k or so for previews with 1k pixels).

The map data and same mapped preview system is used to show live preview pixels, using the live preview.
Both 2d and 3d. Right now this is just on the editor, but I hope to unify this with the live preview on top.

4 Likes
#20

You’ve been busy!!! :exploding_head:

Can’t wait to try it.

1 Like