Repeatedly entering fail-safe mode

My PB keeps on encountering repeated errors and entering failsafe mode, and I’m not sure why.

I think this first started happening due to insufficient power: I have a 300 LED strip that I was powering from a USB C port (I had the global brightness limited, but probably not enough). This seemed to work fine until I made the rendering code more complex. Presumably this increased CPU usage and hence power draw, leading to errors.

Now, however, I am running the LEDS and PB from a 40A power supply, which should be more than enough. I am still pretty regularly encountering repeated errors leading in to failsafe mode. I can’t figure out why. Power should no longer be a concern. It’s certainly possible that I’m doing something bogus in my pattern (divide by zeros or something?) leading to errors; not sure if that would trigger the failsafe condition though.

I’m not sure how to debug this further, I don’t see any way to look at logs of what kinds of errors I’m seeing. Any thoughts on how to diagnose?

Hey! Can we start by asking which firmware version you’re running? Some older ones have bugs that can trigger a failsafe for certain patterns…

Sure thing. It was on 3.47. I just updated it and it’s now on 3.51. I haven’t seen the issue yet but I’ve not had it on the new version that long yet.

It could still be power related. How long, and how thick is the wire going to pixelblaze? High currents can drop voltage over relatively short distances!

good question. the wires are about 5-6 feet long, and 14 AWG (except for a short length at the connectors). however i am now powering the pixelblaze and the leds separately (that is, same power supply/ground but two separate runs of wiring). so i don’t think power is the issue (unless the pb draws more power than 300 leds but that seems unlikely…).

i haven’t had a chance to test things out now that i’ve upgraded to 3.51 - i should do that and see if the issue is still there. won’t have time for another day or two, though.

ok, was just able to test out on 3.51 - still happening. :frowning:

ok, i figured it out and i feel dumb.

my code was using some arrays to compute moving averages of accelerometer readings (to experiment with a hacky smoothing operation to make some transforms look nicer). i was not initializing the values of the arrays, i was assuming they were default-initializing to a zero numerical value. but i’m guessing that’s not true, they’re actually default-initialized to null. so when i compute the average value of the array entries i get a math error.

i tested this by explicitly initializing the array entries to 0 and it is no longer crashing.

i didn’t realize what was going on because the mode appeared to work. it was in fact experiencing an error on the first frame render, but was able to draw the frame. my pattern is static except for rotation due to gravity so it appears to be working.

i’m not 100% on my analysis, but initializing array values to 0 has definitely fixed it. in another language, the code i wrote would crash the renderer before it could possibly draw any pixel values, but i’m guessing that javascript is permissive and lets the code keep running despite the errors.

Unless I crept a bug in to some code, variables and arrays should init with zeros. However it is possible to run out of pattern memory. It doesn’t garbage collect, so if you keep making them it will eventually run out.

In any event those should all result in at worst no pattern and a friendly error message in the editor.

It will attempt to keep rendering, if possible.

Can you dm me your pattern that was causing issues? If there is a bug that can crash PB I want to find it!

@wizard i dm’d you the code, thanks for offering to take a look at it.

i mentioned this in my message, but for public posterity:

  • i tested the offending pattern with the pb, but this time with no leds attached so only the pb itself is drawing power. one time the pb entered failsafe mode, and a different time it did not (that i could tell), but eventually became unreachable, presumably crashed. the device also became pretty warm to the touch, not sure if that’s relevant.

  • apart from the moving-average stuff i mentioned, the pattern is pretty math intensive (it does a bunch of matrix multiplication for pixel coordinate rotation - i’m aware that there’s native functions to do this but i haven’t started using them yet). it’s possible that the pattern is just too cpu intensive which somehow causes problems (overheating?), and the moving-average stuff i mentioned is just a red herring.

Replied to the DM.

For anyone else happening upon this thread this ended up being the same problem as Pixelblaze keeps restarting - #13 by wizard

Lots of small arrays being allocated, getting past memory checks, and causing a crash. Will be fixed in the next version!

Pattern CPU use doesn’t significantly contribute to heat, it would only make the framerate drop. Pixelblaze runs as fast as possible, so even simple patterns load up the CPU to 100% and it just spits out more FPS. Most of the power consumption and heat comes from WiFi.

1 Like

thanks so much for diagnosing the issue! the issue was a little non-obvious to me because the array construction was implicit - that is, i wasn’t making a new array like a = array(3), but rather by copying the accelerometer array a = accelerometer - if you don’t look too closely that just looks like a pointer assignment. :slight_smile:

You can totally do that, assignment doesn’t allocate memory. It’s acts just like a reference or copying a pointer. It was the ‘a = array(3)’ in beforeRender, making a new array each animation frame.

hah, misread my own code. apologies.

1 Like