Another neat project idea hampered by not using a PixelBlaze

Didn’t create the program just edited to fit the coat. Thank you Scott Marley for all the video content and free program which made this possible! Links: Scott Marley’s Mask and the XY generator. Hardware is all finished but going to add more patterns.

Sewed in 792 LEDs WS1228B, 3D printed parts to stabilizes the LEDs so they don’t bend around/break, 3 USB chargers (~21000mAh I believe, one is bigger than the other 2) for power, Esp32, button, switch attached to a 6 relays to turn LEDs off/on, and 6 capacitors to keep power spikes at bay

And another one by someone else:
http://xuth.net/jim/coat/

In both cases, nifty builds, but if they’d used a PB, they’d have 2D and 3D mapping, sound and motion reaction, and so on. I feel like these are great idea but then the code just limits them to way less in action.

1 Like

Those both have some cool patterns but I shudder to imagine how long it took to sort them out. The live-coding dev cycle of the PB makes all the difference to me.

2 Likes

I made a jacket similar to this some years ago (~640 LEDs, 40000mAh battery, various sensors), running about 70 FastLED patterns on it. I totally agree that being able to power it with a PB would have made it far easier to create better and more complex patterns, and I wish the PB had been around at the time as I’m sure it would have saved me a lot of late nights!

That said… after having gone from that jacket project to another couple that do use a PB, I’m still quite torn between the tradeoffs required. Here’s a few examples of where I’ve encountered hurdles with the PB:

  • All code on the PB is per-pattern only. I’ve found this hugely limiting at times. Examples:
    • Code reuse. My jacket codebase is around 12,000 LoC, with a lot of shared functionality across patterns (e.g. geometry, boids, global palettes, fonts, scrolling etc). No way I’d want to cut’n’paste (and maintain…) that on a PB.
    • PB can’t skip or navigate between patterns programmatically or based on external events. E.g. my jacket has a flame sensor. When it detects a cigarette lighter it can switch to a flame effect :grin:. Similarly, there’s a gesture sensor which allows switching to a pattern that flows in the direction your hand was waved. Also has a rotary encoder to skip forward/back/reset.
    • Crossfades and other types of behaviours that operate a level above individual pattern code (speed controls, palette changes, …) aren’t possible.
  • Can’t reuse the same pattern multiple times with different parameters (though with the existing playlist functionality this might be quite easy to add?).
  • PB has only one global map. Sometimes I wanted my jacket to behave as a cylinder, sometimes a flat matrix, sometimes custom mapping.
  • The sensor expansion board microphone doesn’t have auto-gain. I get frustrated mucking around with PID controllers in each sound reactive pattern. My jacket had a MAX9814, which does a great job of dealing with that in hardware.
  • PB has no ability to manage/throttle max power output. This has caused me frustration with certain patterns that momentarily draw too much power at a certain point in their cycle. Rather than beef up the power supply (in my case expensive and impractical) or reduce global brightness just to cater to these brief peaks, I usually end up changing the pattern code until it isn’t as bright, which can be a somewhat time consuming process.
  • Can’t access the underlying LED buffer on the PB. For some situations this is really useful, e.g. whenever multiple passes/layers are required, for applying global effects (e.g. blur/fade/scroll), for blending patterns.
  • Bitmaps, fonts etc. I realise there are hacks to achieve this on PB but they’re not what I’d call practical.
  • Source control of pattern code is painful on the PB.

Now, please don’t take the above as me hating on the PB! Quite the opposite, I love the wonderful balance of power and simplicity PB brings. I also realise Rome wasn’t built in a day and that a fair few of the points I’ve listed above have workarounds or are on the roadmap. But if I were to start a project like this again today, I’d still have to consider long and hard the trade-offs required if I wanted to benefit from the easy to use, powerful programming model and live coding goodness that we’ve all come to love :laughing:.

As far as the difference in programming difficulty between e.g. FastLED and PB goes, I’ve found it really depends a lot on the type of pattern. For pixel/bitmap oriented stuff I’ve found it can go either way. E.g. the “Spiral Twirls” and “Spectrum Analyser” in the Pattern Library were ported from my jacket code, plus there’s various PB patterns I’ve ported the other way, all without too much difficulty. Where PB really starts to shine (excuse the pun) is with irregular pixel layouts, 3D maps, and shader-like patterns. It’s great to see people like you and zranger1 starting to explore more in this direction.

6 Likes

Actually I agree with most of your issues…

And don’t get me wrong, sometimes FastLED/WLED or another solution is the way to go.

That said, this is a good list of things, some of which we’ve discussed ways to improve…

2 Likes

Totally agreed, and I’ve hit (and enjoyed hacking around) many of those issues.

Fortunately almost all are a simple matter of programming, and it’s only a matter of time before either the electromage crew lock themselves in the basement with a pallet of mountain dew and don’t come out until it’s all done, they open source some layers of the PB (e.g. they focus on the low-level functions in PB JS, and make it easy to replace the entire Browser JS front-end) so we can do it ourselves, or a hardware-agnostic open source implementation becomes installable on the PB.

1 Like

Some form of library or include would help here. We’re battling the problem that it’s a compiled language, and the binary is actually what runs. So it’s non trivial.

Ben only recently added playlists. This seems like a good expansion of that.

Another playlist functionality in some ways… But crossfade is a whole different thing, would require running two patterns at once.

Global speed has been discussed repeatedly, while Pallettes needs some fleshing out (likely in userspace code) so we can better define how that could/should work. The FastLED method is nice, and basically some implementation of that could be written, but of course patterns would need to take advantage of it.

Multiple variable setting… You could clone a pattern, then change the default/saved variables, and playlist that, yes.

Multiple saved maps, with a way to switch between them.

For your example though:
Mapping a cylinder and mapping a matrix are pretty identical. The cylindrical is just adding a z to the xy. So Render2D in the matrix patterns and Render3D in the cylinder ones.

Add that to the Sensor Board 2.0 wish list. I really want a better way to “bucket” sounds, for example.

I thought global brightness did this. You’re saying you want to limit on a power basis, but allow.max brightness of some LEDs so long as the power consumption remains below X?

Yeah, we’re definitely starting to explore the realm where buffer access in some way would make a lot of sense and use.

Without strings, it’s all very limited font wise.
Includes would help a lot here.

Bitmaps and any sort of large array seem to be a growth area for sure.

Yeah, but since it’s a UI geared to on the fly coding, even having a single error grinds the editor to a halt. You can’t even save the pattern code in that state.

Maybe we need a PB friendly add-on (ala userscript/bookmarklet/extension) that adds Editor enhancements like “git push”? You’d configure your personal copy, and it would add UI buttons to do those things?

Once you have a epe, we’ve now got code to explode that and also build it back again… So “exported” code is pretty much covered for git/etc. It’s the actual code editor where you lack it.

If we had a full PB emulator, with a programming environment, that would be good too.

Ah the wish list grows.

1 Like

Better would be if the variables were saved per playlist entry rather than per pattern but I don’t know how things work under the hood so have no idea how difficult that would be to implement.

Not sure this is correct? Displaying a pattern designed for a 2D matrix on a 3D cylindrical mapping isn’t the same result as you’d get from wrapping that 2D matrix around the cylinder.

Exactly. The global brightness isn’t a great proxy for power consumption, what’s really required is a sum of estimated power usage of each LED. FastLED has setMaxPowerInVoltsAndMilliamps() which is pretty effective (though still not perfect for me with 8 LED channels and 4 independent 5V 3A USB power feeds, one for every two channels!).

I’m thinking more about advantages like: being able to browse/restore code history, make backups/clones, sharing/collaboration, coding offline, QA/prod/experimental branches, … all the usual stuff VCS brings really. Your script/bookmarket idea is an interesting possible solution for sure…

I agree with your responses overall. All these things are solveable given the right time/effort/priorities.

2 Likes

All depends on how you define the axes.

By that standard, Z is used for X or Y, and angle is the other, while Radius is the new Z… If you are just doing a cylinder with no radius, it’s really only 2D anyway (angle and height)

So wrapping a strand around a can or bending a matrix around it: 2D, or at least 3D where radius is a constant.

Thank you for the list, those are all excellent points, and for the kind words. Please indulge me a few comments/ideas for context/planning purposes :grin:

(hack: you could tie a GPIO to the BTN pin, leave it floating normally and pulse low/output and back to input to trigger pattern advancement)

I want to expose an API for that. I need to brainstorm some API concepts, open to ideas! The sequencers have the concept of play/pause/next, and playlists can be activated by index. Switching to a specific pattern requires the ID either directly or indirectly by scanning e.g. by name.

I think in many ways this API would be best served in global code/shared code with its own state, otherwise each pattern would need to use the API to make something that could implement programatic pattern control. Perhaps “custom sequencers” can be coded up that run much like a playlist does.

Alternatively, importing patterns and delegating to them through a coordinating main pattern could do much the same without actually switching which pattern PB was running at the time. e.g.:

import {render as render1, beforeRender as beforeRender1, sliderBar} from "awesome pattern"
import {render as render2, beforeRender as beforeRender2, sliderFoo} from "cool pattern"

export {sliderFoo} from "cool pattern" //re-export a slider

sliderAwesome(.5)  //set a control from 'awesome pattern' programmatically

export function beforeRender(delta) {
  beforeRender1(delta) // let 'awesome pattern' do it's thing
  beforeRender2(delta) // let 'cool pattern' do it's thing
  //set mode based on time, events, etc
  mode == time(1) < .5 ? 1 : 2
}

export function render(index) {
  if (mode == 1) {
    render1(index) //we should run 'awesome pattern'
  } else {
    render2(index) //we should run 'cool pattern'
  }
}

Sure, this is basically making you implement the sequencer and doesn’t interact with the UI sequencer stuff, but would give you a JS-like way of both code-reuse and compose/control patterns in any way you can imagine. Would allow code to implement sequencers, cross faders (with a little pixel composition support), segment mappers, libraries, etc.

I left the PI controller in pattern code since I wasn’t 100% sure it was the best, and wanted to allow for fixed gain since even the best AGC can cause problems if folks are using the line-in jack in a controlled environment.
If there is an auto-gain solution that is well-liked it could be added to PB (maybe a setting?), or even pushed down to the expander firmware.

There is no buffer (yet, but perhaps there could be). It was a core design goal of PB originally, mostly to allow lots of LEDs on APA102 without memory as things were tight on the ESP8266. The rendering architecture pipelines rendered pixels to a driver so that pixels can be rendered concurrently while sending data. I added an optional buffer for the WS2812 driver since the timing is tight on them, but the other LED types (and anything using output expanders) don’t buffer pixels on PB.

Obviously patterns are free to implement buffers in arrays, much as one would have to do when coding LEDs outside of PB. That said, I totally agree that a PB supported pixel buffer could make things easier + faster that arrays.

The intention here is to limit max brightness % in settings for power for a consistent appearance and static power management. Conversely, dynamic power management alters what appears on LEDs to stay within limits (often frame-by-frame before sending data to LEDs). I think both approaches have benefits/drawbacks and would ideally offer both options.

Implementation-wise since PB does not buffer the pixels, there is no way to limit total power draw dynamically in a holistic way - with only knowledge of power consumption from previously rendered and drawn pixels of the current frame, it would have to blank out future pixels once the power limit was reached. It would look very glitchy and so hasn’t been added yet.

2 Likes

On the roadmap. There’s a bit of work I need to do around the VM and rendering engine. Ideally I could spawn multiple light weight VMs, and even run them across both cores on the ESP32. If they are small enough I could run multiple patterns to implement multi-segmentation support along with cross-fading, and global code as well.

Forgot to mention, I have planned and left myself room to add controls for playlist items. The idea being that each playlist item would have its own controls state. So you could imagine adding multiple copies of a base pattern and adjusting each as desired. Combine that with multiple playlists, and you start to have a system were the pattern list becomes more of a library that can be instantiated and customized without the headache/cost of cloning.

2 Likes

Ah, good point, and something we can deal with in userspace for now:

Assuming the array(s) (either HSV or RGB) are prepped for display and your render runs thru them, you could have an array sum, and then array alter to reduce the total power consumption to a standard level. This would allow having very bright pixels and likely dim proportionally.

Should be easy to do a demo of this

I wrote a trivial Chrome extension the other day. It was a cinch after I waded through the chaotic documentation, inadequate error reporting, and incorrect stackoverflow answers. I could imagine it being used to enhance the PB UI non-intrusively, but we’d have to do this cooperatively to keep it maintainable.

The total power output could be adjust based on the previous frame, but that wouldn’t work for strobing patterns. Here’s that buffering demo you asked for … from my sword code, also including clamping, and yes it was easy!

// Calculate the sum of all pixels in the framebuffer ...
  total = 0
  for (n = 0; n < hPc; n++) {
    fb_Red[n] = min(1,fb_Red[n])
    fb_Green[n] = min(1,fb_Green[n])
    fb_Blue[n] = min(1,fb_Blue[n])
    total += fb_Red[n] + fb_Green[n] + fb_Blue[n]
  }
  // ... and scale them all down if necessary!
  if (total > max_total) {
    adjust = total / max_total
    for (n = 0; n < hPc; n++) {
      fb_Red[n] *= adjust
      fb_Green[n] *= adjust
      fb_Blue[n] *= adjust
    }
  }
2 Likes

Ah, good point! I guess that’s kicking the can down the road a little since it means all 3D patterns would need to be coded up in terms of polar rather than cartesian coordinates. Definite advantage for supporting the 2D patterns for free though!

Sounds good - if it helps, my jacket code looks pretty similar, which has worked well enough for my purposes. Pseudocode:

class EffectsController {
  // The total number of effects. In PB terms this would be equivalent
  // to the playlist length, rather than number of patterns.
  void effectCount();
  // Selects a particular effect
  void set(int index);
  // Move forwards/backwards a given number of effects, wrapping if needed.
  void setRelative(int index);
  // Convenience method, just calls setRelative(1)
  void next();
  // Convenience method, just calls setRelative(-1)
  void previous();
  // The currently selected effect
  int current();
  /// Output the next frame for the currently selected effect
  void render();
}

There’s a few other methods for controlling things like crossfade settings, and render() takes care of the crossfade automatically. I suspect it’s probably better to separate out the render/crossfade stuff though in a more polished/flexible API.

Gut feel is that it would be better to have a global code block that can (among other things) control the sequencer. I guess that might be more work for you to implement, but it forces clear separation of responsibilities, prevents any issues with e.g. circular imports, and because there’s a different context it would mean functions/variables/etc could be provided to the user that suit the global context better (e.g. I’m not sure beforeRender() and render() are the best entry points here). That said, your example code would certainly do the trick and provide plenty of flexibility.

I don’t have too much to offer here other than to say the AGC on the MAX9814 has worked pleasantly well for me - but I haven’t pushed it too hard and I imagine it has its limits and won’t keep everyone happy.

Ah I see! I didn’t know that, I can see why that makes things a bit more challenging! Not the end of the world because as you say the user code can always do the buffering.

Understood and makes sense. This also isn’t the end of the world, as I mentioned I’ve generally been able to modify/tweak any problematic patterns to work around this. The one case where I gave up trying to solve it was for a sound reactive pattern that could get excessively bright for large jumps in noise levels. Unlike most patterns that don’t react to external inputs, it wasn’t possible to test this pattern and be sure it would never exceed the power limit so I gave up on it rather than risk tripping the powerbanks at some random time.

That sounds awesome! It’ll certainly make for smooth crossfades compared to splitting the rendering load across a single core. Mind you, with a VM per core you could probably also split the rendering load of a single pattern, giving near double performance for CPU intensive ones?.. :grin:

IMHO this is absolutely the right way to go.

Doesn’t this result in the pixels being scaled up rather than down?

2 Likes

Suppose total=10 and max_total=20. Then total/max_total=0.5 and the multiplication divides by 2. The key optimization here is that I do a single division and lots of multiplication, instead of dividing every time which is more expensive. These are the kinds of details which save cycles. Now that there is an Array.map feature in v2 I can speed it up even more!

Yes, but your logic above says “if total is greater than max_total”… But yes the principle is correct, even if that code isn’t. Exactly as I figured you’d do. And yes, array functions should speed it up as well.

facepalm

I guess I meant max_total/total and that would explain why it sometimes still blew a fuse.

Thanks :crazy_face:

Here’s another awesome project…
But they use a RaspberryPi and Processing?

Wanted to come back to this, especially since I was playing with @Sunandmooncouture 's map last night.

Because @wizard’s transformation API includes 3D rotation, you can essentially swap any axes for others. So let’s say your led build has something complex, using the mask as a real world example.

Making the Z axis run essentially horizontal thru the mask makes the mapping simple, but many existing patterns might look better if for example, the X axis was the horizontal axis.

Yes, you could remap everything by changing the mapping code to put the X values where the Z values.are, etc.

Or you could rewrite the render3D to swap x and z values…

Or you could just do a transformation API rotate, and boom, in one or two commands (hint: rotate on the y axis, a 90 degree change will swap X and Z) the entire map is reoriented.

In fact, it’s a good argument why you want all of your axes to be in the same units. So a polar coordinate map fails this, as R is distance and A is an angle. A cylindrical adds Z as a distance, so swapping R and Z is possible (unsure if a rotation can do this, but I’d suspect yes)

A spherical coordinate system has the opposite problem, R plus 2 angles.

I need to build a model of a sphere and a cylinder to play with these, now.

But anyway, if you put XYZ coordinates into a map, you can essentially rotate so whichever axis you want (say horizontal along the mask) is where you want it … so a given pattern does the right thing. So taking a sound reactive pattern which normally does the spectrometer along the X-axis, and the volume/ bar height on the Y… If you wanted the bars to run along the Z, you could easily rotate it with one line, no pattern rewrites needed.

@wizard, consider adding a polar<->cartesian conversion into the API? That would allow rotation in any direction, as well as avoiding the userspace conversion. (to be clearer, you’d have to take 2 axes, one distance and one angle, and convert to 2 distance) which would solve the cylinder issue. (Still pondering the spherical issue… I thought it would require 4d, and it might)

1 Like

https://www.fearlessnight.com/hourglass/index.html

Hourglass done without a PB. Using a PB would be cooler :wink:

2 Likes