Color palette support?

Patterns and Code
18

Nice. Using array iterative functions, instead of the for loop, and moving the RGB call into the function, you can make it even shorter and cleaner. (I’ll do it with 3.19b shortly.)

Additionally:
It seems a huge waste to define 4 values of 256 (position, R,G,B) as a 4 value array, given that PB stores numbers in 16.16 and can compact those 4 numbers into 1 32 bit value, since the function could extract the 4 8bit values easily enough.
Plus in code, you could also write it as 2 0x hex numbers (since we can’t do hex decimals/fractionals directly), which means it’s still slightly human readable, if you know hex color codes. Seems better than the binary option with 32 1s and 0s. Decimal is still more human readable, though, so likely I’ll figure out a clean way to populate the single array.

Yes, for HDR LEDs, you’d want more bits… but for ‘normal’ usage, reducing the size would mean you could store more palettes in far less space (using less variables too)

So I’ll implement this as tightly as I can, in the hopes that @wizard will just adopt the code/method and add a basic palette selection into firmware. (Assuming people want this, of course)

@wizard, any chance of adding lerp() as built in to 3.19?

19

Is it just me, or do you wish instead of random seeming names, pallettes were sorted/ordered by some sort of general scheme?

I don’t mind seeing Lava, but Red Lava would at least sort with other reds. Red Orangey is fine too. Even Red Yellow…

Bluegreens… I’m fine with Blue Ocean, Blue Clouds, Blue Atlantica, etc.

Redgreens… Etc

Rainbow

Browns, Yellows…

At least then I’d have an idea without having to look up a palette color scheme or “try it out blindly” (over and over and over…)
If I’m wanting a pallette for a Fire effect, looking at a handful of Red palettes is easy, not scrolling thru dozens to try to pick out the ones that might work.

In that vein, when I crib these (and I will), I’m gonna rename them appropriately.

1 Like
20

If the palette bands are expressed as RGB values, they’re already sortable…

Off the top of my head, if sorting was done based on the component values of the bands presented in descending order (dropping any zeroed components) then similar color transitions would sort out typographically to be near one another:

Bands Description
bFF->b80g40 BlueOnly to BlueGreen
bFF->g80b40 BlueOnly to SeaGreen
bFF->gFF BlueOnly to GreenOnly
b80g40->gFF BlueGreen to GreenOnly
g80b40->gFF SeaGreen to GreenOnly
gFF->bFF GreenOnly to BlueOnly
rFF->r80b80 RedOnly to Purple
rFF->r80b80->bFF RedOnly to Purple to BlueOnly
2 Likes
21

+1 for lerp(); I use it all over the place for scaling sliders and wave functions.

…and I like the improved preview renderer for WLED. Does Pixelblaze generate a preview once after each edit, or does it send the preview repeatedly over the websocket? Other things like Firestorm and @zranger1’s python library seem to have trouble communicating to PB when the editor is open.

22

I like that, but any rainbow ones would get missed, etc. But I suspect any autosort will. I just want names to be grouped into human categories, instead of spread spectrum :slight_smile:

23

I believe the ‘generating’ delay we see is as it re-renders the jpg preview on compile, but the ‘live’ feed? @wizard, is that available other than via the UI? WLED has a /json/live url, which is how Scott is grabbing data in his script

24

And it does:

//  palettes copied directly from http://dev.thi.ng/gradients/
var paletteGreenRed  = [ [ 0.5, 0.5, 0], [0.5, 0.5, 0], [0.5, 0.5, 0], [0.5, 0, 0] ];
var paletteRainbow1 = [ [0.500, 0.500, 0.500], [0.500, 0.500, 0.500], [1.000, 1.000, 1.000], [0.000, 0.333, 0.667]];
var paletteHarvest =  [ [ 0.5, 0.5, 0.5], [0.5, 0.5, 0.5], [1.0, 1.0, 0.5], [0.8, 0.9, 0.3] ];

//  Calculate and render the palette color at a particular point.
function paletteAt(t, palette) {
  _r = palette[0][0] + palette[1][0] * cos(PI2 * ((palette[2][0] * t) + palette[3][0]));
  _g = palette[0][1] + palette[1][1] * cos(PI2 * ((palette[2][1] * t) + palette[3][1]));
  _b = palette[0][2] + palette[1][2] * cos(PI2 * ((palette[2][2] * t) + palette[3][2]));
  rgb(_r, _g, _b);
}

export function render(index) {
  t = time(0.05);
  x = index/pixelCount;
  if (t < 0.3) paletteAt(x, paletteRainbow1);
  else if (t < 0.6) paletteAt(x, paletteGreenRed);
  else paletteAt(x, paletteHarvest);
}

Nice. Thanks @wizard and thanks @scruffynerf!

25

@pixie, are you running the editor and Firestorm/python on the same machine? If so, once the editor is up, that’d stop any other program from connecting from the same IP address. The web browser very sensibly doesn’t want to share its open sockets with other apps.

If you need to have the editor and another websocket connection going at the same time, you can set up multiple IP addresses on the same NIC and run Firestorm, etc. on the secondary IP.

I do this all the time. It’s handy for separating traffic for various tasks and services. It also gives you an easy way to get more than one port 80 on a machine, for programs that absolutely insist on that.

On recent Linux w/GUI, just go to the IPv4 settings in the Network manager and add as many addresses as you need.

It’s not much more complicated in Windows, but it’s harder to find to the right place to do it. Here’s a walkthrough of a couple of methods.

To summarize, you’ll need to go to the TCP/IP v4 properties dialog for your network adapter., switch yourself to a static IP address, then click the “Advanced” button and you can add addresses from there. Or you can just use one of the handy command line options in the walkthrough above.

26

Nice, I’m actually wondering if reversing the way this is put together makes sense…

Using A,B,C, and D do a [R, G, B], so you end up with 4 3-arrayed (RGB) values,
and you calculate a value for each of R, G, B.
But then http://dev.thi.ng/gradients/ offers a global tweak as well, which is kinda nifty.

What if it’s done as R, G, B, and then store A,B,C,D for each? That’s 3 values with an array of 4 in each, but now, the color channels are easily separated… so I could modify the R channel as a whole array, rather than parts of 4 arrays. Not to mention that A,B,C,D are all relative to PI.

Either way, considering seeing if compacting these into a 16.16 also makes sense…
That would make an entire palette reducible down to just a scant 3 or 4 single values. That seems an amazing reduction… and totally worthwhile.

“oh, that palette? It’s 0x12345678, 0x345AE444, 0x13405544, 0xFF00EEDD”

27

In the end, I didn’t bother with bit-packing the band values, because array initializers make the palette code short and sweet enough on its own:

function LERP(percent, low, high) { return low + percent * (high - low); }
//  A palette is a variable number of bands, each containing a startIndex, R, G, and B component.
function fastLedPaletteAt(v, palette) {
  v = clamp(v << 8, 0, 255); 
  for (idx=0;idx<palette.length;idx++) {
    if (v <= palette[idx][0]) { //  We're at the beginning of the band.
      if (v == 0) { rgb(palette[idx][1] >> 8, palette[idx][2] >> 8, palette[idx][3] >> 8); }
      else {  //  We're in the middle of this band, so LERP to find the appropriate shade.
        scale = (v - palette[idx-1][0]) / (palette[idx][0] - palette[idx-1][0]);
        rgb(LERP(scale, palette[idx-1][1], palette[idx][1]) >> 8, LERP(scale, palette[idx-1][2], palette[idx][2]) >> 8, LERP(scale, palette[idx-1][3], palette[idx][3]) >> 8);
      }
      break;
    }
  }
}

// Gradient palette "quagga_gp" originally from http://soliton.vm.bytemark.co.uk/pub/cpt-city/rc/tn/quagga.png.index.html; converted for FastLED with gammas (2.6, 2.2, 2.5)
var paletteQuagga = [ [ 0,   1,   9,  84], [ 40,  42,  24,  72], [ 84,   6,  58,   2], [ 168,  88, 169,  24], [211,  42,  24,  72], [255,   1,   9,  84] ];
// Gradient palette "scoutie_gp", originally from // http://soliton.vm.bytemark.co.uk/pub/cpt-city/rc/tn/scoutie.png.index.html; converted for FastLED with gammas (2.6, 2.2, 2.5)
var paletteScoutie = [ [ 0, 255,156,  0], [127,   0,195, 18], [216,   1,  0, 39], [255,   1,  0, 39] ];


//  Pattern
export function render(index) {
  t = time(0.05);
  x = index/pixelCount;
  if (t < 0.5) fastLedPaletteAt(x, paletteScoutie);
  else fastLedPaletteAt(x, paletteQuagga);
}
2 Likes
28

@zranger1, I don’t think that’s the case.

A websockets client such as the one inside a webbrowser can only connect to a single server at a time, but a websockets server such as the one inside the PB’s webserver can handle multiple clients. If a server gets confused by multiple client connections from the same IP address, then it’s not identifying connections appropriately – that is to say, uniquely.

At any rate, PB can currently handle simultaneous connections from the same Windows computer (web UI inside Edge, and your python library connecting from a WSL session), but the python connection frequently disconnects with an error message if the preview band is running in the Editor window; it doesn’t seem to mind if the preview band is running atop the Pattern List page.

Also I forget sometimes that I have an editor window open and open a second, but that makes the response time slow down so drastically that I eventually notice and check the other browser tabs…

29

This is interesting. You can… I just tested it… open a browser window to a Pixelblaze from multiple different browsers on the same machine simultaneously. I had Edge, Chromium and Firefox going, and everything appears to be working, all simultaneously watching variables change.

I’ll look into this - it’d be interesting to know if the browsers were doing anything different with the socket, or just using a better websocket library. (And actually, being able to run the Python client while you had a browser window open didn’t work at all when I started. Python just refused to connect if the browser already had a websocket connection. This is an improvement!)

30

I took a closer look half an hour ago and the python library was throwing an exception from inside a call to waitForEmptyQueue(1000) following a call to setActivePattern(). I replaced the waitForEmptyQueue(1000) call with a sleep(1000) and it’s run without error since…

1 Like
31

I don’t think this should be the case. Older versions had only global state to track settings for whether or not preview data and updates should be sent, but this has improved, and wouldn’t have prevented a WS connection anyhow. Sure each connection does add load, and there is a system limit of 5 (? IIRC) simultaneous connections. That includes any zombie connections waiting to time out. Older versions would also boot every connection if any stalled, but these are all resolved to the best of my knowledge.

@Scruffynerf yes preview data for the first 100 pixels are streamed over the websocket if it’s enabled for that connection. These come in as binary preview frames. WLED seems to do something similar these days, but as a hex string in JSON.

At some point I will lift the 100 pixel restriction and use the pixel map to render 2d/3d live previews. There will be some limit, and I might do something like sending a subset by skipping pixels or something.

1 Like
32

@Wizard, thanks for keeping up with the small, edge-case network issues! Things have been a lot smoother in the last couple of firmware versions.

And thanks for localizing the Python problem, @pixie. I’ll see if I can find a graceful way to make waitForEmptyQueue() behave itself if the connection momentarily stalls, and will get it checked in ASAP. I’ve got a couple of bug fixes sitting around waiting to go into PyPl anyway.

33

So something akin to that script could be written to poll the PB, generating a 100pixel graphic. I’m not sure that’s worthwhile given that we already get the preview jpg as linked above (5 seconds at 30fps) But it’s good to know it’s possible. In the case of WLED, it’s more critical as documentation, since the default patterns and palettes aren’t really changable.

34

So I tried to pack this cosine gradient code down into 16.16 (8 bits for each of 4 arguments, per color channel) , and it turns out the rounding errors of going to 8bit cause enough of a problem (due to cosine and multiplication, small changes quickly multiply effect) that it stops working as expected, so the palette is just wrong.

I’d still like to avoid the arrays in arrays thing though. so I’ll likely just continue to play with a way to optimize this, but attempt #1 to pack tightly was a failure. Weirdly, I’d see exact matches on red, but totally off on green and blue channels, despite that each was doing the exact same math.

35

If you want to get rid of the nested arrays, just remove the interior square brackets and cross-multiply the array indices:

var paletteRainbow1 = [ 0.500, 0.500, 0.500, 0.500, 0.500, 0.500, 1.000, 1.000, 1.000, 0.000, 0.333, 0.667];

//  Calculate and render the palette color at a particular point.
function paletteAt(t, palette) {
  _r = palette[0] + palette[3] * cos(PI2 * ((palette[6] * t) + palette[9]));
  _g = palette[1] + palette[4] * cos(PI2 * ((palette[7] * t) + palette[10]));
  _b = palette[2] + palette[5] * cos(PI2 * ((palette[8] * t) + palette[11]));
  rgb(_r, _g, _b);
}

But unless you’re desperately short of space, why bother trying to byte-pack them? Readability and the ability to cut-and-paste coefficients from the palette designer webpage are features worth having…

2 Likes
36

Yeah, the single array approach was my next step too. And I really liked the idea of packing it because ideally, you could store a couple of dozen patterns by default and use little space to do so. The 1/4 size was intriguing, but as it turns it, it’s compression is too great and the loss is noticable.

Having played with both FastLED palettes and IQ’s cosine gradients last night:
The FastLED ones are mostly nice for compatibility and porting and being able to say “ok, you want PB to run all of the WLED patterns, we can do that” but fairly limited usage beyond that.
But the IQ ones are hugely useful for PB and work so well, I’m going to likely implement in many uses, because dropping it in as a replacement for H (which right now, is really where most of our pattern color variations come from, just changing the value of hue in some smooth way) seems like it’s loaded in powerful options.

Making a variant of the default time+hue equals rainbow pattern, but having a huge variety of easy to select color options beyond just rainbow seems like a good demo of that. Yes, you could easily write a similar pattern that did smooth variation only in hue between particular values of blue and green for example, but that seems more work than 12 numbers and one quick function.

37

So I got a bit to review @jeff’s utility code with fresh eyes, now that I’ve played with WLED/FastLED (shortened to WLED, hereafter) palettes and IQ’s palettes. And it’s interesting because Jeff’s approach is more PB centric (0…1, including rollover), but isn’t really compatible with many WLED palettes (which use an explicit palette map, with up to 255 positions, so you can specify a color at position 0, and then say position 128, and it’s a smooth transition between them, but then the next step might be 150, with a smooth transition to there. I think Jeff’s approach is 8 steps, but it’s also HSV compliant, while the WLED and IQ methods are mostly RGB, by default (I believe they could work with HSV, in many cases).

The IQ method of cosine gradients is very much a ‘quick and dirty’ method, but its not really suited to specific custom palettes, but more of a mathematical offering of many palettes. This works from a PB perspective in that it’s using 0…1, it’ll wrap since it’s using cosine math, and it’s relatively light in terms of definition. A mere 12-16 values should be enough. So an IQ palette is absolutely a winner in terms of “I don’t want a rainbow, what else do you have?” type palettes, where you want to replace a color ‘flow’ with something else.

WLED palettes, being focused on 8bit, are essentially tables of position+RGB, with a minimal size of 2 (0[start] and 255[end]) with as many midpoints as you’d like to add (between 1-254). It would be easy enough to modify all of those values to 0…1, so you’d have something akin to Jeff’s palettes, but more flexible, and more ‘standardized’ ie lots of WLED/FastLed users, but also sites like cpt-city use the usual RGB 0-255 ranges but can also use 0…1 for position.
For an example, you compare different formats for sd | Blue-Gloss 0
you’ll notice that the .pg and .c3g versions (among others) are using 0…1 for positioning.

The negative to all of these is that most of them don’t cycle/loop between 0 and 1, so you end up with a ‘hard edge’ if you try to wrap it. This isn’t always an issue, depends greatly on usage. But it does mean it’s not always a clean replacement for just wrapping ala hsv()

As for which (if any) is better, it’ll depend greatly on how (and who) wants to implement palette support in whatever patterns. For example, if you’re porting WLED patterns, you’ll almost certainly want to support palettes as it expects, if you want it to look ‘like the real thing’. But if you are modifying existing patterns that are ok with wrapping, thanks to the use of hsv(), then a wrapping palette method makes more sense.

Moving forward, I think a lot will depend on how cleanly the implementation is, and how ‘featureful’ it is. If we end up with some form of ‘library’ include, then it might not matter, but in the short term, if you need to include all of the “palette code” into your pattern, then a minimal amount of code is going to win the day in most cases. I wouldn’t want a short pattern to require twice as much code for palettes as the new code itself. If Ben decides to include some form of palette support in the firmware at some point, this might change, but… until we demonstrate what’s really useful and would benefit from firmware including, I’m not sure it’s clear what would be worth stashing into firmware.

Including the existing WLED palettes into a port of WLED patterns/effects makes huge sense. I’ll discuss the WLED porting effort elsewhere (there is a thread for it).

If it’s possible to define lots of ‘wrapping’ palettes via IQ’s method, and even make random palettes viable, or have a robust selection in a quick/small bit of code, is that ‘better’ than having some discreet non-linear palettes that require something akin to the WLED/cpt-city style ‘positioning’ method? I don’t know the answer, and there may not be a definitive answer, it probably depends on usage and preference… but we’ll see. My hope is we flesh out “both” methods and see what get used and how. And to that end, I’d like to see @Jeff’s utility code tweaked to support more of the WLED method, along with code above, so we have a solid ‘positional’ palette library. and the IQ method (which is definitely shader friendly) will also get a solid ‘library’ approach, and we can just see what is more useful.