Sliders and pickers are persisted a short while after the value stops changing. When a pattern loads, they are called with the saved values after the main body runs, before the first animation frame. They are input only functions. I had planned output only functions (ones that return a value) for UI controls for display.
With the persistentName function example, I wouldn’t know what state to persist unless I poke behind the curtain and looked at the function definition to see that myVariable is being set. Ideally the interface for persistence is intentionally visible, something exported, or an API invoked from the code.
An API to push new control values count be interesting!
Using the term sticky to refer to the new function:
Sliders, pickers, AND stickies persist a short while after the value stops changing. In the case of stickies, it’s displayed as a numeric value (optional?) but otherwise behaves like a slider or picker. When a pattern loads, they are called with the saved values after the main body runs, before the first animation frame. They are input only functions
So I add this to my pattern:
export var mode = 0 // default to zero
export function stickyMyMode(v){
mode = v%5
}
export function beforeRender(delta){
isPressed = checkButton() // this example function checks a pin to see if a button is pressed and returns 1 or 0
if (isPressed) {
// Increment my mode variable thru 5 different modes, wrapping at 5 (so 0-4)
stickyMyMode(mode++)
}
}
export function render(index){
if (mode == 0)
Etc etc etc ..
}
On pattern load, MyMode is initialized with the saved value, and the function is called ONCE, before the first animation frame. Which populate the mode variable, changing it from 0 (default) to whatever was saved. You don’t care about mode, but do care about stickyMyMode, just like you persist sliderMySlider
From a syntax point of view, I think I’d prefer either a decorator:
// variables declared on the line below this will be automatically saved
// @persist
var saveThis;
or an additional keyword at the declaration:
persist var x = 42;
export persist var x = 42;
etc.
From a behavioral standpoint, can you automatically load persistent variables when the pattern starts, and
save them right before it’s unloaded?
Right now, saving control variables causes some display glitching, so it’s not something I think you’d really want to do often while the pattern is running.
I’d rather make it a function … Consistent with the current way sliders and pickers work
A slider is an interface to a single value (0…1)
Everytime you adjust the slider, once you stop, it persists.
A picker is a 3 dimensioned slider. Different UI, but really just 3 values (all 0…1)
This would be a slider with 1 dimension again but no UI, and the only way to change it is to invoke the function. If we get the full 16.16 range, awesome… But I could live with 0…1
That would be fine too. I’m good with whatever makes for the least expensive, most solid and reliable implementation.
It occurs to me that this would be a good way to get a start on that file system/library/palettes thing – in addition to just “persistent”, you could have a “common” variable area. Any pattern could grab a variable from there by name.
What happens if you invoke the sliderMySlider function with a value inside a pattern explicitly?
Does it:
A) call the function?
B} change the UI to match?
The problem with this is that calling a function just runs the function. Calling stickyMyMode(mode++) just runs stickyMyMode, there’s nothing intercepting the argument.
UI Controls work as input only functions because the system knows the value (either because a user poked it in the UI, or a previous value was saved).
To make it work the same way, a system API would be needed for setting the value of “MyMode” (or perhaps any control?) – which I imagine would then invoke stickyMyMode() with the new value, and persist it silently in the background when it stopped moving.
Yes, the UI sends down new slider values. After a while, it sends them with a save command.
When a pattern loads, the engine reads saved controls and calls the appropriate functions.
Push a value into persistence (no PB > flash commands)
Those seem the minimal requirements going forward. Nobody is asking (that I know of) for PB>UI commands, though a bunch of us want more UI options like buttons/switches (on/off) which is really a binary slider (so UI change but no functionality change).
So some way to push a value into persistence is the real ask here. Given concerns of flash wearout… How does slider persistence (which is basically move the slider, when you stop, single byte is saved in flash on a request from UI) aboid wearout? Solely by not having save constantly?
So the key to avoiding accidentally wearing out the flash is to make persistent variables avoid constant saving.
In my mind, this absolutely means NOT using a var approach. People tend to use vars for changing variables and accidentally using a rapidly changing variables would cause rapid flash writes, unless some sort of throttle existed.
Function still makes more sense to me:
We can control how often we call a function, like on a button push, hopefully with a key debounce.
Abuse is possible, but it would be easy to add protections since we can write code to avoid rapid writes. We could try to control frequency of write, or time out (delta?)
A Save argument to make it explicitly push to flash, seems pointless, I can write a normal function otherwise to do the above, and within it call the Persistence function, rather than make the persistence function not always persist.
Simple is better. Warn people that this function should be called infrequently to avoid wear.
So I think we’re back to my original request:
We’re asking the PB to persist Name as value.
You don’t need to worry about myVariable
And we never get access to Name, just can send a byte (0…1) or a 16.16 to it, knowing it’ll last between runs of the pattern.
This could give, as an example of other uses, a pattern run counter. First time thru your PB, the counter is zero. Not as part of the looping code but as part of setup code, the counter is incremented, So each run of the pattern is one higher (32k runs). This could be used as a seed value so every run of pattern is unique (mod 32k). Or every 5 runs, do something, as an example.
Absolutely still abusable, so maybe you need to add a way to disable persist saves? global Setting?
Where is the slider/picker value stored? Its not in the .epe, right? (Is a loaded pattern still in flash as .epe, or unpacked, or what?)
It’s connected to the pattern ID? So sliders don’t collide with similarly named slider of another pattern?
What if there was a wildcard storage? Where name collision could happen? Then global persistent sliders could happen? So…
export function gsliderGlobalSpeed(v)
would store the slider value in the wild card/global storage instead of the pattern ID storage bucket… And all patterns would share that value.
Actually - the var approach - using a decorator or keyword near the variable declaration, then dealing with persistence only at pattern load/unload time - makes it so you can save users from doing catastrophically clueless things with the persist function, like calling it from render(). They will do that. You know they will.
So, ideally, the eventual solution wouldn’t require the user to know or care much about the inner life of an ESP32.
Also, we might want a way, besides deleting the pattern, to get rid of stored variables. Possibly just letting a pattern call clearPersistCache() or something would be enough.
The epe is generated on the fly when you want to export a pattern and is in a portable JSON format that is easier to parse. It includes most of the important portable bits from the pattern like the source code, preview, name, etc, but doesn’t include the compiled binary of the pattern which isn’t necessarily designed to be portable between PB variants (e.g. v2 and v3).
The actual patterns on PB are stored at /p/<id> in an efficient packed binary structure and controls data is stored at /p/<id>.c in JSON in a separate file. Controls data are much smaller than the pattern and almost always fit inside one filesystem block and cause a minimal amount of flash wear.
Perhaps, but it is easier to think of the value as a single thing, and useful in a network scenario. I have heard that when the brightness setting changes (from any client), that a websocket message should be emitted so that other clients can update their UI state. If your code pushed a new state for a control, it would make sense to reflect that in the UI so that they were consistent.
That still doesn’t give the engine a way to get the value if you call that from inside your code, unless it does something sneaky in the compiler/VM. It would have to inject code with API calls into your function definition, or set traps to intercept the function call itself. Exporting a var/function just exposes the name and variable (in this case, the var holds a function), and for controls that is enough to work out the UI control type and name, and call that function with some data.
It is possible with a single exported function, but to do this with a single function would be complicated. First lets look at how a pair of functions could work:
var foo = 32 // default value when the pattern initializes
export function loadPersistentMyName(valueFromStorage) {
foo = valueFromStorage // this happens IF there was a previously saved value
}
export function savePersistentMyName() {
return foo // when this function is called, we return the value we want to save
}
The trick/downside here is that the MyName has to be the same between both functions or it will mysteriously break. There’s no link between the foo variable and the MyName persistent control outside of the function implementation – the engine is unaware of foo, only a persistent thing called “MyName” and the 2 load/save functions for it.
The engine would call loadPersistentMyName() after the main code initialized with a previously saved value, or it won’t call it at all, just like a control.
At some point or points in the future, the engine could call savePersistentMyName() and save the return value for a persistent value called “MyName”, probably along with controls data.
To combine these into a single function, things get more complicated. This function has to handle both cases:
var foo = 32 // default value when the pattern initializes
export function loadOrSavePersistentMyName(action, maybeValueFromStorage) {
if (LOAD_ACTION == action) {
foo = maybeValueFromStorage // a loading action is called IF there was a previously saved value
} else if (SAVE_ACTION == action) {
return foo // when this action is called, we return the value we want to save
}
}
To my eye, this is much more complex and requires more boilerplate code, with the only benefit being that you don’t have to worry that “MyName” is consistent across a pair of functions. This handler function is doing too may things, and doesn’t behave like a proper function anymore (conditionally returning a value, or having no return at all).
A single var approach could be done, and thinking about it a bit more, should be a naming convention + export so that PB code could continue to be compatible with JS, but might have a bit too much or too little magic.
The engine could change the value just after initialization code ran, and read it when necessary, and would have the default value during pattern init. The code wouldn’t have a way of knowing if/when the value changed and potentially invalid values could be injected, and would have to ensure the variable wasn’t holding intermediate values when it could be read for saving. The pattern code would have to be more defensive about how it used that variable everywhere.
I liked the saveSlot / readSlot idea, or another storage api like function as it solves the problem, allows greatest flexibility, and doesn’t add some maybe unexpected behavior.
Another interesting set of issues: What can be stored and how much? Single values? (That would be my preference for an initial version) Arrays? (What happens in the case of nested arrays?)
Are there limits on per-pattern storage, both in terms of size and number of variables? Is there a limit on overall variable storage space?
I’m still not fond of giving users the opportunity to beat their flash ram to death at 100fps via an unrestricted API. It’ll result in a ton of service calls.
Sure, those of us who are already experienced programmers can be trusted to restrict the rate, but as we’ve seen from the latest tutorial poll, there are a lot of people who are more into the LED art construction side of things. They’re really important to Pixelblaze’s future. Probably more so than we are. It helps them out if we can keep the “friction” of coding - and the possibility of bricking your Pixelblaze - as low as possible.
I actually agree this is the best approach to avoid unintended flash wear by rogue programmers who use any user initiated method. If @wizard can make it seamlessly behind the scenes, that would help. Not perfect, as if you ran a pattern and the PB rebooted/powercycled, you’d lose the current value but at least any intentional change away would record the current state.
Assuming this hypothetical storage api saveSlot could protect the flash by committing when necessary or at an interval - however I don’t think it’s a huge cause for concern it’s not like the beginner first time pattern is going to be looking to this api.
I was thinking of implementing it so that the save slot values were held in memory until the pattern gets unloaded, so you could write to them as often as you like without worrying. Similar to the load/save handler functions but API driven instead of magic exports.
I’m thinking it would be suitable for perhaps a dozen or so scalar values. The example use case is for storing a few modes/settings that change from button presses.
These should be items that you want to be persistent, and ideally based on user input, not intended as a way to capture the intermediate pattern state or pixel buffer so that it seamlessly resumes when switching patterns. Doing that could cause excessive flash wear e.g. if used in a repeating playlist.
Which reminds me, in general all of the persistence in PB today originates from a user interaction. There’s nothing automated that can wear flash out intentionally.
Perhaps this use case can be solved in another way entirely.
What if you could define a pin as a hardware button control that supported a stepped mode selection. That would be a whole lot less button handling code, and the persistence could happen automatically as currently done with UI controls. It’s not quite as flexible, but this seems like a common use for a button.
pin = 26
numModes = 4
pinMode(pin, INPUT_PULLUP)
persistentModeButton(pin, FALLING, numModes, modeChangeHandler)
function modeChangeHandler(mode) {
// called after init with saved values, or any time button is pressed
// mode ranges from 0 to numModes-1 before wrapping back to 0
}
It’s like registering an on change callback, but the system can do all the button debouncing, state incrementing, and persistence logic. Setting numModes to 2 would give you a simple toggle button.
If I go this route, I would add a non-persistent modeButton() version of the API as well.
The caveat here is that this would allocate some resources, so it wouldn’t be suitable for reconfiguring on the fly unless it was changed to a register / unregister kind of API pair.
I like persistentModeButton()/modeButton() a lot! Bet you’d see a lot more button-controlled builds.
Let me switch sides here, and give my use case for the saveSlot() approach too. It’s pretty simple – it would make the Multisegment pattern (and similar things) actually workable without an external controller to save the per-segment parameters.
Right now, it uses one controller to select a segment, and at least one other to set parameters for that segment. When you leave the pattern, the control settings are saved, but the accumulated segment settings are not. With even the ability to save a few scalar values, they could be.
I’m not sure what you’re proposing here, as my use case wasn’t pin or button based.
I tweaked a handful of settings (sliders and pickers) and I leave the pattern and come back, and it’s restored. The use case here is to have a way to save a NON-slider or picker value, and have exactly the same behavior. Perhaps the code wants to record a state, but my example use case above is also good: play counter, or rotating between modes, sans user interaction
So here’s a concrete example using current playlist as the driver:
I build a playlist of 3 patterns.
Pattern one is normal, call it P.
Pattern two has a mode flip… each time it runs, it either does A or B
Pattern three has a mode flip between 3 states, D, E & F
When I play the playlist, the behavior would be
PADPBEPAFPBDPAEPBF (it would then repeat)
This is a pretty simple example but I could come up with more complex (so please don’t ask me to just build the above sequence as the playlist…) My point is, doing a playlist of this level is entirely depending on a way for a pattern to record a value that sticks thru load and is updated on unload by the code. And yes, I’m aware that running this playlist would mean a flash write likely happens on 2/3s of the pattern unloads.
It could be argued that isn’t much, or is too much.