// @persist
var myPersistentVar = 34 //34 is a default used first time it runs
Or naming convention on exported vars like UI controls?
export var persistentMyVar = 34
Maybe numbered slots for saving data?
var mySetting = readSlot(0, 34)
saveSlot(0, mySetting)
Perhaps a control-like handler function pair, one called on startup with saved values, and another called when the pattern is going to be shut down that returns the value to save?
How should it interact with live editing? Should it persist, and if so you’d need a way to reset it if you wanted to change what/were something was stored, right?
In memory only, gone between restarts, or persistent on flash? Scalar values only, or arrays too? What about flash wear? I wouldn’t want to add something that could easily wear out flash on accident.
How do slider or picker values persist between restarts? Wouldn’t it be something similar? Are slider/picker values saved in flash memory?
In my case, another solution could be to somehow “pass” the variable to the slider/picker. I have set a button to advance hue by .2 once pressed, hue was set to 0.3 via the slider, now it becomes .5.
For the sake of economy, could it be just one single variable with a particular name that could retain 1 value from every pattern?
Can you discuss how it currently saves the slider/colorpicker variables that do persist?
I was thinking that could be used, except the only time the function is invoked is when the slider/picker is actually moved.
My first thought is literally the same sort of function, except in this case, it needs to be explicitly invoked, ie:
export function persistentName(value to pass in){
// This is called only when someone invokes it.
//On startup only, it is called too
myVariable = value
}
So myVariable is set on start of pattern, and otherwise is only set when the function is called.
Sliders and pickers are persisted a short while after the value stops changing. When a pattern loads, they are called with the saved values after the main body runs, before the first animation frame. They are input only functions. I had planned output only functions (ones that return a value) for UI controls for display.
With the persistentName function example, I wouldn’t know what state to persist unless I poke behind the curtain and looked at the function definition to see that myVariable is being set. Ideally the interface for persistence is intentionally visible, something exported, or an API invoked from the code.
An API to push new control values count be interesting!
Using the term sticky to refer to the new function:
Sliders, pickers, AND stickies persist a short while after the value stops changing. In the case of stickies, it’s displayed as a numeric value (optional?) but otherwise behaves like a slider or picker. When a pattern loads, they are called with the saved values after the main body runs, before the first animation frame. They are input only functions
So I add this to my pattern:
export var mode = 0 // default to zero
export function stickyMyMode(v){
mode = v%5
}
export function beforeRender(delta){
isPressed = checkButton() // this example function checks a pin to see if a button is pressed and returns 1 or 0
if (isPressed) {
// Increment my mode variable thru 5 different modes, wrapping at 5 (so 0-4)
stickyMyMode(mode++)
}
}
export function render(index){
if (mode == 0)
Etc etc etc ..
}
On pattern load, MyMode is initialized with the saved value, and the function is called ONCE, before the first animation frame. Which populate the mode variable, changing it from 0 (default) to whatever was saved. You don’t care about mode, but do care about stickyMyMode, just like you persist sliderMySlider
From a syntax point of view, I think I’d prefer either a decorator:
// variables declared on the line below this will be automatically saved
// @persist
var saveThis;
or an additional keyword at the declaration:
persist var x = 42;
export persist var x = 42;
etc.
From a behavioral standpoint, can you automatically load persistent variables when the pattern starts, and
save them right before it’s unloaded?
Right now, saving control variables causes some display glitching, so it’s not something I think you’d really want to do often while the pattern is running.
I’d rather make it a function … Consistent with the current way sliders and pickers work
A slider is an interface to a single value (0…1)
Everytime you adjust the slider, once you stop, it persists.
A picker is a 3 dimensioned slider. Different UI, but really just 3 values (all 0…1)
This would be a slider with 1 dimension again but no UI, and the only way to change it is to invoke the function. If we get the full 16.16 range, awesome… But I could live with 0…1
That would be fine too. I’m good with whatever makes for the least expensive, most solid and reliable implementation.
It occurs to me that this would be a good way to get a start on that file system/library/palettes thing – in addition to just “persistent”, you could have a “common” variable area. Any pattern could grab a variable from there by name.
What happens if you invoke the sliderMySlider function with a value inside a pattern explicitly?
Does it:
A) call the function?
B} change the UI to match?
The problem with this is that calling a function just runs the function. Calling stickyMyMode(mode++) just runs stickyMyMode, there’s nothing intercepting the argument.
UI Controls work as input only functions because the system knows the value (either because a user poked it in the UI, or a previous value was saved).
To make it work the same way, a system API would be needed for setting the value of “MyMode” (or perhaps any control?) – which I imagine would then invoke stickyMyMode() with the new value, and persist it silently in the background when it stopped moving.
Yes, the UI sends down new slider values. After a while, it sends them with a save command.
When a pattern loads, the engine reads saved controls and calls the appropriate functions.
Push a value into persistence (no PB > flash commands)
Those seem the minimal requirements going forward. Nobody is asking (that I know of) for PB>UI commands, though a bunch of us want more UI options like buttons/switches (on/off) which is really a binary slider (so UI change but no functionality change).
So some way to push a value into persistence is the real ask here. Given concerns of flash wearout… How does slider persistence (which is basically move the slider, when you stop, single byte is saved in flash on a request from UI) aboid wearout? Solely by not having save constantly?
So the key to avoiding accidentally wearing out the flash is to make persistent variables avoid constant saving.
In my mind, this absolutely means NOT using a var approach. People tend to use vars for changing variables and accidentally using a rapidly changing variables would cause rapid flash writes, unless some sort of throttle existed.
Function still makes more sense to me:
We can control how often we call a function, like on a button push, hopefully with a key debounce.
Abuse is possible, but it would be easy to add protections since we can write code to avoid rapid writes. We could try to control frequency of write, or time out (delta?)
A Save argument to make it explicitly push to flash, seems pointless, I can write a normal function otherwise to do the above, and within it call the Persistence function, rather than make the persistence function not always persist.
Simple is better. Warn people that this function should be called infrequently to avoid wear.
So I think we’re back to my original request:
We’re asking the PB to persist Name as value.
You don’t need to worry about myVariable
And we never get access to Name, just can send a byte (0…1) or a 16.16 to it, knowing it’ll last between runs of the pattern.
This could give, as an example of other uses, a pattern run counter. First time thru your PB, the counter is zero. Not as part of the looping code but as part of setup code, the counter is incremented, So each run of the pattern is one higher (32k runs). This could be used as a seed value so every run of pattern is unique (mod 32k). Or every 5 runs, do something, as an example.
Absolutely still abusable, so maybe you need to add a way to disable persist saves? global Setting?
Where is the slider/picker value stored? Its not in the .epe, right? (Is a loaded pattern still in flash as .epe, or unpacked, or what?)
It’s connected to the pattern ID? So sliders don’t collide with similarly named slider of another pattern?
What if there was a wildcard storage? Where name collision could happen? Then global persistent sliders could happen? So…
export function gsliderGlobalSpeed(v)
would store the slider value in the wild card/global storage instead of the pattern ID storage bucket… And all patterns would share that value.
Actually - the var approach - using a decorator or keyword near the variable declaration, then dealing with persistence only at pattern load/unload time - makes it so you can save users from doing catastrophically clueless things with the persist function, like calling it from render(). They will do that. You know they will.
So, ideally, the eventual solution wouldn’t require the user to know or care much about the inner life of an ESP32.
Also, we might want a way, besides deleting the pattern, to get rid of stored variables. Possibly just letting a pattern call clearPersistCache() or something would be enough.
The epe is generated on the fly when you want to export a pattern and is in a portable JSON format that is easier to parse. It includes most of the important portable bits from the pattern like the source code, preview, name, etc, but doesn’t include the compiled binary of the pattern which isn’t necessarily designed to be portable between PB variants (e.g. v2 and v3).
The actual patterns on PB are stored at /p/<id> in an efficient packed binary structure and controls data is stored at /p/<id>.c in JSON in a separate file. Controls data are much smaller than the pattern and almost always fit inside one filesystem block and cause a minimal amount of flash wear.
Perhaps, but it is easier to think of the value as a single thing, and useful in a network scenario. I have heard that when the brightness setting changes (from any client), that a websocket message should be emitted so that other clients can update their UI state. If your code pushed a new state for a control, it would make sense to reflect that in the UI so that they were consistent.
That still doesn’t give the engine a way to get the value if you call that from inside your code, unless it does something sneaky in the compiler/VM. It would have to inject code with API calls into your function definition, or set traps to intercept the function call itself. Exporting a var/function just exposes the name and variable (in this case, the var holds a function), and for controls that is enough to work out the UI control type and name, and call that function with some data.
It is possible with a single exported function, but to do this with a single function would be complicated. First lets look at how a pair of functions could work:
var foo = 32 // default value when the pattern initializes
export function loadPersistentMyName(valueFromStorage) {
foo = valueFromStorage // this happens IF there was a previously saved value
}
export function savePersistentMyName() {
return foo // when this function is called, we return the value we want to save
}
The trick/downside here is that the MyName has to be the same between both functions or it will mysteriously break. There’s no link between the foo variable and the MyName persistent control outside of the function implementation – the engine is unaware of foo, only a persistent thing called “MyName” and the 2 load/save functions for it.
The engine would call loadPersistentMyName() after the main code initialized with a previously saved value, or it won’t call it at all, just like a control.
At some point or points in the future, the engine could call savePersistentMyName() and save the return value for a persistent value called “MyName”, probably along with controls data.
To combine these into a single function, things get more complicated. This function has to handle both cases:
var foo = 32 // default value when the pattern initializes
export function loadOrSavePersistentMyName(action, maybeValueFromStorage) {
if (LOAD_ACTION == action) {
foo = maybeValueFromStorage // a loading action is called IF there was a previously saved value
} else if (SAVE_ACTION == action) {
return foo // when this action is called, we return the value we want to save
}
}
To my eye, this is much more complex and requires more boilerplate code, with the only benefit being that you don’t have to worry that “MyName” is consistent across a pair of functions. This handler function is doing too may things, and doesn’t behave like a proper function anymore (conditionally returning a value, or having no return at all).
A single var approach could be done, and thinking about it a bit more, should be a naming convention + export so that PB code could continue to be compatible with JS, but might have a bit too much or too little magic.
The engine could change the value just after initialization code ran, and read it when necessary, and would have the default value during pattern init. The code wouldn’t have a way of knowing if/when the value changed and potentially invalid values could be injected, and would have to ensure the variable wasn’t holding intermediate values when it could be read for saving. The pattern code would have to be more defensive about how it used that variable everywhere.
I liked the saveSlot / readSlot idea, or another storage api like function as it solves the problem, allows greatest flexibility, and doesn’t add some maybe unexpected behavior.
Another interesting set of issues: What can be stored and how much? Single values? (That would be my preference for an initial version) Arrays? (What happens in the case of nested arrays?)
Are there limits on per-pattern storage, both in terms of size and number of variables? Is there a limit on overall variable storage space?
I’m still not fond of giving users the opportunity to beat their flash ram to death at 100fps via an unrestricted API. It’ll result in a ton of service calls.
Sure, those of us who are already experienced programmers can be trusted to restrict the rate, but as we’ve seen from the latest tutorial poll, there are a lot of people who are more into the LED art construction side of things. They’re really important to Pixelblaze’s future. Probably more so than we are. It helps them out if we can keep the “friction” of coding - and the possibility of bricking your Pixelblaze - as low as possible.
I actually agree this is the best approach to avoid unintended flash wear by rogue programmers who use any user initiated method. If @wizard can make it seamlessly behind the scenes, that would help. Not perfect, as if you ran a pattern and the PB rebooted/powercycled, you’d lose the current value but at least any intentional change away would record the current state.
Assuming this hypothetical storage api saveSlot could protect the flash by committing when necessary or at an interval - however I don’t think it’s a huge cause for concern it’s not like the beginner first time pattern is going to be looking to this api.