Improving sound reactive patterns

Patterns and Code
1

I’ve been making a bunch of sound reactive patterns for my pavilion, and while it works, I feel that a lot of them are not really as responsive as I’d like them to be. I’m stuck between either way too much reactivity or very little at all. Tuning various parameters helps, but I still feel that I’m not really getting as much oomph as I’d like.

I typically will have a pattern where sound will change some of the following

Movement speed of an object
Color
Saturation
Spawn rate
Fade rate

right now I do something like the following to get a normalized sound level

export var meansound=0
export var peaksound=0
function soundavg(insound){ //insound is typically maxFrequencyMagnitude
multval=100
meansound=clamp((meansound+insound/multval)/(1+1/multval),0,100)
peaksound=clamp(insound/meansound,0.1,5)
}

meansound approaches the average level of maxFrequencyMagnitude, and then peaksound is the instantaneous ratio of maxFrequencyMagnitude to the average level. My patterns typically run at 15-20 fps, and peaksound is peaky, but I just don’t get an exciting enough reaction. Does anyone have any good tips on what to do? Maybe I need to scale peaksound in a non linear way, or have some kind of persistence to it greater than a single pass through a loop?

2

Hey! Have you been using the example PI controller in your patterns? I tried to document it more verbosely here, and I believe that version shipped on the v3 if that’s more convenient for you.

3

Yeah, initially I used that but I went away from it for a few reasons

  1. It produces an output that isn’t really intuitive to me
  2. It is focused on the total intensity output, which often doesn’t really correlate to the effect i’m trying to achieve.

for example here is a pattern i’m working on

image
image690×537 32.1 KB

There are a number of drops that move around the upper portion and then fall down the center portion. The drops might change their hue/saturation/speed as a function of sound intensity, but the total brightness is mostly fixed.

The soundavg function I posted above has a couple of aspects I like.

  1. meansound gets close to the actual mean level of the sound at any given time
  2. peaksound is normalized to that level, so I get an output value that is only related to the sound itself, it has no coupling to the state of the leds, but does adjust for different sound levels.

Using this allows my to adjust things like speed and color which are not really coupled to brightness. My issue really seems to be that I’m not getting enough contrast on my sound output. I definitely get some, but I want it to kick more.

Maybe I should replace
peaksound=clamp(insound/meansound,0.1,5)

with something more like this

peaksound=clamp(insound/meansound-1,0.1,5),

so that I only return values when the sound is above average. But I’m also wondering if anyone has an understanding of how energyAverage and maxFrequencyMagnitude tend to behave with respect to music. How peaky can they be compared to their average values, and what is the best way of making them stand out?

4

It depends on music genre of course, but for picking out rapid changes, maybe a moving average of energyAverage over 10 or so frames would provide better contrast with maxFrequencyMagnitude?

Also, you could try picking a specific frequency region and looking at its contribution to overall amplitude.

  • The bass and/or bass drum usually carry the beat. Most of their energy is around 80-250hz.

  • Midrange is likely to be crowded and more uniformly loud. Human voices (not counting opera singers) are usually found in the range 100-1500hz. So are guitars and a lot of the usable range of keyboard, brass and wind instruments (all of which top out at about 4khz).

  • Above that, it varies a lot by genre. There can be a lot of cymbals and other percussion harmonics, which might be interesting to look at in terms of providing off beat and/or syncopated counterpoint to the main beat. There’s not generally a huge amount of energy up there – the perceived loudness is a human perception thing, so you’ll likely have to scale like mad.

In general, average amplitude will be more constant than you’d expect, and local peak amplitude might not be that much greater than average because of compression, and also the volume normalization performed by all the online music services.

There are lots of marketplace incentives for engineers to make sure their tracks are at least as loud as the next guy, with the result that the average level of recordings has converged very tightly on a near constant 0db.

3 Likes