Sound + Non sound pattern

Say I wanted to run a non-sound reactive pattern when it is quiet. But when sound is present above a certain threshold, Pixelblaze changes to one of the sound reactive patterns. Then, when sound falls below the threshold, Pixelblaze reverts to the previous non-sound reactive pattern.

Before I try to dive into the code, does it seem doable to combine different patterns in this way? Or would I be wasting my time trying to make it work?

Thanks!
-Ken

1 Like

Yes, seems completely doable.
Make a variable for ā€œheardsoundā€, and conditionally run your sound code instead of ā€œnormalā€ if itā€™s active. Perhaps with a timeout (have I heard sound within the last 5/10/30 seconds? Do sound otherwise do nonsoundā€¦)

Some patterns will be easier to adapt than others, of course.

Do share your results and code, including if you get stuck on it.

1 Like

Thanks for the tip!
Iā€™ll definitely post when I have something to share :slight_smile:

With the help of my lovely assistant, we have made some progress on integrating sound and no-sound patterns. We combined ā€œRainbow Meltā€ and ā€œSpectromatrix Optimā€. It seems to be working reasonably well using ā€œenergyAverageā€ to set a threshold to determine whether the sound or no-sound pattern should run.
The problem weā€™re running into is that we havenā€™t found a way to include a timeout before reverting to the no-sound pattern. We tried using ā€œsetTimout()ā€, but we are getting an ā€œundefined symbolā€ message. As it stands now, the sound pattern ā€œflashesā€ intermittently when there is sound.

I was also wondering if there was a way to limit the frequency input to a specific range (the human voice for example).

Any suggestions for addressing these issues would be greatly appreciated!

  //start rainbow melt (NO SOUND) pattern variable(s)
hl = pixelCount/2
//end rainbow melt (NO SOUND) pattern variable(s)


//start spectromatrix optim (SOUND) pattern variables and functions

//This pattern uses the sensor expansion board
//it supports pixel mapped configurations with 2D or 3D maps       

                                                                  //what are frequency ranges of frequencyData array elements
                                                              
                                                                  //how do setTimeout() documentation below shows some time functions
                                                                  //but the pixelblaze needs to be connected to internet to get time?
                                                                
export var frequencyData

//start - SR added - built in variables 
export var maxFrequency 
export var maxFrequencyMagnitude
export var energyAverage  //used for threshold for heardsound

//SR created - true or false variable based on energyAverage above or below set threshold in beforeRender function
export var heardsound

//end - SR added / created

width = 14
zigzag = true

averageWindowMs =500

fade = .6
speed = 1
zoom = .3
targetFill = 0.15
var pic = makePIController(1, .1, 300, 0, 300)

var sensitivity = 0
brightnessFeedback = 0

//start - SR created 
// export var sum //sums all values in frequencyData array in for loop in beforeRender function
// export var avg //finds average of all values in one loop through values in frequenceData array in beforeRender function
//end - SR created

var averages = array(32) //COULD USE THIS ARRAY AS RANGE FOR SOUND/NO SOUND THRESHOLD? -sr
pixels = array(pixelCount)
vals = array(32) //COULD USE THIS ARRAY AS RANGE FOR SOUND/NO SOUND THRESHOLD? -sr

// Makes a new PI Controller
function makePIController(kp, ki, start, min, max) {
  var pic = array(5)
  pic[0] = kp
  pic[1] = ki
  pic[2] = start
  pic[3] = min
  pic[4] = max
  return pic
}

function calcPIController(pic, err) {
  pic[2] = clamp(pic[2] + err, pic[3], pic[4])
  return max(pic[0] * err + pic[1] * pic[2],.3)
}

//end spectromatrix optim pattern variables and functions

export function beforeRender(delta) {
  
  //through if statement for sound pattern set-up
  t1 = time(1)
  t2 = time(.6)
  
  wt1 = wave(t1 * speed)

  sensitivity = calcPIController(pic, targetFill - brightnessFeedback / pixelCount);
  brightnessFeedback = 0
  
  dw = delta / averageWindowMs
  
  // avg = 0.000000
  // sum = 0.000000
  
  for (i = 0; i < 32; i++) {
    averages[i] = max(.00001, averages[i] * (1 - dw) + frequencyData[i] * dw * sensitivity)
    vals[i] = (frequencyData[i] * sensitivity - averages[i]*2) * 10 * (averages[i] * 1000 + 1)
    // sum += frequencyData[i] * 10
    if(energyAverage >= 0.001000){ //<-- THIS NEEDS TO BE ADJUSTED FOR THRESHOLD FOR SOUND PATTERN VS NO SOUND PATTERN AND ALSO CAN PLAY
    //AROUND WITH averages AND vals ARRAYS FOR DIFFERENT RANGES TO PLAY WITH
    heardsound = true
  }
    else { //below sets up non-sound pattern
      // setTimeout(render, 30000) //gives 'undefined symbol setTimeout' error; uncertain if calling render function is correct
      
      heardsound = false
      
      t1 =  time(.1)
      t2 = time(0.13)
  }
      
  }
  // avg = (sum / 32) * 1000
  // if(avg >= 0.000700){ //<-- THIS NEEDS TO BE ADJUSTED FOR THRESHOLD FOR SOUND PATTERN VS NO SOUND PATTERN AND ALSO CAN PLAY
  //   //AROUND WITH averages AND vals ARRAYS FOR DIFFERENT RANGES TO PLAY WITH
  //   heardsound = true
  // }
  // else { //below sets up non-sound pattern
  //   heardsound = false
  //   t1 =  time(.1)
  //   t2 = time(0.13)
  // }
  
}


//start spectromatrix optim only functions
export function render3D(index, x, y, z) {
  var  i, h, s, v
  
  i = triangle((wave((x+z)*zoom + wt1) + wave((y+z)*zoom - wt1)) *.5 + t2) * 31

  v = vals[i]

  h = i / 60 + t1 
  v = v > 0 ? v*v : 0
  s = 1 - v
  pixels[index] = pixels[index] * fade + v
  v = pixels[index];

  brightnessFeedback += clamp(v, 0, 2)
  hsv(h, s, v)
}

//support 2D pixel mapped configurations
export function render2D(index, x, y) {
  render3D(index, x, y, 0)
}

//end spectromatrix optim only functions

//this pixel mapper shim will work without a pixel map or on older Pixelblazes
//it calculates x/y based on a 2D LED matrix display given a known width and height
export function render(index) {
  if(heardsound){ //renders sound pattern
    
    var width = 8, height = 8
    var y = floor(index / width)
    var x = index % width
    //comment out this next line if you don't have zigzag wiring:
    x = (y % 2 == 0 ? x : width - 1 - x)
    x /= width
    y /= height
    render2D(index, x, y)
  }
  else { //renders non-sound pattern
    c1 = 1-abs(index - hl)/hl
    c2 = wave(c1)
    c3 = wave(c2 + t1)
    v = wave(c3 + t1)
    v = v*v
    hsv(c1 + t2,1,v)
  }
    
}

Hi @Asphalt.Astronaut,
Awesome!

Pixelblazeā€™s language is based off of JavaScript (ES6) syntax, but with a subset of the language features available. You wonā€™t find many of the JavaScripty things there, while others are different. For example you call sin(a) instead of Math.sin(a), and array(size) instead of new Array(size). APIs like setTimeout, setInterval donā€™t exist yet, but I hope to add something similar in the future. The full list of supported features and APIs is available with the documentation just below the editor (scroll down on the page).

Another one of the differences is that all numbers in Pixelblaze are a 16.16 fixed-point numbers. This can handle values between -32,768 to +32,767 with fractional accuracy down to 1/65,536ths.

The beforeRender(delta) can be used to keep track of time passing. The delta parameter gives you the time that has elapsed in milliseconds since the last beforeRender call. You can accumulate that up to keep track of time.

Hereā€™s a seconds (not milliseconds) setTimeout implementation that you can use:


// these variables hold data used by the timer functions
// put them at the top of your code
var maxTimers = 10 
var timerFns = array(maxTimers)
var timerTimes = array(maxTimers)

/**
 * Schedule a function to run after some time (in seconds) elapses.
 * If successful, a non-negative id is returned and can be used 
 * to cancel it using clearTimeout(id). Returns -1 on error.
 */
function setTimeout(fn, timeout) {
  var nextTimer = -1
  //find an empty timer slot
  for (var i = 0; i < maxTimers; i++) {
    if (!timerFns[i]) {
      nextTimer = i
      break
    }
  }

  if (nextTimer >= 0) {
    timerFns[nextTimer] = fn
    timerTimes[nextTimer] = timeout
  }
  return nextTimer
}

/**
 * Cancel a timeout by id
 */
function clearTimeout(id) {
  if (id >= 0) {
    timerFns[id] = false
  }
}

/**
 * Update timers based on elapsed time, and call any callbacks.
 * Call this from beforeRender(delta)
 */
function updateTimers(delta) {
  var seconds = delta/1000
  //check timers
  for (var i = 0; i < maxTimers; i++) {
    //see if there is a function for this slot
    if (timerFns[i]) {
      //reduce times until zero
      timerTimes[i] = max(0, timerTimes[i] - seconds)
      //see if timer is up
      if (timerTimes[i] == 0) {
        var fn = timerFns[i]
        //blank out this slot so we don't keep executing it and it can be reused
        timerFns[i] = false 
        fn() //run the function
      }
    }
  }
}

To use, youā€™ll call updateTimers(delta) inside your beforeRender implementation. You can register a function (or even a lambda) to run later, and cancel them. A common pattern is to cancel the old timer and re-schedule it every time an even occurs to know some time after the last one happened.

You could use this code, and fill out what you want to happen in stoppedHearingSound(). Call heardSound() every time your code detects sufficient sound to start/reset the timer.

function stoppedHearingSound() {
  //do something 30 seconds after the last time sound was heard here
}

var timerId = -1
function heardSound() {
  //cancel a previous timer if one was set
  if (timerId != -1)
    clearTimeout(timerId)
  //set a timer to trigger in 30 seconds
  timerId = setTimeout(() => {
    timerId = -1
    stoppedHearingSound();
  }, 30)
}
1 Like

The only thing @wizard didnā€™t address was the frequency question. The sound ā€œbucketsā€ were documented somewhere in the forumā€¦ Iā€™ll see if I can find the post.

If you see activity in your choice of whatever buckets you decide fit best, thatā€™ll be the answer.

1 Like

Hi Wizard.
Thanks for the guidance. Unfortunately the coding is a little over our head (Iā€™m a complete novice and she is fresh out of a coding bootcamp) and so we werenā€™t able to get it to work. It still responds to sound for an instant, and then immediatly goes back to the no-sound pattern.
It still looks cool though.
Hereā€™s a link to a short vid showing what it looks like (Iā€™ll also post something in the show and tell)
Cheers!
https://photos.app.goo.gl/cUPPHyX11srsZP1TA

Do join our new coding academy that starts tomorrow. And you post code, Iā€™d be happy to help figure out why it wasnā€™t working as expected

1 Like

So, Iā€™m revisiting this idea with the hope of getting some help to push it over the finish line.
A little context: the installation is for a multi day event. I donā€™t expect there will be people around the art piece the entire timeā€¦ likely there will be times when people in the area will activate the sound response, and other times when no one will be around. The intent is to have sound response when people are around, but when no one is around, the PB reverts to the no-sound portion of the code after a specific timeout (30 sec or so). This is so that when people are far away, they will be drawn towards the piece. This is less likely when the PB is running in sound reactive mode as the light response is subtle when there is no sound present.

Thanks to all the wonderful help Iā€™ve received so far, the code seems to be running as intended except for one aspectā€¦ the 30 second timeout that keeps the code in sound mode before returning to no-sound mode. Right now the code jumps to sound reactive mode when there is sound (as expected), but then jumps right back to non-sound mode without a timeout. The effect is a glitchy jump back and forth between the modes. The video posted previously kind of shows this, but the glitchy effect is much more pronounced when there is talking or music with short interim quiet periods.

So any help resolving this timeout problems would be greatly appreciated. Below is code I have cobbled together from the previous suggestions, but Iā€™m not sure what Iā€™m doing wrongā€¦

// these variables hold data used by the timer functions
// put them at the top of your code
var maxTimers = 10
var timerFns = array(maxTimers)
export var timerTimes = array(maxTimers)

export var timerId = -1

//start rainbow melt (NO SOUND) pattern variable(s)
hl = pixelCount/2
//end rainbow melt (NO SOUND) pattern variable(s)

//start spectromatrix optim (SOUND) pattern variables and functions

//This pattern uses the sensor expansion board
//it supports pixel mapped configurations with 2D or 3D maps

export var frequencyData

//start - SR added - built in variables
//export var maxFrequency
//export var maxFrequencyMagnitude
export var energyAverage //used for threshold for heardsound

//SR created - true or false variable based on energyAverage above or below set threshold in checkThreshold() function
export var heardsound

//end - SR added / created

width = 14
zigzag = true

averageWindowMs =500

fade = .6
speed = 1
zoom = .3
targetFill = 0.15
var pic = makePIController(1, .1, 300, 0, 300)

var sensitivity = 0
brightnessFeedback = 0

var averages = array(32)
pixels = array(pixelCount)
vals = array(32)
/**

  • Schedule a function to run after some time (in seconds) elapses.
  • If successful, a non-negative id is returned and can be used
  • to cancel it using clearTimeout(id). Returns -1 on error.
    */
    function setTimeout(fn, timeout) {
    var nextTimer = -1 //id
    //find an empty timer slot
    for (var i = 0; i < maxTimers; i++) {
    if (!timerFns[i]) {
    nextTimer = i
    break
    }
    }

if (nextTimer >= 0) {
timerFns[nextTimer] = fn
timerTimes[nextTimer] = timeout
}
return nextTimer
}

/**

  • Cancel a timeout by id
    */
    function clearTimeout(id) {
    if (id >= 0) {
    timerFns[id] = false
    }
    }

/**

  • Update timers based on elapsed time, and call any callbacks.
  • Call this from beforeRender(delta)
    */
    function updateTimers(delta) {
    var seconds = delta/1000
    //check timers
    for (var i = 0; i < maxTimers; i++) {
    //see if there is a function for this slot
    if (timerFns[i]) {
    //reduce times until zero
    timerTimes[i] = timerTimes[i] - seconds // max(0, timerTimes[i] - seconds)
    //see if timer is up
    if (timerTimes[i] == 0) {
    var fn = timerFns[i]
    //blank out this slot so we donā€™t keep executing it and it can be reused
    timerFns[i] = false
    fn() //run the function //before render delta
    }
    }
    }
    }

//To use, youā€™ll call updateTimers(delta) inside your beforeRender implementation. You can register a function (or even a lambda) to run later, and cancel them. A common pattern is to
//cancel the old timer and re-schedule it every time an even occurs to know some time after the last one happened.

//You could use this code, and fill out what you want to happen in stoppedHearingSound(). Call heardSound() every time your code detects sufficient sound to start/reset the timer.

function checkThreshold(){
if(energyAverage >= 0.001000){ //ADJUSTS THRESHOLD FOR SOUND PATTERN VS NO SOUND PATTERN

   heardSound() 
 }
 else {
  
   stoppedHearingSound()
 }

}

function stoppedHearingSound() {
//do something 30 seconds after the last time sound was heard here
heardsound = false
//setTimeout()

//cancel a previous timer if one was set
// if (timerId != -1)
// clearTimeout(timerId)
// //set a timer to trigger in 30 seconds
// timerId = setTimeout(() => {
// timerId = -1
// heardsound = false
// checkThreshold();
// }, 30)
return heardsound
}

var timerId = -1
function heardSound() {
heardsound = true
// //cancel a previous timer if one was set
if (timerId != -1)
clearTimeout(timerId)
//set a timer to trigger in 30 seconds
timerId = setTimeout(() => {
timerId = -1
//heardsound = true
stoppedHearingSound();
}, 30)
return heardsound
}

// Makes a new PI Controller
function makePIController(kp, ki, start, min, max) {
var pic = array(5)
pic[0] = kp
pic[1] = ki
pic[2] = start
pic[3] = min
pic[4] = max
return pic
}

function calcPIController(pic, err) {
pic[2] = clamp(pic[2] + err, pic[3], pic[4])
return max(pic[0] * err + pic[1] * pic[2],.3)
}

//end spectromatrix optim pattern variables and functions

export function beforeRender(delta) {

updateTimers(delta)
//checkThreshold()

//if(heardsound){
if(energyAverage >= 0.001000){ //<-- THIS NEEDS TO BE ADJUSTED FOR THRESHOLD FOR SOUND PATTERN VS NO SOUND PATTERN AND ALSO CAN PLAY
// //AROUND WITH averages AND vals ARRAYS FOR DIFFERENT RANGES TO PLAY WITH
// // heardsound = true
heardSound()
//through if statement for sound pattern set-up
t1 = time(1)
t2 = time(.6)

wt1 = wave(t1 * speed)

sensitivity = calcPIController(pic, targetFill - brightnessFeedback / pixelCount);
brightnessFeedback = 0

dw = delta / averageWindowMs

// avg = 0.000000
// sum = 0.000000

for (i = 0; i < 32; i++) {
averages[i] = max(.00001, averages[i] * (1 - dw) + frequencyData[i] * dw * sensitivity)
vals[i] = (frequencyData[i] * sensitivity - averages[i]*2) * 10 * (averages[i] * 1000 + 1)
// sum += frequencyData[i] * 10
// if(energyAverage >= 0.000800){ //<-- THIS NEEDS TO BE ADJUSTED FOR THRESHOLD FOR SOUND PATTERN VS NO SOUND PATTERN AND ALSO CAN PLAY
// //AROUND WITH averages AND vals ARRAYS FOR DIFFERENT RANGES TO PLAY WITH
// // heardsound = true
// heardSound()
}
}
else { //below sets up non-sound pattern
// setTimeout(render, 30000) //gives ā€˜undefined symbol setTimeoutā€™ error; uncertain if calling render function is correct

  stoppedHearingSound()
  //heardsound = false
  
  t1 =  time(.1)
  t2 = time(0.13)

}

// avg = (sum / 32) * 1000
// if(avg >= 0.000700){ //<-- THIS NEEDS TO BE ADJUSTED FOR THRESHOLD FOR SOUND PATTERN VS NO SOUND PATTERN AND ALSO CAN PLAY
// //AROUND WITH averages AND vals ARRAYS FOR DIFFERENT RANGES TO PLAY WITH
// heardsound = true
// }
// else { //below sets up non-sound pattern
// heardsound = false
// t1 = time(.1)
// t2 = time(0.13)
// }

}

//start spectromatrix optim only functions
export function render3D(index, x, y, z) {
var i, h, s, v

i = triangle((wave((x+z)*zoom + wt1) + wave((y+z)*zoom - wt1)) *.5 + t2) * 31

v = vals[i]

h = i / 60 + t1
v = v > 0 ? v*v : 0
s = 1 - v
pixels[index] = pixels[index] * fade + v
v = pixels[index];

brightnessFeedback += clamp(v, 0, 2)
hsv(h, s, v)
}

//support 2D pixel mapped configurations
export function render2D(index, x, y) {
render3D(index, x, y, 0)
}

//end spectromatrix optim only functions

//this pixel mapper shim will work without a pixel map or on older Pixelblazes
//it calculates x/y based on a 2D LED matrix display given a known width and height
export function render(index) {
if(heardsound){ //renders sound pattern

var width = 8, height = 8
var y = floor(index / width)
var x = index % width
//comment out this next line if you don't have zigzag wiring:
x = (y % 2 == 0 ? x : width - 1 - x)
x /= width
y /= height
render2D(index, x, y)

}
else { //renders non-sound pattern
c1 = 1-abs(index - hl)/hl
c2 = wave(c1)
c3 = wave(c2 + t1)
v = wave(c3 + t1)
v = v*v
hsv(c1 + t2,1,v)
}

}

Also, It looks like I missed the PB coding academy. Is there another one upcoming?

Thanks in advance!

Weā€™ll be doing another in the fall, I suspect.

If you could edit your post and select all of the code and hit the <> button in the editor, it makes it all much easier to read/understand

// these variables hold data used by the timer functions
// put them at the top of your code
var maxTimers = 10 
var timerFns = array(maxTimers)
export var timerTimes = array(maxTimers)

export var timerId = -1

//start rainbow melt (NO SOUND) pattern variable(s)
hl = pixelCount/2
//end rainbow melt (NO SOUND) pattern variable(s)


//start spectromatrix optim (SOUND) pattern variables and functions

//This pattern uses the sensor expansion board
//it supports pixel mapped configurations with 2D or 3D maps       

export var frequencyData

//start - SR added - built in variables 
//export var maxFrequency 
//export var maxFrequencyMagnitude
export var energyAverage  //used for threshold for heardsound

//SR created - true or false variable based on energyAverage above or below set threshold in checkThreshold() function
export var heardsound

//end - SR added / created

width = 14
zigzag = true

averageWindowMs =500

fade = .6
speed = 1
zoom = .3
targetFill = 0.15
var pic = makePIController(1, .1, 300, 0, 300)

var sensitivity = 0
brightnessFeedback = 0

var averages = array(32) //COULD USE THIS ARRAY AS RANGE FOR SOUND/NO SOUND THRESHOLD? -sr
pixels = array(pixelCount)
vals = array(32) //COULD USE THIS ARRAY AS RANGE FOR SOUND/NO SOUND THRESHOLD? -sr
/**
 * Schedule a function to run after some time (in seconds) elapses.
 * If successful, a non-negative id is returned and can be used 
 * to cancel it using clearTimeout(id). Returns -1 on error.
 */
function setTimeout(fn, timeout) {
  var nextTimer = -1 //id
  //find an empty timer slot
  for (var i = 0; i < maxTimers; i++) {
    if (!timerFns[i]) {
      nextTimer = i
      break
    }
  }

  if (nextTimer >= 0) {
    timerFns[nextTimer] = fn   
    timerTimes[nextTimer] = timeout 
  }
  return nextTimer
}

/**
 * Cancel a timeout by id
 */
function clearTimeout(id) {
  if (id >= 0) {
    timerFns[id] = false
  }
}

/**
 * Update timers based on elapsed time, and call any callbacks.
 * Call this from beforeRender(delta)
 */
function updateTimers(delta) {
  var seconds = delta/1000
  //check timers
  for (var i = 0; i < maxTimers; i++) {
    //see if there is a function for this slot
    if (timerFns[i]) {
      //reduce times until zero
      timerTimes[i] = timerTimes[i] - seconds // max(0, timerTimes[i] - seconds)
      //see if timer is up
      if (timerTimes[i] == 0) {
        var fn = timerFns[i]
        //blank out this slot so we don't keep executing it and it can be reused
        timerFns[i] = false 
        fn() //run the function  //before render delta
      }
    }
  }
}



//To use, youā€™ll call updateTimers(delta) inside your beforeRender implementation. You can register a function (or even a lambda) to run later, and cancel them. A common pattern is to 
//cancel the old timer and re-schedule it every time an even occurs to know some time after the last one happened.

//You could use this code, and fill out what you want to happen in stoppedHearingSound(). Call heardSound() every time your code detects sufficient sound to start/reset the timer.

function checkThreshold(){
     if(energyAverage >= 0.001000){ //ADJUSTS THRESHOLD FOR SOUND PATTERN VS NO SOUND PATTERN 
       
       heardSound() 
     }
     else {
      
       stoppedHearingSound()
     }
}


function stoppedHearingSound() {
  //do something 30 seconds after the last time sound was heard here
  heardsound = false
 //setTimeout()
 
  //cancel a previous timer if one was set
  // if (timerId != -1)
  //   clearTimeout(timerId)
  // //set a timer to trigger in 30 seconds
  // timerId = setTimeout(() => {
  //   timerId = -1
  //   heardsound = false
  //   checkThreshold();
  // }, 30)
 return heardsound
}

var timerId = -1
function heardSound() {
  heardsound = true
  // //cancel a previous timer if one was set
  if (timerId != -1)
    clearTimeout(timerId)
  //set a timer to trigger in 30 seconds
  timerId = setTimeout(() => {
    timerId = -1
    //heardsound = true
    stoppedHearingSound();
  }, 30)
  return heardsound
}


// Makes a new PI Controller
function makePIController(kp, ki, start, min, max) {
  var pic = array(5)
  pic[0] = kp
  pic[1] = ki
  pic[2] = start
  pic[3] = min
  pic[4] = max
  return pic
}

function calcPIController(pic, err) {
  pic[2] = clamp(pic[2] + err, pic[3], pic[4])
  return max(pic[0] * err + pic[1] * pic[2],.3)
}

//end spectromatrix optim pattern variables and functions

export function beforeRender(delta) {
  
  updateTimers(delta)
  //checkThreshold()
  
  //if(heardsound){
  if(energyAverage >= 0.001000){ //<-- THIS NEEDS TO BE ADJUSTED FOR THRESHOLD FOR SOUND PATTERN VS NO SOUND PATTERN AND ALSO CAN PLAY
  //   //AROUND WITH averages AND vals ARRAYS FOR DIFFERENT RANGES TO PLAY WITH
  //   // heardsound = true
     heardSound()
  //through if statement for sound pattern set-up
  t1 = time(1)
  t2 = time(.6)
  
  wt1 = wave(t1 * speed)

  sensitivity = calcPIController(pic, targetFill - brightnessFeedback / pixelCount);
  brightnessFeedback = 0
  
  dw = delta / averageWindowMs
  
  // avg = 0.000000
  // sum = 0.000000
  
  for (i = 0; i < 32; i++) {
    averages[i] = max(.00001, averages[i] * (1 - dw) + frequencyData[i] * dw * sensitivity)
    vals[i] = (frequencyData[i] * sensitivity - averages[i]*2) * 10 * (averages[i] * 1000 + 1)
    // sum += frequencyData[i] * 10
    // if(energyAverage >= 0.000800){ //<-- THIS NEEDS TO BE ADJUSTED FOR THRESHOLD FOR SOUND PATTERN VS NO SOUND PATTERN AND ALSO CAN PLAY
    // //AROUND WITH averages AND vals ARRAYS FOR DIFFERENT RANGES TO PLAY WITH
    // // heardsound = true
    // heardSound()
  }
  }
    else { //below sets up non-sound pattern
      // setTimeout(render, 30000) //gives 'undefined symbol setTimeout' error; uncertain if calling render function is correct
      
      stoppedHearingSound()
      //heardsound = false
      
      t1 =  time(.1)
      t2 = time(0.13)
  }
      
  
  // avg = (sum / 32) * 1000
  // if(avg >= 0.000700){ //<-- THIS NEEDS TO BE ADJUSTED FOR THRESHOLD FOR SOUND PATTERN VS NO SOUND PATTERN AND ALSO CAN PLAY
  //   //AROUND WITH averages AND vals ARRAYS FOR DIFFERENT RANGES TO PLAY WITH
  //   heardsound = true
  // }
  // else { //below sets up non-sound pattern
  //   heardsound = false
  //   t1 =  time(.1)
  //   t2 = time(0.13)
  // }
  
}


//start spectromatrix optim only functions
export function render3D(index, x, y, z) {
  var  i, h, s, v
  
  i = triangle((wave((x+z)*zoom + wt1) + wave((y+z)*zoom - wt1)) *.5 + t2) * 31

  v = vals[i]

  h = i / 60 + t1 
  v = v > 0 ? v*v : 0
  s = 1 - v
  pixels[index] = pixels[index] * fade + v
  v = pixels[index];

  brightnessFeedback += clamp(v, 0, 2)
  hsv(h, s, v)
}

//support 2D pixel mapped configurations
export function render2D(index, x, y) {
  render3D(index, x, y, 0)
}

//end spectromatrix optim only functions

//this pixel mapper shim will work without a pixel map or on older Pixelblazes
//it calculates x/y based on a 2D LED matrix display given a known width and height
export function render(index) {
  if(heardsound){ //renders sound pattern
    
    var width = 8, height = 8
    var y = floor(index / width)
    var x = index % width
    //comment out this next line if you don't have zigzag wiring:
    x = (y % 2 == 0 ? x : width - 1 - x)
    x /= width
    y /= height
    render2D(index, x, y)
  }
  else { //renders non-sound pattern
    c1 = 1-abs(index - hl)/hl
    c2 = wave(c1)
    c3 = wave(c2 + t1)
    v = wave(c3 + t1)
    v = v*v
    hsv(c1 + t2,1,v)
  }
    
}

I was wondering how to do that. Is this correct?

1 Like

Yes, but you could have edited your original post/comment. (I do that often, sometimes updating posts a handful of times)

Soā€¦ I installed this, and first thing I notice is that when I speak or make sound, the ā€˜heardsoundā€™ variable goes false to true, but then goes false again.
Still reading the code, and trying to figure it out. But I suspect itā€™s just overly complicated.

What we want to do:
If weā€™ve heard sound, flag a variable that says we have heard sound (heardsound), and begin doing a sound pattern, and start some sort of countdown timer (30 seconds or so), and then if the timer expires without having heard more sound (so a heard sound resets the timer back to 30), then switch the heardsound to off, and start a non-sound pattern until we hear a sound again.

Yes?

Iā€™m going to rewrite this, with the above in mind, and try and make it simple to follow.

Click to view rewrite of code
// Rewrite of Asphalt.Astronaut's spectromatrix which either does sound based pattern or non-sound pattern
// This pattern uses the sensor expansion board
// supports pixel mapped configurations with 2D or 3D maps, or an unmapped matrix       

// Re-engineered by Scruffynerf, for clarity and a much simpler way to do the detection/timing

// Variables all up top for easy access

// pixel related
var pixels = array(pixelCount)
var width = sqrt(pixelCount) // assumes a square, if you aren't a square matrix, replace with correct value
var height = sqrt(pixelCount) // assumes a square, if you aren't a square matrix, replace with correct value
var zigzag = true  // if your matrix goes back and forth, and isn't mapped already. 
var hl = pixelCount/2  // half the matrix, for splitting it down the middle

// sound related
export var heardsound // have we heard a sound?  If true, we'll do sound, if false, we won't.
export var levelofdetection = 0.001000      //ADJUSTS THRESHOLD FOR SOUND PATTERN VS NO SOUND PATTERN 
export var timer // counts down
var timerreset = 30000 // how long should we count down?  If you need more than 30 seconds, you'll run into 16bit limit
var sensitivity = 0
var averages = array(32) //COULD USE THIS ARRAY AS RANGE FOR SOUND/NO SOUND THRESHOLD? -sr
var vals = array(32) //COULD USE THIS ARRAY AS RANGE FOR SOUND/NO SOUND THRESHOLD? -sr

export function sliderLevel(v){
  levelofdetection = v/1000
}

// pattern timing
var t1,t2,wt1

// built in variables for sound, exported for debugging
export var frequencyData
export var maxFrequency 
export var maxFrequencyMagnitude
export var energyAverage  //used for threshold for heardsound

// led pattern related, best done with sliders to fine tune
export var averageWindowMs = 500
export var fade = .6
export var speed1 = 1
export var speed2 = .6
export var zoom = .3
export var targetFill = 0.15
export var brightnessFeedback = 0

// initial setup
var pic = makePIController(1, .1, 300, 0, 300)

//sliders to adjust the above things, comment out a slider if you find a desired sweet spot and hardcode it above
export function sliderWindow(v){
  averageWindowMs = v * 1000
}

export function sliderFade(v){
  fade = v
}

export function sliderSpeed1(v){
  speed1 = v * 2 + .01
}

export function sliderSpeed2(v){
  speed2 = v * 2 + .01
}

export function sliderZoom(v){
  zoom = v * 2 + .01
}

export function sliderFill(v){
  targetFill = v
}

function checkThreshold(){
  if(energyAverage >= levelofdetection){ 
    heardSound() 
  // } else {
  //  noSound()
  }
}

function heardSound() {
  // we crossed the threshold, so start the timer, and do the sound pattern
  heardsound = true
  timer = timerreset
}

function noSound() {
  // we aren't using this at this point
}

// Makes a new PI Controller
function makePIController(kp, ki, start, min, max) {
  var pic = array(5)
  pic[0] = kp
  pic[1] = ki
  pic[2] = start
  pic[3] = min
  pic[4] = max
  return pic
}

function calcPIController(pic, err) {
  pic[2] = clamp(pic[2] + err, pic[3], pic[4])
  return max(pic[0] * err + pic[1] * pic[2],.3)
}

function updateTimer(delta){
  if (heardsound) {
    timer = timer - delta
    if (timer <= 0) {
      // time expired, no sound heard for 30 seconds (or whatever the countdown is), do nonsound instead
      timer = 0;
      heardsound = false;
    }
  }
}

export function beforeRender(delta) {
  
  updateTimer(delta);

  t1 = time(speed1)
  t2 = time(speed2)
  wt1 = wave(t1)

  checkThreshold();

  sensitivity = calcPIController(pic, targetFill - brightnessFeedback / pixelCount);
  brightnessFeedback = 0
  dw = delta / averageWindowMs
  
  // avg = 0.000000
  // sum = 0.000000
  
  for (i = 0; i < 32; i++) {
    averages[i] = max(.00001, averages[i] * (1 - dw) + frequencyData[i] * dw * sensitivity)
    vals[i] = (frequencyData[i] * sensitivity - averages[i]*2) * 10 * (averages[i] * 1000 + 1)
    //  THIS CAN BE ADJUSTED FOR THRESHOLD FOR SOUND PATTERN VS NO SOUND PATTERN AND ALSO CAN PLAY
    //  AROUND WITH averages AND vals ARRAYS FOR DIFFERENT RANGES TO PLAY WITH
    // avg = (sum / 32) * 1000
    // sum += frequencyData[i] * 10
    // if(energyAverage >= 0.000800) 
    // if(avg >= 0.000700)
  }
}

export function render3D(index, x, y, z) {
  if (heardsound) {
    spectromatrix(index,x,y,z)
  } else {
    nonsoundpattern(index)
  }
}

//support 2D pixel mapped configurations, faking a 3rd dimension of z=0
export function render2D(index, x, y) {
  render3D(index, x, y, 0)
}

// if no 2d or 3d map exists, plain render will be used, so we'll fake a matrix, then call the 3D render
export function render(index) {
  //  this pixel mapper shim will work without a pixel map or on older Pixelblazes
  //  It calculates x/y based on a 2D LED matrix display given a known width and height
  var y = floor(index / width)
  var x = index % width
  if (zigzag) {
    x = (y % 2 == 0 ? x : width - 1 - x)
  }
  x /= width
  y /= height
  render3D(index, x, y, 0)
}

// finally, the actual pattern code

function nonsoundpattern(index){
  c1 = 1-abs(index - hl)/hl
  c2 = wave(c1)
  c3 = wave(c2 + t1)
  v = wave(c3 + t1)
  v = v*v
  hsv(c1 + t2,1,v)
}

function spectromatrix(index,x,y,z){
  var i, h, s, v
  i = triangle((wave((x+z)*zoom + wt1) + wave((y+z)*zoom - wt1)) *.5 + t2) * 31
  v = vals[i]
  v = v*v
  h = i / 60 + t1 
  s = 1 - v
  pixels[index] = pixels[index] * fade + v
  v = pixels[index];
  brightnessFeedback += clamp(v, 0, 2)
  hsv(h, s, v)
}

What did I do?
I reorganized greatly, trying to compartmentalize pieces into sections.
I removed the entire fancy timer section, itā€™s hugely overkill in this case. If you needed a dozen timers, calling different things at different rates, the old code might make sense. We just want one timer, so why bother with a complex system.
So now the logic is just the above.
Iā€™ve left the whole ā€˜non mappedā€™ code in place, but honestly, itā€™s overkill too. Either you have a matrix (or something mapped to at least 2D, or you donā€™t.) The nonmapped shim is at the end, just in case, though. I think it still works, but I was using a mapped matrix.
I added a pile of sliders, some of which you probably want to remove once you find the value sweet spot, but this way, I could play with values, and see what happened when I did.
I didnā€™t really touch your actual sound reactive code. I might (likely will) redo/improve on this with a handful of different sound reactive options (and a slider to select which, or to change randomly every once in a while, maybe when itā€™s quiet?), but for now, you can now play with the code you had, and try adding things like playing with frequency buckets, sums, averages, MaxFrequency, and so and so onā€¦

Sometimes, simpler is better, and itā€™s easy to get lost in the weeds when too much is there. The timer code is a good example, it was a complex piece of code, and really, the desired behavior could have been done by it, but a simple method does it just as well, and itā€™s so easy to see what it does now.

4 Likes

Wow! This is amazing! Thank you so much!
Iā€™m going to play with it tonight and let you know how it goes.
Canā€™t wait to play with the sliders :slight_smile:

3 Likes

Do post some photos of your actual setup. I always enjoy code running on live LEDs.

1 Like