Since the sensor board handles all of the sound processing, I’m wondering how hard it would be to implement a version of colorchord’s also (described above, it’s v1, but v2 is the same principle as stated in readme for https://github.com/cnlohr/colorchord )
Basically, assign colors to notes based on frequency (so all As are one color, all C sharps another, etc).
Anyone tried this yet? Don’t want to reinvent the wheel…
Not that I’ve seen yet, but two things to check out:
If you know the note that will be triggered instead of listening to determine it, this becomes easier, and I’m reminded of Wizard’s own Synthia.
However, getting the sensor board to detect notes at chromatic precision might not be possible without tweaking the FFT code. The table of frequencies shows 2-5 bins per octave instead of the 12 you’d need to detect all notes. Now, the maxFrequency resolution of 39Hz becomes possibly viable for half-step tone detection above F5, but it would be a stretch when so many fundamentals are below it.
It’s really cool what they do though, calculating the key signature! Thanks, I had never seen this before.
Thanks, I hadn’t dug into the sensor board details yet, but it definitely looks like it doesn’t do enough to emulate the colorchord’s, it will be at best a rough approximation. Detecting the loudest frequency is good, and would be a start…
We’ll see if close is enough.
@wizard I see stm32 code in the colorchord GitHub, and it looks like the sensor board is similar. I don’t know enough to know if the board is capable of
(1) being reprogrammed [Given that you did open source it, I’d guess yes]
and (2) if processing the sound in this way would be exclusive (only capable of either the fft or dft), or potential for both at once? (Which could add functionality, an octave aware returned array (so 12 entries) rather than the (very mixed) 32 frequency bins now?
If only one or the other, perhaps a v2 someday that had a jumper to select?
Mostly because having played with colorchord, adding this color/note mapping seems like a natural fit… (there is an Android app for those who want see how it works…
https://play.google.com/store/apps/details?id=org.cnlohr.colorchord)
I’ll still play with implementing some form of it as is, but the nature of the large frequency bins seems to work against that being very fruitful/functional.
Right, the sensor board has enough resolution to make a really nice EQ/spectrograph display easily, but won’t get individual notes.
Synthia runs an FFT of 8192 w/ 44KHz stereo audio, but also runs on a desktop computer class CPU (or a newer raspberry pi). For contrast, the sensor board starts with a pair of FFT of 32 + 512, and compresses that down on a log scale to 32 buckets.
The MCU used isn’t nearly as powerful as the STM32’s that color chord uses (or the ESP8266). It might work with some modification / limitation. It could be hacked perhaps to get closer to what you are looking for. Definitely not a drop-in replacement.
You could hack a colorchord running microcontroller to spit out the sensor board format, and instead of 32 buckets, you could send up to 32 tracked notes.
So the STM32 in the sensor board is a Cortex M0, the STM32s in Cnlohr’s code are Cortex M4s…
With the v3 upgrade from a esp8266 to a esp32, I wonder if an upgraded sensor board is on the horizon, with a better processor (now that dual core M0+ boards like the Raspberry Pico are coming out for $4). [@wizard did open source this board so potentially if someone wanted to make a compatible board using newer tech, they could…]
Improved FFT would be sweet, but I wonder if any other sensors could be improved or added?
Ideas? What sensors could we add that people might use?
All kinds of sensors, especially i2c as I can hang a bunch off of a bus. There is plenty of CPU headroom on the M0 I use for more sensors. A few GPIO too. I would switch to M4s when I can find a good supply of them, but STM32 supply chain is crazy right now with lead times growing to half a year and more.
I’d love to hear what sensors would be good to include. For myself, a future version would switch to a 6 or 9 axis accel + gyro and maybe magnometer.
After some discussion with ColorChord devs, it sounds like it might well be possible to make the DFT for CC and the note folding (down to 24 bins, 12 notes and 12 half steps) work on the sensor board MCU. Is there an easy way to reprogram the board? Still early in my exploring, no rush.
Yeah, SWD pins are on the board. You can solder .1" pin headers for dev. Code is in github. Just need a SWD/st-link programmer.
@wizard, what’s your dev environment like for the STM boards? I’ve never worked with them, and I’d like to give it a try.
There’s an stm32 IDE, eclipse based. It’s quite decent, and the easiest of all ARM chip dev environments I’ve seen to get started.
Thanks! That’s ideal – already using Eclipse for a great many jobs around here.