Yes, that 3 quick flash LED sequence at startup means Setup mode. If you press the button, it should cycle through patterns, and if holding it did put it in setup mode, then physically the button is OK.
A snag/bug - holding the button while it’s already in setup mode doesn’t work and won’t clear the nonvolatile memory. It will turn the LED solid on, but no quick flashing. This is a bug sorry! To work around it configure it in client or AP mode and then hold the button.
Look for this sequence when entering SETUP mode: Long BLIiIINK, flash+flash+flash, resets, BLINK, flash+flash+flash, pause, solid dim.
Another possible snag, when you connect to a wifi network in setup mode it can take several seconds for it to fully connect, and it will only save the settings once it has successfully connected. After connecting, a few seconds later it will drop its setup AP network. I would wait for the setup network to go away before disconnecting power. If you remove power immediately after configuring it, it might not have saved yet.
Assuming that wasn’t it…
If you put it in AP mode, does that stick? I’m wondering if it’s an interaction with a router. Another thing to try if you have a 2nd PB handy, put the 2nd PB in AP mode and connect the problem one as a client to it. Does it remember wifi if connected to this different network? If it remembers those settings, perhaps switching it back to your wifi might work now that settings for client + AP have been written.
In the other cases where folks reported this I replaced their PB and the new unit worked for them. But when I got the return I could not reproduce the issue here. Putting it into setup mode and connecting it to my wifi worked fine and was remembered.
I know thats a bit of headache to go through, I’m trying to narrow down where the problem might be.