I saw there was a new firmware for the BMC up on the ASUS support site, so I downloaded it and tried to do a firmware upgrade over the ASMB8-iKVM web interface (which I have been using for a long time). I went to the firmware upgrade screen and clicked on the upgrade button, expecting to be asked to upload the firmware image, but instead I got a screen saying that the upgrade was in progress... with what image I don't know.
I waited a while and nothing happened, so I tried to log into the iKVM interface again, but got nothing. Tried also using SuperMicro's IPMIView (which I also have used many times), it said the device was offline and could not connect.
Tried a full power down of the workstation, unplugged it from power, waited a minute, plugged it back in, waited a while, still could not get into the iKVM. I then booted into Ubuntu 18.04 and used ipmitool to figure out what could be going on (this is now on the workstation itself, not remotely). I could access the IPMI fine and the selftest returned PASSED. The IP settings had not changed. I tried a BMC cold reset, it appeared to execute normally, I waited a bit, still could not connect remotely by iKVM (web or IPMIView).
I then booted up a FreeDOS bootable USB and used the ASUS-provided Yaflash.exe (in the firmware update zip) to see what could be going on. Again I could reach the IPMI fine and it read its version number, which showed 1.14. Presuming this meant 1.14.51 which is what I had installed a few months ago and was working mostly OK.
I then noticed that the new firmware I had just downloaded said "ASMB9" in its filename, as opposed to ASMB8, which is what this board actually has... is this a typo? Why would they post ASMB9 firmware on the Z10PE-D16 WS's download page? Ignoring this for the time being...
I found the already installed 1.14.51 firmware image and used Yaflash.exe to reflash it. The flash took about 15 minutes and completed normally. I then did a full power down and unplug again, then started it back up, waited, and tried to connect again... still not reachable remotely over iKVM. Tried to access it again locally and it was still reachable, passed the self-test, IP settings had not changed (correctly, as I requested to preserve the configuration), but still no iKVM remote access.
What can I do at this point? Do I attempt to flash the new firmware that says ASMB9 on it?! Could that well and truly brick it?
Please save me the "don't upgrade if it works"... I know. I made a mistake. But here we are, and I am looking for some way to fix this without having to disassemble this entire workstation and RMA the motherboard. Has anyone else seen this behavior where the web interface upgrade functionality bricks your BMC before you even load up the firmware image?