This can also be an actual feature from a cold boot. I had an abit mobo that did it for 6 years from day one, it was a power on self test sort of thing. They released a bios that stopped it because people complained, but it was not an actual "issue" it was by design, i didn't bother to update my bios as it didn't bother me and it ran fine without a single issue overclocked for 6 years, often on 24/7 for half or more of that time so i didn't actually reboot very often, but i never had any problem with it.
10 years later my Asrock z370 K6 does this as well, with a brand new seasonic plantinum power supply, and it did it with my previous cheap temp Corsair Hx600 or whatever power supply. On this system i'm pretty sure it is due to my manual ram overclock, though just using an XMP profile can probably do it as well. In a non scientific sort of guess i believe it's checking and applying your ram profile and if so cpu overclock. It's something like that. I read about it long ago and forgot because it basically wasn't an actual problem. It could also have something to do with C states or something. No doubt someone will correct me if it is wrong.
Anyway, you said you don't "overclock", but you probably use an XMP profile, which is in fact an overclock on your ram. Also on mine it only does it when i shut off the power at my power supply. If i just shut down normal and power on later it does not do it. As far as i'm concerned in general it's not a problem if you are experiencing the same exact thing. I fyour power supply was actually goiing bad, you would have issues during normal usage, or certainly when gaming or doing anything power intensive.