You'll want a bootable memory test so you can make sure any settings that pass training aren't so flaky they'll corrupt your OS install. The free version of
Memtest86 will work fine for this purpose; you'll need an empty usb drive to make the bootable media. Once you've created that, and before you safe and exit with any new UEFI settings, make sure that USB drive is the only boot device enabled, until you've got the settings you want to test further (in Windows) dialed in.
Settings wise (use the advanced view to make this easier), first thing is to get some baseline values that should POST/train correctly, which you can then clock up until you encounter issues.
For voltages, I'd start with: 1.3 VDDIO, 1.4 DRAM VDD, 1.4 DRAM VDDQ, 1.8 DRAM VPP, 1.15 VDD_SOC, 1.1 VDD_MISC, 0.85 on both VDDG CCD & IOD, and 1.1 VDDP. Set the DRAM Frequency to 8000MT/s, Infinity Fabric Frequency (FCLK) to 2000MHz, and UCLK DIV1 MODE to UCLK=MEMCLK/2. All of those settings should be in the main OC Tweaker page. Most of the stuff under "External voltage settings" should be fine at AUTO, though if you see any options to adjust VRM PWM frequencies, maxing these out, as long as you don't exceed 700kHz, may be prudent.
Under "DRAM Timings Configuration:, try matching the timings (
not the drive strengths/termination resistances, nor clocks I have already specified above) that I'm using in
this post, with the following exceptions: tRCD can be 48 (at 8000MT/s), tRFC2 376, tRFCsb 296, tRDWR 16, and tWRRD 2 (note I don't believe that tRFC2 and tRFCsb aren't actually used, yet, but I fill them in anyway) . GDM will almost certainly need to be enabled for 8000MT/s+ without more voltage. There is a submenu ("Data Bus Configuration") with the output driver impedances and terminations resistances in it. You can try leaving these AUTO for now, but if that doesn't work, manually setting RTT_NOM to Disabled, all the other DRAM ODT impedance values to 40ohm (/6), DRAM DQ drive strength to 48ohm (the highest value), ProcODT to 28.2ohm, and Processor DQ drive strength to 34ohm, and Processors CA drive strength to 30ohm.
Next place for memory timings is under the "Advanced\AMD Overclocking\DDR and Infiity Fabric" menu. You're going to need to enable DDR5 Nitro Mode and want to set RX Data to 1, TX Data to 3, and Control Line to 1. You can probably get away with disabling robust training and keeping the RX/TX Bust Lengths both at 1x. There is another submenu here called "DDR PMU Training", set the values there like
this,
except for ARdPtrInitVal, which should be "
2".
Save and exist with those settings, with the boot device as the MemTest86 USB drive. It will likely take a significant amount of time to train these settings initially, then another few minutes for MemTest86 to start up. You can manually set it to run two passes of tests 6, 7, 8, and 9. If it passes, boot into Windows, grab a ZenTimings screen shot and post it here, then start using Windows tests (TM5, y-cruncher FFTv3/4, and VT3, etc) to look for instabilities.