Ubuntu 20.04 freeze on any display changes or unplugging external display

[palexdev]

I have the same issue
I'm on KDE Neon 5.23, Kernel 5.11.0.38-generic, Nvidia 470.74, X.org 1.20.11, X Protocol 11.0

I can't connect the monitor before the first login, otherwise the UI freezes and fans ramp up a little, same issue happens if I disconnect the monitor with the laptop on.
The only way to recover from these freezes is to force a power off by holding the power button

Logs attached