I'm also having the same issue, I don't use a switch. It seems every day this router needs to be rebooted to get connection again. With the older FWs it was every 2 days.
Connection type: PPPoE
Primary device: Series X
I'm on the latest firmware offered in early access. I've also shut off all power options that enable the Xbox when in sleep. All the IPV6 options are off too.