I have the same type of button delay with EA's NHL and Madden series. I have tried many different routers and modems.
I'm from the US and Time Warner Cable / Spectrum is the ISP I am stuck with. Ping doesn't seem to be the issue with me
either. This is very confusing.
Oh and yeah 20ms on a HDTV is about as good as it gets. I'm using a BENQ gaming monitor and it has 10ms input delay.