The closest server to you is the Warsaw server so I would expect that the 23ms you're seeing is to that server specifically and otherwise you're connecting to a server further away. So yes in theory getting the router to utilize the Geo-Filter would allow you to force the Warsaw server more consistently. However, I will provide some caveats so you have the full picture:
I'm presuming that the Warsaw server is the server giving you 23ms, it might be that in private matches it is relying on a P2P connection instead in which case this wouldn't be accurate and you're actually seeing 23ms to a person in your match instead.
If that is true then your normal ping could already be to the Warsaw server or another and so may not be able to be lowered.
The Warsaw server isn't one I see very often so I couldn't say if it is one that is frequently used for online play. For example, the Milan & Madrid servers are notoriously hard to force games on, even for people that live close to them so it's not a guarantee you'd always be able to force it.
.png "Like")