Should I return the Nighthawk XR500?

[Zippy]

12 minutes ago, Netduma Fraser said:

Sorry to hear that, we do hope you'll consider us again in future after more firmware updates.

As we can't reproduce it it's pretty hard for us to give advice on what to do. I would suggest downgrading to a previous firmware and seeing if you still experience the issue.

Okay.. Fraser my friend im losing this warm fuzzy feeling with this router.. I just don't want to hold on for hopes that the next firmware will fix it.. Im all about supporting you guys but I have a bad feeling about this.. Ive had many routers over time and none have been quite like this one.. Fraser would you be so kind as to give me a link to the firmware this router had when it was new out of the box? I feel that particular firmware may have been the best.. I would like to try that.. 

Zippy

[Netduma Fraser]

Don't worry, we're always continuing to hammer out any bugs and improve the router! Here's two, the first is the oldest which was the first and the second is the one that came very quickly after. Let us know what you find and that can help us understand if it's something that's always been there or if it's been introduced.

https://www.netgear.com/support/product/XR500.aspx#Firmware Version 2.0.0.28 

http://www.downloads.netgear.com/files/GDC/XR500/XR500-V2.1.0.6.zip

[Zippy]

1 hour ago, Netduma Fraser said:

Don't worry, we're always continuing to hammer out any bugs and improve the router! Here's two, the first is the oldest which was the first and the second is the one that came very quickly after. Let us know what you find and that can help us understand if it's something that's always been there or if it's been introduced.

https://www.netgear.com/support/product/XR500.aspx#Firmware Version 2.0.0.28 

http://www.downloads.netgear.com/files/GDC/XR500/XR500-V2.1.0.6.zip

Thank you kind sir! I will keep you guys posted on my experience/results.. Give me a couple of days.. I just want to give myself enough time in case should something go wrong..

Zippy..

[xr500user]

It's a firmware bug.

Clients are jumping onto the router before router dhcp server for local LAN is up and running

(this includes wifi clients)

I believe it's the sequence of startup for some - with different people experiencing different things depending on ISP, LAN setup, etc.

If you have the router up and running with the latest firmware, you can attempt to do a nice reboot but prevent all clients from jumping on..

Turn off the Wifi (to prevent wifi clients from jumping on)

It has to be done from a LAN client.   Turn off router Wifi. Issue a router reboot.

As soon as you issue the reboot, wait a few seconds and disconnect all cables from behind router (WAN, LAN ethernets) Wifi should be off prior.

Router reboots ..

Wait until it's fully booted

Plug in the WAN cable and wait until it's got Internet.

Wait a tiny bit after it has Internet, then plug in your local network ethernets, then turn wifi back on

This is just one issue.. it can cause weird things like bad dhcpd epoch times (since dhcpd running before actual time known) so IP's get locked to certain macs and they never ever expire, so when the device IP ever changes devices may be associated with multiple IPs in duma databases, and if another device happens to reconnect without renewing its lease (with it's old IP) dhcpd may gave it out to someone else who requests -- so now device manager has two different ip, but thinks it should be for a different mac addr

for others - WAN dhcp (udhcpc) is up, but local dhcp (dhcpd) is not (yet), so client requests dhcp and gets an error (since already assigned) so defaults to 169.xx and client is locked there, no ip

what needs to be done is run through all startup scripts on the router and make sure you hold back local LAN and wifi until WAN is fully up and ntp time is set

then start dhcpd, sleep, then start lan and then start wifi radio and make sure to clear any saved information in device manager databases prior to reboot

 

so it still doesn't fix bugs like sticky IPs based on last connection type for device manager, this is when last connection type was wifi for mac xx:xx:xx:xx:xx:xx and then becomes LAN for same mac.  device may stick on either side or be online or offline, but it may make your network stable enough to use for the time being, when qos devices get marked and are not thought to be online, its a problem

You can also turn QoS completely off in settings next to Anti-Bufferbloat (disable QoS), do the reboot sequence above and leave it like that. if all is good for a day or two (so any old info expires), then turn QoS back on and see if everything behaves  - there is no way to set dhcp lease time in XR500 so its set to 1 day by default, although there is no reason why this can't be changed manually - to really know everything is working nice is to take a look at the dhcp.leases file and make sure it looks good, and not any strange epoch times, but most won't do this

unfortunately until its fixed it will continue to some degree over time on those with effected configurations , and i believe it is part of the reason why xbox is having trouble, since it sleeps and wakes and sleeps and wakes, and if at any point something goes wrong in this process and it gets marked for QoS when its online but thought offline it can't maintain connection to its authentication servers because QoS KILLS flows that it doesn't know about in TIME_OUT state and blocks ACK (there's more to it then that, but just an idea), its watching conntrack and theres' a bug there too -- I believe they will fix it, but when -?- don't know -- maybe soon!

 

[Guest Killhippie]

On 1/23/2019 at 11:53 PM, Zippy said:

Thank you kind sir! I will keep you guys posted on my experience/results.. Give me a couple of days.. I just want to give myself enough time in case should something go wrong..

Zippy..

Be aware that by downgrading you are loosing the security updates, there has been a few and I would not want to run online to long without the latest patches, just thought I would mention that. 

[Netduma Jack]

Thanks XR500user, I think the developers are aware of what you're reporting but I've passed it on to them regardless.

[xr500user]

yeah, i believe they are aware - but i think not only are there some quirks in device manager (both views) and QoS match tracking which need to be addressed, its netgears base firmware causing some issues to boot

the issue is very apparent when using wifi extenders.  it seems to work for the most part correctly when the wifi extender is in wifi 'extender' mode.  client connects to wifi extender and gets a virtual mac addr, then gets allowed onto the local XR wifi.  device manager is good at picking up this virutal mac and merging it with the ip of the actual device (so it knows say iphone X is ab:cc:dd:xx:xx:xx and also cc:xx:dd:xx:xx:xx with IP 192.168.1.x)   -- but when extender is in AP mode there are no virtual macs assigned when connected to its wifi, and backhaul is ethernet so its just a pass thru the local LAN (mac addr is true) depending on first connection time and state, this is the issue.  if device first connects to an AP, its considered on Local LAN (since pass thru to ethernet) if it jumps off AP wifi and goes to true XR wifi then there's another problem since device manager holds last location (LAN or Wifi) and it will just stay on LAN as online and won't move to wifi side, sometimes marking gets stalled since nothing is triggered until a dhcp event and there are no dhcp events if the client is still in lease time, it won't ask for a new ip. this is one of many scenarios that can start a sh*t storm since device manager can't assume device once on wifi, always on wifi with the same mac addr -- issue that's effecting here is also effecting qos tracking module, if it can't know the device is online and active how can it mark it? it doesn't, so old info may still be in there with prior marks and wrong ip (if it switched) connections get dropped (connection tracking) and it lingers with a reboot and only gets worse after each reboot as the dhcp.leases files gets more and more out of sync along with dumas tracking of them so to say. sometimes if the client leaves the network from wifi, but was thought to be on LAN the ip isn't being cleared by device manager so theres your "Error cannot delete device because its online"  - its odd though that table view knows the ip isn't there sometimes (show's N/A) but Tree will still show the IP as online even when offline.  now xbox sleeping and waking over and over and over would work in a perfect world but if anything gets some bad info it causes the lost connectivity issue. i'm pretty sure xbox sleep mode (instant on) has no issues on R7800, there is still some legacy things interfering with duma os they have not closed it all down yet. device manager does trigger some events on dhcp renew and other modules also rely on the same information gathering method, so if its wrong, you know problems happen, some may not materialize to state where a user can notice and bitch about,  but clearly you see, some are

 

the startup sequence (bootup) causes issues in  larger networks where multiple devices have different lease expirations.. and if the lease time is hosed in the XR (for whatever startup reason) it can cause problems because once clients notice a connection is available it will request a renew, or just start thinking hey im ok - im in lease time, and start working as normal.   XR kernel has not yet determined its mac addr (and may not know the true ntp time yet so ntp should be set immediately after WAN is up) so it will assign all 00's (incomplete) to that IP it sees data coming from or requested dhcp from, and when the router actually is up and running the device may hit a renew period or request again and oh no that IP is in use, give me a new one, so the router will assign a new IP address to the client, but qos markings are for the old IP - another sh*t storm, and if the IP has a weird lease time of like 1969 or 1970 forget it, its ip will never ever release for use

some way the router needs to fully boot and and acquire WAN ip and internet access,  duma os needs to fully start (with empty databases and tracking) and then LAN needs to start and then Wifi radio on.  at different points of the startup process these things are seen for a split second as active (192.168.1.1) and devices think everything is ok and either request dhcp or mantain what they had before the reboot then interface may get moved to br0 interface but the damage has been done to the files and databases (bad data in there) it may only effect certain devices with just bad luck timing so that's why the more devices the more chance it has to occur.. and if a device actually tries to renew for some it may not get an ip and just give up and set itself to 169.xx.xx.xx - sometimes it works after a reboot so people think its fixed, but its not just had a lucky start up and could occur again

yeah it will result is longer boot up times before everything is started, but so be it, just let people know it may be 2,3+ minutes before internet is available on reboot, if you imagine a boot up with multiple APs (which may have 10 clients behind them) all assaulting the dhcp server as soon as it sees some activity from the gateway and certain things are not yet known to the kernel or duma

theres other things too that are going on causing issue, its probably best if duma just takes control of the kernel if they just can't get it to work, too many chefs in the kitchen

xr700 is just one more complication since the 10G port and the ability to aggregate or use 10G port as uplink or WAN -

duma bugs need to addressed, and netgear startup needs to be optimized for all scenarios .. its hard to fix bugs until netgear startup is working optimally, lol. catch .22 .. especially if work arounds were applied for prior bugs that were corrected but now after being corrected cause new bugs, a real hair puller

 

[Zippy]

On ‎1‎/‎25‎/‎2019 at 8:12 AM, xr500user said:

yeah, i believe they are aware - but i think not only are there some quirks in device manager (both views) and QoS match tracking which need to be addressed, its netgears base firmware causing some issues to boot

the issue is very apparent when using wifi extenders.  it seems to work for the most part correctly when the wifi extender is in wifi 'extender' mode.  client connects to wifi extender and gets a virtual mac addr, then gets allowed onto the local XR wifi.  device manager is good at picking up this virutal mac and merging it with the ip of the actual device (so it knows say iphone X is ab:cc:dd:xx:xx:xx and also cc:xx:dd:xx:xx:xx with IP 192.168.1.x)   -- but when extender is in AP mode there are no virtual macs assigned when connected to its wifi, and backhaul is ethernet so its just a pass thru the local LAN (mac addr is true) depending on first connection time and state, this is the issue.  if device first connects to an AP, its considered on Local LAN (since pass thru to ethernet) if it jumps off AP wifi and goes to true XR wifi then there's another problem since device manager holds last location (LAN or Wifi) and it will just stay on LAN as online and won't move to wifi side, sometimes marking gets stalled since nothing is triggered until a dhcp event and there are no dhcp events if the client is still in lease time, it won't ask for a new ip. this is one of many scenarios that can start a sh*t storm since device manager can't assume device once on wifi, always on wifi with the same mac addr -- issue that's effecting here is also effecting qos tracking module, if it can't know the device is online and active how can it mark it? it doesn't, so old info may still be in there with prior marks and wrong ip (if it switched) connections get dropped (connection tracking) and it lingers with a reboot and only gets worse after each reboot as the dhcp.leases files gets more and more out of sync along with dumas tracking of them so to say. sometimes if the client leaves the network from wifi, but was thought to be on LAN the ip isn't being cleared by device manager so theres your "Error cannot delete device because its online"  - its odd though that table view knows the ip isn't there sometimes (show's N/A) but Tree will still show the IP as online even when offline.  now xbox sleeping and waking over and over and over would work in a perfect world but if anything gets some bad info it causes the lost connectivity issue. i'm pretty sure xbox sleep mode (instant on) has no issues on R7800, there is still some legacy things interfering with duma os they have not closed it all down yet. device manager does trigger some events on dhcp renew and other modules also rely on the same information gathering method, so if its wrong, you know problems happen, some may not materialize to state where a user can notice and bitch about,  but clearly you see, some are

 

the startup sequence (bootup) causes issues in  larger networks where multiple devices have different lease expirations.. and if the lease time is hosed in the XR (for whatever startup reason) it can cause problems because once clients notice a connection is available it will request a renew, or just start thinking hey im ok - im in lease time, and start working as normal.   XR kernel has not yet determined its mac addr (and may not know the true ntp time yet so ntp should be set immediately after WAN is up) so it will assign all 00's (incomplete) to that IP it sees data coming from or requested dhcp from, and when the router actually is up and running the device may hit a renew period or request again and oh no that IP is in use, give me a new one, so the router will assign a new IP address to the client, but qos markings are for the old IP - another sh*t storm, and if the IP has a weird lease time of like 1969 or 1970 forget it, its ip will never ever release for use

some way the router needs to fully boot and and acquire WAN ip and internet access,  duma os needs to fully start (with empty databases and tracking) and then LAN needs to start and then Wifi radio on.  at different points of the startup process these things are seen for a split second as active (192.168.1.1) and devices think everything is ok and either request dhcp or mantain what they had before the reboot then interface may get moved to br0 interface but the damage has been done to the files and databases (bad data in there) it may only effect certain devices with just bad luck timing so that's why the more devices the more chance it has to occur.. and if a device actually tries to renew for some it may not get an ip and just give up and set itself to 169.xx.xx.xx - sometimes it works after a reboot so people think its fixed, but its not just had a lucky start up and could occur again

yeah it will result is longer boot up times before everything is started, but so be it, just let people know it may be 2,3+ minutes before internet is available on reboot, if you imagine a boot up with multiple APs (which may have 10 clients behind them) all assaulting the dhcp server as soon as it sees some activity from the gateway and certain things are not yet known to the kernel or duma

theres other things too that are going on causing issue, its probably best if duma just takes control of the kernel if they just can't get it to work, too many chefs in the kitchen

xr700 is just one more complication since the 10G port and the ability to aggregate or use 10G port as uplink or WAN -

duma bugs need to addressed, and netgear startup needs to be optimized for all scenarios .. its hard to fix bugs until netgear startup is working optimally, lol. catch .22 .. especially if work arounds were applied for prior bugs that were corrected but now after being corrected cause new bugs, a real hair puller

 

xr500user you definitely know the inner workings far past what I know.. My biggest issue is that this effects my QoS.. Which is extremely noticeable when playing a game. So with that said if one goes through the setup like you stated above does this help with things staying correct and not getting all screwed up? Or are we still really uncertain what really triggers this so called bug?  If I do a hard reset start and jump online and play some games they play very well.. But it seems to last about a day or less.. If I do a simple reboot sometimes it helps and sometimes it doesn't and things can even become worse.. But when a clean factory reset is done everything seems fine again.. Until I trigger this unknown.. The QoS gets all messed up again.. This is like  running around in a circle..  When this also takes place if I disable my QoS while still playing during a game it recovers a bit. But obviously not as nice as if it was when the QoS is enabled and performing correctly.. When I do a factory reset and pay attention to my packets being prioritize my download packets are always more then my upload packets. Which is correct because when playing a game we are downloading more then uploading.. But when I trigger this unknown bug my download packets come to a crawl and basically stop! So at that time the router is no longer doing its job and is useless.. And again if I do a factory reset after this takes place things are fine and my download packets are being prioritized once again and gaming is all good..

 After reading what you have said above doesn't sound to promising and really this router sounds like a blender of issues that could and can unfold.. While Netduma maybe aware of some of these issues this sounds like these issues are very deep and could even be unfixable.. That's my gut feeling.. Its strange I have two friends of mine that have both the R1 and the XR500.. And both swear the R1 works better. Im wondering if this is why they say that.. Currently both of them are not using the XR500.. They are using the R1.. Im starting to think I should have bought an R1!! LOL All I want to do is game not fiddle with this router day in and day out just to play a few good games.. If you have anything else you want to share or ideas to try please feel free to share!  Thanks!

Zippy..

[Netduma Fraser]

Zippy what is your physical setup like? Also gaming uses primarily upload so it's working correctly when upload packets are more rapid than download

[xr500user]

7 hours ago, Zippy said:

xr500user you definitely know the inner workings far past what I know.. My biggest issue is that this effects my QoS.. Which is extremely noticeable when playing a game. So with that said if one goes through the setup like you stated above does this help with things staying correct and not getting all screwed up? Or are we still really uncertain what really triggers this so called bug?  If I do a hard reset start and jump online and play some games they play very well.. But it seems to last about a day or less.. If I do a simple reboot sometimes it helps and sometimes it doesn't and things can even become worse.. But when a clean factory reset is done everything seems fine again.. Until I trigger this unknown.. The QoS gets all messed up again.. This is like  running around in a circle..  When this also takes place if I disable my QoS while still playing during a game it recovers a bit. But obviously not as nice as if it was when the QoS is enabled and performing correctly.. When I do a factory reset and pay attention to my packets being prioritize my download packets are always more then my upload packets. Which is correct because when playing a game we are downloading more then uploading.. But when I trigger this unknown bug my download packets come to a crawl and basically stop! So at that time the router is no longer doing its job and is useless.. And again if I do a factory reset after this takes place things are fine and my download packets are being prioritized once again and gaming is all good..

 After reading what you have said above doesn't sound to promising and really this router sounds like a blender of issues that could and can unfold.. While Netduma maybe aware of some of these issues this sounds like these issues are very deep and could even be unfixable.. That's my gut feeling.. Its strange I have two friends of mine that have both the R1 and the XR500.. And both swear the R1 works better. Im wondering if this is why they say that.. Currently both of them are not using the XR500.. They are using the R1.. Im starting to think I should have bought an R1!! LOL All I want to do is game not fiddle with this router day in and day out just to play a few good games.. If you have anything else you want to share or ideas to try please feel free to share!  Thanks!

Zippy..

have you tried just disabling QoS and just leave it off prior to/and after the fresh reboot? (just disable QoS, reboot, leave it off)   just to see if it works better? turning it on and off when its been up for a while creates some corruption if something is not right gets in there.  i know this because of changing some things behind duma os back and forcing/sending dummy DHCP events to device manager while trying to figure out exactly what its doing (which script triggers, etc) after an event to pinpoint the failure.. at one point I was unable to turn off QoS fully since I had removed rules that it was looking for to delete when being turned off, lol.  I have yet to factory reset once since getting this router though.. I always have been able to restore it to a very stable working state manually.

rebooting the way i said is probably the cleanest reboot you are going to get, but yes whatever is triggering it may pop up again and throw everything out of wack (monkey wrench in the machine) the problem is the constant rebooting everyone is doing without that method creates more problems on each reboot - you never know what you're going to get, especially if some IPs get locked up on a reboot (ones that you've assigned statically) so if the static device requests an IP it can't get its permanently assigned one back so it may get a different one or just decide to up and quit and settle with a 169 non usable ip .. and then as time goes on it just gets more and more out of sync with other settings (qos, port forwards you may have set, etc) i don't think that it is unfixable, it just needs to be fixed. its the illusion that the lan port isn't working, but it is, just the device is not   i have faith they will do it. they got some smart guys there, and what they are doing is not an easy task.. i give mad props to the cross.. it's advanced traffic shaping for dummies on the interface, but deep down so much stuff is going on behind the scenes and things need to be in order for it all to come together and one link in the chain fails.. its a cascade of small failures, just a nasty bug that i'm sure they will deal with it. something has to give with netgear

[Zippy]

7 hours ago, Netduma Fraser said:

Zippy what is your physical setup like? Also gaming uses primarily upload so it's working correctly when upload packets are more rapid than download

 Hey Fraser, My setup isn't anything out of the norm. Very basic.. cable modem (non puma) to router (xr500). That's it.. And I only have 4 devices attach to my router.. One xbox, 2 labtops, and one iPhone.. iPhone and one labtop is wireless while the xbox and other labtop is hardwired. That's it.. Nothing fancy going on here. Xbox isn't on standby mode either..Its on energy saving which is full shutdown.. Now on to your part about packets that isn't the case. I can play a handful of games when things are fine and packets are being prioritized very smoothly in both directions.. You can physically see this.. But whatever takes place the download packets come to a halt and even the upload packets slow down to a crawl.. And remember this is happening after several games have been played.. My download rate while playing a game ranges between 580-640 kbps.. My upload ranges at around 280 kbps.. Which is normal..  We always download more then uploading when playing a game..  I think even you may have mentioned this when we were discussing this about port prioritization awhile back.. Do you understand what im saying? The reason I bring this up is because this is another way to tell when things start to get messed up in the router.. Packets quit prioritizing.. And this is part of QoS which even xr500user has mentioned that QoS can get corrupted and does get corrupted for me..

Zippy

[Zippy]

22 minutes ago, xr500user said:

have you tried just disabling QoS and just leave it off prior to/and after the fresh reboot? (just disable QoS, reboot, leave it off)   just to see if it works better? turning it on and off when its been up for a while creates some corruption if something is not right gets in there.  i know this because of changing some things behind duma os back and forcing/sending dummy DHCP events to device manager while trying to figure out exactly what its doing (which script triggers, etc) after an event to pinpoint the failure.. at one point I was unable to turn off QoS fully since I had removed rules that it was looking for to delete when being turned off, lol.  I have yet to factory reset once since getting this router though.. I always have been able to restore it to a very stable working state manually.

rebooting the way i said is probably the cleanest reboot you are going to get, but yes whatever is triggering it may pop up again and throw everything out of wack (monkey wrench in the machine) the problem is the constant rebooting everyone is doing without that method creates more problems on each reboot - you never know what you're going to get, especially if some IPs get locked up on a reboot (ones that you've assigned statically) so if the static device requests an IP it can't get its permanently assigned one back so it may get a different one or just decide to up and quit and settle with a 169 non usable ip .. and then as time goes on it just gets more and more out of sync with other settings (qos, port forwards you may have set, etc) i don't think that it is unfixable, it just needs to be fixed. its the illusion that the lan port isn't working, but it is, just the device is not   i have faith they will do it. they got some smart guys there, and what they are doing is not an easy task.. i give mad props to the cross.. it's advanced traffic shaping for dummies on the interface, but deep down so much stuff is going on behind the scenes and things need to be in order for it all to come together and one link in the chain fails.. its a cascade of small failures, just a nasty bug that i'm sure they will deal with it. something has to give with netgear

xr500user everything you explained there is exactly what I have experienced and have been trying to explain to Fraser.. Thanks so much for pointing things out more.. A few things I want to touch on also is you are correct that when things get more and more out of sync other things get effected also.. UPnP becomes effected , if one is using port forwarding that also gets effected. Things really can snowball.. Right now my UPnP shows active for a device that isn't even online.. Just to check this I unplugged the ethernet cord to the device and checked it the very next day and it was still saying active.. Once I see this that also is another sign of trouble.. Its so hard to trouble shoot this because idk what is actually working correctly in this router and what isn't. But since so many things can and do get effected it gets very troubling.. So at this point are you saying that I should disable all QoS and leave it off for good until they find a possible fix?   Thanks again for all your help in this.. And please keep sharing if you discover more!  I sure hope the Duma team can recreate these issues..

Thanks! Zippy..

[xr500user]

i would shoot for having it on if everything is stable, as everything works better when its on (unless its not working and causing you issues of course) if you turn it off then you lose the ability to see what the breakdown of the traffic is in network monitor area (which only seems to show the top 5 consumers at any one time, if a device is not there and you have more then 5 devices on your network - this doesn't mean its not working, and doesn't see the device, it just isn't consuming enough to be in the top 5) when qos is on you can click the upload or download bars on the device in network monitor and it will open a breakdown to the right of it of what it thinks the traffic is (games, media, web, etc) and if you go one step further and click on the color on the circle it will show additional breakdowns. if you click the upload and download bar on total usage line (first on the list) -overall usage- you can see breakdowns like what apps overall are consuming when you click the color on the circle that you want to see  (twitch, bittorrent, SSL traffic, apple, microsoft, ssl, media streaming, youtube, netflix, etc) when its off, this info will just be unknown traffic.  qos tries to prioritize whats more important and should go before someone else , in a perfect world it works. it classifys and then marks certain connections with priorities and if the connection for example say is thought to be offline in another module, for whatever reason (a monkey) you know we have an issue here , sometimes these classifications stick, even through a reboot and some chaos starts, combined with the dhcp thing, etc it could be a mess quick.. you wanna know why nobody else is doing what they are doing -  this because its a real pain in the ass and pretty difficult to get it to work for everyone and every network scenario, but credit goes where credit is due they are the only one this close to perfecting it

upnp is another animal, netgear always had issues with it in one form or another (reporting wise), upnp is supposed to open ports for you automatically so you don't have to do the port forwarding.  but the requesting application is thought to be trusted if on the local LAN so it goes ahead and opens the port in the iptables firewall on the router.. and after a certain time its supposed to remove it, just because it shows there its most likely to be closed if after the limit..as other things on the kernel take care open connections that should not be lingering. its the upnp daemon netgear chooses  just not updating its upnp_pmlist file (whats reflected on the admin interface), and sometimes it doesn't update it on creation either.  there's a lot more information inside this file that they could show you like what app requested the upnp but they just don't show it (like Toredo, Skype, identd, etc) its in there, they are just not showing it on the interface (keeping it simple for the user) another issue is the version of upnp they are using to do all this is possibly custom modified over the years (at least we hope!) and may not have all the recent changes/fixes - future versions fixed issues of removing items from this list (just look at the changelog for miniupnpd) they did have a fix for closing the ports on the RFC on the list, and in addition to that...

2018/05/02:
  option to store remaining time in leasefile

which could be useful so you can restart miniunpd or reboot and not lose active upnp ports if they should still be open and devices were never switched off during the reboot, also could help in tracking these and fixing the list with a watcher if a monkey does jump in

but this requires netgear to update its daemons and modules, something they don't seem to want to do and nobody ever has given answer to that question as to why?...

but rest assured just because its not reporting correctly it is opening/closing the ports as it should because things wouldn't work at all if it wasn't.  the kernel has many NAT helpers built into it that do a lot of the work of upnp and bypassing upnp it seems, because I see them being managed through a different way.. certain apps like Skype will show because its old school requesting..or does not have a helper. some apps are only going to request a port once and not again until at least the time limit expires, but if you close an app and reopen it will request again and show on the list (or reboot the xbox so it requests them again)

if anything ever jumps out at me i will report of course, and i am no expert in this, just dabbling and like to know how these things are working (and when they don't)

[Netduma Fraser]

The developers are aware so I think it's best for me to leave it to them as it sounds like you've tried everything I would have suggested

[Zippy]

6 hours ago, Netduma Fraser said:

The developers are aware so I think it's best for me to leave it to them as it sounds like you've tried everything I would have suggested

Thanks Fraser for all your help though!   I just want you to know I do appreciate your help. Regardless of the outcome. 

Zippy.

[Zippy]

On ‎1‎/‎27‎/‎2019 at 12:46 AM, xr500user said:

i would shoot for having it on if everything is stable, as everything works better when its on (unless its not working and causing you issues of course) if you turn it off then you lose the ability to see what the breakdown of the traffic is in network monitor area (which only seems to show the top 5 consumers at any one time, if a device is not there and you have more then 5 devices on your network - this doesn't mean its not working, and doesn't see the device, it just isn't consuming enough to be in the top 5) when qos is on you can click the upload or download bars on the device in network monitor and it will open a breakdown to the right of it of what it thinks the traffic is (games, media, web, etc) and if you go one step further and click on the color on the circle it will show additional breakdowns. if you click the upload and download bar on total usage line (first on the list) -overall usage- you can see breakdowns like what apps overall are consuming when you click the color on the circle that you want to see  (twitch, bittorrent, SSL traffic, apple, microsoft, ssl, media streaming, youtube, netflix, etc) when its off, this info will just be unknown traffic.  qos tries to prioritize whats more important and should go before someone else , in a perfect world it works. it classifys and then marks certain connections with priorities and if the connection for example say is thought to be offline in another module, for whatever reason (a monkey) you know we have an issue here , sometimes these classifications stick, even through a reboot and some chaos starts, combined with the dhcp thing, etc it could be a mess quick.. you wanna know why nobody else is doing what they are doing -  this because its a real pain in the ass and pretty difficult to get it to work for everyone and every network scenario, but credit goes where credit is due they are the only one this close to perfecting it

upnp is another animal, netgear always had issues with it in one form or another (reporting wise), upnp is supposed to open ports for you automatically so you don't have to do the port forwarding.  but the requesting application is thought to be trusted if on the local LAN so it goes ahead and opens the port in the iptables firewall on the router.. and after a certain time its supposed to remove it, just because it shows there its most likely to be closed if after the limit..as other things on the kernel take care open connections that should not be lingering. its the upnp daemon netgear chooses  just not updating its upnp_pmlist file (whats reflected on the admin interface), and sometimes it doesn't update it on creation either.  there's a lot more information inside this file that they could show you like what app requested the upnp but they just don't show it (like Toredo, Skype, identd, etc) its in there, they are just not showing it on the interface (keeping it simple for the user) another issue is the version of upnp they are using to do all this is possibly custom modified over the years (at least we hope!) and may not have all the recent changes/fixes - future versions fixed issues of removing items from this list (just look at the changelog for miniupnpd) they did have a fix for closing the ports on the RFC on the list, and in addition to that...

2018/05/02:
  option to store remaining time in leasefile

which could be useful so you can restart miniunpd or reboot and not lose active upnp ports if they should still be open and devices were never switched off during the reboot, also could help in tracking these and fixing the list with a watcher if a monkey does jump in

but this requires netgear to update its daemons and modules, something they don't seem to want to do and nobody ever has given answer to that question as to why?...

but rest assured just because its not reporting correctly it is opening/closing the ports as it should because things wouldn't work at all if it wasn't.  the kernel has many NAT helpers built into it that do a lot of the work of upnp and bypassing upnp it seems, because I see them being managed through a different way.. certain apps like Skype will show because its old school requesting..or does not have a helper. some apps are only going to request a port once and not again until at least the time limit expires, but if you close an app and reopen it will request again and show on the list (or reboot the xbox so it requests them again)

if anything ever jumps out at me i will report of course, and i am no expert in this, just dabbling and like to know how these things are working (and when they don't)

Hey xr500user I have a question about the breakdown of the Network Monitor.. When I click on the device for a breakdown on what is being prioritized mine comes up with unknown.. This is for my xbox.. And this is with QoS enabled.. What are your thoughts on this?  I never have seen anything stating gaming traffic. Both download and upload.. Im guessing I triggered the bug again.. What should this say when im gaming? Im very curious about what it is suppose to say.. At no time have I ever seen it say gaming traffic..

Thanks! 

Zippy.

[Netduma Jack]

13 hours ago, Zippy said:

Hey xr500user I have a question about the breakdown of the Network Monitor.. When I click on the device for a breakdown on what is being prioritized mine comes up with unknown.. This is for my xbox.. And this is with QoS enabled.. What are your thoughts on this?  I never have seen anything stating gaming traffic. Both download and upload.. Im guessing I triggered the bug again.. What should this say when im gaming? Im very curious about what it is suppose to say.. At no time have I ever seen it say gaming traffic..

Thanks! 

Zippy.

Hi Zippy - I can pitch in on this. All services are categorised using our DPI, but if there's a service running which isn't yet categorised it'll show up as 'unknown' or 'uncategorised'. The traffic / data transfer will still be happening perfectly fine, it's just not correctly displayed on the Network Monitor. Just to clarify, which game are you playing when it comes up as an unknown service?

[Zippy]

3 hours ago, Netduma Jack said:

Hi Zippy - I can pitch in on this. All services are categorised using our DPI, but if there's a service running which isn't yet categorised it'll show up as 'unknown' or 'uncategorised'. The traffic / data transfer will still be happening perfectly fine, it's just not correctly displayed on the Network Monitor. Just to clarify, which game are you playing when it comes up as an unknown service?

Hey Jack   The game that im playing is Halo 5.  When I click on the bar say for upload a circle does open up to the right.. Then I click on the circle again to see what is transferring and another circle  comes up and says unknown.. This is with QoS enabled..  Hope this makes sense. 

Thanks Jack!

Zippy.

 

[Netduma Fraser]

It would mostly show when playing PC games when we can detect the individual engines being used like Steam and Unreal. It wouldn't say Halo or anything along those lines.

[Zippy]

Okay Thanks! I just figured it would say something other then the whole circle in red that is all unknown.. My ASUS ROG Router has a similar feature as this.. It would tell me all apps and what not tied to it.. Like Microsoft, Halo, and other attachments.. Thanks again!

Zippy.

[Netduma Fraser]

Interesting that it would say Halo given the packets from the console are likely encrypted

[Zippy]

4 hours ago, Netduma Fraser said:

Interesting that it would say Halo given the packets from the console are likely encrypted

It had things in there like Halo, Microsoft, SSL.. and others but I cant think of all of them.. But yes it did have Halo and other things attached to it.. It was really very interesting..  It must have something in the router that knew what it was because when one would hookup the xbox it would display it automatically and identify it as an Xbox.. And in the QoS it would display it as such and give further breakdowns from there.. It also has a section for prioritizing ports.. And even goes into depth of how much a port is transferring data.. There are a lot of similarities between the ROG and XR500..

[Netduma Jack]

8 hours ago, Zippy said:

It had things in there like Halo, Microsoft, SSL.. and others but I cant think of all of them.. But yes it did have Halo and other things attached to it.. It was really very interesting..  It must have something in the router that knew what it was because when one would hookup the xbox it would display it automatically and identify it as an Xbox.. And in the QoS it would display it as such and give further breakdowns from there.. It also has a section for prioritizing ports.. And even goes into depth of how much a port is transferring data.. There are a lot of similarities between the ROG and XR500..

No idea how they could've classified the game specifically when it's all encrypted - sounds like they might've used some dodgy means to achieve that! Still, we're going to keep improving our DPI anyway, and we'll work on giving more and more detail to folks who want it  Lots of cool features down the road...

[Zippy]

7 hours ago, Netduma Jack said:

No idea how they could've classified the game specifically when it's all encrypted - sounds like they might've used some dodgy means to achieve that! Still, we're going to keep improving our DPI anyway, and we'll work on giving more and more detail to folks who want it  Lots of cool features down the road...

Hey Jack I do look forward to the other features and improvements.. One of my most favorite things I like about my ASUS is that it had port prioritization very similar to the XR500.. But my biggest issue with the XR500 is the only port arrangement that triggers the light to come on in the XR500 is 3074 source and 3544 destination UDP.. Any other arrangement doesn't do anything.. And the problem with that is key ports the server hosting the game are using are not triggering it.. If you have a answer for that im all ears! One thing I would recommend is to have a viewable portable of what ports are actually being used in both directions..  This would be a good tool to have so one can source target specific ports only.. And not use an arrangement of ports.. Source targeting a port is much more efficient then using a arrangement of ports.. Plus its also very handy when one is trying to troubleshoot a specific game..  

Thanks Jack!

Zippy.

[Netduma Fraser]

DumaOS Classified Games / Games Console service covers the port range that any game will use so it should light up. That's a good idea, probably one for a bit further down the line once we've achieved the features we want to implement first.