Router randomly lost internet connection and couldn’t access it

[Netduma Fraser]

That's very interesting, they should send that unit back to devs to debug! Thanks for the additional info. It's a shame issues like these are completely out of our control.

[Gouldin]

Well in good news, netgear came to same conclusion as me and are replacing it as it seems the flash chip is buggered on it. Hopefully xr500 number 2 will be stable to go along with the awesome pings it was getting

[Fathomless]

Update: 

Cable company technician did not come yet, I keep my fingers crossed (Technical will be tasked to bring upstream signal up to 46dbmv for 4 channels in my case Arris spec calls for 45-51dbmV ). In mean time I stopped loosing internet connection, do not have to do modem reboots everyday, somehow I did 3 things at the same time and do not know what helped.

I noticed interesting pattern on the XR500 though, it would be useful if netduma team could include addition network KPIs I will describe pattern later.

 

Changes I did:

1. Downgraded internet speed from 1gbps to 150mbps, I looked at the network monitoring provided XR 500 I only have spikes to 90mbps, should not impact me short term; I kept  DOCSIS 3.1 protocol, I have a plan to go back to 1gbps and I still use all 32 downstream channels . Speed test captures 180mps, that is expected though for 150mbps plan.

2. Arris SB8200 modem has two Ethernet ports, I switched from the first port to second. (assuming that interfaces are managed interfaces potentially could make a difference; I do not think that this step can fix the problem )

3. Called cable company to send busting signal to my modem to increase upstream signal I was hopping that I can increased power by 2dbmV (Comcast cannot do that).  Right now my upstream signal levels fluctuates between 36dbmV-38dbmV, signal to noise ratio just ok 38db so low upstream signal can contribute to renegotiation of the upstream modulation standards/protocols and/or potentially connection drop.

 

Tech. have to go find amplifier on the Comcast's line and tune it down so Comcast monitoring would find that signal is low and would increase output on the modem ,  signal travels across CM(cable modem)->cable -> fiber->cable -> cmts, and would have amplifier and attenuators, modulator and demodulators, so I have to keep my fingers crossed that tech is high caliber and also can read Spectrum diagrams, since I discovered unusual spectrum when I looked at spectrum analyzer.

 

 

appendix:

1. I selected my next modem it will be Netgear CM1000, coming Wednesday. Why netgear?

- has the same chipset as Arris SB8200(Motorola company owned) and Motorola MB8600

- Comcat and Arris both have poor customer support they put customer in the middle, while have contractual obligations and marketing reinforcing the sales.

 

2. During troubleshooting I experienced pattern when XR500 UI response would be slow, speed tests would start on, but low throughput is like 5-7mps, wireless communication was slow, latencies were high

- I looked at CPU - normal 20-30% combined

- looked at through put and it looked normal

- rebooted the modem since i was waiting for that ..... solved router performance, issue.

 

Would it be possible to have additional Key Performance Indicators (KPI) such as: concurrent connection(inbound/outbound); connections total inbound/outbound?

It is difficult to troubleshoot performance issues with XR500, device is complex enough and has component of router, component of switch, component of WIFI, and on top of it runs software firewall like software with packet inspection which we all like ; from another side - setup of additional equipment to do network captures does make sense for the home and it requires additional skills as well, so it would simplify my life if I just had a few monitors.

[Netduma Fraser]

Fantastic to hear you're not experiencing that issue anymore, hopefully it stays that way. We could certainly look at those features if it's something people want!

[Fathomless]

Update:

 

Had another repro, related to activity on XR500,

1. ran WIFI speed test via the Access point (R6700), run two of them at the same time.

2. did it a few times, as a result XR500, was not able to route traffic anymore, all devices on the access point were down it killed the Switch portion of the router again

3. did speed tests via WIFI(XR500), had 3 instances running at the same time was hoping to kill WIFI, WIFI was stable internet speed tests were ok. (did not notice very high CPU levels on the xr500)

4. Reboot the XR500 as I always do in this case

5. After reboot lost internet connection, I could talk to devices which connected on the XR500 but no internet

6. rebooted the modem everything came back

 

So in this case reboot of the router took down the modem

 

I am not sure but it could be that both devices are suffering from the similar bug, under high traffic.

[Netduma Fraser]

Thanks for the info, someone else I think had something similar to the WiFi tests taking it down. I'm not sure why that is but I couldn't reproduce it. Will investigate further.

[Fathomless]

I guess it would be difficult to reproduce, the interesting thing is that XR500 is stable with FIOS provider, I do not know what to think is it placebo effect? or maybe I am testing more with Comcast.

In my home I do not have big impact since I can pick and chose what provider to use, and what devices can be on the FIOS network and what devices on the cable.

But I guess normally people would use only one provider at the time and that could be potential problem down the road. 

 

There were 2 repros in one day, I do not know if you have to have Arris SB8200 to repro it since I do not know yet what XR500 sends to the modem and how modem replies and if that communication is causing a fault, I can tell more later this week since I am switching to Netgear CM1000, that modem had reports of connectivity issue on the Ethernet ports as well. I think Netgear resolved Ethernet connectivity issues for the CM1000 modem, if not I will join two cases.

 

Here is how it happened for me this time, in case if you would try to reproduce failure:

1. Modem - disconnected while I was streaming from multiple devices, and all other appliances would try to get to internet. 10-20 min disconnect time for the modem.

2. Modem connected and Router established connection automatically as it should. 

3. First issue happened within an hour(did not noticed exact time), I renewed IPTV archive streaming and amazon fire tv reported connectivity to the internet interrupted, switched to XR500 WIFI and streaming started again.

4. Rebooted XR500 after verifying that xbox could not connect as well.

5. reboot resolved the issue until later that night where I was playing with download test to see if XR500 network counters are reflecting right bandwidth. one of the tests was to run two speed tests from different browsers, and on that test I lost connection again.

5. Verified that XR500 WIFI is still working well.

6. Rebooted the router, that did not fix the internet, check if communication across WIFI and Ethernet interfaces happening. 

7. Rebooted the SB8200 modem.

8. Issues fixed, did not come back yet, but also I did not have much time to do continues speed tests, it was much faster to kill these devices when I had 1gbps.

 

I did final reboot of devices in sequence:

1. shutdown router and modem at the same time

2. turn the router up

3. when router is fully up, bring the Modem up.

4. after this sequence my modem and router are stable, go figure what is going on and why it is stable now

[Netduma Fraser]

You're very thorough, this is much appreciated! Have you been able to update your cases with all this extra information? I wouldn't have thought a different modem would cause the issue though.

[Fathomless]

I will update the NG case after new modem installation is done, hopefully end of the week. I might see new patterns as well.

[Fathomless]

I do not think my case will stand - both issues fixed, I am updating my case with Netgear.

 

Installed new modem this time netgear CM1000, as a result I have stable internet, upstream power improved. XR500 behaves now, I cannot repro any cases I had,  stress test is stable as well.

 

Recap for the issue with Arris sb8200: time to time it will loose Ethernet connectivity that is level 2 OSI model (arp table becomes corrupt), CM1000 had similar case but they fixed it promptly; notable: I see CM1000s upstream power is higher, compare to Arris sb8200.

 

XR500 might be loosing Ethernet connectivity on level 2 OSI model as well, the question is can netgear XR500 team look at the case with CM1000 and try correlate underlining issues.

Another question: Can netgear provide SOP and debugging tools, so people can upload finding to netgear?

 

Here is side by side stats in case if anybody else has similar pattern with internet disconnect and resolution.

 

Arris SB8200:

Downstream  power across 32chanells(docsis 3.1): -1dbmv to 0dbmv (within specification)

Signal to noise ratio: 38db - ok

Upstream power across 4 channels: 37dbmv dropping bellow 35dbmv at times. Arris SB8200 support wants to have 45dbmv-51dbmv for 4 channels, Comcast could bring the power up, they will do company to company investigation how to match specifications.

 

Netgear CM1000:

Downstream: -1dbmv -ok

Signal to noise: 39db - ok

upstream power: 40.3 dbmv - ok (specification from netgear calls for 38-48dbmv)

[Netduma Fraser]

Great to hear your issue has been resolved, we're in the process of debugging so hopefully we can get this resolved for others shortly.

[Bmann]

Great information here this has made my decision clear. I am also replacing my Arris 8200 and will step down (meaning speed docsis 3.0)to a Netgear cm600 since I dont have gigabit service anyways and the Arris was overkill for me in the first place. I am also on a very similar setup with Comcast being my ISP. I have read other posts on other fourms that the Netgears line of cable modems seem to be very stable with the combination of XR500.

 

On a side note I am seeing a daily t3 time out in my arris just one time though and usually like middle of night or day when no one is online.

[Netduma Fraser]

Great information here this has made my decision clear. I am also replacing my Arris 8200 and will step down (meaning speed docsis 3.0)to a Netgear cm600 since I dont have gigabit service anyways and the Arris was overkill for me in the first place. I am also on a very similar setup with Comcast being my ISP. I have read other posts on other fourms that the Netgears line of cable modems seem to be very stable with the combination of XR500.

 

On a side note I am seeing a daily t3 time out in my arris just one time though and usually like middle of night or day when no one is online.

 

Great do let us know how you get on! While it would be a relief that it's an issue not caused by the XR500 we can't tell everyone who has the issue to get a new modem so hopefully we can do a workaround.

 

That is probably them doing updates/tests, they will usually do these in the middle of the night we're it's most likely to impact the least amount of people.

[Fathomless]

Great information here this has made my decision clear. I am also replacing my Arris 8200 and will step down (meaning speed docsis 3.0)to a Netgear cm600 since I dont have gigabit service anyways and the Arris was overkill for me in the first place. I am also on a very similar setup with Comcast being my ISP. I have read other posts on other fourms that the Netgears line of cable modems seem to be very stable with the combination of XR500.

 

On a side note I am seeing a daily t3 time out in my arris just one time though and usually like middle of night or day when no one is online.

 

I heard that CM600 was stable as well, personally I liked Arris/Motorola brand they made great modems, but now they switched to poor customer support.

 

From other prospective, my decision was based on requirements: docsis 3.1, I would like to have 32 channels, I want stable chipset.  In this configuration only one modem works for me CM1000. 

 

You would have to check levels of power. Downstream and upstream, verify numbers against specification rule of thump good downstream ranges are: -7/8 +7/8dbmv, upstream would depend on signal to nose ratio and if you have 40db signal/noise It could be enough to have 38/40dbmv, then verify correctible errors and uncorrectable errors.

Correctible error is something CMTS and your modem would fix on protocol level, uncorrectable errors are mostly retransmission due to corruption sign of the line issues or there is a nose. Comcast techs might be able to help with uncorrectable errors, might take 2-3 attempts, it literally should be 0 in good line, or you might have some uncorrectable errors during maintenance on the line.

I just looked at 3 day modem run and one channel has 18 correctable errors, others in single digits or 0, that is very insignificant.

 

Interesting that XR500 now consumes less RAM, 10-20%less, when it was connected to Arris it was consuming more then 55%(kernel and NedumaOS) now it seams to be noticeably bellow 47, and I tried to see stress test impact- there was no impact on RAM or CPU.

[Bmann]

I heard that CM600 was stable as well, personally I liked Arris/Motorola brand they made great modems, but now they switched to poor customer support.

 

From other prospective, my decision was based on requirements: docsis 3.1, I would like to have 32 channels, I want stable chipset.  In this configuration only one modem works for me CM1000. 

 

You would have to check levels of power. Downstream and upstream, verify numbers against specification rule of thump good downstream ranges are: -7/8 +7/8dbmv, upstream would depend on signal to nose ratio and if you have 40db signal/noise It could be enough to have 38/40dbmv, then verify correctible errors and uncorrectable errors.

Correctible error is something CMTS and your modem would fix on protocol level, uncorrectable errors are mostly retransmission due to corruption sign of the line issues or there is a nose. Comcast techs might be able to help with uncorrectable errors, might take 2-3 attempts, it literally should be 0 in good line, or you might have some uncorrectable errors during maintenance on the line.

I just looked at 3 day modem run and one channel has 18 correctable errors, others in single digits or 0, that is very insignificant.

 

Interesting that XR500 now consumes less RAM, 10-20%less, when it was connected to Arris it was consuming more then 55%(kernel and NedumaOS) now it seams to be noticeably bellow 47, and I tried to see stress test impact- there was no impact on RAM or CPU.

 I just hooked up the CM600 (Broadcom chip) and have owned a CM 1000 last year and in my case it did not improve my set-up that much not that at that time I had any issues.  A year ago my issues were outside to the tap. So I worked with the Comcast line engineer and watched him replace it at the same time had a comcast tech inside making sure signal levels were ok etc.. A week prior I had a new line from tap to house run. Also the tech installed a drop amplifier which is really to make sure tv and internet work properly.  I dont watch tv anymore and usually bypass it and run direct to modem which is ideal anyways.  right now I do have the amplifier enabled and will plan to disconnect it if I any further issues to rule it out of the equation.

 

While hooking up the CM600 upon booting up the X500 everything came back online but I did disconnect once when I was messing around with setting the ISP to static but that to be expected   So now I will monitor this over the weekend to see if anything changes.  I play Xbox and Destiny2 which is crappy P2P and since hooking the X500 I cannot do strict mode in geo filter Destiny kincks me.  So for me right now my main thing with this router is the bufferbloat function to take advantage of.  My previous Asus 3100 had the custom script author Fresh from dls fourms and fq codel.  Is was working but I was hoping this would be a step better time will tell.

 

Thanks for all the great info.

[Netduma Admin]

For Destiny 2 you just need to disable strict mode and it should work great. Don’t stop using the Geo-Filter as it will force local players

[Bmann]

Ok router just went offline no connections active hardwired or wifi. Its not the new Netgear CM600 I plugged it in direct to PC and internet came right up. So my uptime has been almost 4 hours. Factory reset time.

[Fathomless]

Do you see Comcast public IP on the router? the reason why i am asking is that Comcast told me that i can only have one IP. If computer got public IP it means that router cannot. I tired to have two IPs, and i requested two for troubleshooting , but they could not accommodate that.....if you can open the case with NetGear, just ask tier 1 to forward you to tier 2, and document issue you have, they will need to debug it.

 

I do not know if Frazier got all debugging info he needs, i promise that i will follow up with my case tomorrow morning(pacific time), but i do not have repro yet.

[Netduma Admin]

Bmann - what setting do you have Anti-Bufferbloat set to? If it's 'High Priority Traffic Detected', could you try it on 'Always/Never' to test if that makes a difference. Thanks.

 

Fathomless - thanks for your help with this. The more info you can collect from the logs etc the easier it will be for NETGEAR to look into

[Bmann]

Do you see Comcast public IP on the router? the reason why i am asking is that Comcast told me that i can only have one IP. If computer got public IP it means that router cannot. I tired to have two IPs, and i requested two for troubleshooting , but they could not accommodate that.....if you can open the case with NetGear, just ask tier 1 to forward you to tier 2, and document issue you have, they will need to debug it.

 

I do not know if Frazier got all debugging info he needs, i promise that i will follow up with my case tomorrow morning(pacific time), but i do not have repro yet.

Yes it is the public ip from Comcast and the pc ip is from the router. Right now after the factory reset it seems to be fine but time will tell. Funny you mention this I have been thinking something was weird with my initial setup. I noticed in the logs that it was trying to get its dhcp from comcast and not from the router initially. This may have been normal but just seemed weird to me since all the previous routers did not seem to behave this way. So in my case i think it has something to do with the way it does its network discovery on initial setup.

 

Forgot to mention i got a new ip since i changed out my cable modem to a netgear cm600. So a fresh start

[Bmann]

Bmann - what setting do you have Anti-Bufferbloat set to? If it's 'High Priority Traffic Detected', could you try it on 'Always/Never' to test if that makes a difference. Thanks.

 

Fathomless - thanks for your help with this. The more info you can collect from the logs etc the easier it will be for NETGEAR to look into

Set to always.Right now no issues after the factory reset but need to give it a day or two to see how it goes.

[Bmann]

Well for awareness just got kicked out completely from Xbox while playing destiny 2 competitive match network is still up and solid.  Cable modem not a single log in the logs so it rock solid but I am noticing thins in the logs on the router preceding this loss of network to Xbox. Nothing looks bad to me. While I was typing this the xbox reconnected itself which I have never experienced before strange maybe I got Ddosed since I was top player. lol Ran dslreports from edge browser A+ A  A+ so no issues   So when I have experienced disconnects I have seen lease changes in the log proceeding the disconnect this is the only common denominator that stands out right now.

 

[admin login] from source 192.168.1.2, Saturday, April 28, 2018 12:28:56

[DoS Attack: SYN/ACK Scan] from source: 91.228.153.17, port 80, Saturday, April 28, 2018 12:28:45

[admin login] from source 192.168.1.2, Saturday, April 28, 2018 12:19:41

[DumaOS] DHCP lease change., Saturday, April 28, 2018 12:19:01

[DumaOS] DHCP new event., Saturday, April 28, 2018 12:19:01

[DHCP IP: 192.168.1.100] to MAC address  Saturday, April 28, 2018 12:19:01

[DumaOS] DHCP lease change., Saturday, April 28, 2018 12:18:45

[DumaOS] DHCP new event., Saturday, April 28, 2018 12:18:45

[DHCP IP: 192.168.1.100] to MAC address , Saturday, April 28, 2018 12:18:45  (my Xbox) I removed the MAC from the log fyi.

[Bmann]

Xbox goes offline again but rest of network is fine and same logged messages .  I am leaning to a DHCP problem I googled DHCP issues and Netgear and I so a bunch of Netgear related issues due to DHCP and other models maybe they still have some bad code.

I checked my lease time from a PC

 

Lease Obtained. . . . . . . . . . : Saturday, April 28, 2018 8:42:19 PM

Lease Expires . . . . . . . . . . : Sunday, April 29, 2018 9:00:19 PM

 

I set my Xbox up as a static address.  I just added my pc to a static address and will leave it on overnight to see if it also disconnects connection.

 

Overnight results:  PC stayed online no disconnects.  Ipad wifi went offline while I was watching youtube also had another ipad go offline because saw it was disconnected this morning.

 

So right now random disconnects both wifi and hardwired devices.  Checked cable modem no logged messages so its been rock solid still.

[Netduma Fraser]

Very interesting, good troubleshooting. Static IP is exactly what you should do. I wouldn't recommend setting them for every device as it could take awhile. I would however recommend extending the time of the DHCP lease and see if you experience the issue less frequently. 

Since you set static for the Xbox has that stopped dropping?

[Bmann]

Xbox has been static the entire time.  How to I extend the lease time ? I am assuming my ISP sets this.

 

I just noticed something weird to me.  So under Settings/Setup/Internet Setup  my config below

 

Get Dynamic from ISP =selected

DNS Get automatically =selected

Router MAC address = Use Default Address = Selected

 

Ok here is what is weird to me my actual MAC printed on Box and on bottom of router are different from what I am seeing in default address?  I would assume the default MAC would equal the routers actual MAC. 

 

Example router without pasting my actual address MAC is lets say is 8A3ADG112345 my router displays this 8A3ADG112346  maybe this is a Netgear thing.