Edit: Issue has now been resolved. Root cause was found to be a policy update related to the 4.27.7 firmware. The issue was not, strictly speaking, with the firmware itself and AT&T has said that they have fixed whatever it was that this firmware was affecting. They rolled my firmware back to 4.25.4 and latency went back to normal, however 4.27.7 should also work now as well.
I’ve been having this issue for about a month and half now where my latency will increase slowly over time until the connection becomes useless and the residential gateway has to be power cycled (typically once per day). I had the RG replaced (originally a BGW320-505, now a BGW320-500), with no change. AT&T sent someone out to check the fiber, splices and connectors from the RG to the main box and while some were cleaned or replaced, it did not fix the issue.
Pinging an external IP from the RG diagnostic interface shows very high latency and the root cause does not appear to be related to my internal network.
Holy, I just dealt with this. Old gateway would do the same thing.
One thing to check, don’t plug your network in at all. Disable your WiFi, then plug a device directly up to the gateway and attempt to ping your gateway via lan. I bet you’ll find that the gateway itself is becoming unstable.
Then further test, do this. While it’s unstable, either turn off the ONT (if you have one) or unplug the fiber from the SFP port on the back of the modem, then all the sudden it becomes stable again. I bet your ping responses to the gateway become normal again.
Do be careful when pulling the fiber out of the SFP port. Too much of a bend can crack it.
If this test shows the results I’m suggesting, I’ll follow this up with instructions on discovering the issue. Let me know!
Edit: Also, I don’t think this is just a simple firmware issue. While a firmware might fix this. I think the latest, 4.27.7 has a flaw exposing it, making it susceptible to a type of dns attack.
Troubleshooting with AT&T lead me to have 4 different gateways sent to me. One of them fortunately had old firmware on it (4.25.4) and it has yet to update, and everything seems to be working. Also, I can confirm the DNS attack…
Just chiming in. I’ve also been dealing with this issue since July 8th. AT&T swapped out the Gateway, but both were on 4.27.7, so it didn’t make a difference.
It seems to be somewhat better now, compared to how it was initially. It typically needed to be rebooted every 30min or so. Now, it lasts about 2.5 hours before becoming unusable.
Judging by the responses, it doesn’t seem like much can be done aside from fishing for a gateway that hasn’t been patched, or just waiting for an update. I’ve since set up a router rebooter (wifi connected outlet that sits between the AT&T gateway and the power) which reboots once it hits the unusable state. Seems to be an okay bandaid until things (hopefully) get patched.
so I have this exact same problem. How I have my gateway setup right now is direct to my PC and direct to my WIFI. When this starts to go crazy if I unplug my PC from the gateway and test internet on wireless devices it works absolutely fine.
also a TX resident and am having the same issue. however in my experience it seems the latency is just generally inconsistent (sometimes as fast as 6ms to as slow as ~1000+ms) rather than a gradual rampup, but i’m going to keep pingplotter running for a few days to confirm. also my RG automatically reboots itself in this case. does yours do this as well or do you have to manually cycle it?
either way, this has been very frustrating and i am hoping the new firmware comes out soon.
Guys, I might have found the cause. There is a NAT table page under the diagnostics tab. When it died yesterday, the number of total sessions in use was around 4k. I identified some of the applications that were causing this and stopped them. Today, I don’t see the latency increase anymore, and the number of sessions in the NAT table is under 500.
So it’s a known problem but when I contacted AT&T live support recently, they make a malicious change to my modem that puts my network security at risk and assured me it’s fixed.
Today their support bot is configured to NOT handle any Internet performance problems. It goes through the restart guide then will only try to sell you AT&T Wireless afterwards. As if I want broken cell service too.
Two technicians have now tried to troubleshoot the issue and have cleaned various connectors and replaced some splices. I’m sure it needed to be done, but did not fix the main issue.
However, today I noticed that when I am connected with a VPN, the latency goes back to normal (well, <100ms anyway) …so what is AT&T doing???
Screenshot showing good latency when connected to VPN, red area is when tech was troubleshooting: https://imgur.com/a/ypHnNSF
(Not sure why I can’t upload the image…) Edit: Fixed!
update 2: spoke too soon it looks like. we got up to 4 days but it rebooted twice just now
update on my end: it seems disabling ipv6 on the gateway has substantially improved the issue.
latency is a lot more consistent now and our gateway uptime is at 3d5h at the time of writing, which is the longest it has stayed up by far. usually it crashes around the 1 day mark
i would recommend trying it out if you don’t have anything that explicitly requires ipv6
it seems like att has bad ipv6 considering there is a thread about disabling ipv6 for att users that has 10k upvotes lol
Joining the conversation here. Experiencing the same for at least last couple of weeks. We have U-Verse TV as well and that seems to hiccup and skip at the same time as the internet drops out. I’ve run some side-by-side pings to the RG and to 8.8.8.8 and when one spikes in latency, so does the other.
BGW320-500 running firmware 4.27.7 and in TX.
I’ve just done a reboot and we’ll see how long that lasts. Did it once before a couple of weeks ago, but didn’t seem to have much effect. I’ll look to disable IPv6 next to see if that helps. If not, I too have some bittorrent traffic consuming some of the NAT table entries and I’ll try turning that off.
Thanks all for the suggestions and troubleshooting so far!
I have BGW320-500 running firmware 4.27.7 have been having this issues for almost two months contacted support 9 times, had tech visit about 5 times, and had the gateway swapped about 4 times but still having the exact latency build-up issue.
In my case, I have about 1000 NAT sessions and the latency builds up about every 24 hours to a point where the service becomes unusable.
I noticed a couple of things
The problem seems to be related to the NAT table
When the latency increases the gateway website also becomes very slow. I suspect it’s related to the NAT table consuming too much memory or CPU which also increases the latency as a side effect.
Since I use pass-thru mode reducing both the max NAT state table size and timeout of connections helps but the problem still happens which leads me thinking the ATT gateway might not be release the NAT state correctly over time in the firmware 4.27.7
I had this same problem for pretty much all of June and July and resolved it by restarting the BGW320-505 each time my primary gateway (a Firewalla) reported packet loss or failed ping tests. Also based in Texas, FWIW. Fortunately, I have a secondary WAN connection to fail over to.
Sometime in the past week or two, I noticed my primary gateway stopped reporting failures from the AT&T WAN connection. Checked a few mins before posting this and it looks like I’m now on firmware version 6.28.7. If anyone’s still having this issue, maybe reach out to tech support asking if this update is available for your gateway.
Sad news - as of 9-4-2024 we’re still experiencing this exact same issue in the Bay Area. Support is useless; no access to architecture engineering team or any dreams like that. We’re stuck, multiple sites, same ABF service, same RGs with 4.27.7 firmware…
I’ve spent weeks on this! This is EXACTLY the issue that started a while ago. Software 4.27.7. I wonder if it auto-updated to this?
ATT tech support is worthless, especially when a reboot ‘fixes’ it.
Same as everyone else - pass-through mode with a PFSense firewall.
