SITE NOTE: We apologize if you’ve had, or are having, trouble

Though it’s entirely, maddeningly out of our direct control, we want to apologize for trouble that some people have had, intermittently, getting to WSB the past few days – and might be having today. Once in a while, WSB, like any website, might experience the classic problem of “the server’s down and nobody can get in.” The server manager fixes it, and everything is OK again.

This is something different.

Apparently there’s some kind of intermittent routing problem between Comcast and our longtime server-management company, WiredTree (now owned by LiquidWeb), and so some – but not all – Comcast users have had trouble reaching our server, some – but not all – of the time. It’s affecting some other websites too, we’ve learned. But as far as we have heard, it’s only Comcast – so if you have some other way of seeing us (CenturyLink, phone, etc.), that should be OK. Again, we apologize, and are working to figure out our options if this isn’t fully resolved very soon.

Mark April 27, 2017 (2:13 pm)

Interesting, looks like only AS7922’s (Comcast) routes are flapping inside Wiredtree’s network. I caught a traceroute while it was dead (WiredTree’s intermediate hops respond fine, but responses from WSB’s server itself are dropped periodically):

Start: Thu Apr 27 14:02:30 2017
HOST: valen                                            Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 50-248-192-206-static.hfc.comcastbusiness.net     0.0%    10    0.3   0.3   0.3   0.4   0.0
  2.|-- 96.120.102.13                                     0.0%    10    8.8   7.9   6.0   8.8   0.6
  3.|-- te-0-7-0-6-sur03.burien.wa.seattle.comcast.net    0.0%    10    8.7   8.1   6.8   9.0   0.6
  4.|-- be-21-ar01.seattle.wa.seattle.comcast.net         0.0%    10    8.6   8.7   7.7  11.3   0.9
  5.|-- be-33650-cr01.seattle.wa.ibone.comcast.net        0.0%    10   10.5  10.4   9.2  13.1   1.1
  6.|-- be-10817-cr02.denver.co.ibone.comcast.net         0.0%    10   35.6  35.7  34.3  36.9   0.6
  7.|-- be-10517-cr02.350ecermak.il.ibone.comcast.net     0.0%    10   58.5  58.4  57.1  59.7   0.7
  8.|-- hu-0-11-0-0-pe03.350ecermak.il.ibone.comcast.net  0.0%    10   57.4  56.7  55.8  57.4   0.0
  9.|-- 50.242.150.130                                    0.0%    10   57.3  61.0  56.1  79.8   7.8
 10.|-- cr1.chi1.wiredtree.com                            0.0%    10   56.3  58.7  56.3  62.5   2.1
 11.|-- vl101.dsr1.chi2.wiredtree.com                     0.0%    10   67.6  58.9  53.0  67.6   4.1
 12.|-- ???                                              100.0    10    0.0   0.0   0.0   0.0   0.0
 13.|-- ???                                              100.0    10    0.0   0.0   0.0   0.0   0.0
 14.|-- ???                                              100.0    10    0.0   0.0   0.0   0.0   0.0
 15.|-- prosperity2.westseattleblog.com                  90.0%    10   60.2  60.2  60.2  60.2   0.0

Around the same time, saw no issue from a VM of mine in LA:

Start: Thu Apr 27 14:03:26 2017
HOST: trident                                         Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- 174.136.96.1                                     0.0%    10    3.7   3.0   1.0  15.5   4.4
  2.|-- 38.142.33.145                                    0.0%    10    1.3   1.5   1.1   2.9   0.5
  3.|-- be2964.ccr21.lax01.atlas.cogentco.com            0.0%    10    1.9   1.7   1.1   3.2   0.6
  4.|-- be2931.ccr21.phx02.atlas.cogentco.com            0.0%    10   12.6  12.8  12.6  13.1   0.0
  5.|-- be2929.ccr21.elp01.atlas.cogentco.com            0.0%    10   21.1  20.7  20.5  21.2   0.0
  6.|-- be3047.ccr22.den01.atlas.cogentco.com            0.0%    10   33.8  33.6  33.4  33.8   0.0
  7.|-- be3036.ccr22.mci01.atlas.cogentco.com            0.0%    10   45.4  45.3  45.0  45.5   0.0
  8.|-- be2832.ccr42.ord01.atlas.cogentco.com            0.0%    10   57.2  57.2  56.9  57.4   0.0
  9.|-- be2766.ccr41.ord03.atlas.cogentco.com            0.0%    10   57.5  57.5  56.9  57.8   0.0
 10.|-- be2409.rcr11.b002281-5.ord03.atlas.cogentco.com  0.0%    10   57.8  57.7  57.5  57.9   0.0
 11.|-- 38.88.42.54                                      0.0%    10   70.3  59.5  57.1  70.3   4.3
 12.|-- lw-eqx-border2-te1-2.rtr.liquidweb.com           0.0%    10   57.3  80.5  57.0 262.9  64.7
 13.|-- 209.59.157.253                                   0.0%    10   64.7  59.9  57.6  67.8   3.7
 14.|-- vl101.dsr1.chi2.wiredtree.com                    0.0%    10   57.4  60.8  57.4  67.2   3.7
 15.|-- prosperity2.westseattleblog.com                  0.0%    10   57.9  57.9  57.6  58.2   0.0

I’d guess the problem is on WiredTree’s side, but it’s hard to tell without a traceroute in the opposite direction. Hopefully they’ll get it sorted soon.

– Mark

WSB April 27, 2017 (2:14 pm)

Thanks for that – the most recent exchange we had with WT/LW asked us for a traceroute but we personally have not experienced the problem (we are wondering if it’s just people on particular Comcast nodes around here and ours in Upper Fauntleroy is somehow not involved) so we can’t do one ourselves … Also interesting that Cogent is mentioned above. WT/LW now tells us that some Comcast and some Cogent users are involved and I hadn’t even heard of Cogent before, didn’t get time to scour their site to see where they serve – TR

Wendy April 27, 2017 (5:45 pm)

I have Comcast, and live on Alki and have had problems for several days.

Me April 27, 2017 (8:46 pm)

Near Schmitz Park Elem., using Comcast, have had problems for days with this website and another website hosted at the same place, I guess, looks like same NS servers.

No problems if we switch to Verizon.

I reported this problem to Comcast today. They opened a ticket but acted like hadn’t heard about it

Lamont April 27, 2017 (10:12 pm)

From the traceroute above it doesn’t look like its a Comcast problem. At least not any kind of Comcast problem that someone at the helpdesk could help with. Unless you can find a CCIE network engineer at Comcast that speaks BGP they are unlikely to be able to help. Since it looks like the packages are getting to wiredtree and getting lost in their network, it appears to be wiredtree’s problem. At any rate their network engineers should be motivated to dig up the previously mentioned Comcast network engineer and sort it all out. But the person you can call up at Comcast won’t be remotely useful for this issue at all.

Me April 27, 2017 (9:59 pm)

DK why, but clearing the cache seems to have unclogged our problem. ATP it works using Comcast.

Iggy April 28, 2017 (12:49 pm)

I’m a Comcast user in Morgan Junction. Today has been the worst day for WSB connectivity. Fortunately, although this past time it took about a dozen tries over about five minutes, I eventually get on. Not having trouble with any other internet sites–just WSB. This is not a complaint, just a FYI, since you’ve run this article.

WSB April 28, 2017 (1:25 pm)

Thanks for the update. Without hearing directly from people, we’re not sure if this is still a problem or not – earlier this week there were some hints in the stats but yesterday looked better and I haven’t had a chance to look yet today … TR

Sorry, comment time is over.