Why the internal network delays, Gmail?
Help (and hi)! I work in higher education and we've been experiencing problems with Google delaying or queuing email for delivery to our domain. Here's some truncated email headers: ** Example 1: X-Received: by 10.237.55.65 with SMTP id i59mr10986018qtb.62.1472137448952; Thu, 25 Aug 2016 08:04:08 -0700 (PDT) Received: by mail-qt0-f175.google.com with SMTP id u25so27419242qtb.1 for <@***>; Thu, 25 Aug 2016 12:05:46 -0700 (PDT) ** Example 2: X-Received: by 10.36.1.75 with SMTP id 72mr5275579itk.40.1472130531887; Thu, 25 Aug 2016 06:08:51 -0700 (PDT) Received: by mail-it0-f48.google.com with SMTP id x131so289116132ite.0 for <@***>; Thu, 25 Aug 2016 11:50:42 -0700 (PDT) In both of these examples, these emails haven't even left Google's internal network yet; I'm getting blamed for these delays, however there is no delay in receiving these emails after they leave Google's network. Are other people having this same problem? I've tested delivery to my network from many outside sources and all SMTP requests go through without delay; this issue seems be exclusive to Google-hosted and Gmail accounts and domains. -- Nate <natemetheny@gmail.com>nate@dopedesign.com
In article <CALtoqtSn9JWUuho7ffT+Ma79PCvpmDCNymnfcYBXsRbebSRDVQ@mail.gmail.com> you write:
Help (and hi)!
I work in higher education and we've been experiencing problems with Google delaying or queuing email for delivery to our domain.
This is a question for Google, not for nanog. Only they know how their network is set up and how their mail servers are managed. R's, John PS: Also keep in mind that sometimes free services are worth what you pay for them.
John, With all due respect, it's S.O.P. for Nanogen to ask the list if anyone else is experiencing a particular problem with some carrier or another. So Nate's question is totally appropriate for this list. I know I've solved several problems by airing them here and getting insight from other list members. -mel beckman
On Aug 26, 2016, at 4:13 PM, John Levine <johnl@iecc.com> wrote:
In article <CALtoqtSn9JWUuho7ffT+Ma79PCvpmDCNymnfcYBXsRbebSRDVQ@mail.gmail.com> you write:
Help (and hi)!
I work in higher education and we've been experiencing problems with Google delaying or queuing email for delivery to our domain.
This is a question for Google, not for nanog. Only they know how their network is set up and how their mail servers are managed.
R's, John
PS: Also keep in mind that sometimes free services are worth what you pay for them.
Hi Mel, There's another mailing list called 'mailop' which is probably more appropriate for email related problems, than NANOG. And in response to Nate:
I was in contact with Google and after some convincing and detailed header information, they acknowledged that they are having internal MX issues and assure me that they will deal with the issue promptly.
Initially they did not even acknowledge that there was a problem, so it took several tiers of support people to finally see the issue.
I look forward to the ongoing exchanges on the list.
Useful to know, but John is right - as cited, working through Google's support process, got somewhere. Further exchanges on NANOG are probably inappropriate. A group like 'mailop' probably has a higher care factor, however. (I would also note that email delays that are demonstratably outside of your network (as headers will show) are very easily painted as something beyond your control, and the nature of email is very much 'best effort', so anyone playing the blame game needs a reality check. Just because email exchanges 'are often' near-instantaneous, does not mean they always will be.) Mark. On 27/08/2016 8:53 a.m., Mel Beckman wrote:
John,
With all due respect, it's S.O.P. for Nanogen to ask the list if anyone else is experiencing a particular problem with some carrier or another. So Nate's question is totally appropriate for this list. I know I've solved several problems by airing them here and getting insight from other list members.
-mel beckman
*snip*
I was working within the limits of what I had available. I apologize if people on the list consider a network and systems administrator reaching out to peers for assistance with a particular problem that is clearly network related is inappropriate for a network operations group list that may or may not have Google or Google affiliated employees or contractors on it. I will use more discretion in the future. -- Sent from a phone. Please excuse the brevity of this message and any typographical errors. On Aug 26, 2016 17:18, "Mark Foster" <blakjak@blakjak.net> wrote:
Hi Mel, There's another mailing list called 'mailop' which is probably more appropriate for email related problems, than NANOG.
And in response to Nate:
I was in contact with Google and after some convincing and detailed header
information, they acknowledged that they are having internal MX issues and assure me that they will deal with the issue promptly.
Initially they did not even acknowledge that there was a problem, so it took several tiers of support people to finally see the issue.
I look forward to the ongoing exchanges on the list.
Useful to know, but John is right - as cited, working through Google's support process, got somewhere. Further exchanges on NANOG are probably inappropriate. A group like 'mailop' probably has a higher care factor, however.
(I would also note that email delays that are demonstratably outside of your network (as headers will show) are very easily painted as something beyond your control, and the nature of email is very much 'best effort', so anyone playing the blame game needs a reality check. Just because email exchanges 'are often' near-instantaneous, does not mean they always will be.)
Mark.
On 27/08/2016 8:53 a.m., Mel Beckman wrote:
John,
With all due respect, it's S.O.P. for Nanogen to ask the list if anyone else is experiencing a particular problem with some carrier or another. So Nate's question is totally appropriate for this list. I know I've solved several problems by airing them here and getting insight from other list members.
-mel beckman
*snip*
In article <CALtoqtQkfEaDXnr1+4yzyDoyuweBG_+QyaQ7Ubhxtmv0JCnTmA@mail.gmail.com> you write:
I was working within the limits of what I had available.
Here's the subscription page for mailop. It's got about as odd a mix of people as nanog, ranging from people with single user linux machines to people who run some of the largest mail systems in the world, including Gmail: https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop R's, John
John, But mailop doesn't have the same odd mix of people as nanog. For example, I'm not on mailop. :) In any event, Nate specifically asked if other nanogers were seeing similar symptoms, which is an entirely appropriate use of this list. -mel On Aug 26, 2016, at 9:26 PM, John Levine <johnl@iecc.com<mailto:johnl@iecc.com>> wrote: In article <CALtoqtQkfEaDXnr1+4yzyDoyuweBG_+QyaQ7Ubhxtmv0JCnTmA@mail.gmail.com<mailto:CALtoqtQkfEaDXnr1+4yzyDoyuweBG_+QyaQ7Ubhxtmv0JCnTmA@mail.gmail.com>> you write: I was working within the limits of what I had available. Here's the subscription page for mailop. It's got about as odd a mix of people as nanog, ranging from people with single user linux machines to people who run some of the largest mail systems in the world, including Gmail: https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop R's, John
On Sat, 27 Aug 2016 10:34:36 -0000, Mel Beckman said:
But mailop doesn't have the same odd mix of people as nanog. For example, I'm not on mailop. :)
And apparently you need to know the secret handshake to get on. After Chrome complained the SSL cert on the subscription page had expired 6 months ago, the site tells me I can't subscribe: Your subscription is not allowed because the email address you gave is insecure. Yay, team?
On Sat, Aug 27, 2016 at 11:24 AM, <Valdis.Kletnieks@vt.edu> wrote:
And apparently you need to know the secret handshake to get on.
I was able to sign-up yesterday, I even saw John's mail about your insecure error. I don't know why I didn't sign up before, my work ITIL is Messaging Manager. -- Joe Hamelin, W7COM, Tulalip, WA, +1 (360) 474-7474
On Sat, Aug 27, 2016 at 01:25:42AM -0000, John Levine wrote:
In article <CALtoqtQkfEaDXnr1+4yzyDoyuweBG_+QyaQ7Ubhxtmv0JCnTmA@mail.gmail.com> you write:
I was working within the limits of what I had available.
Here's the subscription page for mailop. It's got about as odd a mix of people as nanog, ranging from people with single user linux machines to people who run some of the largest mail systems in the world, including Gmail:
I know they're mailops, and not tlsops, but surely presenting a cert that didn't expire six months ago isn't beyond the site admin's capabilities? - Matt
On Aug 27, 2016, at 6:46 PM, Matt Palmer <mpalmer@hezmatt.org> wrote:
On Sat, Aug 27, 2016 at 01:25:42AM -0000, John Levine wrote:
In article <CALtoqtQkfEaDXnr1+4yzyDoyuweBG_+QyaQ7Ubhxtmv0JCnTmA@mail.gmail.com> you write:
I was working within the limits of what I had available.
Here's the subscription page for mailop. It's got about as odd a mix of people as nanog, ranging from people with single user linux machines to people who run some of the largest mail systems in the world, including Gmail:
I know they're mailops, and not tlsops, but surely presenting a cert that didn't expire six months ago isn't beyond the site admin's capabilities?
I tried again, ten months later. Still broken :-( Is there a replacement site I'm missing out on?
Fun fact about letsencrypt certs, they expire after a month or so. Looks like the site admin never noticed/cared to update it (since 2016), even though there's a nice little helper program to auto-update them that you can throw in a cronjob (or scheduled task, if you're into IIS) and forget about Ed Pers -----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Lyndon Nerenberg Sent: Sunday, June 11, 2017 6:27 PM To: NANOG list <nanog@nanog.org> Subject: mailops https breakage
On Aug 27, 2016, at 6:46 PM, Matt Palmer <mpalmer@hezmatt.org> wrote:
On Sat, Aug 27, 2016 at 01:25:42AM -0000, John Levine wrote:
In article <CALtoqtQkfEaDXnr1+4yzyDoyuweBG_+QyaQ7Ubhxtmv0JCnTmA@mail.gmail.com> you write:
I was working within the limits of what I had available.
Here's the subscription page for mailop. It's got about as odd a mix of people as nanog, ranging from people with single user linux machines to people who run some of the largest mail systems in the world, including Gmail:
I know they're mailops, and not tlsops, but surely presenting a cert that didn't expire six months ago isn't beyond the site admin's capabilities?
I tried again, ten months later. Still broken :-( Is there a replacement site I'm missing out on?
How else would one maintain government control over free encryption certificates? -- ˙uʍop-ǝpısdn sı ɹoʇıuoɯ ɹnoʎ 'sıɥʇ pɐǝɹ uɐɔ noʎ ɟı
-----Original Message----- From: NANOG [mailto:nanog-bounces@nanog.org] On Behalf Of Randy Bush Sent: Tuesday, 20 June, 2017 17:34 To: Edwin Pers Cc: NANOG list Subject: Re: mailops https breakage
Fun fact about letsencrypt certs, they expire after a month or so.
90 days
Both. Either. Take your pick Ed Pers From: Seth Mattinen Sent: Tuesday, June 20, 8:06 PM Subject: Re: mailops https breakage To: nanog@nanog.org On 6/20/17 16:57, Keith Medcalf wrote: > How else would one maintain government control over free encryption certificates? So Let's Encrypt is run by the Illuminati now? Or is it Freemasons? It's hard to keep track.
In article <m2o9tiutc7.wl-randy@psg.com> you write:
Fun fact about letsencrypt certs, they expire after a month or so.
90 days
Well, yes. That's why highly skilled and experienced administrators such as yourself set up the automatic renewal scripts at the same time they install the initial cert, right? I use the acme.sh shell script. It ain't hard. R's, John
Im thankful Nate posted. Gmail isnt a small system that affects only a small percentage of people worldwide, and therefore a perfect candidate for a mail- specific list that many (and many nanoggers like me) arent part of, for lack of additional bandwidth in life. However, gmail not working (similar to 8.8.8.8/.4.4 or 4.2.2.2 not working) shouldnt be on a mail-only or dns-only ops list im not part of: when 8.8.8.8 doesnt work, the complaints appear as "is there a power failure at your datacentre?" to "my website is down!" - same deal with gmail, it's that big. While I cant say exactly what that cutoff line is for mail and dns issues being postable to nanog or not, I definitely know gmail is pretty much the top of the pile (for now). Thanks Nate! /kc On Fri, Aug 26, 2016 at 05:31:59PM -0600, Nate Metheny said:
I was working within the limits of what I had available.
I apologize if people on the list consider a network and systems administrator reaching out to peers for assistance with a particular problem that is clearly network related is inappropriate for a network operations group list that may or may not have Google or Google affiliated employees or contractors on it.
I will use more discretion in the future.
-- Sent from a phone. Please excuse the brevity of this message and any typographical errors.
On Aug 26, 2016 17:18, "Mark Foster" <blakjak@blakjak.net> wrote:
Hi Mel, There's another mailing list called 'mailop' which is probably more appropriate for email related problems, than NANOG.
And in response to Nate:
I was in contact with Google and after some convincing and detailed header
information, they acknowledged that they are having internal MX issues and assure me that they will deal with the issue promptly.
Initially they did not even acknowledge that there was a problem, so it took several tiers of support people to finally see the issue.
I look forward to the ongoing exchanges on the list.
Useful to know, but John is right - as cited, working through Google's support process, got somewhere. Further exchanges on NANOG are probably inappropriate. A group like 'mailop' probably has a higher care factor, however.
(I would also note that email delays that are demonstratably outside of your network (as headers will show) are very easily painted as something beyond your control, and the nature of email is very much 'best effort', so anyone playing the blame game needs a reality check. Just because email exchanges 'are often' near-instantaneous, does not mean they always will be.)
Mark.
On 27/08/2016 8:53 a.m., Mel Beckman wrote:
John,
With all due respect, it's S.O.P. for Nanogen to ask the list if anyone else is experiencing a particular problem with some carrier or another. So Nate's question is totally appropriate for this list. I know I've solved several problems by airing them here and getting insight from other list members.
-mel beckman
*snip*
-- Ken Chase - math@sizone.org Toronto Canada
Hi,
administrator reaching out to peers for assistance with a particular problem that is clearly network related is inappropriate for a network
clearly network related? people have an interesting expectation of email - expecting instant delivery. you might check their level of expectation....the SLA etc define service availability but email delivery is pretty much 'best efforts of all parties involved in the transaction' - ideally it gets there quickly...but it could take up to 72 hours. google have several status dashboards that you can check/monitor. generally, if you have an issue with a particular service on the internet, contact them directly. dont use a 3rd party mail list - they *might* be aroudn on it but its not their official service desk contact point ;-) alan
Hi,
I was working within the limits of what I had available.
Google offer several trouble shooting tools for their service too, you might want to look at their toolbox eg https://toolbox.googleapps.com/apps/messageheader/ (part of their 'why is my email slow to deliver?' process) alan
Thanks for all the feedback related and unrelated to the problem. I'm aware of many available troubleshooting tools and considered this one of them, but I've been shown that this, albeit appropriate, forum, was not a good choice to solicit technical assistance. I consider the matter closed. -- Sent from a phone. Please excuse the brevity of this message and any typographical errors. On Aug 27, 2016 11:17, <A.L.M.Buxey@lboro.ac.uk> wrote:
Hi,
I was working within the limits of what I had available.
Google offer several trouble shooting tools for their service too, you might want to look at their toolbox eg
https://toolbox.googleapps.com/apps/messageheader/
(part of their 'why is my email slow to deliver?' process)
alan
participants (14)
-
A.L.M.Buxey＠lboro.ac.uk -
Edwin Pers -
Joe Hamelin -
John Levine -
Keith Medcalf -
Ken Chase -
Lyndon Nerenberg -
Mark Foster -
Matt Palmer -
Mel Beckman -
Nate Metheny -
Randy Bush -
Seth Mattinen -
Valdis.Kletnieks＠vt.edu