Re: Best TAC Services from Equipment Vendors
when a TAC engineer wanted to bounce our Voice VLAN SVI in the middle of an *airport* production day. I about turned over my desk trying to wrest the remote control session back from him before he hit enter on the shut. Since then, I have had to go through a not insignificant evaluation period of TAC engineers before I let them take control of a remote session, and it is now simply pure instinct to log SSH sessions.
Picture it, Cisco TAC, on a troubleshooting call, runs 'no ip routing' and hits enter before our engineer could scream "NO" at 11:30AM on a core L3 on a college campus. RCA afterwards: 1. "Always log all terminals (we prefer SecureCRT) from Windows bastion host to OneDrive or Google Drive" 2. New CiscoTAC TACACS login created allowing Enable but Denying "configure" as a command. When you troubleshoot, you log in as CiscoTAC. The CiscoTAC tacacs profile description in Clearpass makes it clear why it's there. I left the curse words out. -J John C. Lyden Associate Director, Network Operations Division of Information Resources & Technology Rowan University
We were one of the earlier adopters of Cisco ACI. Any issues with ACI were automatically escalated to an engineer that could fix almost anything. Now ACI tickets seem to go though a generic queue and the tech doesn't even know how to spell ACI. We continue to have the same type of failure with Cisco DNA Center and TAC has to engage the business unit nearly every time to fix it. Sometimes it is like presenting a case to the supreme court to get the business unit to engage. They collect so much data that I wonder if it would be easier to ship the servers to them. Curtis Parish 615.494.8861 Senior Network Engineer ----IF CLASSIFICATION START---- ----IF CLASSIFICATION END----
In light of this thread's contents, I have to give a shout out to Nokia TAC. Maybe because we buy a lotta stuff and have a lotta maint contracts, but they don't do things like what has been mentioned. Of course, I see some stuff from Level 1 folks where I think 'whaaat???' but they haven't done anything like what I have heard on this thread in the past 5 years I have been using them. Even for 'informational' tickets they respond quickly. scott
Hello, Anyway AWS Direct Connect Partner for US region able to link up with me please? Thanks! Best Regards, Edy
The only issue,that I have experienced, with Nokia TAC is throwing stuff from team to team and before you get things done, each team has blamed the other team. On Thu, 14 Mar 2024 at 01:43, scott via NANOG <nanog@nanog.org> wrote:
In light of this thread's contents, I have to give a shout out to Nokia TAC. Maybe because we buy a lotta stuff and have a lotta maint contracts, but they don't do things like what has been mentioned. Of course, I see some stuff from Level 1 folks where I think 'whaaat???' but they haven't done anything like what I have heard on this thread in the past 5 years I have been using them. Even for 'informational' tickets they respond quickly.
scott
Ultimately, I think every vendor will have customers with good experiences, and customers with not-so-good experiences. Without sufficient data, it is hard to say which vendor, in general, does a good job for their customer base re: TAC. What I will say, albeit anecdotally also, is that TAC experience with the vendor will often be good if you have access to an SE on a regular basis (i.e., one or two in your country of abode). It will even be better if that SE is naturally diligent. Which means that oftentimes, it will come down to one person, to give you a sense of how well a large organization like an OEM treats your business. And if that one person were to leave and move on, the gorilla that is the OEM could come crashing down on you in ways you didn't think was possible. So yes, while it is possible to institutionalize processes and such, like with everything else in life, your experience is very likely to come down to one or two people that care deeply enough to (want to) make a difference. For me, if I have ever have to have a relationship with a vendor, it is a priority that I establish a close relationship with someone on their sales and engineering team, as that will determine how much support I get if I ever run into trouble. And OEM's that were once great at this can quickly become the worst you've ever seen, if those two people were no longer there or stopped caring. In other words, there are no forever-guarantees, and it comes and goes in cycles. Mark.
I've been reading the "${VENDOR}'s support has really gotten worse lately" threads for pretty much every vendor for the past twenty years. That's not to say they've all been wrong. But it reminds me of those quotes you'll see about how "these kids today are awful and society is going to pot" and then the big reveal is that it was written in the 1950s, or 1920s, or just before the peak of Rome, or something like that. The general tendency for people to view the past as the good ol' days. My most memorable Cisco TAC disaster story. Taking away "configure" from TAC wouldn't have saved us. The guy simply reloaded the switch without asking. The core switch for a building with hundreds of end users. In the middle of the day. The building with most of the C-level execs. Our management was pi-i-i-issed. That got escalated pretty high, pretty quickly. And quick policy change that we did not give TAC keyboard control. This was about ten years ago. On Tue, Mar 12, 2024 at 7:47 AM Lyden, John C <lyden@rowan.edu> wrote:
when a TAC engineer wanted to bounce our Voice VLAN SVI in the middle of an *airport* production day. I about turned over my desk trying to wrest the remote control session back from him before he hit enter on the shut. Since then, I have had to go through a not insignificant evaluation period of TAC engineers before I let them take control of a remote session, and it is now simply pure instinct to log SSH sessions.
Picture it, Cisco TAC, on a troubleshooting call, runs 'no ip routing' and hits enter before our engineer could scream "NO" at 11:30AM on a core L3 on a college campus.
RCA afterwards:
1. "Always log all terminals (we prefer SecureCRT) from Windows bastion host to OneDrive or Google Drive" 2. New CiscoTAC TACACS login created allowing Enable but Denying "configure" as a command. When you troubleshoot, you log in as CiscoTAC.
The CiscoTAC tacacs profile description in Clearpass makes it clear why it's there. I left the curse words out.
-J
John C. Lyden Associate Director, Network Operations Division of Information Resources & Technology Rowan University
participants (7)
-
Crist Clark -
Curtis L. Parish -
Lyden, John C -
Mark Tinka -
Pascal Masha -
Pui Ee Luun Edylie -
scott