These days the development of software is a rapidly changing process. The introduction of cloud hosting and containerization has made big applications running in your server room a thing of the past. Where before you would maintain one or two big projects with a team of engineers dedicated to a single project. Now you have hundreds of small services split into bite-sized chunks with developers working on several projects. In the perfect world, every software project will have a full suite of unit, integration and system tests in place. Even in these rare cases, it is still nearly impossible to avoid an escalation or critical performance issue when you are running at scale in production. In my role as a technical support engineer, I have seen all manner of escalations. They can come in the form of; From a support engineers perspective, here are a few tips I have learned over the years on how to approach an escalation.
The initial customer contact in support is crucial. Letting the customer know their issue is under investigation reassures the customer and takes the pressure off the support engineer. The initial response from support can often be automated, these days using a bot for first-line support is often standard practice. A simple example of an email would be: “Dear {{ticket.requester.name}}, Thank you for reaching out to the {{team.name}}, a ticket has been created for you with the Ticket ID – {{ticket.id}}. A support engineer will get back to you as soon as possible and work with you to resolve your issue. Thank you for your patience. Sincerely, {{engineer.name}}” This simple email lets the customer know their issue is assigned to an engineer and the team is willing to help. You could enhance this email by adding elements such as: If the support case comes in via phone or chat, this is not so much of an issue. The customer is getting live responses from the support engineer.
Investigating an issue can be simple, unfortunately this is often not the case. If the product you support is complicated, escalations could be caused by a wide variety of factors. The goal of the support engineer is to find the route cause of the issue quickly, with as little time to set up a reproduction case as possible. When the customer sees an issue, getting their logs, screenshots and environment data will often show the root cause of the problem. You can get this information through email or chat, but a call is usually the quickest way to access this information. Having a template email or call script to get this information in place will ensure a new support engineer can gather all the essential data. They can use this data to consult with the other engineers if they need assistance. With this information, an engineer will be able to report the issue without needing to reproduce the issue themselves, drastically reducing the time taken to resolve the issue.
When you have all the information, you may need from the customer; It is often best to halt direct communication when you are investigating the issue. If you are on a call or chat with the customer, filtering through log files and attempting to reproduce the issue can be a demanding process. Often it is better to convert the call or chat with the customer into an email. The customer will typically understand if you tell them you will get back to them when you know more about the issue. Having to only focus on finding the issue removes any distractions. Even if finding the issue is your sole task, it may take hours or in extreme cases days to find the exact cause.
So now you have your issue, this process could have taken you five minutes or five days. At this point, the development team must be notified. How the team is notified will change based on the process within the engineering team. Often the two conventional approaches are to raise an alert in a chat room or create an engineering ticket. For a severe/blocking issue such as a service being unavailable, raising an alert in the office chat will alert the engineers, and they can restart the services as appropriate. It may be they already know there is an issue and are investigating. For an issue in the product, like missing data or a page in the UI erroring; A ticket to fix the product must be created. The priority of an issue is often not something you as a support engineer can control, but you can advise. The priority is based on several factors, such as: For example; You should consider these factors when creating a ticket and your rationale for your suggested priority should be documented on the ticket. This will give the engineers some context for when they will start work on the issue.
As soon as the issue has been reported, it becomes something you as a support engineer has no control over. Until the developers resolve the issue, all you can do is keep the customer updated on changes. It is essential to let the customer know the issue has been reported. If you have a ticket ID or public issue board providing them with the link to this means they can check the status themselves whenever they need. Promising hard deadlines on when the issue will be resolved can be risky. If an issue is more complicated than it initially appears the fix may be delayed which could potentially annoy the customer. Being vague on a fix date is often the best solution, using “as soon as possible”, “in the next release” or “we hope to fix the issue within the week” are good alternatives to utilise. Having a specific inbox or state for a ticket waiting on the development team gives you an easy If it has been a while since the ticket was created, it is often beneficial to let the customer know they have not been forgotten. When the developers have resolved the issue, you can finally notify the customer and resolve the ticket. This entire process could take less than an hour, or it could take months. However long it takes you to get the satisfying feeling of making the customer happy.
Tip 1: Reassure the customer
Tip 2: Gather all the information you need
Tip 3: Where possible process the issue into an email
Tip 4: When reporting the issue, be clear on its priority
Tip 5: Keeping the customer up to date on the issue