Last month I attended a 2-day conference called DevOpsDays London and also spoke to delegates on the dxw stand.
At the conference, Kate Whalen spoke about how to filter unimportant notifications in her talk titled, ‘How to Leverage AWS Features to Secure and Centrally Monitor Your Accounts’. She suggested creating your own custom dashboard specific to the team’s needs. This allows your team to focus on the highest priority, so the person on-call isn’t being disturbed by low-priority alerts. This would be useful to implement in our own operations team as we’re constantly digging for relevant information.
Kate also suggested having separate communication channels. One for discussing current on-call incidents and another for system alerts.
Again, I believe this would be useful to implement at at dxx as it could speed up our response times resolving incidents as well as encourage clearer communication with the team.
In his talk, “Don’t Panic!”, Euan Finlay spoke about developers who are about to join the on-call team. However, his advice could apply to anyone joining on-call for the first time. In order to become better at dealing with incidents more efficiently, he suggests that you must familiarise yourself with your alerts system and have access to the latest documentation. He echoed Kate Whalen’s suggestion of sorting out alerts which are not actionable.
Before diving straight into your first on-call incident, Euan encourages you to take a deep breath and ask yourself the following:
What are the major impacts of the incident?
- What have you tried already?
- Check the basics. (e.g. logs, outages, documentation)
- If you’re still stuck, call for support.
- Communicate with your teams and customers, depending on impact level.
- Most importantly, do an incident review to learn from and not blame.
- Create follow-up actions or a list of recommendations to prevent the incident from occurring again.
If you’re in charge of your team, it’s better to give praise to the person on-call for trying to resolve the issue rather than reprimanding them for their efforts as it may hinder their confidence for handling future incidents.
The afternoon kicked off with open spaces for unconference discussions. I attended a session called ‘Home learning setups for people who hate books’, which appeared to have attracted those with an alternative learning style. I found this relevant as my learning style is a mixture of visual and tactile. Therefore, I prefer to keep reading to a minimum as a dyslexic.
I thought I was the only person within the group who wanted to work on projects at home while attempting to replicate my office environment. However, the common obstacles seemed to be not knowing what to work on as a project in order to develop the skills set for work.
I would recommend attending DevOpsDays London even if you aren’t a developer or operations person. There was a good balance between technical and non-technical talks amongst like-minded people. Did I forget to mention about the free swag and the superb catering?!
Also published on Medium.