Debugging can lead engineers into the dark corners of a codebase, giving them time to gain a deep understanding of some section of code, but this vital information — the bug and the fix — often stays in the brain of the engineer who worked on the issue.
What happens if a similar bug appears in the future? Do you task another engineer with shipping a fix or is it best to pull the original engineer away from their existing work? In this blog, we’ll dive into best practices for sharing knowledge within the engineering organization for startups.
Bug dives: Unifying Engineers and Developer Success Teams
At Nylas, we use collaborative bug dives to help educate our team about bugs, debugging strategies and the associated parts of the codebase that caused them. During a bug dive, the engineer who solved the bug explains their approach and ultimate solution to the engineering and developer success (Nylas’s version of customer success) teams. This is beneficial not just for other engineers, but even more so for the technical customer success team to get more context on the best way to communicate about issues other users might experience and how they might diagnose similar technical issues in the future.
How to Structure a Bug Dive Meeting
For bug dives at Nylas, the engineering and customer success teams gather together every two weeks to share learnings. We use Zendesk to track customer issues, so the engineer who is spearheading the bug-dive will share the Zendesk ticket with the team to add context, or, if the bug was reported internally, they’ll show the relevant logs, error messages, or stacktraces.
Then, we’ll discuss:
- The engineer’s initial hypothesis about the bug
- How they were able to replicate the issue
- If the original hypothesis was disproven, how was it disproven
- What debugging strategies were used
- What monitoring tools were consulted
Finally, they present the ultimate fix and walk through the code that solved it.
The engineering and customer success teams leave with a better understanding of that part of the codebase and gain exposure to new debugging techniques. All bug dive presentations are archived in Dropbox, where they are easily discoverable for anyone to reference in the future.
Tips for Making Bug Dives Successful
- Schedule bug dives at a regular cadence
- To keep up attendance, choose a specific time every two weeks and add it to the team calendars. It’s a good idea to select a time before or after another event so that engineers won’t have to pull themselves away from existing work. At Nylas, we have bug dives right after lunch and it’s a great way to digest and recharge before heading back to work.
- Come up with a creative name and summary about why the bug is interesting.
- Share this when messaging about the bug dive to build excitement for the topic. You will likely get lower turnout for an event titled “Bug dive about db queries” than for events with captivating bug specific titles like “What happens when you query the database for an object named X?”. For the first title, engineers will likely think “I already know about db queries, I’m not going to waste my time” but the second title will pique their interest.
- Anyone can lead a bug dive!
- You can think of bug dives as mini technical talks. Giving technical talks and communicating challenging ideas are important skills for all engineers to have. A bug dive is a great, low stakes way to practice and help engineers develop their public speaking skills for larger audiences. Don’t restrict bug dives to senior engineers, it can be nice role reversal for junior engineers to teach more senior engineers a thing or two.
How Bug Dives are Different than Traditional Knowledge Transfer Sessions
Bug dives go really deep on a specific topic. Because it’s incredibly hard to convey a deep level of understanding on a large system in 30 minutes, bug dives are more focused and cover a narrow slice of the system in detail. Instead of high level discussions, bug dives dive into the code for a particular system.
Learning about bugs in the code exposes the team to neglected corners of the codebase. Only challenging or unexpected bugs are presented at bug dives. The content covered in a bug dive is dictated by where the bug occurred, which means that the content of bug dives is often lesser known systems. Bug dives help the team get acquainted with the code in more peripheral systems that often deserve more attention.
The engineer presenting the bug doesn’t have to do more work. Apart from organizing information, since the engineer just solved this bug, they are already familiar with their hypothesis and the holes in it. They will probably have some notes, have bookmarked the relevant monitoring pages, and can easily pull up the code changes.
This process celebrates the hard work that the engineers did to debug the issue. Bug crushing can be a laborious and thankless process — you’re fixing something that shouldn’t be broken in the first place. But bug dives provide an arena to celebrate the engineer’s grit. The engineer can bring the audience through the woes of subpar code and bad monitoring and end with the satisfying conclusion of the fix.
Bug dives are self-sustaining. Because bug dives are both celebratory of the presenting engineer and beneficial for the audience, they are self-sustaining. There will always be more bugs. The engineers who figures out a bug will always want to share their feat and be celebrated. And the rest of the engineering organization will always be curious about the cause of the unruly bug and what debugging strategies were used to identify it.
Apart from small questions, it’s not often that you get to learn from engineers on other teams. Bug dives unify the engineers by providing an arena for cross-team knowledge transfer. This unity can lead to more productive decisions about the design and execution of our entire product.
At Nylas, our engineers like to share their perspective on a variety of industry-related topics. We’d love to hear about your bug dive stories and what’s been successful for you. For more about Nylas, and to join the discussion on twitter, click here.