Enabling teams to warp factor 10

One of my strengths is enabling teams. Seeing spots where some documentation here or a tool there will help move the team forward and increase their velocity. This is something that is hard to hire for, or call out that you do well on a resume, but is really important for building effective and efficient teams.

Being successful at it requires the ability to step back and look beyond the immediate technical problems to see where people are struggling or wasting time. It also requires good communication skills to ask the right questions and engage the right players to solve them. And, quite frankly, sometimes it involves working outside of an established process and asking for forgiveness instead of permission.

For me, this is just an extension of performance. Instead of optimizing a piece of software or hardware it’s optimizing teams.

What does enablement look like?

What does it actually mean to “enable teams”? How does this business-speak translate into the real world? How can we get a team to warp factor 10?

What follows are some examples of how I’ve enabled teams by creating tools, writing documentation, and leading teams. Keep in mind that through all of this I was officially a consulting-level validation engineer in the performance team.

Enabling your team

The EMC Isilon performance team is one of the most geographically-distributed teams we have with developers in multiple US timezones (including Hawaii) and countries (including Denmark, Italy, and France). Our master SVN server, however, is located in Seattle. Our remote developers were having a heck of a time doing SVN branch merges on their local workstations across the WAN — the WAN latency being the bottleneck. Merges could take hours or even timeout and fail altogether.

To fix this I took an older hardware client with a 10GigE network interface, threw in some spare SSDs (we are a storage company after all), and put it up in the Seattle lab for remote devs to use for their merges. This significantly decreased the time it took to do branch merges, not to mention reducing the headache of doing it. This was so successful that I put similar clients in our remote labs for other developer-related activities that take place in those locations. These remote shared workstations have since become popular with other teams inside the engineering organization as well.

Take away: Enabling teams can be a very low-level activity, it doesn’t always have to be grand and visionary. Something as simple as playing the IT guy can significantly improve the day-to-day life of your team.

Enabling your team by enabling others

The EMC Isilon OneFS operating system is built on top of FreeBSD. OneFS 8.0 upgrades the underlying FreeBSD version from 7 to 10. The development process used to do the upgrade prevented developers from testing their incremental code changes in the standard Isilon regression framework until almost code-complete and provided no way to leverage any existing test automation. A prior effort had a similar testability problem and was ultimately unsuccessful. Moving to FreeBSD 10 was important to the organization as a whole, but particularly so for the performance team as we kept hitting limitations of FreeBSD 7 when trying to address performance problems.

Seeing the success of the FreeBSD 10 effort as vital to unblocking my team’s goals I volunteered to help. I created a lightweight isolated virtual framework built on top of VMware Workstation that allowed developers to run a small set of regression tests on their branch to validate it before they merged it. Validation engineers were able to work alongside developers to add tests to this framework as code was developed. This approach enabled the validation team to test the code as it was dropped in instead of waiting until the end when the new product was capable of being run inside the standard Isilon framework.

Take away: Enabling your team may mean taking a step back, seeing how it depends on others, and assisting them directly.

Enabling your team by enabling the larger organization

In mid-2014 EMC Isilon started pushing towards a more DevOps-centric model and I jumped at the opportunity to help the organization build better products faster. For internal purposes, we have been using virtual nodes running on VMware Workstation for many, many years. These enable developers to quickly spin up nodes on their workstations, test something out, and break them down without waiting for physical hardware to become available or waiting for the hardware to reimage. The downside is that these only ran on a developer’s workstation, require beefy workstations, and limit developers to 3-node clusters.

Seeing how underutilized virtual nodes were, and yet how powerful they are for both development and test, I volunteered to lead the team to make virtual nodes more accessible. I did so because I believed I had a unique understanding of the interdependencies involved and the skills to make it successful. Management agreed and I ended up equal parts team lead, project manager, developer, and tester for the DuctTape team. Together we made virtual clusters running on centrally-hosted ESXi systems a reality. Now both developers and testers can spin up a virtual cluster of any size running almost any build at the push of a button. This capability was delivered just in time for the final testing of the FreeBSD 10 code upgrade (mentioned above) before it could be run on hardware rigs. It was later incorporated into our standard regression wheel to supplement the physical hardware rigs already in use, further increasing our test concurrency.

Take away: Enabling your team can mean taking a huge step back, looking at organization-wide limitations, evaluating your skill set against those required to address those limitations, and putting yourself out there to try.

Incremental enablement

Developing for Distributed Proofreaders has always been challenging due to the need for very specific (and antiquated) middleware and the use of CVS as source control. Developers struggled to get a working development environment set up to test their changes and then no way to effectively have multiple changes in-flight due to CVS’s poor branch support.

During my 9-month sabbatical I iteratively addressed these by:

  • moving the project to git
  • documenting recommended workflows for the project centered around user branches to have multiple changes in-flight
  • creating a development VM with all supporting middleware already installed and a base set of data populated for testing
  • conducting two git training bootcamps using the VM to get developers up to speed on how to use git in the new development model

These have already paid off and we have several new developers test-driving code changes.

I’m in the process of introducing unit tests for existing classes and establishing them as a best-practice for future classes and refactorings.

Take away: Enabling teams is an incremental process. Once you have one roadblock removed, look afresh and see what the next one is. You don’t have to solve it all at once: incremental progress is progress all the same. And don’t overlook the importance of good documentation!

Not in my job description

None of the above roles were in my “job description” as a performance validation engineer, but I saw problems that I could help fix and fixed them.

In the case of the FreeBSD merge and the DevOps work, I volunteered to take on the role which then became my job for several months. I was able to identify and articulate to management how I was well-qualified to help address the problem and that addressing the problem was how I could best serve the overall business at that time.

I don’t know of anyone whose sole job description is a “team enabler”. Instead, it is everyone’s responsibility to find those areas around them where they can help their teammates work more quickly, efficiently, and thereby increasing the velocity of the team overall.

No one works in isolation

In the modern tech world everyone works on a team of some sort — no one works in isolation. The ability to identify what is tripping up a team and taking the initiative to fix it is an important skill for an individual contributor and a hard requirement for any team lead or manager.

Cultivating the skill of team enablement increases your value to any organization. Being able to highlight it on your resume, discuss it with recruiters, and communicate it during interviews will increase your hireability.

Life: quantity vs quality

If you had a choice between increasing the quality of your life at the expense of quantity, which one would you choose? Would you rather live better for less amount of time or live less-well for longer?

It’s a hard question to answer. Unlike the witches in Discworld, we don’t know when we’re going to die which makes the trade-off hard to calculate. And how much better would life have to be such that you’d be willing to have less of it? I expect the answer varies for every person, but my guess is that optimally the answer is somewhere in the middle of both spectrums. Few people would opt to die tomorrow for one glorious day today, or suffer every day in order to prolong their life for an indeterminate amount of time.

The question isn’t just philosophical, we implicitly answer it every day based on hundreds of little decisions: do I snack on the carrots or the chocolate cake? Others, like those with hard-to-cure medical conditions, make it explicitly: do I undergo treatment to possibly extend my life while decreasing the quality or do I live the time I have left with the best quality possible?

If you had asked me this question a few years ago I would have said I preferred quantity over quality. Who doesn’t want to see mankind living among the stars or honest-to-goodness hoverboards? During my sabbatical I’ve decided that while I’m not going out in a blaze of glory tomorrow, I’m solidly on the side of quality.

Yes, I’ll have dessert after (or for!) dinner tonight, thank you. The Corolla has a better safety rating than the Miata? Screw it, I’m renting the Miata. Why am I stressing out working 60+ hours at work when I could work 40 less-stressful hours and have a life I enjoy more? Yes, I’ll rent the downstairs unit as well as the upstairs one after our dumb-ass neighbor moves out despite the expense because it will make it much more enjoyable to live here. And in the future if I ever have to make a medical decision with the quality/quantity trade-off, quality already has some points in its favor.

What about you? Where do you fall on the quality vs quantity spectrum?