A leave of absence from EMC Isilon – the skinny

The prior post gave a high-level overview of why I’m taking a leave of absence from EMC Isilon, but I wanted to take a moment and write down some of the details of my burn-out for myself and to share with friends.

The downward trajectory of my work-life balance started when I left my cozy position in the performance team to work on BSD10 merge effort in February of last year. When I started on that team there was no test plan and the only tester was a developer who didn’t want to be a tester but was one anyway. The BSD10 merge was said to be critical to the success of the business, yet management wasn’t willing to move any testing resources, experienced or not, over to it. Instead, they pushed us to use contractors which produced crappy code and required extreme hand-holding.1

Despite this, I rallied the test resources we had, created a test plan, worked with the developers on a viable code merge/validation plan, and created test infrastructure to enable automated code testing. I felt like I had moved a mountain. 2

If I had rolled off that effort back to the performance team I probably would have been ok. Instead, in August I put my neck out on The Factory whose mission was to change how we develop our products.

The Factory was charged with architecting whole new systems but had no high-level people with those skills on it — except me (and I’m not an experienced architect). It was tasked with coding up those systems but had no developers — except me (and I’m a tester, not a developer). The code being developed needed to fit into the existing test infrastructure, yet no one with that knowledge was on the team — except me (this one I totally own as me). When I pushed back saying we needed more developers than just me, the only people we were given were from other teams that did not want them and only one of which was a developer. When I pushed again saying that wasn’t enough, we got told to use contractors — who produced poor quality code and only increased the technical debit. All the while management was saying that this effort was critical in keeping Isilon relevant. We were pushed to increase our velocity but given no experienced resources.

I pushed very hard — designing systems, writing code, testing code, herding the team — and we were able to get DuctTape up and running in time to save the BSD10 team’s ass. We enabled them to test their code using virtual nodes since they were not yet running on hardware. Everyone who uses DuctTape loves it. The nodes are fully integrated into our test system and Just Work. I felt like I had moved a mountain again.

For the longest time I was frustrated with the work of some members of the team, until I realized that they are performing at the career level they are at. You don’t expect an intern to architect your internal deployment strategy, why should I expect Consultant-level code from Senior-level people?

And here I am burnt out. Perhaps I would have been less burn out if management had ponied up resources for these teams working on “critical” projects but not staffing them as such. Unless this trend is changed and they invest in the people on the internal infrastructure teams (both more bodies as well as some higher-level people) the entire organization is going to implode.

1 Eventually they brought Ngie back which was the saving grace of the BSD10 merge, their contributions can not be overstated.

2 The BSD10 merge was late merging into HEAD, but came in a very high quality, thanks partially to the work I did early in their effort.

Published by


I'm a gay geek techie space nerd living in Seattle, WA.

One thought on “A leave of absence from EMC Isilon – the skinny”

  1. Sam stuff, different day

    I don’t know how involved you were when I was doing BSD10 merge. Among other things I was also told it was critical to success and we could have anything we wanted. I wanted two full-time test resources, one at the Principal level. Nope.

    I actually encouraged Ngie to take that other job, hoping the complete lack of any test resource would prod them into giving us the resources we needed. Apparently not.

    Honestly I’m just disappointed. I think the complete lack of follow-through on the promise of “whatever you need” speaks volumes about the org. Especially when the ask is actually very reasonable and, in almost every engineer’s opinion is actually necessary for success.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s