Establishing a New Function

Problem

Until fairly recently, Niche’s entire QA team worked on both manual and automated testing. Every QA analyst wrote and executed manual test plans, while also developing numerous automated test suites. Our QA analysts coded thousands of automated tests over a period of several years.

While I am proud of what we were able to accomplish as a small team, this “everyone does everything” approach ultimately proved unscalable. This is natural -- methods that worked for a company of 20 (that is, Niche at the time I joined the company) won’t necessarily work for a company of over 200 (which is where Niche stands now).

As the company scaled, the QA team struggled to keep up with our increased automation needs; our suites grew outdated and less reliable over time, with ever-larger gaps in total coverage. The QA team’s split focus slowed the rate at which we could evolve in both manual and automated testing; there was precious little space for individuals to gain the mastery necessary to innovate in either area. Realizing that we were falling behind, especially on the automation front, I took some time to reassess the state of the team, the scope of its responsibilities, and how we could improve the situation.

Actions taken

For years, our automation suites had been growing with a number of known deficiencies, none of which could receive adequate attention due to limited expertise and resources. After some deliberation, I decided that we needed to establish a separate function for automation specifically. In this new departmental structure, our QA analysts, previously responsible for testing in all forms, would now focus primarily on manual testing, while this new role on the team -- the Software Engineer in Test -- would be dedicated to automation.

I believed that establishing a new functional team of automation-focused software engineers would yield myriad improvements. It would allow the QA department to set more rigorous standards for automation code quality, and to address both the known and unknown deficiencies we had in our current test suites. It would also free up cognitive space for the QA Analysts to get more creative in their functional and exploratory testing. Moreover, this change would make it easier to hire for both roles, which was a must given Niche’s rapid growth.

Confident that a separate function was necessary, I presented my proposal to the CTO. I laid out the value proposition for a new function, the basics of a job description, key responsibilities, and a rough career ladder. I also did some research and, in collaboration with the CTO and our HR team, helped to define a salary band for the role. With those numbers attached, I had a full-fledged proposal of what the cost to the business was, which could be weighed against the opportunities it provided. That allowed the value to be properly assessed, and it was decided that my proposal was worthwhile.

To start, I moved one member of the QA analyst team over to be our first Software Engineer in Test. This person was someone with a CS degree who had made significant contributions to our automation suites, and who had shown the leadership potential necessary to help me further shape the new role. They were highly qualified for the job, and to date had merely lacked the opportunity to focus fully on automation due to their manual testing responsibilities. I put them in place and spent the better part of the year working with them to define the role in action and further refine its responsibilities. At this point, we began the search to hire additional Software Engineers in Test externally. We developed a hiring workflow and by the start of the following year, we were able to hire two more people in the role.

From there, the new Software Engineer in the Test group was able to begin revitalizing our automation efforts. We are still in the early stages of the team’s maturity at this point, but I can see its promise paying off, sprint by sprint, at an increasing rate. Thankfully, Niche’s leadership was supportive and patient, given that it took over a year for things to start gelling.

Lessons learned

• While I believed in the value of specialization, we couldn’t afford it as a smaller company. However, once we started to grow, I thought that we should provide people with opportunities to develop mastery. There is great value in having people who are utility players and can do a lot of different things, but it is enormously difficult to build an entire team of utility players. By giving people the opportunity to specialize and narrow their focus while deepening it, I enabled a greater variety of ways for people on my team to flourish and shine.
• We did our hiring for the Software Engineer in Test role entirely during the pandemic. We tried to emphasize face-to-face interaction, introducing a collaborative coding exercise as part of the interview process. It was a good test of my skills as a communicator, and especially now that I am working with a fully remote team, I cannot emphasize enough the importance of synchronous face-to-face communication while vetting candidates.
• I quickly learned the value of having team members whose expertise exceeded my own, and the importance of trusting in that expertise. Shortly after creating this new team, they brought to my attention that our automation suite’s problems were deeper and more complex than I had realized. We were using certain patterns and tools that were not meeting our needs; we were succeeding in spite of them, but not because of them. The team recommended that it would be better to restart from scratch with new tooling, rather than optimize what we had. I took their recommendations into advisement, which instantly strengthened trust on their end, and opened the door to innovations I could not have driven on my own.