Hello there! You know, one of the roles that is often found in system implementation projects is that of the test manager. Who does a test manager do? Many of my readers are sometimes confused about the roles and responsibilities of the test manager.
In this article, I want to describe what a test manager does and the value he or she adds to a project. A test manager role, in my opinion is critical because it covers quality – quality of the software being implemented.
Usually, he or she will coordinate large teams of testers to test and debug software so that the product rolled out is of the absolute highest quality. Let’s discover more below.
The test manager is a critical role in a system implementation a project
First up, let’s try to understand what’s the definition of a test manager, who is also sometimes called a test lead or lead tester.
Definition: A test manager leads a team to improve product quality by implementing structured testing processes and tools.
Notice the terms used in my definition.
Leads a team. A test manager must be able to lead a team. Testing teams for large system implementation projects can comprise up to 50 to 100 testers. That’s a huge number of people to coordinate and supervise.
Improve product quality. The KPI for a test manager is product quality. This is usually the number of defects uncovered and fixed in a product. While a “product” in this context is usually meant to be a software application – a test manager can be deployed not just for software projects, but also things like construction, engineering, etc. Any product that demands quality assurance will usually require a test manager.
Structured testing processes and tools. A test manager must be supported by good processes and tools. The rate at which defects are found (which can number into the hundreds in a day), plus the number of test scenarios and depth of analysis usually requires industrial strength testing processes and systems. Usually, a software (e.g. HP QualityCenter) is deployed to support testing.
Beyond the definition above, there are a two points which need to be highlighted about the test manager role – the stress level and pre-requisites.
The stress level. A test manager’s job is stressful. Put it this way – his or her job is really the “last line of defence”. All defects must be uncovered and fixed by the testing team before the product ships out the door.
If you think about it, in a system implementation project, all the Business Analysts, Developers, etc. who are involved in the earlier stages of a project – can theoretically do a bad job and “no one will really know”. UNTIL the testing phase. The testing phase is where the buck stops. All “hidden” problems in requirements, development and so forth – will come out. And stakeholders will be shocked to find that the software is not meeting their expectations, etc.
That kind of scenario helps you understand why a test manager’s job is very stressful. It’s not for the weak of heart.
Pre-requisities. What are the pre-requisites for a test manager role? Well – to me, the only pre-requisites are project management and deep analytical and problem solving skills. Business domain, solution or industry knowledge will a definite plus
Yes – a test manager in fact is something like a Business Analyst. He or she needs to project manage testers – just like a Lead BA manages junior BAs. He or she needs to analyze and resolve defects, which means problem solving skills are critical.
2. Roles and Responsibilities
Right, let’s now try to understand more about the roles and responsibilities of a test manager.
- Developing a test strategy and plan
- Developing test cases
- Leading a testing team to execute test cases
- Managing defects uncovered
- Reporting on testing progress
- Obtaining sign-off on the testing phase
Let’s take a look at these roles and responsibilities in turn.
Developing a test strategy and plan. A test manager must, before doing any detailed work, develop a proper test strategy and plan.
A test strategy is a paper document that lists out the key objectives, approach and timeline for testing. It may also include details of testing processes and tools, along with any persons responsible for executing them.
The test plan is usually a VERY detailed plan that lists out who, what and when tests happen. Who is supposed to run a particular test script, using what data and when it should be run. When certain batch jobs should be run to ensure test data “catches up” to the current test date. If you’ve managed system testing before, you’ll know exactly what I mean 🙂
Developing test cases. Test cases are also the responsibility of the test manager. What’s a test case, you ask? Here’s my definition.
Definition: A test case is a well-defined, documented series of steps to executive a specific business function in a system, and includes details of expected results.
Test cases can easily run into the thousands for a large project.
On one core insurance project I was on, we had over 2,000 test cases! Two thousand! Documenting that alone took all of four months.
The test manager should work with business stakeholders and also IT to ensure that all steps – down to what button to click and what will be shown on the screen – are written into test case documents. When there’s absolute clarity in a test case, it’s also then easier for a tester to execute the test case.
Leading a testing team to execute test cases. A test manager must coordinate many resources – playing the function of a project manager. He or she needs to get the testing environments and locations set up. He or she should also get testers to check in and ensure they run their assigned test cases their assigned dates. Which is why good test managers usually have strong project management skills as well.
A test manager leads teams to execute test cases
Managing defects uncovered. Defects uncovered during the testing process need to be fixed. But, as you can imagine – there could be hundreds of defects detected in a day.
Most of these will have differing severities and need to be fixed with different levels of urgency. And most of the time, the test manager will need to interact with business users, the PM, vendors and in-house IT to resolve the defects. It requires a very organized mind and a lot of to rigor to keep things in control.
Case Study: I’ve been the test manager for a core insurance implementation in Malaysia before. I oversaw a team of twenty testers situated in Kuala Lumpur. Each day, as they executed their test cases, I had to log into a test management system to look at defects they’d raised. Any critical defects would be despatched out for fixing ASAP. Also, I’d hole a daily defect call to ensure everything – from users to vendors to IT – were in synch as to which defects were to be fixed, re-tested, re-opened and closed. It was quite a crazy time but I learnt a lot from that stint.
Reporting on testing progress. A test manager also needs to compile and report on the status of testing. For example – how many test cases have been run in a particular day. Or, how many critical defects have closed out this week. Senior management usually need these metrics to judge if testing is progressing well.
Obtaining sign-off on the testing phase. Finally, one very important role of the test manager is to obtain sign-off on the testing phase. Those of you who have been in testing before will know that this is VERY HARD to do. Many defects may still be open by the time the testing phase ends. Non-critical defects can left open, but workarounds will need to be documented. And stakeholders usually don’t like workarounds 🙂 – which means they won’t sign off. Sometimes, if a sign-off is too difficult, the overall PM or a senior sponsor for the project may need to come in to highlight that sign-off is required and that workarounds must be accepted.
3. Testing Tools and Techniques
A key point about the test manager is that he or she cannot run alone. He or she needs to have testing processes and tools in place to facilitate testing.
There are two things here – processes and tools. Let’s talk a bit about each.
Testing processes. Testing processes must always be clearly documented, e.g.
- Who sets up the test environments. Who writes test cases
- Who is controlling the system “test date”
- Who is running the testing batch jobs
- Who is raising defects
- Who is managing the fixing of defects
- Who is classifying the severity of defects
These questions MUST be answered very early in a project. If you wait till near the testing phase to do this, it’ll be too late.
Tip: When I was a test manager, I’d draw out testing processes using “swim lane” diagrams – these are immensely useful in providing a visual representation of who does what (and when) during testing.
Testing tools. The next thing I should touch on are testing tools. Most of the test managers I know are familiar with one or two industrial grade testing tools. I’m talking about software like HP QualityCenter and JIRA.
If you’ve been in test management, you might already be familiar with these tools. They are absolutely CRITICAL in managing test cases and defects. The system will track who has run which test case and also allows screenshots or notes to be attached to the associated defect log. Test cases are usually crafted and then loaded into these testing tools for centralized control and reference.
If you’re looking to apply for a role of test manager, I’d suggest you get familiar with one of these tools.
Wrapping Up …
I hope the above has helped you understand more about what a test manager does. Indeed, the test manager is like a Project Manager cum Business Analyst. He or she needs to lead large testing teams to ensure quality, and having business domain or industry knowledge helps with the development of test cases and the management of defects.
Many test managers I know move on to become Lead BAs or project managers – as these are the natural career choices coming out of a testing role. And most do a very good job – probably because the stress of testing prepares you will for almost anything!
That’s all I have for now. If you have any queries on testing or the test manager role, do feel free to drop me a note. I’ll be glad to help.
Until next time, have fun learning more about testing and test management!