This is the overview of Test Management Tools currently used by our Brightest consultants:
→ TestRail was the al time favorite, both in usage % as in preference.
→ xRay was the least popular tool, although it isn’t the least used tool at our customers.
As a tester – gatekeeper of qualitative software – we don’t always have the luxury to choose the best Test Management Tool for the environment and tooling eco system we work in. Of course, you can tighten a screw with a knife, but it’s not the best option. Often, our consultants must use that knife instead of the correct screwdriver. So, they must be inventive and bend the Test Management Tool to adjust it to their needs.
With the evolutions in software development, testing can’t stay behind. More and more tests must run more frequent and faster than ever to ensure consistent quality in continuous evolving software. That’s why managing and automating your tests becomes more important than ever.
This means, good integration with (most) test automation tools to follow up their automatically triggered executions, is almost inevitable nowadays. Since this gives you direct insights in what’s going on with your software before any change is deployed into production at any time.
Of course, you can tighten a screw with a knife, but it’s not the best option.
Automation is only one part of the puzzle. We must schedule, follow-up and evaluate the results of our test efforts. This brings us to the importance of decent reporting and the possibility to tweak these reports to the needs of the different stakeholders. Gathering all information about all kinds of testing and test phases, in a transparent way ànd in one place, will enable your team to make better decisions and improve their way of working. Which immediately impacts the ROI of software development.
When developers create new features, they want to know about the status and progress of testing these new features. The follow-up and building of those features are often done in other tools then Test Management Tools. So, your Test Management Tool must be able to ‘speak’ with the development tools to enable the best teamwork and get all the benefits out of your tooling.
Here’s an overview of the top 5 tools used by our consultants and their ability to integrate seamlessly with the most popular development tools:
Test Management tool
Azure DevOps TestPlan
Part of Azure DevOps
When our consultants are asked about what they expect from a good Test Management Tool, these are their top 5 answers:
So, how do the top 5 tools used by our consultants rate on these features? ↓
Test Management Tool
▪ Meets the top 5 features:
▪ Non-standard markdown
▪ Auto complete feature
▪ Reporting is difficult
|Azure DevOps Test Plan|
▪ Seamless integration with Azure DevOps (easy teamwork with developers)
▪ Export is cumbersome
▪ Complex handling for simple tasks
▪ Seamless integration with JIRA (easy teamwork with developers)
|▪ Focus on unit and user story (exploratory) testing (reuse is difficult)|
Consultants can’t always choose the tools they work with, which is a big advantage, because they learn to use many different Test Management Tools integrated with various development or automation tools. This way we discover new gems and / or get creative in ways to bend the toolset our way.
Here are some tips and tricks of our experts to help you when you come across one of these tools:
▪ Learn their many keyboard shortcuts and its specific markdown
▪ Learn basic SQL since it’s needed for decent analysis and reporting
Azure Test Plan
▪ Create a separate test-plan for each application / project as a (master) regression set, which you don’t use to execute your tests.
▪ Use the term ‘run status’ for the execution status in the current run
▪ There is no way to group tests and link that group to a feature. You must do this test by test or work in cycles per epic/features.