Testing Enterprise-Grade Software:
Enterprise-grade software cannot only be a major development challenge, but also a formidable challenge for one’s QA & Testing team.
Are there any specific problem areas to pay attention to or specific QA approaches that can help?
Our QA Team Knows the Answer
“Enterprise software…” The phrase conjures up a bunch of associations, ranging from sprawling legacy enterprise systems to development challenges that spell years of arduous work and just as much hassle.
While a lot has been said about the approaches and techniques that facilitate enterprise software development, less attention is paid to the testing peculiarities of enterprise-grade IT projects.
Simultaneously, being, usually, implemented by more than one development team using a boggling variety of technologies, these projects become bug-infested shortly after the kick-off, thereby threatening to turn what was, initially, seen as a long-term cash cow into a resounding flop (and, eventually, a cash crunch for both software providers and their clients).
Can anything be done to help avoid time losses, extra costs, flunked deadlines and dilapidated projects?
Our QA folks answer this one in the positive, and they’ve pretty much seen it all when it comes to enterprise stuff.
Here’s what they’ve told us.
As we’ve mentioned, enterprise-grade software projects tend to represent major and variegated affairs. Multiple project teams with varying qualifications, programming styles, and work habits create a diverse panoply of functionality that seldom stays unaffected by all these factors.
Wide-ranging team locations, different languages and different time zones all add to the roster of factors that impact the project quality, while making the informal communication between the members of the different project teams a lot more difficult.
Do not undertake the testing of any of the system functionality without an agreed-upon documentation package. Among other things, this package can include:
A Test Plan – a document that describes the testing process in a formal way and enables the QA team to prepare in advance for the testing to be performed.
With a Test Plan, one can easily list the project requirements and make sure they are sufficiently clear prior to the beginning of the testing process.
Similarly, one can determine the kinds of testing that will need to be employed, and then come up with a list of the tools and techniques that will be required to conduct them. Thus, the team will be able to prepare the required testing environments, distribute the testing tasks between themselves, and define the input and output testing criteria already at this point.
What are the benefits of doing all the above beforehand with the help of a Test Plan and not when the testing is just about to start or, still worse, already underway?
They are quite many and quite significant:
You can determine and try to eliminate the factors that are likely to ill-impact the quality of your testing or, even, make this testing entirely impossible.
You can pre-order and start using earlier any lacking tools and environments that may take long to fine-tune when you receive them.
You can estimate the testing time more precisely, and, thus, be more precise in your forecasting and reporting. Your task allocation will be a lot more efficient as well.
You can continually adjust the testing process on the fly with a view to minimizing the risks involved.
You can forecast the risks that are likely to arise and have plenty of time to ponder/recommend ways of mitigating them.
You can make the testing process more transparent for your development team and other project stakeholders.
A Test Report, – a document that informs one about how much in compliance with the project requirements a product is.
In some instances, a Test Report can also provide the details of the testing that has been performed, such as, for example, the time spent, bugs that persist and still require fixing, and the kinds of testing that have been used to date.
A Test Report can be used to determine whether a product is ready for release.
While being, on average, an amalgamation of diverse (Web, backend, mobile) functionality in itself, almost any enterprise application is also planned to integrate and interact with multiple external systems. This generates a multitude of related defects, strewn along the “seams.”
When faced with the testing of an enterprise software application, one should:
Pay special attention to Integration Testing and conduct a sufficient amount of this kind of testing.
Make sure that the QA & Testing team, involved with the project, has a sufficient amount of Integration Testing experience. Actually, the more, the better.
It may prove hugely helpful to involve your Business Analysts in your Integration Testing. Their knowledge and holistic view of the system can help pinpoint the problem or prevent some adjacent system areas from being impacted.
Your Business Analysts should create as many Business Cases for testing purposes, as possible, prior to the kick-off of your Integration Testing. This will promote an integrated testing approach and help ensure that the different parts of the system functionality, involved in a business process, interact with one another and support this business process as planned.
We could also recommend using an awesome y BA tool, invented by our Business Analysts and called
In short, Integration Case Studies describe how different systems interact within a workflow, initiated by one of these systems. The system, which initiates the workflow, provides a data input, thereby causing the other systems to interact with one another.
Our QA Team has collaborated with our BA Team on several enterprise projects through the medium of Integration Use Cases, and this really has given our testing efficiency a huge boost.
All the required system integration must be prepared and arranged for before the solution going in production. If the integration needs to involve an external system that cannot be integrated with before the production phase, the party that owns this system must provide real-time data, required for the integration, beforehand.
The normally wide-ranging functionality and large number of developers and other experts involved make it difficult to trace bugs to those responsible, keep track of what has been done in order to fix them and manage the outcomes. Sometimes, it is, even, hard to identify the developer, who is responsible for or just familiar with the problem area and to whom the bug should or can be assigned.
Try to increase the amount of end-to-end testing and, from the beginning, try to involve those QA engineers, who are skilled in end-to-end testing.
It may help to use two differently experienced QA teams simultaneously: one, more experienced in business value-related testing and another one more versed in development related-testing.
Any complex enterprise system provides a truckload of diverse functionality and there is always a plethora of features to be tested.
In the majority of instances, these features look the same to a QA team in terms of their business value to the client. They either process them mechanically one by one, or make the blunder of giving precedence to what is bulky and complicated, instead of making the functionality that is more important or, even, mission-critical their priority.
The feature set behind a small button that appears on a mediocre-looking screen and is hidden a in the bowels of your system may be the one the rest of this system is just of no use without. Simultaneously, something big and intricate may well be something your client can go without for the next couple of years.
You can request the Business Analysts working on your project to make a detailed presentation of the project to your QA Team. This will help your QA engineers to gain a better understanding of the business value of the different parts of the system to the end user.
You should create a knowledge base for the project and make it available to your QA Team. This knowledge base should contain all existing project-related documentation and be supervised and regularly updated by your Product Owner.
Your Product Owner should put on your QA Team’s radar any newly appeared functionality that has a greater-than-average testing priority.
The functionality of any enterprise solution covers a large number of quite intricate and complex business cases and workflows. Only real-time data can allow you to test your system’s ability to adequately support them.
However, this data is, often, hard or, even, impossible (such as, for instance, in the case of the Defense and Healthcare industries) to obtain, and many clients will baulk at the idea of putting it at the disposal of their enterprise software contractors.
As a result, enterprise software vendors are, frequently, compelled to use artificial data sets that do not only fall short of ensuring the expected system performance, but may also leave large parts of the functionality, including those that are already in production, untested. The latter may happen because sufficiently large and representative volumes of artificial data are, usually, difficult to create.
Try to use as much real-time data, as possible.
If you are a client, talk to your vendor about the possibility of providing them with real-time scrambled data and, if required, request their assistance in preparing this data.