I remember reading an article on how Quality Engineering was well beyond and more complex than software testing. In the course of reading that article, it struck me that the author had a totally different understanding of those two terms.
I wanted to examine them and explore some ideas.
Software Quality Engineering, simply put, is applying the practices of Quality Engineering, as a discipline, to the development and creation of software. It is part of a defined quality program focused on how software is made. It focuses on the process, goals, measurement, compliance to the agreed goals established by the development team for the project.
Unfortunately, like most things in software, the formal, intended meaning of concepts have been forgotten or ignored. People will hear a term that sounds familiar and apply their understanding to it. If you use your favorite search engine and look up “Software Quality Engineering” most of the responses you get will be around software testing. For most people in software, a “Quality Engineer” does software testing.
Let us look at Software Testing then. Again, a significant problem is people hear a term and apply their understanding of similar terms to the new one. The result is confusion.
Common View of Testing
A view which some consider to be the "real" or "correct" view, is that testing validates behavior. Tests "pass" or "fail" based on expectations and the point of testing is to confirm those expectations.
The challenge of introducing the concept of “Quality” with this conception of testing brings in other problems. It seems the question of "Quality" is often tied to a "voice of authority.” For some people that "authority" is the near-legendary Jerry Weinberg: "Quality is value to some person." For others the “authority” is Joseph Juran: "fitness for use."
How do we know about the software we are working on? What is it that gives us the touch points to be able to measure this?
There are the classic measures used by advocates of testing as validation or pass/fail:
These may shed some light on testing or on the perceived progress of testing for some organizations. Many organizations will assert with great confidence that these measures tell them all they need to know about their testing.
However, they speak nothing about the software itself or the quality of the software being tested. Nor do they tell us anything meaningful about the testing that is being done.
One response, a common one, is that the question of the “quality of the software” is not a concern of “testing,” that it is a concern for “quality engineering.” Thus, testing is independent of the concerns of overall quality.
Good testing, like everything involved in creating good software, takes disciplined, thoughtful work. Following the precise steps dictated is not testing. It is following a cookbook recipe or a script. Testing takes consideration beyond the simple, straightforward path.
When people ask me what testing is, my working definition is:
Software testing is a systematic evaluation
of the behavior of a piece of software,
based on some model.
By using models that are relevant to the project, epic or story, we can select appropriate methods and techniques in place of relying on organizational comfort-zones. If one model we use is “conformance to documented requirements” we exercise the software one way. If we are interested in aspects of performance or load capacity, we’ll exercise the software in another way.
There is no rule limiting a tester to using a single model. Most software projects will need multiple models to be considered in testing. There are some concepts that are important in this working.
When it comes to “documented requirements,” they serve as information points for us. Many times, they make reasonable starting points for good, meaningful testing.
To perform good testing we need good communication. Real communication is not documents which are emailed back and forth. Communication itself is shared and is bi-directional. It is not a lecture or a monologue. Communication requires conversation. This helps make sure all parties are in alignment.
Good testing looks at the reason behind the project. We need to be able to identify and understand what the intended change will do. We need to be able to show how the business problem is addressed by the change being implemented.
Good testing looks to understand the impact of what is being done to the broader application and software system. It is not enough to identify the intended impact areas. We must be able to illuminate areas within the system which may be impacted, whether intended or not. We must also identify people who may find their work processes changed by the change in software.
Good testing looks at these reasons and purposes for the changes and compares them to the team and company purpose and values. Are they in alignment with the mission, purpose and core values of the organization? Good testing includes a willingness to report variances in these fundamental considerations beyond requirements and code.
Good testing can exercise the design before a single line of code is written. Good testing can help search out implied or undocumented requirements to catch variances before design is finalized.
Good testing can help product owners, designers and developers in demonstrating the impact of changes on people who will be working with the software. Good testing can help build consensus within the team as to the very behavior of the software.
Good testing can navigate between function level testing to broader aspects of testing, by following multiple roles within the application and evaluating what people using or impacted by the change will experience.
Good testing can help bring the voice of the customer, internal and external, to the conversation when nothing or no one else does. Good testing challenges assurances. It investigates possibilities and asks questions about what is discovered.
Good testing challenges assumptions and presumptions. It looks for ways in which those assumptions and presumptions are not valid or are not appropriate in the project being worked on.
Good testing serves the stakeholders of the project by being in service to them.
How Does Testing Serve Stakeholders? Testing provides information.
Applying test approaches to design can help reveal weak points in the design, before any code is written. Doing the same thing with documented requirements can head off problems in design before design begins.
Then there is the reason the work is being done in the first place. Applying the same mindset or approach to evaluate the business problem being addressed, before requirements are considered, can help clarify thinking around not only the reason driving the work, but also define some of the expected outcomes. By doing this from the very inception, before the project is actually a project, many of the problems which come up later can be reduced if not avoided altogether.
In that sense, the idea of “quality engineering” takes on a third definition. That is, software quality engineering is building quality software products from the very inception. To make that happen takes very similar skills to what is required for good software testing.