«  View More Posts

Myths (and Truths) About Software Testing

May 29th, 2023 | 10 min. read

By Milan Polacek

As you know, software testing is a crucial part of software development – but there are several myths surrounding it that still need to be debunked if you need testing to align with your expectations. By drawing a line between fact and fiction around software testing, you can make better decisions about your testing strategy and have a more realistic approach to achieving those goals. 

Here at Pricefx, as a leading cloud-native pricing software vendor, we’ve seen some common misunderstandings around software testing arise repeatedly that we’d like to help address in the hopes of promoting better cooperation both between software vendors and their customers and across internal teams.  

In this article, we’ll outline the most common myths we’ve heard about software testing, and close out with a few truths you can take with you, too, to help you drive better future discussions around testing.  

Myth #1: Software Testing Inherently Improves the Quality of the Software  

Consider this common situation in software testing: software testers find a bug, report it, and have it assigned to a developer. But since deployment of the software to the customer is planned for next week, the stakeholder decides to postpone all bug fixing and deploy the software with the reported bug still unresolved.  

Not all bugs found during testing will be fixed when they should. On the contrary, many won’t be – as we’ve seen in this example, this is usually dependent on whether the project timeline allows for it. Given the quality of your software is the same before and after finding a bug if it’s ignored, no, software testing will not always improve the quality of the software.  

Instead, the purpose of software testing is to find out the most critical information about the software (which could include bugs) so the right decisions can be made by stakeholders as soon as possible.  

Myth #2: Automated Tests Can Cover Everything Now and Manual Testers Aren’t Needed  

While this is a nice idea in theory, we should consider the bigger picture and ask a few questions first: 

Do you have a large data set you need to test? If yes, then you should really think about test automation, because it’ll help carry out test regression testing faster and with more test data than you can do manually. However, you still should think about whether someone is needed to evaluate the reports from automated tests, report bugs and modify automated scripts. 

Who will create those automated tests? Not every tester is able to create them because they usually require special technical skills, so you’ll need to think about whether you have the appropriate resources for this.  

Is our tested application stable? If not, keep in mind that automated testing is much more expensive than manual testing, as automated tests must be modified and upgraded again and again. 

Do you have the right tool for automation and are you ready to pay for licenses? Test automation needs specific tools to carry out, and while some can be affordable or even free, there are many that can come at a high cost. First, try out the tool and check whether it’s suitable for the software, and based on this, evaluate whether your company can afford to take it on.  

As you know, test automation is a complex topic. While it can bring many efficiencies to your testing process, it can also get to be very expensive – much more than if you would continue with a manual testing approach. So, we suggest considering your project requirements to decide the best course of action. 

  

Myth #3: It’s Easy to be a Good Software Tester  

We like the saying: “If you want to be a good tester, learn to think like one, not look like one”.  

The reality is that being a good tester requires a mindset and set of abilities not everyone has. To be successful in testing, you’ll need to embody a whole host of things, including:  

  • Analytical skills to understand the software 
  • Technical skills to understand the finer details of the solution 
  • Attention to detail while simultaneously seeing the big picture 
  • Critical thinking 
  • Communication skills 
  • A good understanding of the customer’s business needs 
  • A good understanding of the project development process  

So, while not everyone can call themselves a great tester overnight, everyone can aim to work on the skills we’ve mentioned to excel in a testing role.  

 

Myth #4: The Purpose of Software Testing Is to Have No Bugs Left in the Software 

If you hear from a stakeholder that they expect zero bugs in the software after testing, be careful; this person either doesn’t understand the complexity of the solution or has very little experience in software development. Eliminating all bugs has never been the goal of software testing, and there are two reasons for this: 

Time is limited  

Software is becoming increasingly complex; to check a seemingly infinite combination of inputs, outputs, and functionalities of a solution today would take ages. Rarely is anyone willing to pay for a lengthy testing period, so testing should aim to be as efficient as possible within the requested timeframe. To do that, testers should prioritize finding the most critical bugs first and then use tools to make sure even what isn’t visible in the user interface is working properly.  

While testers do their best to ensure the software fulfills the client’s requirements and is without major bugs, it is near-impossible to guarantee the software is bug-free.  

 

Not everything can be controlled   

Have you noticed how many programs are running on your computer at once? There’s your operating system (OS) (for example, Windows or MacOS), your internet browser, your CRM and ERP solutions and a handful of other systems which are completely out of our control. Not only are their updates also out of our control, we can’t even choose if we want to install them. After all, you can’t be sure if this will affect the functionality of the software.  

So with that said, while testers can do their best to make sure things run smoothly, there’s always a chance a bug will be found outside of the software that causes it to stop working. 

Now that we’ve outlined the most common myths we’ve heard around software testing, we’ll get into some common beliefs that happen to be true and explain why that’s the case.  

 

Myth #5: Testers Aren’t Needed on a Project, Developers Can Do Unit Testing And That’s Enough 

This one usually comes up when there’s pressure to lower the project price. At first glance, it’s a reasonable suggestion, right? In fact, in some cases this approach can work, especially when a company already has testing experience on IT projects and is able to test everything on their own. Typically, however, this approach doesn’t work well.  

Why? It comes down to how developers and testers are different across these areas: 

Testing coverage: In general, a developer only tests the requirements he developed and simple scenarios. Testers also deal with more complex scenarios, testing edges cases (i.e. those which are extreme or uncommon) and running regression tests of several requirements together. When testers aren’t working on the project, customers can expect to find more bugs. 

Test management: Testers typically document their work, using a test management tool (like Xray for Jira) to create test cases and document results to be easily checked or used later on when needed. These created test cases can be also used by a customer as a baseline for testing. When testing is carried out by developers, these elements of test management are generally missing on projects. 

Help with User Acceptance Testing (UAT): As pricing software customers are generally not experienced software testers, they often need help when the UAT phase comes around; at this stage, they tend to have questions like: 

While software testers can help with these questions and support a smoother UAT phase, developers normally can’t help here – and companies are left to deal with UAT on their own. If mismanaged, the UAT phase can be a source of significant risk on the project. 

Myth #6: Testing after Software Upgrades is Unnecessary  

Every software solution needs upgrading from time to time. There are different reasons for this: it might require new features, bug fixes, security patches, user interface (UI) upgrades, to name a few.  

So, the first important point to accept here is that software upgrades are normal. But is testing after upgrades necessary? Well, not always in the case of low-priority software. However, in the case of solutions that are critical to a company and contain a lot of business data and complex calculations, it’s important to make sure the priority business cases are working correctly after upgrades.  

While we don’t recommend testing the entire software from scratch, business-critical use cases should be retested after an upgrade to ensure that that everything is working properly. In case test cases were created for the UAT phase, some of those cases can be used, depending on your selected use cases. In the long term, test automation can help speed up testing after upgrades, as the business cases don’t change very much.  

 

Truth #1: Software Testing Makes the Project More Expensive  

Yes, it’s true that software testing, and more specifically getting a team together, is an added cost; when there are QA specialists or software testers in your project team, somebody has to pay for them, right? However, this way of thinking could lead to questioning the value of other roles in the project team, such as the project manager or solution architect. To take this logic further, you might even ask yourself, isn’t the best solution to have one senior developer lead the project, communicate with customers, propose and develop the solution, and demo it for the customer – something like superman? Well, that isn’t how it works.  

Every role in the project team has its own purpose and requires different skills. If you remove any role from the team, this increases the risk of finding bugs in the UAT stage, delivering something different than expected, and missing delivery deadlines, among other things. Is this a risk you really want to take?  

 

Truth #2: Software Testing is Part of Quality Assurance (QA) 

This comes down to a simple terminology issue, so we’ll explain the difference between the two here. Software testing is focused solely on testing a particular software solution. Software testers are most concerned with the software specifications, conducting test analysis, preparing and testing test cases, reporting bugs. On the other hand, QA not only covers software testing but also involves the whole testing process for the project, from clarifying requirements, defining the testing strategy, creating reports for customers, and even making software demos and assisting with UAT testing. 

The tasks involved in software testing and quality assurance differ from company to company, so it’s a good idea to clarify expectations upfront. In general, expectations for the QA team are higher than those of software testers, especially when it comes to communication skills and responsibilities.  

 

Learn More About What Software Testing Involves 

By debunking common myths about software testing (and sprinkling in a dose of truth), we hope this article will promote more productive collaboration among software vendors and customers as well as more effective teamwork within organizations.  

If you’d like to learn more on the subject, consider heading to our article on User Acceptance Testing (UAT), the final testing stage in a software project, to find out why UAT exists and its main objectives.  

CTA-What-is User-Acceptance-Testing-and-Why-Do-You-Need-it

 

 

Milan Polacek

QA Competency Lead , Pricefx

Milan Poláček is currently the QA Competency Lead for Customer Solutions at Pricefx. Milan has over 20 years' experience in the software testing field and gained his experience by testing medical devices and then on large projects for banks and telco companies as testing leader during the development phase and supporting the UAT phase. On the weekends you will find Milan playing with his dog, walking and traveling around the Czech Republic or working on his self-development.