The Ultimate Beginner’s Guide to Acceptance Testing

What is Acceptance Testing?

Acceptance testing my friends is like the grand finale of a fireworks show. 

It’s the last phase in the software testing process where we check if the system is ready for delivery. 

In simpler terms, it’s like a final rehearsal before the big show. 

We test whether the software meets all specified requirements and determine if it’s ready for delivery.

This type of testing is performed by clients or end-users who use real-world scenarios to see if the software behaves as expected. 

If it passes this stage then voila! The software is ready to go live and we’re all set to cutover to PRODUCTION.

What do I know about User Acceptance Testing?

Now you might be wondering, “Why should I listen to this person?” 

I’ve been in the software testing industry for over 15 years, working on various projects across different stages of the software testing life cycle. 

User Acceptance Testing is certainly one of those phases.

I’ve seen systems crash and burn during acceptance tests and others that have sailed through smoothly. 

I’ve learned from each experience and honed my skills in planning and executing effective acceptance tests.

What Will You Learn From This Article?

In this article let’s take a deep into acceptance testing. Let’s take a look at what it involves, how to prepare for it, how to execute tests effectively and what documentation you need. 

We’ll also discuss:

• The teams involved in acceptance testing
• How to define clear acceptance criteria
• How acceptance testing gets signed off

I’m hoping this will be an exciting journey for you  into the world of acceptance testing.

Understanding Acceptance Testing

A Typical Project Checklist

Below is a typical acceptance testing project checklist.  This may vary from each project but you can use this as a guide to support your project.

Checklist Item	Description
Define User Stories/Use Cases	Outline what exactly needs to be tested based on discussions with stakeholders. You can capture this in a project test plan or test strategy document.
Prepare Test Plan	Detail how tests will be conducted, including resources required and timelines.
Develop Test Cases	Create detailed instructions on how each requirement will be tested.  You should include input (steps) and output (expected results) for each case. The end user’s should ideally create these to prove their end to end business flows.
Map Test Cases to Requirements Traceability Matrix (RTM)	Ensure every requirement is covered by at least one test case to guarantee complete coverage. As a Test Lead on a project, you should ideally support the business users as they may be unfamiliar with this document.
Define Acceptance Criteria	Specify what must be achieved for a requirement to be considered ‘done’.  This should align with stakeholder expectations. You can capture the agreed criteria in the test strategy document and capture if it has been accepted in the test exit report.
Assemble Test Team	Identify who will perform the tests. This is likely to be end-users or clients of the system.  Ensure they are trained and understand their roles. When you plan resourcing, make sure you factor in any vacations, and actual time they can test the system.
Set Up Test Environment	Ensure the test environment mirrors the production environment as closely as possible to ensure accurate test results.
Set Up User Access for the Test Environment	Ensure that users have been set up with their roles and permissions. You’re likely to have issues when you start testing but it’s better to reduce the risk of user issues as early as possible. Ensure your users also confirm they can access the system before testing commences.
Schedule Testing Activities	Plan when each test will take place, considering any dependencies between tests.  Consider time for testing cycles, retesting and any regression testing as a result of releases.
Review and Approve Documents	Have all relevant documents reviewed and approved by stakeholders before beginning testing activities. This includes user stories/use cases, test plans, and test cases. Also factor in post testing activities such as test exit reports and any closure activities.
Prepare Defect Management Process	Establish a process for logging, tracking and resolving defects found during test execution. Consider how many times per week you want defect triage calls, key resources that will need to be involved and the process you will use for defect resolution. If you are working with external vendors, it is important that you think of a way you can work together to ensure your defect lists are all aligned and up to date.
Prepare Test Exit Report	Document a summary of testing activities including defect status, coverage of requirements and recommendations for going into PRODUCTION.  .
Plan Sign-off Process	Determine who will sign off on the tests (usually business stakeholders) and under what conditions they’ll do so. You may also find that other teams within your organisation will need to sign off such as IT Security, Infrastructure, Architecture, OPerations, P&C etc.

The Teams Involved

Acceptance testing involves two main teams: developers and users (or clients). 

Developers create the system based on specified requirements while users test whether these requirements have been met satisfactorily.

The end users are the ones that usually provide the requirements in the first place so they’re essentially signing off the final product.

Now, there is a slight caveat to two main teams.  If the UAT team isn’t familiar with how to test then you as a Testing specialist will need to roll your sleeves up and support.

This is something that I’ve done numerous times on various projects across my career where me and my team have had to facilitate the end users with test design and execution for acceptance testing.

This will need to be clearly defined in your test procedure and approach documentation.

Preparing for Tests

Preparation is key when it comes to acceptance tests. 

Firstly, you’ll need well-defined user stories or use cases that outline what exactly needs to be tested. 

If you are a business owner, think about your current business as usual (BAU) tasks will be tested in the new system. That’s exactly what I tell my UAT users.

Next up your end users will be required to design test cases which detail how each requirement will be tested.  Think of them as your roadmap during your test execution journey!

These are typically prepared by business analysts, product owners and end users.

Executing Tests

When executing tests, your end users will be simulating real-world scenarios using test cases as their guide. 

They’ll record any discrepancies they find between expected and actual results. Anything that deviates from an expected output will be raised as a defect, or bug as it’s otherwise known. 

Documentation Required

Documentation will play a crucial role in acceptance testing.  

As someone who is used to working in a highly regulated environment, I cannot stress how important this is.

I always tell my team;. 

“If we were to be audited (internally or externally), would you have all the sufficient paperwork to back up everything you’ve done?”

It’s a powerful statement and generally changes the mindset of the person, regardless of how much of a nightmare governance can be.

Key documents may include; 

1. user stories/use cases (which outline requirements), 
2. test plans (which detail how tests will be conducted), 
3. test cases (which guide testers during execution)  
4. defect reports (which record any issues found).

Note: I’m only including key documents that are relevant for UAT only.  Project level documents such as the Test Strategy, Requirements Traceability Matrix (RTM) and RACI matrices should all be completed prior to this stage.

Defining Acceptance Criteria

Acceptance criteria is what needs to be achieved for a requirement to be considered ‘done’.

They’re usually defined by product owners or business analysts based on discussions with stakeholders.

What your end user is essentially telling you with the acceptance criteria is “without this being completed, we will not sign off for product delivery”.

It’s kind of a big deal so you should always know what the acceptance criteria is.   

Signing Off Tests

Once all defects have been addressed and retested successfully, it’s time for sign-off! 

This means that stakeholders agree that all requirements have been met satisfactorily based on their predefined acceptance criteria.

Frequently Asked Questions About Acceptance Testing

1. What is acceptance testing?
   Acceptance testing is like the final dress rehearsal before the big show. 

It’s the last phase in the software testing process where we check if the system meets all specified requirements and is ready for delivery. In simpler terms, it’s like a final examination before graduation.

For example, let’s say you’ve built an app for online shopping. 

During acceptance testing, you’d check whether all functionalities; from product search to checkout work as expected under real-world conditions. 

If everything goes smoothly and meets your predefined criteria (like no crashes or delays during checkout), then congratulations! 

Your app has passed acceptance testing and is ready to go live.

2. Who performs acceptance testing?
   Acceptance testing is performed by clients or end-users who use real-world scenarios to see if the software behaves as expected. 

They’re like our audience during a dress rehearsal – they give us feedback on whether our performance (or in this case, our software) meets their expectations.

For instance, in the case of our online shopping app, actual users would test various features like product search, adding items to cart and checking out to ensure everything works seamlessly.

In an organisation where you’ve upgraded your financial systems, it makes sense that the Finance team would be suited to test and accept the system.  

Since you end users are also subject matter experts (SMEs), it makes sense that they are best suited to conduct acceptance testing.  

3. What are user stories/use cases?
   User stories or use cases are like scripts for a play as they outline what exactly needs to be tested. 

I’d loosely compare this to a high level test scenario where you have a specific end goal in sight

They’re typically prepared by Business Analysts or product owners based on discussions with stakeholders.

Let’s take an online shopping app example. 

A user story could be something like: “As a customer, I want to search for products so that I can find what I’m looking for quickly.” 

This gives clear direction on what functionality needs to be tested (product search) and why it’s important (for quick and easy product discovery).

4. What are test cases?
   Test cases are detailed instructions that guide testers during the test execution process.  

Think of them as your roadmap during your journey! 

They detail how each requirement will be tested and what constitutes a pass or fail.

Each test case will have an input (test steps) and an output (expected result).

For example, based on our earlier user story about product search functionality in an online shopping app, a test case might involve entering different keywords into the search bar and checking if results are displayed “as expected”.

5. What documents are required for acceptance testing?
   Documentation plays a crucial role in acceptance testing. 

For example, writing test cases, capturing evidence and writing test summary reports. 

The governance process is a key part of your role in supporting user acceptance testing.

Key documents include; 

• user stories/use cases (which outline requirements), 
• test plans (which detail how tests will be conducted), 
• test cases (which guide testers during execution) and 
• defect reports (which record any issues found).

For instance, while conducting acceptance tests for your online shopping app, you’d need user stories outlining key functionalities (like product search), detailed test plans explaining how these functionalities will be tested along with specific test cases guiding testers through each step of execution.

6. What are defects or bugs?
   Defects or bugs refer to discrepancies between expected and actual results that users find when executing tests.

Let’s take an example from our online shopping app again. Suppose one of the functionalities was that customers should receive an email confirmation once they place an order but during tests no such emails were received. 

This discrepancy would be logged as a defect as it deviates from expected behaviour.

7. How do you define acceptance criteria?
   Acceptance criteria define what must be achieved for a requirement to be considered ‘done’. 

They’re usually defined by product owners or business analysts based on discussions with stakeholders.

Using our previous example about searching products in an online shopping application: The acceptance criteria could include things like – “The application should return results within 2 seconds” or “The application should handle at least 1000 simultaneous searches”. 

These criteria set clear expectations about what needs to happen for this particular requirement to pass.

It could be either a checklist or formally in a document such as the Test Strategy.

8. When does sign-off occur in acceptance testing?
   Sign-off occurs once all defects have been addressed and retested successfully indicating that all requirements have been met satisfactorily based on predefined acceptance criteria.

So in the context of our online shopping application: Once all functionalities have been tested according to their respective test-cases without any critical defects being found, the stakeholders can sign-off indicating their satisfaction with the system’s readiness.

As a Test Manager, what you’re looking at is the current level of confidence of the system. If the confidence is high and there are relatively low severity defects then it’s very likely the system will be signed off.

You should use the entire test execution phase(s) to gauge confidence levels.

9. Why is documentation important in acceptance testing?
   Documentation guides you through each step of the process helping you record any issues that arise along the way. Think of it like your GPS during your journey.

In context of our ongoing example. 

Documenting every step from defining user-stories, use-cases till signing off ensures transparency and traceability throughout the process making sure nothing slips through cracks!

If you work in a highly regulated environment, documentation will be mandatory as part of your key controls. 

10. Can I perform acceptance testing without clear user stories/test cases?
    While technically possible, I would not recommend it. 

Without clear user-stories/test-cases you won’t know exactly what needs to be tested which could lead to coverage gaps and missed defects.

Imagine trying to navigate an unknown city without a map.  That’s what performing acceptance testing without user-stories feels like!

You can conduct exploratory testing without any test cases or user stories as part of UAT but it should not be mutually exclusive.

Wrapping Up: Acceptance Testing Demystified

So there you have it! 

I’ve taken a deep dive into the world of acceptance testing for you. Explored what it is, why it’s important, how to prepare for and execute tests effectively.

I’ve also discussed the key players involved in this process, from developers who build the system based on specified requirements to users or clients who test whether these requirements have been met.

Remember, documentation is crucial in this stage of software testing. 

It’s like your GPS during your journey, guiding you through each step and helping you record any issues that pop up along the way.

Defining clear acceptance criteria is another critical aspect of acceptance testing. 

These are your goal posts that determine when a requirement can be considered ‘done’.

Once all defects have been addressed, retested successfully and your users have confidence in the system, it’s time for sign-off!

Acceptance testing may seem daunting at first, but with careful planning and execution (and a little help from this guide), you’ll be well on your way to ensuring that your software is ready for delivery.

V00.01 – 30/7/2023 Started in OpenAI Playground GPT4.