Created Writing high-quality testing instructions (markdown)
Alejandro López Ariza
parent
764da30198
commit
076e654d68
|
|
@ -0,0 +1,80 @@
|
||||||
|
## What is a test?
|
||||||
|
|
||||||
|
A test is a method that we can use to check that something meets certain criteria. It is typically defined as a procedure which contains the steps required to put the system under test in a certain state before executing the action to be checked. Therefore, a test consists of the following stages:
|
||||||
|
- **Preconditions:** All the steps that need to be performed to put the system in the desired state before executing the action we want to check. A test could have many preconditions.
|
||||||
|
- **Action:** This is the exact step that causes the change we want to check in the system. It should be only one because each test should ideally cover one thing at a time.
|
||||||
|
- **Validation:** Relates to the steps to be performed in order to validate the result of performing the action in the system. A test could validate more than one thing.
|
||||||
|
|
||||||
|
For example, in the process of adding an item to the cart:
|
||||||
|
- The **preconditions** would be all the steps involved in:
|
||||||
|
- The product creation process.
|
||||||
|
- Logging as a shopper.
|
||||||
|
- Heading to the shop page where the products are listed.
|
||||||
|
- The **action** would be clicking the *“Add to cart”* button in the desired product.
|
||||||
|
- The **validation** stage would include checking that the cart icon (if any) shows 1 more item and the product we selected is now included in the cart.
|
||||||
|
|
||||||
|
Specifying what are the preconditions, action and validation can be quite beneficial when understanding the scope of a test, because:
|
||||||
|
- The **preconditions** describe what we have to do so that we can execute the test successfully.
|
||||||
|
- The **action** lets us know the purpose of the test, in other words, it is the key to understanding what we need to test.
|
||||||
|
- The **validation** stage lets us know what to expect when executing the test.
|
||||||
|
|
||||||
|
In this context, we will refer to testing instructions as the tests we need to execute in order to validate that the changes delivered in a Pull Request or Release work as expected.
|
||||||
|
|
||||||
|
## Flow to write good testing instructions
|
||||||
|
1. **Outline the user flows** you want to cover.
|
||||||
|
2. **Define the environment** where the testing instructions should be executed (server, PHP version, WP version, required plugins, etc), and start writing the testing instructions as if you were starting from a fresh install.
|
||||||
|
3. Identify the **preconditions**, **action** and **validation** steps.
|
||||||
|
4. Write **as many preconditions as you need** to explain how to set up the state of WooCommerce so that you can execute the desired action to test every flow.
|
||||||
|
1. Try to be detailed when explaining the interactions the user needs to perform in WooCommerce.
|
||||||
|
2. If there are several preconditions for a user flow that is explained in a public guide, feel free to simply link the guide in the testing instructions instead of writing several steps. For example, *"Enable dev mode in WooCommerce Payments by following the steps mentioned [here](https://woocommerce.com/document/woocommerce-payments/testing-and-troubleshooting/dev-mode/)"*.
|
||||||
|
5. Write **the action step**, which should cover the specific action that we want to test as part of this user flow.
|
||||||
|
6. Write **as many validation steps** as needed in order to assess that the actual result meets expectations.
|
||||||
|
1. Bear in mind to check only the steps needed to validate that this change works.
|
||||||
|
|
||||||
|
|
||||||
|
### Considerations for writing high-quality testing instructions
|
||||||
|
- Define the testing instructions in a way that they can be **understood and followed by everybody**, even for people new to WooCommerce.
|
||||||
|
- Make sure to describe every detail and **avoid assuming knowledge**, the spectrum of readers might be wide and some people would not know the concepts behind what is being assumed. For example, instead of saying *“Enable the [x] experiment”*, say something like:
|
||||||
|
```
|
||||||
|
- Install the WooCommerce Beta Tester plugin.
|
||||||
|
- Go to `Tools > WCA Test Helper > Experiments`.
|
||||||
|
- Toggle the [x] experiment.
|
||||||
|
```
|
||||||
|
- Always try to explain in detail **where the user should head to**, for example instead of saying “Go to the Orders page as admin”, say “Go to [url]” or even “Go to WooCommerce > Orders”.
|
||||||
|
- Try to use real test data. For example, instead of saying *"Enter a name for the product"*, say something like *"Enter 'Blue T-Shirt' as the product name"*. This will make it more self-explanatory and remove potential doubts related to assuming knowledge.
|
||||||
|
- Make sure you **keep your testing instructions updated** if they become obsolete as part of a new commit.
|
||||||
|
- If the testing instructions require to add custom code, please **provide the code snippet**.
|
||||||
|
- If the testing instructions require to install a plugin, please **provide a link to this plugin, or the zip file** to install it.
|
||||||
|
- If the testing instructions require to hit an API endpoint, please provide the **link to the endpoint documentation**.
|
||||||
|
- Ideally **provide screenshots and/or videos** that supported what the testing instructions are explaining.
|
||||||
|
|
||||||
|
|
||||||
|
## Examples
|
||||||
|
### Good quality testing instructions
|
||||||
|
#### Example 1
|
||||||
|
<img width="905" alt="Screen Shot on 2022-12-29 at 13-32-23" src="https://user-images.githubusercontent.com/45979455/213682695-3dc51613-b836-4e7e-93ef-f75078ab48ac.png">
|
||||||
|
|
||||||
|
#### Example 2
|
||||||
|
<img width="813" alt="Screen Shot on 2022-12-29 at 13-39-19" src="https://user-images.githubusercontent.com/45979455/213682778-b552ab07-a518-48a7-9358-16adc5762aca.png">
|
||||||
|
|
||||||
|
### Improving real testing instructions
|
||||||
|
|
||||||
|
In this section, you will see some real examples of testing instructions that have room for improvement (before) and how we can tweak them (after).
|
||||||
|
|
||||||
|
#### Before
|
||||||
|
<img width="726" alt="Screen Shot on 2022-12-29 at 14-21-28" src="https://user-images.githubusercontent.com/45979455/213682262-25bec5c3-154c-45ec-aa3d-d3e07f52669e.png">
|
||||||
|
|
||||||
|
#### After
|
||||||
|
<img width="1203" alt="Screen Shot on 2022-12-29 at 16-10-22" src="https://user-images.githubusercontent.com/45979455/213682303-1b12ab97-f27a-41cb-a8db-da8a78d18840.png">
|
||||||
|
|
||||||
|
#### Improvements
|
||||||
|
<img width="1203" alt="Screen Shot on 2022-12-29 at 16-10-22 (1)" src="https://user-images.githubusercontent.com/45979455/213682323-0ecc998d-69ab-4201-8daa-820b948315e8.png">
|
||||||
|
|
||||||
|
#### Before
|
||||||
|
<img width="369" alt="Screen Shot on 2023-01-11 at 13-28-47" src="https://user-images.githubusercontent.com/45979455/213682396-8c52d20e-1fca-4ac1-8345-f381c15a102a.png">
|
||||||
|
|
||||||
|
#### After
|
||||||
|
<img width="654" alt="Screen Shot on 2023-01-11 at 13-36-29 (1)" src="https://user-images.githubusercontent.com/45979455/213682480-c01e0e84-5969-4456-8f43-70cbb8509e8d.png">
|
||||||
|
|
||||||
|
#### Improvements
|
||||||
|
|
||||||
Loading…
Reference in New Issue