Tuesday, 22 May 2012

A practical take on Exploratory/Session Based Testing

General Overview and Definition:
As companies look for more cost effective and agile testing methods Exploratory (or Ad Hoc testing) is coming to the fore. I like James Bach’s definition best: ‘Exploratory testing is essentially simultaneous learning, test design and test execution.’ The approach relies on your instinct as a tester and requires a high level view of the functionality.
However, planning an exploratory testing session is key. Planning should be high level and based on the key aspects of the functionality and your testing toolset.

Planning - A Practical Example:
A Software Development House needs to release a piece of functionality to a client. The client insists that it is tested by the end of the day, bugs notwithstanding.
This gives you, the tester, and one working day to test it. For the example this is 6 hours of test time, with half an hour for planning, and a further hour to tidy up and ask questions/raise defects.
The functionality is available for test and has passed initial deployment checks.
The piece of functionality regards creating an online account with an e-Commerce site has 7 requirements, split over three functional areas. This document assumes you have one or the other!
It is recommended that you add a priority and a size to each requirement or functional area. The priority may or may not be provided but you can apply your own common sense to the issue:

Functional Area
Requirement
Priority
Size
Account Details
User can add a Username
High
Medium

User can set a Password
High
Large

User can set Address Details
Medium
Medium

User can add a Nickname
Low
Small
Add Payment Methods
User can add a Credit Card payment method
High
Large

User can add an e-Wallet payment method
Medium
Large
Post Account Creation
User is logged in after creating an Account
High
Small

The key is the priority and the size; we can use this to work out a strategy for allocating time. This is done subjectively:

Requirement
Priority
Size
Percentage of TimeAllocated
Time Allocated in Hours
User can add a Username
High
Medium
15
1
User can set a Password
High
Large
25
1.5
User can add a Credit Card payment method
High
Large
25
1.5
User is logged in after creating an Account
High
Small
10
0.6
User can set Address Details
Medium
Medium
10
0.6
User can add an e-Wallet payment method
Medium
Large
10
0.6
User can add a Nickname
Low
Small
5
0.3

You now have a high level plan which communicates the priority order of testing and how much time this will take. Communicate this to the stakeholders (development team, project team etc) and you are ready to get started.

Exploratory Testing – Executing the Plan:
So we have decided our plan and testing can begin.
The first requirement is ‘User can add a Username.’ There is an hour allocated for this.
Obviously all the requirements for Account Details are linked (you may not be able to submit an Account Creation Request without some of the other items but our focus for the session is on the username. ‘Stub’ out the other fields with values you think should work.
Structure as follows:
User Interface – How is the UI presented? Is there any validation? Any help prompts? Can I interrogate the CSS,JavaScript, and Flash (if web based)?
Middleware – How does my username get to the DB? Is it a messaging system? Web service? Does it validate on its way through?
Database/Storage – Where/what is the DB? What are primary keys? Does my username need to unique? How do the database rules translate to the UI?
As you generate and execute these high level test ideas, note a sentence that describes what you have tested (try in a text file it easily communicable and you don’t need to rewrite them afterwards). Note any defects/questions which occur under the Test Idea. Do this incrementally as they occur, rather than storing them up.
Leave yourself sometime at the end to communicate your coverage of each requirement outwards, clarify any questions,and decide which of your potential defects need to be logged.

Resources:
Basic Definition

Key Aspects of Exploratory Testing

Exploratory vs Experience Based Testing

Testing without a Map – when there are no requirements!

Sunday, 20 May 2012

Let the machine check it, I get bored easily

In the spirit of my new blog I thought I would start with a trick I learn't the other day. I really don't like repetitive testing and I think its our duty (strong words I know) to find interesting ways to get over these challenges.

So I was charged with a piece of API testing. It was a simple sign up process, which set the user into a particular status. The status wasn't returned by the API response  (just a success or failure) however so I needed to go off to the front end to check it.

I had been toying with using SoapUI and Selenium together for some time so I thought what the hell and attempted to hack the two together in a relatively graceful fashion (as we do as testers sometimes). I came up with this approach:
  1. Download the latest Selenium Standalone server.
  2. Put it in here - SoapUIRoot\Bin\Ext
  3. Fire up Soap UI, add a Suite>Test>Groovy Step
  4. Learn Selenium Webdriver (you'll need to one day, may as well get on with it)
In terms of Webdriver I kept it really simple:

import org.openqa.selenium.By
import org.openqa.selenium.WebDriver
import org.openqa.selenium.WebElement
import org.openqa.selenium.ie.InternetExplorerDriver
import org.openqa.selenium.support.ui.ExpectedCondition
import org.openqa.selenium.support.ui.WebDriverWait
  
        // Create a new instance of the Internet Explorer driver
        // Notice that the remainder of the code relies on the interface,
        // not the implementation.
        WebDriver driver = new InternetExplorerDriver()

        // And now use this to visit the site
        driver.get("https://mysite.project.qa.com/login")

        //Get past the Certificate Problem
        WebElement override = driver.findElement(By.id("overridelink"));
                   override.click();

         //Enter Username and Password, Login
         WebElement username = driver.findElement(By.id("username"));username.sendKeys("User");
         WebElement password = driver.findElement(By.id("password"));password.sendKeys("Password1");
         WebElement signin = driver.findElement(By.id("submit"));signin.click();
                  
         //Find the Customer with the number
         WebElement custno =    driver.findElement(By.id("CustomerID"));custno.sendKeys(${#TestCase#CustID});
         
        //Assert the Status
        assertTrue(isElementPresent(By.linkText("Status is Live")));
         
         //Close the browser
         driver.quit();
 I came out with a test that looked like this:
  1. Groovy step to generate a random username and GUID
  2. Property transfer to get them into a HTTP step
  3. HTTP Post to the API (and capture the Customer ID)
  4. Transfer step for Customer ID
  5. Groovy step to run Selenium and use the Customer ID to find the sign up and check the status
And there we are! One very boring bit of testing automated forever. Testing and analysis of new stuff is fun, checking old stuff is a bit dull. Automate now to enjoy the future more.