Copyright © 2011-2014 John Ferguson Smart.

Online version published by Wakaleo Consulting.

This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States license. For more information about this license, see creativecommons.org/licenses/by-nc-nd/3.0/us/.

Java™ and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc., in the United States and other countries.

Eclipse™ is a trademark of the Eclipse Foundation, Inc., in the United States and other countries.

Apache and the Apache feather logo are trademarks of The Apache Software Foundation.

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and Wakaleo Consulting was aware of a trademark claim, the designations have been printed in caps or initial caps.

While every precaution has been taken in the preparation of this book, the publisher and authors assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein.

2. Introducing Thucydides

Thucydides (Thoo-SID-eh-dees) is a tool designed to make writing automated acceptance and regression tests easier ( Refer to Pt. 3 ). It provides features that make it easier to organize and structure your acceptance tests, associating them with the user stories or features that they test. As the tests are executed, Thucydides generates illustrated documentation describing how the application is used based on the stories described by the tests.

Thucydides provides strong support for automated web tests based on Selenium 2 ( http://docs.seleniumhq.org/projects/webdriver/ ), though it can also be used effectively for non-web tests ( ? ).

Thucydides was a Greek historian ( http://en.wikipedia.org/wiki/Thucydides ) known for his astute analysis skills who rigorously recorded events that he witnessed and participated in himself. In the same way, the Thucydides framework observes and analyzes your acceptance tests, and records a detailed account of their execution.

3. Basic concepts of Acceptance and Regression Testing

To get the most out of Thucydides, it is useful to understand some of the basic principles behind Acceptance Test Driven Development. Thucydides is commonly used for both Automated Acceptance Tests and Regression tests, and the principles discussed here apply, with minor variations, to both.

Acceptance Test Driven Development, or ATDD, is an advanced form of Test Driven Development (TDD) in which automated acceptance criteria — defined in collaboration with users — drive and focus the development process. This helps ensure that everyone understands what features are under development.

One of the important things about ATDD is the idea of "Specification by Example". Specification by Example refers to the use of relatively concrete examples to illustrate how a system should work, as opposed to more formally written specifications.

Let’s look at an example. In many projects, requirements are expressed as simple stories along the following lines:

In order to earn money to buy a new car
As a car owner
I want to sell my old car online

If we were implementing an online car sales web site that helps people achieve this goal, we would typically define a set of acceptance criteria to flesh out this story. For example, we might have the following criteria in our list of acceptance criteria:

  • Car owner can place a standard car ad online

  • Car owner can place a premium car ad online

  • Car ad should display the brand, model and year of the car

and so on.

A tester planning the tests for these acceptance criteria might draw up a test plan outlining of the way she expects to test these criteria. For the first criteria, she might start off with a high-level plan like the following:

  • Go to car ads section and choose to post a standard car ad

  • Enter car details

  • Choose publication options

  • Preview ad

  • Enter payment details

  • See ad confirmation

Each of these steps might need to be broken down into smaller steps.

  • Enter car details

    • Enter car make, model and year

    • Select options

    • Add photos

    • Enter description

These steps are often fleshed out with more concrete details:

  • Enter car details

    • Create an ad for a 2006 Mitsubishi Pajero

    • Add Air Conditioning and CD Player

    • Add three photos

    • Enter a description

For our purposes, Regression Tests can be defined as end-to-end tests that ensure that an application behaves as expected, and that it continues to behave as expected in future releases. Whereas ATDD Acceptance Tests are defined very early on in the piece, before development starts, Regression Tests involve an existing system. Other than that, the steps involved in defining and automating the tests are very similar.

Now different project stakeholders will be interested in different levels of detail. Some, such as project managers and management in general, will be interested only in which application features work, and which need to be done. Others, such as business analysts and QA, will be interested in the details of how each acceptance scenario is implemented, possible down to the screen level.

Thucydides helps you structure your automated acceptance tests into steps and sub-steps like the ones illustrated above. This tends to make the tests clearer, more flexible and easier to maintain. In addition, when the tests are executed, Thucydides produces illustrated, narrative-style reports like the one in [fig-test-report].

figs/test-report.png
Figure 1. A test report generated by Thucydides

When it comes to implementing the tests themselves, Thucydides also provides many features that make it easier, faster and cleaner to write clear, maintainable tests. This is particularly true for automated web tests using Selenium 2, but Thucydides also caters for non-web tests as well. Thucydides currently works well with JUnit and easyb - integration with other BDD frameworks is in progress.

4. Getting started with Thucydides

4.1. Creating a new Thucydides project

The easiest way to start a new Thucydides project is to use the Maven archetype. Three archetypes are currently available: 1. one for using Thucydides with JUnit, 2. another if you also want to write your acceptance tests (or a part of them) using easyb, and 3. finally one more to help you write acceptance tests in jBehave.

In this section, we will create a new Thucydides project using the Thucydides archetype, and go through the essential features of this project.

From the command line, you can run mvn archetype:generate and then select the net.thucydides.thucydides-easyb-archetype archetype from the proposed list of archetypes. Or you can use your favorite IDE to generate a new Maven project using an archetype.

$ mvn archetype:generate

xxx: remote -> net.thucydides:thucydides-easyb-archetype (Thucydides automated acceptance testing project using Selenium 2, JUnit and easyb)
xxx: remote -> net.thucydides:thucydides-jbehave-archetype (Thucydides automated acceptance testing project using Selenium 2, JUnit and JBehave)
xxx: remote -> net.thucydides:thucydides-simple-archetype (Thucydides automated acceptance testing project using Selenium 2 and JUnit)

$ Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): 359:xxx  ( Enter correct value )
$ Choose a number:xx:xx  ( Enter last project value )
...
Define value for property 'groupId': : com.mycompany
Define value for property 'artifactId': : webtests
Define value for property 'version':  1.0-SNAPSHOT: :
Define value for property 'package':  com.mycompany: :
Confirm properties configuration:
groupId: com.mycompany
artifactId: webtests
version: 1.0-SNAPSHOT
package: com.mycompany
 Y: :
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2:33.290s
[INFO] Finished at: Fri Oct 28 07:20:41 NZDT 2011
[INFO] Final Memory: 7M/81M
[INFO] ------------------------------------------------------------------------

This will create a simple Thucydides project, complete with a Page Object, a Step library and two test cases, one using JUnit, and one using easyb. The actual tests run against the online dictionary at Wiktionary.org. Before going any further, take the project for a spin. First, however, you will need to add the net.thucydides.maven.plugins to your plugin groups in your settings.xml file ( present under maven installation directory ):

<settings>
   <pluginGroups>
       <pluginGroup>net.thucydides.maven.plugins</pluginGroup>
       ...
      </pluginGroups>
  ...
</settings>

This will let you invoke the Maven thucydides plugin from the command line in the short-hand form shown here. Now go into the generated project directory, run the tests and generate the reports:

$ mvn test thucydides:aggregate

This should run some web tests and generate a report in target/site/thucydides directory (open the index.html file).

If you drill down into the individual test reports, you will see an illustrated narrative for each test similar to the one shown in [fig-test-report]

Now for the details. the project directory structure is shown here:

+ src
   + main
      + java
         + com.mycompany.pages
            - DictionaryPage.java

   + test
      + java
         + com.mycompany.pages
            + requirements
               - Application.java
            + steps
               - EndUserSteps.java

      + stories
         + com.mycompany
            - LookupADefinition.story

This project is designed to provide a starting point for your Thucydides acceptance tests, and to illustrate some of the basic features. The tests come in two flavors: easyb and JUnit. easyb is a Groovy-based BDD (Behaviour Driven Development) library which works well for this kind of test. The sample easyb story can be found in the LookupADefinition.story file, and looks something like this:

using "thucydides"

thucydides.uses_default_base_url "http://en.wiktionary.org/wiki/Wiktionary:Main_Page"
thucydides.uses_steps_from EndUserSteps
thucydides.tests_story SearchByKeyword

scenario "Looking up the definition of 'apple'", {
    given "the user is on the Wikionary home page", {
        end_user.is_the_home_page()
    }
    when "the end user looks up the definition of the word 'apple'", {
        end_user.looks_for "apple"
    }
    then "they should see the definition of 'apple", {
       end_user.should_see_definition_containing_words "A common, round fruit"
    }
}

A cursory glance at this story will show that it relates a user looking up the definition of the word apple. However only the “what” is expressed at this level – the details are hidden inside the test steps and, further down, inside page objects.

If you prefer pure Java tests, the JUnit equivalent can be found in the LookupADefinitionStoryTest.java file:

@Story(Application.Search.SearchByKeyword.class)
@RunWith(ThucydidesRunner.class)
public class LookupADefinitionStoryTest {

    @Managed(uniqueSession = true)
    public WebDriver webdriver;

    @ManagedPages(defaultUrl = "http://en.wiktionary.org/wiki/Wiktionary:Main_Page")
    public Pages pages;

    @Steps
    public EndUserSteps endUser;

    @Issue("#WIKI-1")
    @Test
    public void looking_up_the_definition_of_apple_should_display_the_corresponding_article() {
        endUser.is_the_home_page();
                endUser.looks_for("apple");
        endUser.should_see_definition_containing_words("A common, round fruit");

    }
}

As you can see, this is a little more technical but still very high level.

The step libraries contain the implementation of each of the steps used in the high-level tests. For complex tests, these steps can in turn call other steps. The step library used in this example can be found in EndUserSteps.java:

public class EndUserSteps extends ScenarioSteps {

        public EndUserSteps(Pages pages) {
                super(pages);
        }

    @Step
    public void searches_by_keyword(String keyword) {
        enters(keyword);
        performs_search();
    }

        @Step
        public void enters(String keyword) {
        onHomePage().enter_keywords(keyword);
        }

    @Step
    public void performs_search() {
        onHomePage().starts_search();
    }

    private HomePage onHomePage() {
        return getPages().currentPageAt(HomePage.class);
    }

    @Step
        public void should_see_article_with_title(String title) {
        assertThat(onHomePage().getTitle(), is(title));
        }

    @Step
    public void is_on_the_wikipedia_home_page() {
        onHomePage().open();
    }
}

Steps appear in the living documentation reports, in a human-readable version of the method name. For example, is_on_the_wikipedia_home_page() will be displayed at "Is on the wikipedia home page".

Step methods can have parameters: these will appear in the reports, so you know what parameters were used with a particular step. You can customize the way a step is displayed by including the text version of the step in the annotation like this:

@Step("a step about a person called {0}, aged {1}")
public void a_customized_step_with_two_parameters(String name, int age) {..}

Page Objects are a way of encapsulating the implementation details about a particular page. Selenium 2 has particularly good support for page objects, and Thucydides leverages this. The sample page object can be found in the HomePage.java class:

@DefaultUrl("http://en.wiktionary.org/wiki/Wiktionary:Main_Page")
public class SearchPage extends PageObject {

              @FindBy(name="search")
        private WebElement searchInput;

        @FindBy(name="go")
        private WebElement searchButton;

        public SearchPage(WebDriver driver) {
                super(driver);
        }

        public void enter_keywords(String keyword) {
                searchInput.sendKeys(keyword);
        }

    public void starts_search() {
        searchButton.click();
    }

    public List<String> getDefinitions() {
        WebElement definitionList = getDriver().findElement(By.tagName("ol"));
        List<WebElement> results = definitionList.findElements(By.tagName("li"));
        return convert(results, new ExtractDefinition());
    }

    class ExtractDefinition implements Converter<WebElement, String> {
        public String convert(WebElement from) {
            return from.getText();
        }
    }
}

The final piece in the puzzle is the Application.java class, which is a way of representing the structure of your requirements in Java form, so that your easyb and JUnit tests can be mapped back to the requirements they are testing:

        public class Application {
            @Feature
            public class Search {
                public class SearchByKeyword {}
                public class SearchByAnimalRelatedKeyword {}
                public class SearchByFoodRelatedKeyword {}
                public class SearchByMultipleKeywords {}
                public class SearchForQuote{}
            }

            @Feature
            public class Backend {
                public class ProcessSales {}
                public class ProcessSubscriptions {}
            }

            @Feature
            public class Contribute {
                public class AddNewArticle {}
                public class EditExistingArticle {}
            }
        }

This is a simple way to get Thucydides to generate the aggregate reports about features and stories. Other possibilities include organizing the directories by capability and feature, or integrating with an external system such as JIRA.

4.2. Setting custom web driver capabilities

You can set custom web driver capabilities by passing a semi-colon separated list of capabilities in the property thucydides.driver.capabilities. For example,

"build:build-1234; max-duration:300; single-window:true; tags:[tag1,tag2,tag3]"

Refer to section Running Thucydides tests from the command line.

In the following sections, we will look at different aspects of writing automated tests with Thucydides in more detail.

5. Writing Acceptance Tests with Thucydides

In this section, we look at the things you need to know to write your acceptance or regression tests using Thucydides in more detail. We will also outline a general approach to writing your web-based acceptance tests that has worked well for us in the past.

  1. Define and organize the requirements or user stories you need to test

  2. Write high level pending tests for the acceptance criteria

  3. Choose a test to implement, and break it into a small (typically between 3 and 7) high-level steps

  4. Implement these steps, either by breaking them down into other steps, or by accessing Page Objects.

  5. Implement any new Page Object methods that you have discovered.

Note

These steps should not been seen as a linear or waterfall-style approach. Indeed, the process is usually quite incremental, with requirements being added to the Application class as they are required, and pending tests being used to defined tests before they are fleshed out.

5.1. Organizing your requirements

To get the most out of automated tests in Thucydides, you need to tell Thucydides which features of your application you are testing in each test. While this step is optional, it is highly recommended.

The current version of Thucydides uses a simple, three-level organization to structure acceptance tests into more manageable chunks. At the highest level, an application is broken into features, which is a high-level functionality or group of related functions. A feature contains a number of stories (corresponding to user stories, use cases, and so on). Each story is validated by a number of examples, or acceptance criteria, which are automated in the form of web tests (sometimes called scenarios). Each test, in turn, is implemented using a number of steps.

Of course this structure and these terms are merely a convenience to allow a higher-level vision of your acceptance tests. However, this sort of three-level abstraction seems to be fairly common.

In the current version of Thucydides, you define this structure within the test code, as (very light-weight) Java classes
[Future versions of Thucydides will support other ways of defining your user requirements.]
. This makes it easier to refactor and rename user stories and features within the tests, and gives a central point of reference in the test suite illustrating what features are being tested. A simple example is shown here. The Application class is simply a convenient way of placing the features and user stories in the one file. Features are marked with the @Feature annotation. User stories are declared as inner classes nested inside a @Feature class.

public class Application {

    @Feature
    public class ManageCompanies {
        public class AddNewCompany {}
        public class DeleteCompany {}
        public class ListCompanies {}
    }

    @Feature
    public class ManageCategories {
        public class AddNewCategory {}
        public class ListCategories {}
        public class DeleteCategory {}
    }

    @Feature
    public class ManageTags {
        public class DisplayTagCloud {}
    }

    @Feature
    public class ManageJobs {}

    @Feature
    public class BrowseJobs {
        public class UserLookForJobs {}
        public class UserBrowsesJobTabs {}
    }
}

6. Defining high-level tests

There are two approaches to automated acceptance criteria or regression tests with Thucydides. Both involve implementing the tests as a sequence of very high-level steps, and then fleshing out those steps by drilling down into the details, until you get to the Page Objects. The difference involves the language used to implement the high-level tests. Tools like easyb are more focused on communication with non-developers, and allow high level tests to be expressed more easily in business terms. On the other hand, developers often find it more comfortable to work directly with JUnit, so if communication with non-technical stakeholders is not a high priority, this might be a preferred option.

In the current version of Thucydides, you can write your tests using easyb (for a more BDD-style approach) or in JUnit using Java or another JVM language (Groovy is a popular choice). Other BDD tools will be supported in future versions. We will discuss both here, but you can use whatever you and your team are more comfortable with.

6.1. Defining high-level tests in easyb

Easyb (http://easyb.org) is a Groovy-based BDD tool. It makes it easy to write light-weight stories and scenarios using the classic BDD-style "given-when-then" structure, and then to implement them in Groovy. The Thucydides easyb plugin is designed to make it easy to write Thucydides tests using easyb.

6.1.1. Writing a pending easyb story

In easyb, you write tests (referred to as "scenarios") that, when using Thucydides, correspond to automated acceptance criteria. Tests are grouped into "stories" - each story has it’s own file.

Scenarios are first written as "pending". These are just high-level outlines, describing a set of acceptance criteria for a particular story in a "given-when-then" structure.

When the tests are executed, pending scenarios are skipped. However they appear in the reports, so that you know what features still need to be implemented. An example of how pending scenarios appear in a Thucydides report can be seen in [fig-story-results-pending].

figs/story-results-pending.png
Figure 2. Pending tests are shown with the calendar icon

Here is an example of a pending easyb story using Thucydides:

using "thucydides"

import net.thucydides.demos.jobboard.requirements.Application.ManageCategories.AddNewCategory

thucydides.tests_story AddNewCategory

scenario "The administrator adds a new category to the system",
{
        given "a new category needs to be added to the system"
        when "the administrator adds a new category"
        then "the system should confirm that the category has been created"
        and "the new category should be visible to job seekers"
}

scenario "The administrator adds a category with an existing code to the system",
{
        given "the administrator is on the categories list page"
        when "the user adds a new category with an existing code"
        then "an error message should be displayed"
}

Let’s examine this story piece-by-piece. First, you need to declare that you are using Thucydides. You do this by using the easyb using keyword:

using "thucydides"

This will, among other things, inject the thucydides object into your story context so that you can configure Thucydides to run your story correctly.

Next, you need to tell Thucydides what story you are testing. You do this by referencing one of the story classes you defined earlier. That’s what we are doing here:

import net.thucydides.demos.jobboard.requirements.Application.ManageCategories.AddNewCategory

thucydides.tests_story AddNewCategory

The rest of the easyb story is just a set of regular easyb pending scenarios. For the moment, there is no implementation, so they will appear as "pending" in the reports:

scenario "The administrator adds a new category to the system",
{
        given "a new category needs to be added to the system"
        when "the administrator adds a new category"
        then "the system should confirm that the category has been created"
        and "the new category should be visible to job seekers"
}

scenario "The administrator adds a category with an existing code to the system",
{
        given "the administrator is on the categories list page"
        when "the user adds a new category with an existing code"
        then "an error message should be displayed"
}

You typically declare many pending stories, preferably in collaboration with the product owner or BAs, at the start of an iteration. This lets you get a good picture of what stories need to be implemented in a given iteration, and also an idea of the relative complexity of each story.

6.1.2. Implementing the easyb stories

The next step is to implement your stories. Let’s look at an implemented version of the first of these scenarios:

using "thucydides"

import net.thucydides.demos.jobboard.requirements.Application.ManageCategories.AddNewCategory
import net.thucydides.demos.jobboard.steps.AdministratorSteps
import net.thucydides.demos.jobboard.steps.JobSeekerSteps

thucydides.uses_default_base_url "http://localhost:9000"
thucydides.uses_steps_from AdministratorSteps
thucydides.uses_steps_from JobSeekerSteps
thucydides.tests_story AddNewCategory

def cleanup_database() {
    administrator.deletes_category("Scala Developers");
}

scenario "The administrator adds a new category to the system",
{
    given "a new category needs to be added to the system",
    {
      administrator.logs_in_to_admin_page_if_first_time()
      administrator.opens_categories_list()
    }
    when "the administrator adds a new category",
    {
       administrator.selects_add_category()
       administrator.adds_new_category("Scala Developers","SCALA")
    }
    then "the system should confirm that the category has been created",
    {
        administrator.should_see_confirmation_message "The Category has been created"
    }
    and "the new category should be visible to job seekers",
    {
        job_seeker.opens_jobs_page()
        job_seeker.should_see_job_category "Scala Developers"
    }
}

Again, let’s break this down. In the first section, we import the classes we need to use:

using "thucydides"

import net.thucydides.demos.jobboard.requirements.Application.ManageCategories.AddNewCategory
import net.thucydides.demos.jobboard.steps.AdministratorSteps
import net.thucydides.demos.jobboard.steps.JobSeekerSteps

Next, we declare the default base URL to be used for the tests. Like the equivalent annotation in the JUnit tests, this is used for tests executed from within the IDE, or if no base URL is defined on the command line using the webdriver.base.url parameter.

thucydides.uses_default_base_url "http://localhost:9000"

We also need to declare the test step libraries we will be using. We do this using thucydides.uses_steps_from. This will inject an instance variable into the easyb context for each declared step library. If the step library class name ends in Steps (e.g. JobSeekerSteps), the name of the variable will be the class name less the Steps suffix, converted to lower case and underscores (e.g. "job_seeker"). We will learn more about implementing test step libraries further on.

thucydides.uses_steps_from AdministratorSteps
thucydides.uses_steps_from JobSeekerSteps
thucydides.tests_story AddNewCategory

Finally we implement the scenario. Notice, that since this is Groovy, we can declare fixture methods to help set up and tear down the test environment as required:

def cleanup_database() {
    administrator.deletes_category("Scala Developers");
}

The implementation usually just invokes step methods, as illustrated here:

scenario "The administrator adds a new category to the system",
{
    given "a new category needs to be added to the system",
    {
      administrator.logs_in_to_admin_page_if_first_time()
      administrator.opens_categories_list()
    }
    when "the administrator adds a new category",
    {
       administrator.selects_add_category()
       administrator.adds_new_category("Scala Developers","SCALA")
    }
    then "the system should confirm that the category has been created",
    {
        administrator.should_see_confirmation_message "The Category has been created"
    }
    and "the new category should be visible to job seekers",
    {
        job_seeker.opens_jobs_page()
        job_seeker.should_see_job_category "Scala Developers"
        cleanup_database()
    }
}

6.2. Defining high-level tests in JUnit

Thucydides integrates smoothly with ordinary JUnit 4 tests, using the ThucydidesRunner test runner and a few other specialized annotations. This is one of the easiest ways to start out with Thucydides, and is very well suited for regression testing, where communication and clarification with the various stakeholders is less of a requirement.

Here is an example of a Thucydides JUnit web test:

@RunWith(ThucydidesRunner.class)
@Story(UserLookForJobs.class)
public class LookForJobsStory {

    @Managed
    public WebDriver webdriver;

    @ManagedPages(defaultUrl = "http://localhost:9000")
    public Pages pages;

    @Steps
    public JobSeekerSteps job_seeker;

    @Test
    public void user_looks_for_jobs_by_key_word() {
        job_seeker.opens_jobs_page();
        job_seeker.searches_for_jobs_using("Java");
        job_seeker.should_see_message("No jobs found.");
    }

    @Test
    public void when_no_matching_job_found_should_display_error_message() {
        job_seeker.opens_jobs_page();
        job_seeker.searches_for_jobs_using("unknownJobCriteria");
        job_seeker.should_see_message("No jobs found.");
    }

    @Pending @Test
    public void tags_should_be_displayed_to_help_the_user_find_jobs() {}

    @Pending @Test
    public void the_user_can_list_all_of_the_jobs_for_a_given_tag() {}

    @Pending @Test
    public void the_user_can_see_the_total_number_of_jobs_on_offer() {}

}

Let’s examine this section-by-section. The class starts with the @RunWith annotation, to indicate that this is a Thucydides test. We also use the @Story annotation to indicate which user story (defined as nested classes of the the @Feature classes above) is being tested. This is used to generate the aggregate reports.

@RunWith(ThucydidesRunner.class)
@Story(UserLookForJobs.class)
public class LookForJobsStory {
    ...

Next, come two essential annotations for any web tests. First of all, your test case needs a public Webdriver field, annotated with the @Managed annotation. This enables Thucydides to take care of opening and closing a WebDriver driver for you, and lets Thucydides use this driver in the pages and test steps when the tests are executed:

    @Managed
    public WebDriver webdriver;

The second essential field is an instance of the Pages class, annotated with the @ManagedPages annotation. This is essentially a page factory, that Thucydides uses to provide you with instantiated page objects. The defaultUrl attribute lets you define a URL to use when your pages open, if no other base URL has been defined. This is useful for IDE testing:

    @ManagedPages(defaultUrl = "http://localhost:9000")
    public Pages pages;

Note that these two annotations are only required for web tests. If your Thucydides test does not use web tests, you can safely leave them out.

For high-level acceptance or regression tests, it is a good habit to define the high-level test as a sequence of high-level steps. It will make your tests more readable and easier to maintain if you delegate the implementation details of your test (the "how") to reusable "step" methods. We will discuss how to define these step methods later. However, the minimum you need to do is to define the class where the steps will be defined, using the @Steps annotation. This annotation tells Thucydides to listen to method calls on this object, and (for web tests) to inject the WebDriver instance and the page factory into the Steps class so that they can be used in the step methods.

    @Steps
    public JobSeekerSteps job_seeker;

6.2.1. Pending tests

Tests that contain no steps are considered to be pending. Alternatively, you can force a step to be skipped (and marked as pending) by using the @Pending annotation or the @Ignore annotation. Note that the semantics are slightly different: @Ignore indicates that you are temporarily suspending execution of a test, whereas @Pending means that the test has been specified but not yet implemented. So both these tests will be pending:

@Test
public void administrator_adds_an_existing_company_to_the_system() {}

@Pending @Test
public void administrator_adds_a_company_with_an_existing_code_to_the_system() {
    steps.login_to_admin_page();
    steps.open_companies_list();
    steps.select_add_company();
    // More to come
}

A test is also considered pending if any of the steps used in that test are pending. For a step to be pending, it needs to be annotated with the @Pending annotation.

Junit assumptions

You can use junit assumptions in your tests or step methods to . Steps where the conditions under assumptions fail are marked as PENDING instead of ERROR. Subsequent steps are also marked as PENDING.

@Test
public void administrator_adds_an_existing_company_to_the_system() {}

@Test
public void administrator_adds_a_company_with_an_existing_code_to_the_system() {
    steps.login_to_admin_page();
    steps.open_companies_list();
    steps.select_add_company();
    // More to come
}
...
@Step
public void open_companies_list() {
  Assume.assumeThat(user.role, is("admin"));
  CompaniesListPage page = pages().get(CompanyListPage.class);
  String companieslist = page.getCompaniesList();

}
...

In the above example, if the assumption in step open_companies_list fails, it and all susbequent steps will be marked PENDING.

6.2.2. Running tests in a single browser session

Normally, Thucydides opens a new browser session for each test. This helps ensure that each test is isolated and independent. However, sometimes it is useful to be able to run tests in a single browser session, in particular for performance reasons on read-only screens. You can do this by using the uniqueSession attribute in the @Managed annotation, as shown below. In this case, the browser will open at the start of the test case, and not close until all of the tests have been executed.

@RunWith(ThucydidesRunner.class)
public class OpenStaticDemoPageSample {

    @Managed(uniqueSession=true)
    public WebDriver webdriver;

    @ManagedPages(defaultUrl = "classpath:static-site/index.html")
    public Pages pages;

    @Steps
    public DemoSiteSteps steps;

    @Test
    @Title("The user opens the index page")
    public void the_user_opens_the_page() {
        steps.should_display("A visible title");
    }

    @Test
    @Title("The user selects a value")
    public void the_user_selects_a_value() {
        steps.enter_values("Label 2", true);
        steps.should_have_selected_value("2");
    }

    @Test
    @Title("The user enters different values.")
    public void the_user_opens_another_page() {
        steps.enter_values("Label 3", true);
        steps.do_something();
        steps.should_have_selected_value("3");
    }
}

If you do not need WebDriver support in your test, you can skip the @Managed and @Pages annotations, e.g.

@RunWith(ThucydidesRunner.class)
@Story(Application.Backend.ProcessSales.class)
public class WorkWithBackendTest {

    @Steps
    public BackendSteps backend;

    @Test
    public void when_processing_a_sale_transation() {
        backend.accepts_a_sale_transaction();
        backend.should_the_update_mainframe();
    }
}

6.3. Adding tags to test cases

You can add arbitrary tags to your tests both in junit and easyb. Tags provide context to tests. A tag has two parts - a type and a name. Thucydides reports categorize tests based on the specified tag types.

Tag types are arbitrary and you can add as many types as you wish. By default, a story tag type is automatically added to each test. An example of tags on Thucydides reports is given in [fig-tags-in-reports]

figs/tags-in-reports.png
Figure 3. Tag types appear on top. Each tag type displays the tag names.

6.3.1. Adding tags to junit tests

Tags are added to junit tests using @WithTag annotation. The following will add a tag of type epic with name "Audit".

If no type is defined, the default tag type is assumed to be feature. In other words, the following two tags are equivalent.

@WithTag has an alternative, more concise syntax using a colon (:) to separate the tag type and name. For example,

or,

Multiple tags can be added using @WithTags annotation or it’s shorter cousin - @WithTagValuesOf. For example,

Using @WithTagValuesOf, the above can be written more succinctly as:

6.3.2. Adding tags to easyb tests

Tags can be easily added to easyb stories in the form of thucydides.tests.<type> to the stories. For example,

6.3.3. Filter tests by tags in jUnit

You can filter tests by tag while running Thucydides. This can be achieved by providing a single tag or a comma separated list of tags from command line. If provided, only classes and/or methods with tags in this list will be executed.

Example:

mvn verify -Dtags="iteration:I1"

or

mvn verify -Dtags="color:red,flavor:strawberry"

6.4. Running Thucydides in different browsers

Thucydides supports all browser-based WebDriver drivers, i.e. Firefox, Internet Explorer and Chrome, as well as HTMLUnit. By default, it will use Firefox. However, you can override this option using the webdriver.driver system property. To set this from the command line, you could do the following:

$ mvn test -Dwebdriver.driver=iexplorer

If you are not using Firefox by default, it is also useful to define this variable as a property in your Maven pom.xml file, e.g.

<properties>
    <webdriver.driver>iexplorer</webdriver.driver>
</properties>

For this to work with JUnit, however, you need to pass the webdriver.driver property to JUnit. JUnit runs in a separate JVM, and will not see the system properties defined in the Maven build. To get around this, you need to pass them into JUnit explicitly using the systemPropertyVariables configuration option, e.g.

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.7.1</version>
            <configuration>
                <systemPropertyVariables>
                    <webdriver.driver>${webdriver.driver}</webdriver.driver>
                </systemPropertyVariables>
            </configuration>
        </plugin>

6.4.1. Chrome switches

Thucydides supports chrome.switches system property to define options for the Chrome driver. This lets you set useful chrome options such as "--homepage=about:blank" or "--no-first-run". You can provide any number of options, separated by commas, e.g.:

$mvn verify -Dchrome.switches="homepage=about:blank,--no-first-run"

6.5. Forcing the use of a particular driver in a test case or test

The @Managed annotation also lets you specify what driver you want to use for a particular test case, via the driver attribute. Current supported values are “firefox”, “iexplorer”, “chrome” and “htmlunit”. The driver attribute lets you override the system-level default driver for specific requirements. For example, the following test case will run in Chrome, regardless of the webdriver.driver system property value used:

@RunWith(ThucydidesRunner.class)
@Story(Application.Search.SearchByKeyword.class)
public class SearchByFoodKeywordStoryTest {

    @Managed(uniqueSession = true, driver="chrome")
    public WebDriver webdriver;

    @ManagedPages(defaultUrl = "http://www.google.co.nz")
    public Pages pages;

    @Steps
    public EndUserSteps endUser;

    @Test
    public void searching_by_keyword_pears_should_display_the_corresponding_article() {
        endUser.is_the_google_home_page();
        endUser.enters("pears");
        endUser.starts_search();
        endUser.should_see_article_with_title_containing("Pear");
    }

    @Test
    @WithDriver("firefox")
    public void searching_by_keyword_pineapples_should_display_the_corresponding_article() {
        endUser.is_the_google_home_page();
        endUser.enters("pineapples");
        endUser.starts_search();
        endUser.should_see_article_with_title_containing("Pineapple");
    }
}

In easyb, you can use the uses_driver directive, as shown here:

using "thucydides"
...
thucydides.uses_default_base_url "http://localhost:9000"
thucydides.uses_driver chrome

...

scenario "The administrator adds a new category to the system",
{
    given "a new category needs to be added to the system",
    {
      administrator.logs_in_to_admin_page_if_first_time()
      administrator.opens_categories_list()
    }
    when "the administrator adds a new category",
    {
       administrator.selects_add_category()
       administrator.adds_new_category("Scala Developers","SCALA")
    }
    then "the system should confirm that the category has been created",
    {
        administrator.should_see_confirmation_message "The Category has been created"
    }
    and "the new category should be visible to job seekers",
    {
        job_seeker.opens_jobs_page()
        job_seeker.should_see_job_category "Scala Developers"
    }
}

In JUnit, you can also use the @WithDriver annotation to specify a driver for an individual test. This will override both the system-level driver and the @Managed annotation’s driver attribute, if provided. For example, the following test will always run in Firefox:

    @Test
    @WithDriver("firefox")
    public void searching_by_keyword_pineapples_should_display_the_corresponding_article() {
        endUser.is_the_google_home_page();
        endUser.enters("pineapples");
        endUser.starts_search();
        endUser.should_see_article_with_title_containing("Pineapple");
    }

7. Writing Acceptance Tests with JBehave

Thucydides is an open source library designed to make it easier to define, implement and report on automated acceptance criteria. Until now, Thucydides tests have been implemented using JUnit or easyb. However the most recent version of Thucydides, version 0.9.x, now lets you write your acceptance criteria using the popular JBehave framework.

7.1. JBehave and Thucydides

JBehave is an open source BDD framework originally written by Dan North, the inventor of BDD. It is strongly integrated into the JVM world, and widely used by Java development teams wanting to implement BDD practices in their projects.

In JBehave, you write automate your acceptance criteria by writing test stories and scenarios using the familiar BDD "given-when-then" notation, as shown in the following example:

Scenario: Searching by keyword and category

Given Sally wants to buy some antique stamps for her son
When she looks for ads in the 'Antiques' category containing 'stamps'
Then she should obtain a list of ads related to 'stamps' from the 'Antiques' category

Scenarios like this go in .story files: a story file is designed to contain all the scenarios (acceptence criteria) of a given user story. A story file can also have a narrative section at the top, which gives some background and context about the story being tested:

In order to find the items I am interested in faster
As a buyer
I want to be able to list all the ads with a particular keyword in the description or title.

Scenario: Searching by keyword and category

Given Sally wants to buy some antique stamps for her son
When she looks for ads in the 'Antiques' category containing 'stamps'
Then she should obtain a list of ads related to 'stamps' from the 'Antiques' category

Scenario: Searching by keyword and location

Given Sally wants to buy a puppy for her son
When she looks for ads in the Pets & Animals category containing puppy in New South Wales
Then she should obtain a list of Pets & Animals ads containing the word puppy
  from advertisers in New South Wales

You usually implement a JBehave story using classes and methods written in Java, Groovy or Scala. You implement the story steps using annotated methods to represent the steps in the text scenarios, as shown in the following example:

public class SearchSteps {
    @Given("Sally wants to buy a $gift for her son")
    public void sally_wants_to_buy_a_gift(String gift) {
        // test code
    }

    @When("When she looks for ads in the $category category containing $keyword in $region")
    public void looking_for_an_ad(String category, String keyword, String region){
        // more test code
    }
}

7.2. Working with JBehave and Thucydides

Thucydides and JBehave work well together. Thucydides uses simple conventions to make it easier to get started writing and implementing JBehave stories, and reports on both JBehave and Thucydides steps, which can be seamlessly combined in the same class, or placed in separate classes, depending on your preferences.

To get started, you will need to add the Thucydides JBehave plugin to your project. In Maven, just add the following dependencies to your pom.xml file:

<dependency>
    <groupId>net.thucydides</groupId>
    <artifactId>thucydides-core</artifactId>
    <version>0.9.2</version>
</dependency>
<dependency>
    <groupId>net.thucydides</groupId>
    <artifactId>thucydides-jbehave-plugin</artifactId>
    <version>0.9.0</version>
</dependency>

New versions come out regularly, so be sure to check the Maven Central repository (http://search.maven.org) to know the latest version numbers for each dependency.

7.3. Setting up your project and organizing your directory structure

JBehave is a highly flexible tool. The downside of this is that, out of the box, JBehave requires quite a bit of bootstrap code to get started. Thucydides tries to simplify this process by using a convention-over-configuration approach, which significantly reduces the amount of work needed to get started with your acceptance tests. In fact, you can get away with as little as an empty JUnit test case and a sensibly-organized directory structure for your JBehave stories.

7.3.1. The JUnit test runner

The JBehave tests are run via a JUnit runner. This makes it easier to run the tests both from within an IDE or as part of the build process. All you need to do is to extend the ThucydidesJUnitStories, as shown here:

package net.thucydides.showcase.jbehave;

import net.thucydides.jbehave.ThucydidesJUnitStories;

public class JBehaveTestCase extends ThucydidesJUnitStories {
    public JBehaveTestCase() {}
}

When you run this test, Thucydides will run any JBehave stories that it finds in the default directory location. By convention, it will look for a ‘stories` folder on your classpath, so `src/test/resources/stories’ is a good place to put your story files.

7.3.2. Organizing your requirements

Placing all of your JBehave stories in one directory does not scale well; it is generally better to organize them in a directory structure that groups them in some logical way. In addition, if you structure your requirements well, Thucydides will be able to provide much more meaningful reporting on the test results.

By default, Thucydides supports a simple directory-based convention for organizing your requirements. The standard structure uses three levels: capabilities, features and stories. A story is represented by a JBehave .story file so two directory levels underneath the stories directory will do the trick. An example of this structure is shown below:

+ src
  + test
    + resources
      + stories
        + grow_potatoes                     [a capability]
          + grow_organic_potatoes           [a feature]
            - plant_organic_potatoes.story  [a story]
            - dig_up_organic_potatoes.story [another story]
          + grow_sweet_potatoes             [another feature]
          ...

If you prefer another hierarchy, you can use the thucydides.capability.types system property to override the default convention. For example. if you prefer to organize your requirements in a hierachy consisting of epics, theme and stories, you could set the thucydides.capability.types property to epic,theme (the story level is represented by the .story file).

When you start a project, you will typically have a good idea of the capabilities you intent to implement, and probably some of the main features. If you simply store your .story files in the right directory structure, the Thucydides reports will reflect these requirements, even if no tests have yet been specified for them. This is an excellent way to keep track of project progress. At the start of an iteration, the reports will show all of the requirements to be implemented, even those with no tests defined or implemented yet. As the iteration progresses, more and more acceptance criteria will be implemented, until acceptance criteria have been defined and implemented for all of the requirements that need to be developed.

figs/jbehave-initial-project.png
Figure 4. A Thucyides project using JBehave can organize the stories in an appropriate directory structure

An optional but useful feature of the JBehave story format is the narrative section that can be placed at the start of a story to help provide some more context about that story and the scenarios it contains. This narrative will appear in the Thucydides reports, to help give product owners, testers and other team members more information about the background and motivations behind each story. For example, if you are working on an online classifieds website, you might want users to be able to search ads using keywords. You could describe this functionality with a textual description like this one:

Story: Search for ads by keyword
In order to find the items I am interested in faster
As a buyer
I want to be able to list all the ads with a particular keyword
in the description or title.

However to make the reports more useful still, it is a good idea to document not only the stories, but to also do the same for your higher level requirements. In Thucydides, you can do this by placing a text file called narrative.txt in each of the requirements directories you want to document (see below). These files follow the JBehave/Cucumber convention for writing narratives, with an optional title on the first line, followed by a narrative section started by the keyword Narrative:. For example, for a search feature for an online classifieds web site, you might have a description along the following lines:

Search for online ads

Narrative:
In order to increase sales of advertised articles
As a seller
I want potential buyers to be able to display only the ads for
articles that they might be interested in purchasing.

When you run these stories (without having implemented any actual tests), you will get a report containing lots of pending tests, but more interestingly, a list of the requirements that need to be implemented, even if there are no tests or stories associated with them yet. This makes it easier to plan an iteration: you will initially have a set of requirements with only a few tests, but as the iteration moves forward, you will typically see the requirements fill out with pending and passing acceptance criteria as work progresses.