API Tests Generation

Tests are generated for each service described in a specification.

There are 2 types of tests that may be generated:

• poke tests (using RestApiPokeTestsGenerator)
• scenario tests (using ScenarioTestsGenerator)

You may generate tests with Java API passing an appropriate generator to Helium instance like

new Helium().from(file).processBy(new ScenarioTestsGenerator(file, output))

Poke tests

The aim of these tests is to check deployment status of service methods and their reaction to incorrect parameters passed in a call. We do it checking whether some method requires any parameters or body. And it if does, we generate a test that sends an invalid request without required data, checking for appropriate response code that should point to client error. Also if you specify example of message fields and set useExamples true, we'll use them for generating tests that send requests with those examples and validate correspondence of response structure to your specification.

To illustrate, for such a specification

service {
  type "UserProfile" message {
    name 'string'
    age 'int32'
  }
  get "person/@id" spec {
    parameters {
      full(type: 'bool', examples: ['false'])
    }
    tests {
      pathExample {
        id '123'
      }
    }
  }
}

2 tests will be generated.

1. Test sending request GET person/123 and expecting a client error to be received
2. Test sending request GET person/123?full=false and checking that response is successful and response body is an object with 2 fields: name and age (since both are required)

Scenario Tests

This type of tests allows you to write scenarios and check remote server behavior behind the API. Such tests are more difficult to write, but it's a very easy task for those who are familiar with Groovy language. Here is an example:

service {
  name "Some API"
  
  // Login request specification
  post "user/login" spec {
    body {
      email 'string'
      password 'string'
    }
    response 'UserProfile'
  }

  // Search request specification
  get "venues/search" spec {
    parameters {
      query 'string'
    }
    response 'VanueList'
  }

  tests {
    
    // Login action
    def login = {
      
      // send login request
      def loginResponse = post "user/login" with {
        body { // specify request body: {"email":"[email protected]","password":"secretpassword"}
          email "[email protected]"
          password "secretpassword"
        }
      }
      
      // check that response is successful
      loginResponse.mustSucceed()
      // check whether session key is provided
      assert loginResponse.httpHeaders.containsKey('SESSION-KEY')
      // store session key in a variable named 'SESSION_ID'
      store "SESSION_ID", loginResponse.httpHeaders['SESSION-KEY']
    }

    // describe a scenario: a test will be generated
    // before: login will make login action be executed before the main scenario body
    scenario "search venues", before: login spec { 
      // send search request
      def searchResponse = get "venues/search" with {
        httpHeaders { 'SESSION-KEY' SESSION_ID } // use stored session key
        parameters { query "Cafe" } // specify request URI parameters (?query=Cafe)
      }

      // when response body is accessed response is checked to be successful
      // also body is parsed and check to correspond to the types specification 
      // let's say our expected response is like [{"name":"Promoted cafe"}, {"name":"Another cafe"}]
      assert !searchResponse.body.empty : "Venues not found" // check that we have not an empty array
      assert searchResponse.body[0].title == "Promoted cafe" : "Promoted cafe is not at the first position!"
    }

  }

}

Results of service methods execution are wrapped HTTP responses. You can:

• check their response code like

someResponse.mustSucceed()
someResponse.mustBeClientError()

• access HTTP status code

someResponse.statusCode == 417

• access HTTP headers like

someResponse.httpHeaders['NAME']

• access method result object, which will be pretended with types validation like

someResponse.body

You may write asserts like:

assert !someResponse.body.name.empty : "Name is empty" // will fail for {"name":""}
assert someResponse.body.venues[0].favorite : "Name is empty" // will fail for {"vanues":[{"favorite":false}]}

If assertion condition is not satisfied, scenario will be interrupted. To avoid interruption you may use report command:

report "Received list does not contain any items"

After invoking report, scenario continues its execution, but the corresponding unit test fails adding provided message to the report.

You can store some values in order to share them between before, after, and main actions of scenario. Use

store "NUMBER", 15

to store the value. It may be accessed via the specified name later

assert NUMBER == 15