Overview
Testify is a Java implementation of Semantic Testing specification. For users, Testify provides a common set of primatives that can be learned once and used everywhere to write test cases. For developers, Testify provides primatives and extension points to enable their libraries and frameworks to be testable.
Benefits
- Testify makes it painless to write unit, integration and system tests. Write simple and isolated test cases to verify your code works as expected without worrying about managing test state.
- Build reusable and composable test components that manage their own state.
- We don’t make assumptions about your technology stack, so you can develop and add new features to Testify without rewriting existing code.
Features
- Uniform Annotations for writing Unit, Integration and System Tests
- Managed Test Case Configuration, Isolation, and Execution
- JUnit 4 Testing Framework Support
- JUnit 5 Testing Framework Support
- Pluggable Mocking SPI (
Mockito
andEasyMock
supported) - Pluggable Dependency Injection Framework SPI (
Spring
,HK2
andGuice
supported) - Pluggable Application Framework SPI (
SpringBoot
,Jersey 2
,gRPC
, andSpring Web MVC
supported) - Pluggable Local Resource SPI (
HSQL
,ElasticSearch
,ZooKeeper
, etc supported) - Pluggable Virtual Resource SPI (
Docker
supported) - Pluggable Remote Resource SPI
- Pluggable Server SPI (
Undertow
Supported) - Pluggable Client SPI (
JAX-RS
client supported) - Pluggable Test Configuration and Wring Validation SPI
- Pluggable Test Inspection SPI
- Pluggable Test Reification SPI
Supported Frameworks
- Spring DI Integration Testing
- SpringBoot System Testing
- HK2 DI Integration Testing
- Jersey 2 System Testing
- Google Guice DI Integration Testing
- gRPC System Testing
Getting Started
Configuration Checklist
- Latest release version is 1.0.5
- Take a look at the change log
- Install JDK version 8 and insure
JAVA_HOME
environmental variable is set - Install Maven version 3.1.1 or above
- Install Git version 2.9.3
- (Optional) Install Docker 1.11.2 - 17.05.0~ce
- Insure formal parameter names of constructors and methods are added to compiler generated class files:
1
2
3
4
5
6
7
8
9
10
11
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.8</source>
<target>1.8</target>
<compilerArguments>
<!-- Enable runtime discovery of parameter names -->
<parameters />
</compilerArguments>
</configuration>
</plugin>
- Insure Testify Java Agent is used by the surefire or failsafe plugins:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
<dependencies>
<dependency>
<groupId>org.testifyproject</groupId>
<artifactId>core</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<!--
Enables the referencing of dependencies as properties. For example:
${org.testifyproject:core:jar}
-->
<artifactId>maven-dependency-plugin</artifactId>
<version>${plugin.dependency}</version>
<executions>
<execution>
<id>properties</id>
<goals>
<goal>properties</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>javaagent:${org.testifyproject:core:jar}</argLine>
</configuration>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<argLine>javaagent:${org.testifyproject:core:jar}</argLine>
</configuration>
</plugin>
</plugins>
</build>
JUnit 4
I find the best way to learn is through experiential learning. A number of examples are presented below to help you get acclimated with Testify and how to write effective unit, integration and systems tests. Please note that the examples are simple and intended to present the reader with the basic concepts of Testify. For complete examples refer to the links at the end of each section.
Unit Testing
Create Unit Test Project from Archetype
The easiest way to get started with unit testing with Testify is to create a new project using Testify’s unit test archetype:
1
2
3
mvn archetype:generate \
-DarchetypeGroupId=org.testifyproject.archetypes \
-DarchetypeArtifactId=junit-unittest-archetype
Example Unit Test
Given a CreateGreeting
class with Map
and RandomUUIDSupplier
as its collaborators:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
public class CreateGreeting {
private final Map<UUID, GreetingModel> store;
private final RandomUuidSupplier randomUuidSupplier;
CreateGreeting(Map<UUID, GreetingModel> store, RandomUuidSupplier randomUuidSupplier) {
this.store = store;
this.randomUuidSupplier = randomUuidSupplier;
}
/**
* Create the given greeting.
*
* @param model the greeting model
*/
public void createGreeting(GreetingModel model) {
UUID id = randomUuidSupplier.get();
store.put(id, model);
}
}
The unit test for the CreateGreeting
class would look:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
@RunWith(UnitTest.class)
public class CreateGreetingTest {
@Sut
CreateGreeting sut;
@Fake
Map<UUID, GreetingModel> datastore;
@Fake
RandomUuidSupplier randomUuidSupplier;
@Test
public void givenMapStoreConstructorShouldNotDoWork() {
//Act
CreateGreeting result = new CreateGreeting(datastore, randomUuidSupplier);
//Assert
assertThat(result).isNotNull();
verifyZeroInteractions(datastore, randomUuidSupplier);
}
@Test
public void givenNullCreateGreetingShouldReturn() {
//Arrange
GreetingModel model = null;
//Act
//Note that since we are using a fake Map to store values an NPE will
//not be thrown but some Map implementations do not allow null values
sut.createGreeting(model);
}
@Test
public void givenExistingGreetingUpdateGreetingShouldUpdate() {
//Arrange
GreetingModel model = mock(GreetingModel.class);
UUID id = UUID.fromString("aa216415-1b8e-4ab9-8531-fcbd25d5966f");
given(randomUuidSupplier.get()).willReturn(id);
//Act
sut.createGreeting(model);
//Assert
verify(randomUuidSupplier).get();
verify(datastore).put(id, model);
}
}
@RunWith? @Sut? @Fake? Okaaay!
The first thing you will notice in the above unit test class is that it is annotated with @RunWith(UnitTest.class)
. UnitTest
is a custom Testify JUnit Runner
implementation that configures, wires, verifies and executes the test class.
The next thing you will notice are two annotations, @Sut
and @Fake
. The @Sut
annotation denotes the System Under Test (SUT) and @Fake
denotes the desire to create mock instances of the CreateGreeting
class’s Map<UUID, GreetingEntity> store
and RandomUuidSupplier randomUuidSupplier
collaborators. In the example above the name of the collaborator field in the test and system under test are not important as Testify performs type based matching and falls back to type and name based matching in case there is ambiguity as to which collaborator we wish to be fake and initialize.
If you’re wondering what is going on under the hood, well, @Sut
and @Fake
annotations provide hints to Testify and behind the scene Testify inspects the test class fields as well as SUT class fields and then:
- determines which test field corresponds to which system under test field
- creates a fake instance of
Map
andRandomUuidSupplier
- creates an instance of
CreateGreeting
class and sets itsMap<UUID, GreetingModel> store
andRandomUuidSupplier randomUuidSupplier
collaborators to the fake instance - initializes the test class
sut
,store
andrandomUuidSuppler
fields.
One other key feature of Testify of note is the fact that new instances of CreateGreeting
and its collaborators are created for each test case. This means your tests run in complete isolation and you do not have to worry about managing state between test runs. Everything is taken care of for you so you can focus on writing your test cases and production code not boilerplate code to manage test state.
For complete unit test examples please take a look at: JUnit Unit Test Examples.
Spring Integration Testing
Create Spring Integration Project from Archetype
To get started with Spring integration testing with Testify create a new project using Testify’s JUnit Spring integration test archetype:
1
2
3
mvn archetype:generate \
-DarchetypeGroupId=org.testifyproject.archetypes \
-DarchetypeArtifactId=junit-spring-integrationtest-archetype
Example Spring Integration Test
Given a GetGreeting
service with an GreetingRepository
collaborator:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
@Service
public class GetGreeting {
private final GreetingRepository greetingRepository;
@Autowired
GetGreeting(GreetingRepository greetingRepository) {
this.greetingRepository = greetingRepository;
}
/**
* Get a greeting with the given id.
*
* @param id the greeting id
* @return the optional containing the greeting, empty otherwise
*/
public Optional<GreetingEntity> getGreeting(UUID id) {
return greetingRepository.findById(id);
}
}
and Spring Data GreetingRepository and GreetingEntity:
1
2
3
4
@Repository
public interface GreetingRepository extends PagingAndSortingRepository<GreetingEntity, UUID> {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
@Entity
@Table(name = "Greetings")
@ToString
@EqualsAndHashCode
public class GreetingEntity implements Serializable {
private UUID id;
private String phrase;
public GreetingEntity() {
}
public GreetingEntity(String phrase) {
this.phrase = phrase;
}
public GreetingEntity(UUID id, String phrase) {
this.id = id;
this.phrase = phrase;
}
@Id
@GeneratedValue(generator = "greetingIdGenerator")
@GenericGenerator(name = "greetingIdGenerator", strategy = "uuid2")
@Column(name = "greeting_id", updatable = false, insertable = false)
public UUID getId() {
return id;
}
public void setId(UUID id) {
this.id = id;
}
@Column
public String getPhrase() {
return phrase;
}
public void setPhrase(String phrase) {
this.phrase = phrase;
}
}
and a Spring Java Config:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
@ComponentScan
@Configuration
@EnableJpaRepositories
@EnableTransactionManagement
public class GreetingConfig {
/**
* An in-memory H2 database data source.
*
* @return the data source
*/
@Bean
DataSource productionDataSource() {
PGSimpleDataSource dataSource = new PGSimpleDataSource();
dataSource.setServerName("production.acme.com");
dataSource.setPortNumber(5432);
dataSource.setDatabaseName("postgres");
dataSource.setUser("postgres");
dataSource.setPassword("mysecretpassword");
return dataSource;
}
@Bean
LocalContainerEntityManagerFactoryBean entityManagerFactory(DataSource dataSource) {
LocalContainerEntityManagerFactoryBean bean =
new LocalContainerEntityManagerFactoryBean();
bean.setDataSource(dataSource);
bean.setPersistenceUnitName("example.greetings");
return bean;
}
/**
* Provides JPA based Spring transaction manager.
*
* @param entityManagerFactory the entity manager factory
* @return jpa transaction manager
*/
@Bean
PlatformTransactionManager transactionManager(EntityManagerFactory entityManagerFactory) {
JpaTransactionManager transactionManager =
new JpaTransactionManager(entityManagerFactory);
return transactionManager;
}
}
The Spring integration test for the GetGreeting
service would look:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
@Module(GreetingConfig.class)
@LocalResource(InMemoryHSQLResource.class)
@RunWith(IntegrationTest.class)
public class GetGreetingIT {
@Sut
GetGreeting sut;
@Real
GreetingRepository greetingRepository;
@Test(expected = InvalidDataAccessApiUsageException.class)
public void givenNullGetGreetingShouldThrowException() {
//Arrange
UUID id = null;
//Act
sut.getGreeting(id);
}
@Test
public void givenNoneExistentKeyGetGreetingShouldReturnAnEmptyOptional() {
//Arrange
UUID id = UUID.fromString("aa216415-1b8e-4ab9-8531-fcbd25d5966f");
//Act
Optional<GreetingEntity> result = sut.getGreeting(id);
//Assert
assertThat(result).isEmpty();
}
@Test
public void givenExistentKeyGetGreetingShouldReturnGreetingEntity() {
//Arrange
UUID id = UUID.fromString("0d216415-1b8e-4ab9-8531-fcbd25d5966f");
//Act
Optional<GreetingEntity> result = sut.getGreeting(id);
//Assert
assertThat(result).isPresent();
}
}
IntegrationTest? @Module? @LocalResource? @Real? Whaaah?!
IntegrationTest
is similar to UnitTest
test runner in that it configures, wires, verifies and executes the test class.
Since this is an integration test we want to verify proper wiring and integration between services and their collaborators. Usually this involves loading a module that defines services we want to test and ideally working with real services.
In above GetGreetingTest
test class the GetGreeting
service and its GreetingRepository
collaborator are discovered by the Spring Java Config class GreetingConfig
using @ComponentScan
annotation. We let Testify know this by annotating the test class with @Module(GreetingConfig.class)
. Behind the scenes Testify creates a new Spring ApplicationContext
, register the GreetingConfig
configuration class and insures that only the service we are testing and its collaborators are initialized.
Since our GetGreeting
service has GreetingRepository
, a Spring Data repository, as a collaborator we need a SQL database to test the service. This is where @LocalResource
annotation can help. We simply annotate the test class with @LocalResource(InMemoryHSQLResource.class)
and Testify takes care of creating an in-memory HSQL database and making it available for testing. Note that all InMemoryHSQLResource
does is provide an in-memory java.sql.DataSource
and java.sql.Connection
which are used to replace all references to DataSource
and Connection
in the Spring application context.
We have already seen @Sut
annotation in action in the unit test example. In the context of a Spring integration test it serves a similar purpose, to let Testify know the field represents the system under test. In this example it happens to be the GetGreeting
service.
As mentioned earlier we typically want to work with real instance of collaborators when writing integration tests and so we annotate the greetingRepository
field with @Real
to let the framework know that we want the real instance of the GreetingRepository
collaborator managed by Spring. Of course, there are times when you do not want to use real instance of collaborators (i.e. credit card processing service), and in those instances you can annotate the collaborator with @Fake
to work with a fake instance (a mock instance) of the collaborator.
What if you want to use the real instance of GetGreeting's
collaborators but want to mock or verify certain methods of the collaborator? Well, you can do that too. You just need to annotate the greetingRepository
field with @Virtual
annotation to work with a delegated mock instance of the GreetingRepository
service that will allow you to mock certain methods and delegate others to the real instance of the collaborator in the Spring application context.
If you are curious about what is going on behind the scenes, Testify:
- inspects the test and system under test fields
- creates a Spring application context and loads the
GreetingConfig
module - configures and starts test resources
- retrieves an instance of the
GetGreeting
services from the application context - initializes the test class’s
sut
andgreetingRepository
fields
As with unit tests you do not have to worry about managing test state. Every Spring integration test case runs in complete isolation and Testify takes care of managing the Spring application context and all scaffolding.
For complete Spring integration test examples please take a look at: Spring JUnit Integration Test Examples
Spring Boot System Testing
Please note that this example requires that you have Docker installed and Docker Remote API enabled. See the Virtual Resources section for details on how to install and configure Docker.
Create Spring Boot Project from Archetype
To get started with Spring Boot system testing create a new project using Testify’s Spring Boot system test archetype:
1
2
3
mvn archetype:generate \
-DarchetypeGroupId=org.testifyproject.archetypes \
-DarchetypeArtifactId=junit-springboot-systemtest-archetype
Example Spring Boot System Test
Given the following Spring Boot Application:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
@SpringBootApplication
public class GreetingApplication {
public static void main(String[] args) throws Exception {
GreetingApplication application = new GreetingApplication();
application.run(args);
}
public void run(String[] args) {
SpringApplication.run(GreetingApplication.class, args);
}
@Bean
DataSource productionDataSource() {
PGSimpleDataSource dataSource = new PGSimpleDataSource();
dataSource.setServerName("production.acme.com");
dataSource.setPortNumber(5432);
dataSource.setDatabaseName("postgres");
dataSource.setUser("postgres");
dataSource.setPassword("mysecretpassword");
return dataSource;
}
@Bean
LocalContainerEntityManagerFactoryBean entityManagerFactory(
EntityManagerFactoryBuilder builder, DataSource dataSource) {
Map<String, Object> properties = new HashMap<>();
properties.put(DATASOURCE, dataSource);
return builder.dataSource(dataSource)
.persistenceUnit("example.greeter")
.properties(properties)
.build();
}
@Bean
ModelMapper modelMapper() {
ModelMapper mapper = new ModelMapper();
Configuration configuration = mapper.getConfiguration();
configuration.setMatchingStrategy(MatchingStrategies.STRICT);
configuration.setFieldAccessLevel(Configuration.AccessLevel.PUBLIC);
configuration.setMethodAccessLevel(Configuration.AccessLevel.PUBLIC);
configuration.setAmbiguityIgnored(false);
configuration.setDestinationNamingConvention(NamingConventions.JAVABEANS_MUTATOR);
configuration.setSourceNamingConvention(NamingConventions.JAVABEANS_ACCESSOR);
return mapper;
}
}
and Spring Data GreetingRepository and GreetingEntity:
1
2
3
4
@Repository
public interface GreetingRepository extends PagingAndSortingRepository<GreetingEntity, UUID> {
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
@Entity
@Table(name = "Greetings")
@ToString
@EqualsAndHashCode
public class GreetingEntity implements Serializable {
private UUID id;
private String phrase;
public GreetingEntity() {
}
public GreetingEntity(String phrase) {
this.phrase = phrase;
}
@Id
@GeneratedValue(generator = "greetingIdGenerator")
@GenericGenerator(name = "greetingIdGenerator", strategy = "uuid2")
@Column(name = "greeting_id", updatable = false, insertable = false)
public UUID getId() {
return id;
}
@JsonProperty
public void setId(UUID id) {
this.id = id;
}
@SafeHtml
@NotNull
@Column
public String getPhrase() {
return phrase;
}
public void setPhrase(String phrase) {
this.phrase = phrase;
}
}
and a Spring REST controller:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
@RestController
public class ListGreetingsResource {
private final GreetingRepository greetingRepository;
@Autowired
ListGreetingsResource(GreetingRepository greetingRepository) {
this.greetingRepository = greetingRepository;
}
@RequestMapping(
path = "/greetings/list",
method = RequestMethod.GET,
produces = MediaType.APPLICATION_JSON_VALUE)
@ResponseStatus(HttpStatus.OK)
public Iterable<GreetingEntity> listGreetings() {
return greetingRepository.findAll();
}
}
The Spring Boot system test for the ListGreetingsResource
above would look:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
@Application(GreetingApplication.class)
@Module(TestModule.class)
@VirtualResource(value = "postgres", version = "9.4")
@RunWith(SystemTest.class)
public class ListGreetingsResourceST {
@Sut
ClientInstance<WebTarget, Client> sut;
@ConfigHandler
void configureClient(ClientBuilder clientBuilder) {
clientBuilder.register(JacksonFeature.class);
}
@Test
public void callToListGreetingsShouldReturnGreetings() {
//Act
Response response = sut.getClient().getValue()
.path("greetings")
.path("list")
.request()
.get();
//Assert
assertThat(response.getStatus()).isEqualTo(OK.getStatusCode());
GenericType<List<GreetingEntity>> genericType =
new GenericType<List<GreetingEntity>>() {
};
List<GreetingEntity> result = response.readEntity(genericType);
assertThat(result).hasSize(1);
}
}
SystemTest? @Application? @VirtualResource? ClientInstance? @ConfigHandler? Oh my!
Once again you will note that we are running the test with SystemTest
test runner. It is similar to UnitTest
and IntegrationTest
that we saw earlier in that it also configures, wires, verifies and executes the test class.
Since this is a system test we want to verify our application works from the client perspective. This means we load and start the Spring Boot application and start communicating with the application via HTTP. In the above code we are using Jersey Client implementation of JAX-RS to call /greetings/list
endpoint.
In ListGreetingsResourceST
example above we load the the application by annotating the test class with @Application(GreetingApplication.class)
. Behind the scenes Testify starts the Spring Boot application and then creates a client instance that is aware of the base URI and port of the servlet container the application is
deployed to. Finally, it injects an instance of ClientInstance
into the test class so that it can be used to call the application’s endpoints from the client’s perspective.
Our ListGreetingsResource
has GreetingRepository
, a Spring data repository, as a collaborator. This means we need a SQL database, preferably our production database, to test the ListGreetingsResource
. This is where @VirtualResource
annotation and virtual resources shines. We simply annotate the test class with @VirtualResource(value = "postgres", version = "9.4")
and Testify takes care of pulling and starting postgres resource (in this instance from Docker registry) and making it available for testing. To start testing with virtual resources you will need to:
- install and configure Docker
- create the
TestModule
and annotate your test class with@Module(TestModule.class)
:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
@Configuration
public class TestModule {
/**
* Create a datasource that takes precedence (@Primary) over the production datasource that
* points to the postgres in the container resource.
*
* @param inetAddress the container address.
* @return the test data source
*/
@Primary
@Bean
DataSource testDataSource(
@Qualifier("resource:/postgres:9.4/resource") InetAddress inetAddress) {
PGSimpleDataSource dataSource = new PGSimpleDataSource();
dataSource.setServerName(inetAddress.getHostAddress());
dataSource.setPortNumber(5432);
//Default postgres image database name, user and postword
dataSource.setDatabaseName("postgres");
dataSource.setUser("postgres");
dataSource.setPassword("mysecretpassword");
return dataSource;
}
/**
* Create and configure a test entity manager bean factory.
*
* @param builder the entity manager builder
* @param dataSource the test data source
* @param applicationContext the application context
* @return an entity manager bean factory
*/
@Primary
@Bean
LocalContainerEntityManagerFactoryBean testEntityManagerFactory(
EntityManagerFactoryBuilder builder,
DataSource dataSource,
ApplicationContext applicationContext) {
Map<String, Object> properties = new HashMap<>();
properties.put(DATASOURCE, dataSource);
properties.put("hibernate.ejb.entitymanager_factory_name", applicationContext.getId());
return builder.dataSource(dataSource)
.persistenceUnit("test.example.greeter")
.properties(properties)
.build();
}
}
Note that the above TestModule
simply creates a new DataSource
based on the virtual resource our test requires via @VirtualResource
annotation. In our ListGreetingsResourceST
test class we requested a postgres database image and so Testify creates a InetAddress
whose name is derived from the virtual resource name and makes it available for injection. We can further configure the underlying client used by the ClientInstance
by adding a method annotated with @ConfigHandler
to the test class which takes client configuration object as a parameter. In this example we happen to be using the Jersey 2 JAX-RS client implementation and so we pass in a ClientBuilder
instance. If you are curious about what is going on behind the scenes, Testify:
- creates a proxy sub-class of
GreetingApplication
- intercepts and stores the application’s Spring application context so that all of its beans are eligible for injection
- intercepts and alters the application’s port to point to a random port
- intercepts and stores the base URL of the server
- configures and starts the virtual resource and adds
InetAddress
with the virtual resource’s name - creates a
ClientInstace
instance that points to the server and add it to the application’s Spring application context - initializes the test class fields annotated with @Sut
As with the other testing levels you do not have to worry about managing test state. Testify will configure, start and inject a client instance before each test and manage the clean up process.
One other feature of note is that Testify supports “In-Container” system testing for Spring. This means you can inject any Spring managed beans in the deployed application into your test class and execute systems much like integration tests. Please note that the use of this feature is highly discouraged due to the fact that you are not performing system testing from the client’s perspective.
For complete Spring Boot system test examples please take a look at Spring Boot JUnit System Test Examples repository.
Spring MVC system test examples can be found in Spring MVC JUnit System Test Examples repository.
HK2 Integration Testing
Create HK2 Project from Archetype
Testify also supports integration testing of HK2 modules and services. To get started create a new project using Testify’s HK2 integration test archetype:
1
2
3
mvn archetype:generate \
-DarchetypeGroupId=org.testifyproject.archetypes \
-DarchetypeArtifactId=junit-hk2-integrationtest-archetype
Example HK2 Integration Tests
From integration testing perspective, testing HK2 modules is similar to integration testing Spring modules. For examples please take a look at HK2 JUnit Integration Test Examples repository.
Jersey 2 System Testing
Create Jersey 2 Project from Archetype
Jersey 2 applications system testing is also supported. To get started create a new project using Testify’s Jersey 2 system test archetype:
1
2
3
mvn archetype:generate \
-DarchetypeGroupId=org.testifyproject.archetypes \
-DarchetypeArtifactId=junit-jersey-systemtest-archetype
Example Jersey 2 System Tests
Testing Jersey 2 application is eerily similar to system testing Spring Boot applications. For examples please take a look at Jersey 2 JUnit System Test Examples repository.
Google Guice Integration Testing
Create Google Guice Project from Archetype
Testify also supports integration testing of Google Guice modules. To get started create a new project using Testify’s Guice integration test archetype:
1
2
3
mvn archetype:generate \
-DarchetypeGroupId=org.testifyproject.archetypes \
-DarchetypeArtifactId=junit-guice-integrationtest-archetype
Example Google Guice Integration Tests
From integration testing perspective, testing Guice modules is similar to integration testing Spring and HK2 modules. For examples please take a look at Guice JUnit Integration Test Examples repository.
gRPC System Testing
Create gRPC Project from Archetype
gRPC applications system testing is also supported. To get started create a new project using Testify’s gRPC system test archetype:
1
2
3
mvn archetype:generate \
-DarchetypeGroupId=org.testifyproject.archetypes \
-DarchetypeArtifactId=junit-grpc-systemtest-archetype
Example gRPC System Tests
Testing gRPC application is eerily similar to system testing Spring Boot applications. For examples please take a look at gRPC JUnit System Test Examples repository.
Local Resources
A local resource is an asset that is managed locally and similar to a deployment environment resource which be drawn on to effectively test assumptions. The easiest way to get started with local resources is to create a local resource provider implementation. In this example we will create an in-memory HSQLDB local resource. Using Testify’s resource provider archetype create a new project:
1
2
3
mvn archetype:generate \
-DarchetypeGroupId=org.testifyproject.archetypes \
-DarchetypeArtifactId=junit-resourceprovider-archetype
Example HSQL LocalResourceProvider
To create a hsql resource provider that can be used in your test cases simply implement the LocalResourceProvider SPI contract:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
public class InMemoryHSQLResource implements
LocalResourceProvider<JDBCDataSource, DataSource, Connection> {
private JDBCDataSource server;
private Connection client;
@Override
public JDBCDataSource configure(TestContext testContext,
LocalResource localResource,
PropertiesReader configReader) {
JDBCDataSource dataSource = new JDBCDataSource();
String url = format("jdbc:hsqldb:mem:%s?default_schema=public", testContext.getName());
dataSource.setUrl(url);
dataSource.setUser("sa");
dataSource.setPassword("");
return dataSource;
}
@Override
public LocalResourceInstance<DataSource, Connection> start(TestContext testContext,
LocalResource localResource,
JDBCDataSource dataSource)
throws Exception {
server = dataSource;
client = dataSource.getConnection();
return LocalResourceInstanceBuilder.builder()
.resource(server, DataSource.class)
.client(client, Connection.class)
.build("hsql", localResource);
}
@Override
public void stop(TestContext testContext,
LocalResource localResource,
LocalResourceInstance<DataSource, Connection> instance)
throws Exception {
server.getConnection()
.createStatement()
.executeQuery("SHUTDOWN");
client.close();
}
}
LocalResourceProvider? Hmmm…
In the above implementation of LocalResourceProvider
contract we are creating a reusable HSQL in-memory database resource provider. Notice that our implementation:
- Specifies 3 type parameters of
LocalResourceProvider
contract, JDBCDataSource (resource configuration), DataSource (the resource), and Connection (resource client) - Does not declare a constructor
- Implements three methods defined by the contract (configure, start, and stop)
- Has state (server and client instances)
Our example is pretty simple because our configuration object and resource object are the same but this may not be the case for a more complex implementations. Regardless of the complexity of an implementation the ultimate goal is to provide a managed, reusable, and disposable resource for our integration and system tests and to enable us to replace production code that relies on the resource (DataSource) and client (Connection) with the ones provided by our LocalResourceProvider
implementation.
As you might guess the configure
method is responsible for creating a pre-configured configuration object that can be refined further via @ConfigHander
prior to starting the resource and execution of our test case. The start
method is responsible for starting the resource and returning a LocalResourceInstance
that encapsulates a resource component and a client component which is configured to communicate with the resource. In this example our resource instance happens to comprises of an in-memory HSQL JDBC DataSource
and Connection
. Finally, the stop method is responsible for gracefully stopping the resource and closing the client.
Now that we have a resource provider implementation we can start using it to provide a database resource for our integration and system tests by annotating our test class with @LocalResource(InMemoryHSQLResource.class)
:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
@Module(GreetingConfig.class)
@LocalResource(InMemoryHSQLResource.class)
@RunWith(SpringIntegrationTest.class)
public class GetGreetingIT {
@Sut
GetGreeting cut;
@Real
GreetingRepository greetingRepository;
@Real
ResourceInstance<DataSource, Connection> resourceInstance;
@Real
DataSource dataSource;
@Real
Connection connection;
@ConfigHandler
void configure(JDBCDataSource dataSource) {
//XXX: You can refine the configuration object of the
//InMemoryHSQLResource implementation
}
@Test
public void givenExistentKeyGetGreetingShouldReturnGreetingEntity() {
//Arrange
UUID id = UUID.fromString("0d216415-1b8e-4ab9-8531-fcbd25d5966f");
//Act
Optional<GreetingEntity> result = cut.getGreeting(id);
//Assert
assertThat(result).isPresent();
}
}
Notice that we are able to inject the resource instance, data source and connection created in our InMemoryHSQLResource
resource provider implementation which can be useful if you wish to directly interact with the resource and the client. Please note that the above example is kitchen-sink example and may not reflect a typical use-case.
For complete LocalResourceProvider implementations and examples take a look at: Example JUnit Resource Provider Local Resource Implementations
Virtual Resources
A virtual resource is disposable asset similar to a deployment environment resource that can be drawn on to effectively test assumptions.
Docker Virtual Resource
Testify provides support for virtual resources based on Docker Containers. Before you can use virtual resources in your integration and system tests you will need to install and configure Docker on your system.
Install Docker
To install Docker please follow the Docker instalation instructions for your OS platform.
Configuring Docker
Testify uses Docker Remote API to pull images and manage Docker containers. By default Docker Remote API is not enabled. To enable Docker Remote API follow the bellow instructions for your OS platform.
Ubuntu 14.04 / Mint 17
- After you install add yourself to the
docker
group and reboot your system:
1
2
3
4
sudo -s -- <<EOC
usermod -a -G docker $USER
reboot
EOC
- Backup any existing docker
daemon.json
configuration file and create a new one that enables local and remote docker API hosts:
1
2
3
4
5
6
7
8
9
10
11
12
13
sudo -s -- <<EOC
mkdir -p /etc/docker
mv /etc/docker/daemon.json "/etc/docker/daemon.json.$(date -d "today" +"%Y%m%d%H%M")" \
2>/dev/null
tee -a /etc/docker/daemon.json >/dev/null <<'EOF'
{
"hosts": [
"unix:///var/run/docker.sock",
"tcp://127.0.0.1:2375"
]
}
EOF
EOC
- Since we are using a
daemon.json
file to configure the docker daemon we need to insure that daemon is not configured by our system docker service file:
1
sudo sed -i 's/ExecStart=.*/ExecStart=\/usr\/bin\/dockerd/g' /lib/systemd/system/docker.service
- Restart the docker service:
1
sudo service docker restart
- Test that the remote api is working:
1
docker -H tcp://127.0.0.1:2375 ps
Ubuntut 16.04 / Mint 18 / Debian 8
- After you install add yourself to the
docker
group and reboot your system:
1
2
3
4
sudo -s -- <<EOC
usermod -a -G docker $USER
reboot
EOC
- Backup any existing docker
daemon.json
configuration file and create a new one that enables local and remote docker API hosts:
1
2
3
4
5
6
7
8
9
10
11
12
13
sudo -s -- <<EOC
mkdir -p /etc/docker
mv /etc/docker/daemon.json "/etc/docker/daemon.json.$(date -d "today" +"%Y%m%d%H%M")" \
2>/dev/null
tee -a /etc/docker/daemon.json >/dev/null <<'EOF'
{
"hosts": [
"fd://",
"tcp://127.0.0.1:2375"
]
}
EOF
EOC
- Since we are using a
daemon.json
file to configure the docker daemon we need to insure that daemon is not configured by the system docker service file:
1
sudo sed -i 's/ExecStart=.*/ExecStart=\/usr\/bin\/dockerd/g' /lib/systemd/system/docker.service
- Restart the docker service:
1
2
3
4
sudo -s -- <<EOC
systemctl daemon-reload
systemctl restart docker
EOC
- Test that the remote api is working:
1
docker -H tcp://127.0.0.1:2375 ps
Fedora 24 / CentOS 7 / RHEL 7 / Oracle Linux 7
- After you install add yourself to the
docker
group and reboot your system:
1
2
3
4
sudo -s -- <<EOC
usermod -a -G docker $USER
reboot
EOC
- Backup any existing docker
daemon.json
configuration file and create a new one that enables local and remote docker API hosts:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
sudo -s -- <<EOC
mkdir -p /etc/docker
mv /etc/docker/daemon.json "/etc/docker/daemon.json.$(date -d "today" +"%Y%m%d%H%M")" \
2>/dev/null
tee -a /etc/docker/daemon.json >/dev/null <<'EOF'
{
"storage-driver": "devicemapper",
"hosts": [
"unix:///var/run/docker.sock",
"tcp://127.0.0.1:2375"
]
}
EOF
EOC
- Since we are using a
daemon.json
file to configure the docker daemon we need to insure that daemon is not configured by our system docker service file:
1
sudo sed -i 's/ExecStart=.*/ExecStart=\/usr\/bin\/dockerd/g' /lib/systemd/system/docker.service
- Restart the docker service:
1
2
3
4
sudo -s -- <<EOC
systemctl daemon-reload
systemctl restart docker
EOC
- Test that the remote api is working:
1
docker -H tcp://127.0.0.1:2375 ps
Windows
- Follow Docker installation and configuration instructions on Microsoft Windows Containers site.
- Insure
c:\ProgramData\docker\config\daemon.json
file defineshosts
value:
1
2
3
{
"hosts": ["tcp://127.0.0.1:2375"]
}
- Restart Docker:
1
Restart-Service docker
Mac OS X
TODO
Docker Remote API and Testify
By default Testify communicates with Docker through Docker Remote API on
non-secure http://127.0.0.1:2375
URL endpoint. If you wish to use an endpoint
with a different IP address and port or are using a Docker-Machine, or want to
use secure-communication then you will need to explictly configure the Docker
Client used by Testify using @ConfigHandler
(not recommended):
1
2
3
4
5
@Config
public void configure(DefaultDockerClient.Builder builder) {
builder.uri("http://192.168.99.100:2375");
//add additional configuration such as registry configuration here
}
Glossary
Annotations
@Sut
An annotation used on single test class field to denote the field type as the system under test (SUT).
@Fake
An annotation that can be placed on unit, integration and system test class field to denote the field as a fake collaborator. A fake collaborator is a mock instance of the collaborator that allows us to mock functionality and verify interaction between the system under test and the fake collaborator in isolation. Note that if the value of the test class fake field is already initialized with:
- a mock instance of the collaborator then this mock instance will be used and injected into the system under test.
- the real instance of the collaborator then a mock instance that delegates to the real instance will be created and injected into the system under test.
@Real
An annotation that can be placed on integration and system test class fields to denote the field as a real collaborator of the system under test.
@Virtual
An annotation that can be placed on unit, integration and system test class field to denote the field as a virtual collaborator. A virtual collaborator is a mock instances that delegate to a real instance of the collaborator of the system under test and are useful if you wish to mock certain functionality and delegate others to the real collaborator instance.
@CollaboratorProvider
An annotation used to specifying a provider of the system under test’s collaborators. This annotation can be placed on a test class or method. This is useful for configuring a system under test whose collaborator(s) can not be faked or virtualized (i.e. a java.net.URL
collaborator which is a final class). Note that if this annotation is:
- placed on a test class method then this method will be called to provide the system under test's collaborators.
- placed on the test class and `CollaboratorProvider#value()` is specified then a method within `CollaboratorProvider#value()` class will be called to provide collaborators for the system under test.
@ConfigHandler
An annotation that can be placed on a test class or test class method to configure various functions before each integration and system test run (i.e. HK2 ServiceLocator, Spring Application Context, required resources, etc). Note that if the annotation is placed on:
- a test class method then the method will be called to perform pre-test run configuration.
- the test class and `ConfigHandler#value()` is specified then the the appropriate configuration method in the `ConfigHandler#value()` class will be called to perform pre-test run configuration.
@Module
An annotation that can be placed on integration and system tests to load a module that contains services before each test run (i.e. Spring’s Java Config, HK2’s AbstractBinder, or Guice’s AbstractModule).
@Scan
An annotation that can be placed on integration and system tests to load a resources that contains services before each test run (i.e. Spring service fully qualified package name, HK2 service locator descriptor classpath).
@Application
An annotation that can be placed on system tests to specify an application (i.e. Jersey 2, Spring Boot, Spring MVC, etc) that should be loaded, configured, started, and stopped before and after each test run.
@LocalResource
An annotation that can be placed on integration and system tests to specify local resources that must be loaded, configured, started, stopped before and after each test case (i.e. an in-memory database). Note that a typical local resource consists of a resource component and resource client component (optional). For example, if a test class requires an in-memory database then the database javax.sql.DataSource
can be thought of as the resource component and the java.sql.Connection
to the DataSource as the resource client component.
@VirtualResource
An annotation that can be placed on integration and system tests to specify virtual resources that should be loaded, configured, started, stopped before and after each test run. This is useful when performing integration and system tests using real production environment (i.e. a real PostgresSQL database or Cassandra cluster).
@RemoteResource
An annotation that can be placed on integration and system tests to specify remote resources that should be loaded, configured, started, stopped before and after each test run. This is useful when performing integration and system tests using real cloud and third party production or sandbox services (i.e. a AWS SQS, GitHub API, Stripe API, etc).
@Fixture
An annotation that can be placed on test field to denote them as test fixture. When placed on a test class field then the value of the field can be initialized or destroyed using Fixture#init()
and Fixture#destroy()
attribute before and after each test run.
@Bundle
A meta-annotation that identifies an annotation as test group. A test group annotation provide the ability to define, group, and use one or more Testify annotations in a reusable manner in your test classes and avoid annotation bloat.
@Discoverable
An annotation that can be placed on service implementation classes to add entries for the service to the META-INF/services
directory and allow the service to be discoverable through the JDK’s java.util.ServiceLoader
service-provider loading facility.
@Name
An annotation that can be placed on field, method, or method parameter to give it a unique name to distinguish it from similar fields, methods or parameters.
@Property
An annotation that can be placed on test class field to inject runtime/configuration properties into a test class.
@Hint
An annotation that can be placed a test class to provide hints to the test class. This is useful when being explicit is necessary due to the presence of multiple implementations of discoverable contracts in the classpath.
FAQ
Why do we need yet another Java Testing Framework?
The intent of this project is not to add yet another testing framework to the Java eco-system or simply re-invent the wheel but to fill a void. There are numerous testing framework out there but none that allow you to write Unit, Integration and System tests quickly and intuitively using the same framework. Testify is built on testing best practices and we feel confident that it will help as you write production and test code. Ultimately we hope you find Testify enables you to be more productive by saving you time.
Who started the project?
Testify Project was started by Sharmarke Aden late 2015 because he was not satisfied with the landscape of Jave Testing Framework. Having written HK2 Testing Framework, contributed to Cargo, and used various testing frameworks (Spring Testing Framework, Arquillian, Selenium, etc), and seen countless of ineffective “unit”, “integration”, and “system” tests Sharmarke decided there has to be a better way.
Do you intend to support my favorite XYZ Framework?
Testify is designed to be modular and extensible. Adding support for a new dependency injection or application framework or any feature for that matter is a fairly straight forward process. Having said that, work has to be prioritized. We will seriously consider every request and if your request is aligned with Testify’s mission and vision we will add it. We am also very open to and would love to have contributions from the community. Please submit a pull request for features and fixes you would like to see in Testify.
Can my Test Class have multiple Resources?
Absolutely! @LocalResource
, @VirtualResource
, @RemoteResource
annotations are repeatable and your test class can require as many resources as it needs ;)
Can my Test Class have multiple Modules
Yes. @Module
annotation is repeatable. Having said that, it is recommended that your test class only import one single module to limit the scope of the test class and maintain modularity. If you find that you need many modules you should think about refactoring.
Isn’t what you call fake really a mock? Why not call it mock?
It is true that there is a distinction between a fake and a mock. Testify indeed conflates the two as described by Martin Flowers. Testify considered and in fact did use @Mock
at inception but a decision was made to change it to @Fake
for the following reason:
- Testify has adopted Semantic Testing and takes systems thinking approach to thinking about what we call collaborators.
- Many testing frameworks (i.e. Mockito, Powermock, etc) already have
@Mock
annotations and developers will have to make a conscious decision as to which annotation to import. This can lead to a number of issues that can slow down development. @Fake
is business people and novice friendly and we want to be as friendly as possible to a wider audience.@Fake
fits nicely with@Real
and@Virtual
lexicon. Collaborators are either fake, real, or virtual (delegated mock) and we believe that these concepts are easier to explain, understand and reason about.- Using
@Fake
is in-line with Testify’s mission of being an easy to understand and easy to use testing framework.
Docker Container based testing is eating my disk space? How do I reclaim disk space?
- Testify tries to delete containers after each test but sometimes may not be able (i.ie the JVM is terminated in the middle of a test case). You can manually clean up the exited containers using the following command:
1
docker rm -v $(docker ps -a -q -f status=exited)
- Docker keeps all images it pulls from the Docker registry on disk. If you are still low on disk space you can manually delete dangling Docker images (file system layer not being referenced) to create disk room. It is a good idea to use fixed version Docker images but if you want to always use the latest version of a Docker image then you may want to periodically clean up dangling images by executing the following command:
1
docker rmi $(docker images -f "dangling=true" -q)
- Docker also keeps volumes of pulled images on disk. If you are still low on disk space and are using Docker v1.19.x and above you can delete dangling volumes using the following command:
1
docker volume rm $(docker volume ls -qf dangling=true)
I’m having issues, can you help?
- If you come across an issue please check if it is a known issue
- File a Bug
- Gitter
- Users Mailing List ([email protected])
- Developers Mailing List ([email protected])