Over the last few years I’ve had the opportunity to work on several Service
Oriented Architecture (SOA) applications. I learned that writing integration
tests for such applications is difficult, but important. The challenge lies in the fact that most SOA applications use
testing approaches that are well suited for monolithic applications, but these
approaches are not always suited for testing SOA applications. It is important
because without integration tests it is far too easy for subtle bugs to creep
into your code base.

What is SOA?

Service Oriented Architecture might be best understood by first
understanding monolithic applications. Your typical long-lived Rails
application tends to be a monolithic application. According to Martin Fowler,
monolith applications:

…are often built in three main parts: a client-side user interface (consisting
of HTML pages and javascript running in a browser on the user’s machine), a
database (consisting of many tables inserted into a common, and usually
relational, database management system), and a server-side application. The
server-side application will handle HTTP requests, execute domain logic,
retrieve and update data from the database, and select and populate HTML views
to be sent to the browser. This server-side application is a monolith – a
single logical executable. Any changes to the system involve building and
deploying a new version of the server-side application.

While there might not be a universally accepted definition of SOA, applications
that adhere to this approach exhibit some common characteristics. SOA is a style
of architecting applications where the underlying structure supports
communication between a collection of loosely coupled services over well-defined
interfaces.

An example might help clarify:

Figure 1.

In this example, there are two applications that comprise the SOA, the home
application and the client application. home has a database and exposes a
RESTful interface. The
client does not have a database and communicates with home via HTTP. For
example, if the client wants a list of all the users, it has to issue a HTTP
request to the /users resource of the home application, home queries its
database for all the users, and home responds to the client with a JSON
respresentation of all the users.

Testing Monolithic Applications

The Ruby community has developed a robust suite of testing tools. My go-to tools
when writing tests for a monolithic Ruby on Rails application are rspec-
rails, factory_girl_rails, and database_cleaner. rspec-rails is the testing framework, factory_girl_rails creates test data and database_cleaner cleans up test data. These tools provide a
straightforward and simple syntax that most Rails developers are accustomed to
using. A simple test that employs these tools looks like:

In spec/models/user_spec.rb

Testing Approaches for SOA

The SOA projects in which I’ve been involved had two things in common:

• They lacked a robust and integrated test suite.
• The developers on these
  projects understood the importance of testing, but were unable to effectively
  utilize existing tooling to do the job.

Why did these projects have these problems in common? They all required state to
exist in one of the other services in order to be tested. In our client/home
example, home’s database must be seeded with records in order for the
client’s tests to run. factory_girl creates test data in applications that
contain a database and database_cleaner deletes test data in applications that
contain a database. Since the client does not have a database, factory_girl
and database_cleaner are not useful to the client and are not always suited
for applications that comprise SOA. In lieu of being able to use factory_girl
and database_cleaner, the following are several approaches I’ve seen employed:

• Stub HTTP requests from client to home.
• Seed home’s test database
  with all the required test data needed for the client’s test suite to run
  prior to running the client’s test suite.
• Create test data in home from
  the client by issuing HTTP requests to home’s public API.

Applying a real-world scenario to each of these approaches helps determine the
efficacy of each strategy and emphasizes why integrating home with the
client’s test suite is important. The scenario is as follows:

Given the architecture described in Figure 1, Developer A is working on the
client and Developer B is working on home. home provides a public
resource, /users, which returns a list of all the users. Developer A creates a
feature and a passing test in the client that displays the last name of all
the users on the client’s index page. Then, Developer B renames the resource
in home from /users to /people and does not tell Developer A. Then,
Developer A creates another feature and a passing test in the client that
displays the number of users on the index page. These two features are
deployed to production.

Stubbing HTTP Requests

The most common strategy I’ve seen employed is to test each individual
application in isolation of every other application. With this approach,
home’s server is not running when the client’s test suite is running. As a
result, the client application can not issue HTTP requests to home and all
HTTP requests from the client to home are stubbed. The biggest benefit of
this approach is that test setup and coordination between the client and home is
reduced because home’s server does not have to be running while the client’s
test suite is running.

However, the biggest disadvantage is that stubbing the HTTP requests from the
client to home instills a false sense of security in the client’s test
suite. If all HTTP requests issued to /users in home from the client
were stubbed to always return a list of all users, then renaming the resource in
home from /users to /people would not result in any test failures when the
client’s test suite is run. Because there were no test failures when the
client’s test was run, the developers would feel confident that the code works
as intended and this code would be deployed to production. Yikes, they introduced a
crisis!

Proponents of stubbing HTTP requests might reply that if the client just
updated the stubbed HTTP requests or re-recorded the cassettes (if using
VCR) then running the client’s test suite would
result in failures. The flaw with that approach is that
the developer has to remember to issue updates—and humans have notoriously bad
memories.

It is worth noting that there is nothing inherently wrong with stubs. They are
often effectively used to isolate units under test. However, integration testing
SOAs (client issuing HTTP requests to home in our example) gives us more
confidence in the system than stubbing would otherwise provide. You might then
ask, “When should I stub HTTP requests?” I suggest that you stub HTTP requests
when you cannot manage the test server, such as with a third-party service. If you can
manage the test server, then I suggest that you should not stub HTTP requests.

Seeding Home’s Database

With this approach, prior to running the client’s test suite, home’s test
database is seeded with all the test data required for the client’s test suite
to pass. The biggest benefit of this approach is that home’s server can be
running while the client’s test suite is running, which allows the client to
issue HTTP requests to home (as opposed to stubbing HTTP responses to
home). As a result, renaming the resource /users to /people would result
in test failures when the client’s test suite is run. This bug would be caught
prior to being deployed to production. Crisis averted!

The disadvantage of this approach is that it quickly becomes an unviable long-
term solution by virtue of its implementation. There are many unforeseeable
problems with this approach, but the main concern is that home’s test database
bleeds state between tests. For example, if 20 tests in the client each created a
user, then there would be 20
users in home’s test database when the client’s test suite finished running. A situation like this will likely give rise to
bugs that are difficult to track down.

Create Data with Home’s Public API

It is certainly possible to create test data in home from the client using
home’s public API. Like the seeding of home’s database approach, the biggest
benefit of this approach is that home’s server can be running while the
client’s test suite is running, which allows the client to issue HTTP
requests to home (as opposed to stubbing HTTP responses to home). As a
result, renaming the resource /users to /people would result in test
failures when the client’s test suite is run and this bug would be caught
prior to being deployed to production. Again, crisis averted!

However, this approach can make creating test data difficult in the client.
This is especially true when the data that needs to be created has complex data
constraints or has complicated associations. In some cases, the test data that
needs to be created might not be able available through the public API. I would
also argue that creating test data via home’s public api unnecessarily tests
home’s resources when the test should instead focus on testing a piece of
application code (i.e., a user signing in). Like seeding home’s test database
strategy, this approach bleeds state between tests.

The Fourth Option

Wouldn’t it be great if there was a fourth option? What if something existed in the
client, like factory_girl and database_cleaner, that enabled the client
to easily create and delete test data in home? remote_factory_girl in
conjunction with remote_factory_girl_home_rails allows the client to create
test data in home’s database. remote_database_cleaner in conjunction with
remote_database_cleaner_home_rails enables the client to delete test data in
home’s test database.

A simple test that employs these tools looks like:

In spec/models/user_spec.rb in client

Like the seeding of home’s database and using home’s public API
strategies, the biggest benefit of this approach is that home’s server can be
running while the client’s test suite is running. The client could issue
HTTP requests to /users (as opposed to stubbing HTTP responses to home)
to retrieve a list of users. Renaming of the resource /users to /people
would result in test failures when the client’s test suite is run. The bug
would be caught prior to being deployed to production. Yay, crisis averted!

However, what sets this apart from the other approaches is that:

• remote_factory_girl and remote_database_cleaner have a similar interface
  and workflow to their counterparts, factory_girl and datbase_cleaner, and
  because these tools are popular among Rails developers, the overhead associated
  with introducing new developers to this approach is arguably lower.
• remote_factory_girl leverages factory_girl, which can be beneficial when
  creating data with complex data constraints or associations.
• remote_database_cleaner leverages database_cleaner so test data does not
  bleed between tests.

With this approach, SOA applications enjoy (almost) all the benefits of
factory_girl and database_cleaner that monolithic applications do!

Create Test Data in Home from Client

remote_factory_girl lives in the client and is configured to issue HTTP
requests to home at the resource /remote_factory_girl/home on a specified
port (4000 in this case). When RemoteFactoryGirl.create is invoked, a HTTP
request is sent to home which includes the factory_girl factory name and
attributes as parameters. remote_factory_girl is configured in the client
like:

In Gemfile in client

In spec/spec_helper.rb in client

remote_factory_girl_home_rails lives in home and exposes the resource
/remote_facotry_girl/home. It creates test data with factory_girl based on
the input provided by remote_factory_girl and responds with a JSON
representation of the record just created. The factory names receieved as
parameters from the client must have a cooresponding factory defined in
home. remote_factory_girl_home_rails is configured in home like:

In Gemfile in home

In config/environments/test.rb in home

In config/routes.rb in home

Note: remote_factory_girl_home_rails can be configured to skip specified
controller actions in home when the client’s test suite is running.

Delete Test Data in Home from Client

remote_database_cleaner lives in the client and issues HTTP requests to
home at the resource /remote_database_cleaner/home/clean on a specified port
(4000 in this case). When RemoteDatabaseCleaner.clean is invoked, an HTTP
request is sent to the resource /remote_database_cleaner/home/clean in home
on the specified port. remote_database_cleaner is configured in the client
like:

In Gemfile in client

In spec/spec_helper.rb in client

remote_database_cleaner_home_rails lives in home and exposes the resource
/remote_database_cleaner/home/clean. When this resource is hit, it cleans
home’s database. remote_database_cleaner_home_rails is configured in home
like:

In Gemfile in home

In config/environments/test.rb in home

In config/routes.rb in home

Note: remote_factory_girl_home_rails can be configured to skip specified
controller actions in home when the client’s test suite is
running.

Running Client’s Test Suite

remote_factory_girl and remote_database_cleaner issue HTTP requests to
home on a specified port (4000 in our example), so home’s server needs to
be running on that port. We also want to ensure that data is created and deleted
in home’s test database (not development), so home’s server needs to be
running in the test environment (assuming remote_factory_girl_home_rails and
remote_database_cleaner_home_rails were included in the test group in the
Gemfile and enabled in config/environments/test.rb). After home’s server
is running, the client’s test suite can be run. Start home’s server like:

$ rails server --environment=test
--pid=/Path/to/home_application/tmp/pids/home_app-test.pid --port=4000

Note: The pid option is not required. However, providing a pid allows
home’s development and test server to be running simultaneously so you do not
have to shutdown home’s development server to run the client’s test suite.

The Takeaway

Integration testing SOA gives us more confidence in our test
suites. Without integration testing, it is far too easy for bugs to creep into
our code base. Although integration testing might be more difficult in the
short term, I believe it pays dividends in the long run.

Working Demo

soa_integration_testing
is a demo that has a client and home application that are configured with
remote_factory_girl, remote
_factory_girl_home_rails,
remote_database_cleaner,
and remote_database_cleaner_home_rails.