Should my tests be @Transactional?

June 25, 2014 - Andreas Eisele

Simple question. The simplest answer to that may be "It depends.".

But let's explore that aspect a little further and please keep in mind there is no absolute truth despite what pages upon pages of heated online discussions might suggest. In addition to that we also have some fully-working, juicy sample code for you to ogle over.

We're talking about @Transactional in a Spring based project context. Obviously the same holds pretty much true for your J2EE project.

So what does @Transactional mean if you annotate your test suite with it? Well it means that every test method in your suite is surrounded by an overarching Spring transaction. This transaction will be rolled back at the end of the test method regardless of it's outcome.

The fallacy

Some people conclude from this that you cannot be sure that your tests actually writes data do the database. But this is not true, what you are seeing instead is very likely the side effect of using an ORM.

Let's start with plain jdbc...

Imagine this scenario: You open a transaction with plain JDBC, then issue a stmt.executeQuery(query) and later on rollback the transaction. Of course that query gets sent to the database. And if you insert some data, the db constraints will be checked and you will also be able to query for it (from the same session) until you rollback. Same holds true for tools like the fantastic JooQ.

...but what about my favourite JPA implementation, Hibernate?

Change of scenes: As a quick reminder, JPA, or say your favourite and beloved implementation of it, Hibernate, has the concept of a session. Just because you add a add/update an object in your session, does not mean it gets saved directly to the database. But why not? Because the session has the concept of a FlushMode. And very likely that flush mode is set to AUTO, so Hibernate can decide when to send queries to the database.
But you could of course also set it to ALWAYS and get pretty much the same database behaviour as using plain jdbc calls.

You usually do not notice the flush mode, because Spring's HibernateTransactionManager makes sure to do a session.flush() before committing every transaction - but the same is of course not happening come rollback time. This means that none/some/all (yep!) modifications that would have hit the database are being removed from the session and are not sent. Which makes your tests potentially fragile. Very fragile.

So, enough theory, what options do we have when it comes down to tests that hit the database?

1. Don't make your tests @transactional at all (but your service layer)

If you want to don't fuzz around too much and just want to make sure your stuff really does get saved to the database, can be read, updated and deleted, then do not make your tests @transactional. This means that all your service layer/persistence layer methods invoked through the tests would start their own transactions (you made them transactional, did you?) and flush the changes upon commit. So you are guaranteed to notice if something blows up on flush.

But you are also guaranteed to have, after a while, a database full of junk test data. Often one can live with that and clean up that data once in a while or after every test-run. If an in-memory database like H2 is also an option, you could also reset/truncate your database after every method run (but parallelizing tests then somewhat gets tricky , but more on that in a different post).

See the below snippet for one approach to this pattern (and have a look at the complete sample code to see a fully working h2 in-memory database reset).

Complete File

// [...]
// note that this test suite is not @Transactional
@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = Application.class)
public class UserRepositoryIntegrationTests {
    private static final String TEST_EMAIL = "test@test.test";
    private static final String TEST_PASSWORD = "password";
    @Autowired
    private UserRepository repository;
    @Test
    public void testCreate() throws Exception {
        User user = new User(TEST_EMAIL, TEST_PASSWORD);
        assertThat(user.isNew(), is(true));
        user = repository.save(user);
        assertThat(user.isNew(), is(false));
        assertThat(user.getId(), is(notNullValue()));
        assertThat(user.getEmail(), is(equalTo(TEST_EMAIL)));
        assertThat(user.getPassword(), is(equalTo(TEST_PASSWORD)));
    }
// [...]
}

2. Manual Flush

A different approach would be to keep the test suite transactional but just manually flush the changes to your database when needed. This is a little more verbose and dirty but in the end you combine the advantages of a transactional test suite with the confidence of knowing that nothing is blowing up further down the stream. It is valuable to note that simply flushing doesn't mean that we commit a transaction, so the rollback after the test method end is still doing it's job.

Complete File

// [...]
// note that this test suite is @Transactional but flushes changes manually
@Transactional
@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = Application.class)
public class UserRepositoryFlushingIntegrationTests {
    private static final String TEST_EMAIL = "test@test.test";
    private static final String TEST_PASSWORD = "password";
    @Autowired
    private UserRepository repository;
    @PersistenceContext
    private EntityManager entityManager;
    @Test
    public void testCreate() throws Exception {
        User user = new User(TEST_EMAIL, TEST_PASSWORD);
        assertThat(user.isNew(), is(true));
        user = repository.save(user);
        entityManager.flush();
        assertThat(user.isNew(), is(false));
        assertThat(user.getId(), is(notNullValue()));
        assertThat(user.getEmail(), is(equalTo(TEST_EMAIL)));
        assertThat(user.getPassword(), is(equalTo(TEST_PASSWORD)));
    }
// [...]
}

( 3. Set the flush-mode to FlushMode.ALWAYS for your tests )

It is in brackets because it works, but of course makes your test code behaviour deviate from your production code behaviour. Use with caution!

4. Do not care

Especially in the cases of integration tests where you just assume that your persistence layer is working (because you tested it before, right?) having your test suites transactional is nice. See below for an example where we are testing the higher service layer instead of the persistence layer and frankly we don't care about correctness of the persistence layer further down (apart from it not blowing up).

Complete File

// [...]
// note that this test suite is @Transactional
@Transactional
@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = Application.class)
public class UserServiceIntegrationTests {
    private static final String TEST_EMAIL = "test@test.test";
    private static final String TEST_PASSWORD = "password";
    @Autowired
    private UserService service;
    private Wiser wiser;
    @Before
    public void setUp() throws Exception {
        this.wiser = new Wiser(2500);
        wiser.start();
    }
    @After
    public void tearDown() throws Exception {
        wiser.stop();
    }
    @Test
    public void testRegister() throws Exception {
        User user = service.register(TEST_EMAIL, TEST_PASSWORD);
        assertThat(user.getEmail(), is(equalTo(TEST_EMAIL)));
        assertThat(user.getPassword(), is(equalTo(TEST_PASSWORD)));
        List<WiserMessage> messages = wiser.getMessages();
        assertThat(messages, hasSize(1));
        WiserMessage message = messages.get(0);
        assertThat(message.getEnvelopeReceiver(), is(equalTo(TEST_EMAIL)));
    }
//[...]
}

The complete test code can be founde here. Give it a try, it is fully functional!