Interoperability Happens - Integration testing and unit testing

Wednesday, September 15, 2010

Also known as, “An exercise in Twitter opinionating”:

Sitting in a meeting, I heard one of the other meeting participants say, “You should do your integration testing before your unit testing in your code.”

Now, I wasn’t (and I’m still not) sure I agree with this or not, and when in doubt, I like to get data points. Twitter to the rescue!

The responses that came back were fascinating, and all over the map:

MarkFreedman

Why? Disagree, unless I'm missing something => RT @tedneward: OH: "You should do your integration testing before your unit testing."

kiranhatti

@tedneward disagree! How could an unstable code (not unit tested)be ready for integration testing

bradwilson

@tedneward Write acceptance tests, so we're all on the same page, then use TDD to get components in place until acceptance tests pass.

bradwilson

@tedneward There is value on an agile team for starting with automated acceptance tests, as a stopping function for development effort.

iceX33

@tedneward mostly agree integration first helps to focus on feature, not losing in details

JasonBarile

@tedneward Depends on the intent of the integration testing. If it's to find gaps in the feature set, maybe. Otherwise, disagree.

justncase80

@tedneward disagree. The order depends on what you're creating.

RockyLhotka

@tedneward I continually intermix them; can't say I care a lot about the distinction either in most cases

secos

@tedneward in the sense that you may want to have some of the integration work done before you write the internals, yes.

My next thought was “how do I capture this for further discussion”, so here we go.

Couple of thoughts that occur to me as I read through the responses:

Are acceptance tests the same as integration tests?
Are either of these at the same level of scope as unit tests?
Does it make a difference which order you approach them?

Love to hear your thoughts.

Wednesday, September 15, 2010 4:32:28 PM (Pacific Daylight Time, UTC-07:00)
Comments [6] |

Wednesday, September 15, 2010 7:39:42 PM (Pacific Daylight Time, UTC-07:00)

My take:

Acceptance tests are tests made by the customer to determine if a piece of software is acceptable (it does what it is meant to do). When automating it, they are built in collaboration with the customer, and usually tests the UI (going all the way down the system), or tests the software immediately below the UI (subcutaneous testing).

Integration tests are tests that go through the boundaries of the components of the system. 'All the way down to the database.' Automated acceptance tests as described above are, also, integration tests.

Unit tests test individual 'units' (a class, or a method) in isolation, separated from the rest of the system. Usually requires mocking techniques to fake dependencies.

If you test first, you will probably get more coverage. 'If something already works, why waste time testing it?'

Personally, I usually design and code first (a class, or a small set of classes), because I think I reach a elegant design faster when not constrained by TDD's red/green/refactor cycle. Sometimes, though, writing some client code first helps me imagine how I want my classes to be used.

Tetsuo

Wednesday, September 15, 2010 8:01:49 PM (Pacific Daylight Time, UTC-07:00)

To elaborate, what I was really thinking was that in some cases the first thing you want to do is to create an actually working application which acts as an integration test. It might not really do much at first but it provides the foundation for the rest of the pieces to rest on. Then slowly flesh out the application horizontally, vigorously adding unit tests as you do so. While that is happening having that one integration test helps. As areas of your application become more real and fixed more integration tests are useful.

Justin Chase

Wednesday, September 15, 2010 11:19:04 PM (Pacific Daylight Time, UTC-07:00)

I´d say: Acceptance Tests are Integration Tests by definition. No customer would accept non-integrated stuff to test ;-)

Then Integration Tests and Unit Tests both are necessary, I´d say. Unit Tests test individual units, Integration Tests test how individual units (and aggregated units) play together. We could live w/o Unit Tests - but that would make bug fixing life pretty frustrating.

If you are unable to test individual units because there are none as in most brownfield apps, then, well, you need to start with Integration Tests. But the goal should be to identify units and make them individually testable. That´s one of the purposes of refactoring.

If you have individually testable units, though, you should start as close to a bug to fix as possible. That is with a Unit Test if possible. If not, then it´s necessarily an Integration Test.

Ralf Westphal

Thursday, September 16, 2010 12:37:09 AM (Pacific Daylight Time, UTC-07:00)

Ted, have you heard of the book "Growing Object-Oriented Software, Guided by Tests" (http://www.growing-object-oriented-software.com/)? It is a wonderful book written by Steven Freeman and Nat Pryce. The first chapter covers all of your questions and explains why you should start with an end-to-end test (acceptance test) and then fill in the necessary objects by test driving the application's design. The difference between acceptance-, integration- and unit-tests are covered on page nine as followed: "Acceptance: Does the whole system work? Integration: Does our code work against code we can't change? Unit: Do our objects do the right thing, are they convenient to work with?". The sample application in part three of the book (almost 200 pages) shows how the authors develop a Rich-Client wich communicates asynchronously over XMPP, completley guided by tests. Developing this app the authors show exactly when which test has to be written (for this app). Of course all three types of tests are covered. I really recommend reading this book. I think it is one of this books which must remain on a dev's desk all the time.

Cheers

Claus Polanka

Thursday, September 16, 2010 12:24:01 PM (Pacific Daylight Time, UTC-07:00)

I saw your tweet about this, but I couldn't figure out how to get into the conversation in a constructive way. Thanks for this blog post.

My thinking is along the lines pointed out by Claus. I started a development project a few years ago and I attempted to have deliverable unit tests as part of it. I got through to near-beta level and became frustrated (and still am) realizing that I didn't have a worked out deployment process that took me through deliverable, integratable builds. This is for a library of Java classes and supporting JNI to some underlying native code, but even libraries have integration.

If I were doing this again, or even finally updating to a full-up open-source beta, I would develop the deployment process, with integration tests and installation tests along with the sort-of bottom up part of building and testing the library elements themselves.

I will have an opportunity over the course of 2011, to try that out and see if I can avoid how I painted myself into a corner by not dealing with this well enough early enough.

I will take a look at the information about the Freeman and Pryce book.

[Aside: Oddly, I also got to a point of complexity in management of my own web sites that I had to sit down and figure out what my deployment model was. I hadn't thought of this as something requiring tests too -- there are only static web pages -- although I am going through a migration and there are obviously tests, I just hadn't been recognizing them as such. So considering integration tests as companion to unit tests may be something to always look for just because there is insight available that way.]

orcmid

Thursday, September 16, 2010 12:32:18 PM (Pacific Daylight Time, UTC-07:00)

Ted, with regard to your additional thoughts, I think that acceptance tests are different from *my* integration tests, although they may be part of someone elses acceptance in an integrated case.

On the other hand, I want to have integration confirmation tests that I deliver as part of my libraries so that someone can tell that they have succeeded in integrating/installing the libraries properly along with *their* deployment. In this case I am giving the adopter of the deliverable a way to confirm successful integration/installation. And that becomes something I test with too.

The additional questions, like the one about integration tests re unit tests, may not be so important in that they have yes-no answers but that they give us something to think about when dealing with a specific situation and our desire to have a reliable end-to-end process for it, all the way out to the adopters installation, integration, maintenance, and removal.

orcmid

Comments are closed.