iOS Tests/Specs TDD/BDD and Integration & Acceptance Testing

by

What are the best technologies to use for behavior-driven development on the iPhone? And what are some open source example projects that demonstrate sound use of these technologies? Here are some options I’ve found:

Unit Testing

Test::Unit Style

  1. OCUnit/SenTestingKit as explained in iOS Development Guide: Unit Testing Applications & other OCUnit references.
    • Examples: iPhoneUnitTests, Three20
  2. CATCH
  3. GHUnit
  4. Google Toolbox for Mac: iPhone Unit Testing

RSpec Style

  1. Kiwi (which also comes with mocking & expectations)
  2. Cedar
  3. Jasmine with UI Automation as shown in dexterous’ iOS-Acceptance-Testing specs

Acceptance Testing

Selenium Style

  1. UI Automation (works on device)

    • UI Automation Instruments Guide
    • UI Automation reference documentation
    • Tuneup js – cool library for using with UIAutomation.

    • Capturing User Interface Actions into Automation Scripts

      It’s possible to use Cucumber (written in JavaScript) to drive UI Automation. This would be a great open-source project. Then, we could write Gherkin to run UI Automation testing. For now, I’ll just write Gherkin as comments.

    UPDATE: Zucchini Framework seems to blend Cucumber & UI Automation! 🙂

    Old Blog Posts:

    • Alex Vollmer’s UI Automation tutorial
    • O’Reilly Answers UI Automation tutorial
    • Adi Saxena’s UI Automation tutorial

  2. UISpec with UISpecRunner

    • UISpec is open source on Google Code.
    • UISpec has comprehensive documentation.

  3. FoneMonkey

Cucumber Style

  1. Frank and iCuke (based on the Cucumber meets iPhone talk)

    • The Frank Google Group has much more activity than the iCuke Google Group.
    • Frank runs on both device and simulator, while iCuke only runs in simulator.
    • Frank seems to have a more comprehensive set of step definitions than iCuke’s step definitions. And, Frank also has a step definition compendium on their wiki.
    • I proposed that we merge iCuke & Frank (similar to how Merb & Rails merged) since they have the same common goal: Cucumber for iOS.

  2. KIF (Keep It Functional) by Square

  3. Zucchini Framework uses Cucumber syntax for writing tests and uses CoffeeScript for step definitions.

Additions

  • OCMock for mocking
  • OCHamcrest and/or Expecta for expectations

Conclusion

Well, obviously, there’s no right answer to this question, but here’s what I’m choosing to go with currently:

For unit testing, I used to use OCUnit/SenTestingKit in XCode 4. It’s simple & solid. But, I prefer the language of BDD over TDD (Why is RSpec better than Test::Unit?) because our words create our world. So now, I use Kiwi with ARC & Kiwi code completion/autocompletion. I prefer Kiwi over Cedar because it’s built on top of OCUnit and comes with RSpec-style matchers & mocks/stubs. UPDATE: I’m now looking into OCMock because, currently, Kiwi doesn’t support stubbing toll-free bridged objects.

For acceptance testing, I use UI Automation because it’s awesome. It lets you record each test case, making writing tests automatic. Also, Apple develops it, and so it has a promising future. It also works on the device and from Instruments, which allows for other cool features, like showing memory leaks. Unfortunately, with UI Automation, I don’t know how to run Objective-C code, but with Frank & iCuke you can. So, I’ll just test the lower-level Objective-C stuff with unit tests, or create UIButtons only for the TEST build configuration, which when clicked, will run Objective-C code.

Which solutions do you use?

Related Questions

  • Is there a BDD solution that presently works well with iOS4 and Xcode4?
  • SenTestingKit (integrated with XCode) versus GHUnit on XCode 4 for Unit Testing?
  • Testing asynchronous code on iOS with OCunit
  • SenTestingKit in Xcode 4: Asynchronous testing?
  • How does unit testing on the iPhone work?

8 Answers
8

Leave a Comment