How to collect users’ task completion times?

I’m not sure what kind of application you’re trying to test (web, mobile, desktop application, etc.), but it seems that you’ll have to choose one of the 3 following options:

  • Time the users yourself
    • Do public field-testing (also known as “guerrilla testing“)
      • e.g. “Hello stranger, would you like a $5 gift card to {local coffee shop}? Our quick user testing will take less than 5 minutes of your time…
    • Bring users on-site to test under observation
    • PROS: Direct observation gives you the chance to ask for additional feedback or elaboration on certain difficulties you may have observed, as well as opens the dialogue for the user to express any other concerns.
    • CONS: Requires face-to-face interaction, so your results may have a location bias. In other words, your results may not be representative of your broader target audience.
  • Modify your application so it automatically collects these analytics for you
    • See Google Analytics for one example
    • PROS: Easy for you; works automatically (assuming you can integrate analytics into your application); usually provides much more insight into many different facets of how users use your applications, which can provide even more benefit.
    • CONS: User behavior not necessarily representative of a purely goal-oriented or task-driven approach to navigating the app, meaning your data may show more casual browsing behavior or distracted users than if you sat them down and observed them directly.
  • Allow users to time themselves, using their own stopwatch
    • This option is perhaps the least reliable, but it’s easy to collect responses from a wide number of people at once
    • PROS: Users can time themselves remotely, so your responses are not limited to users with whom you can physically interact.
    • CONS: Your data has to be taken with a grain of salt, as you can only trust that your users have completed the requested tasks, and have timed themselves accurately.
  • As suggested by @ralien, you could utilize KLM (or “Keystroke-level modeling”) (demonstration)
    • Allows you to estimate the time to complete the task without involving users at all.
    • PROS: Cheap, and provides results which may be good enough to satisfy your requirements.
    • CONS: Offers a theoretical analysis of an application’s interface and organization, so may not align completely with users’ actual experience.

In any situation, you’ll likely have to incentivize your users in some way—most people don’t like to spend their free time doing user testing just for the fun of it.

You may Also Like:

None found

Leave a Comment