MTurk vs Utest Crowdsourcing — Which is Best?

by Leslie Handmaker from Friendiversary

If you’ve been reading blogs, work in a large company, or are a small business owner, chances are you are at least somewhat familiar with the term ‘crowdsourcing’. If not, here are the basics. Crowdsourcing takes tasks which are typically done in-house and outsources them to an on-demand workforce. The labor costs are less expensive than outsourcing to a firm offshore and there is no long term commitment. Additionally the work can be completed quickly due to the large pools of workers signed up to take on tasks. Workers are registered and make themselves available to take on as little or as much work as they choose. They make their decisions on factors which include the amount of time they have, the amount of money they can earn, the ease of the work, and their level of interest. Not every task is perfect for each worker, which is where the crowdsourcing model shines. There are so many workers available and willing to work on portions of a project that companies can get accurate and quick results at affordable rates.

A couple of sites are doing an excellent job of making crowdsourcing easy for companies.
1. Amazon’s Mechanical Turk, www.Mturk.com, offers great solutions for achieving what they describe as ‘Human Intelligence Tasks’ aka HITs. There are tens of thousands of registered workers available to work on small quick tasks. As an example let’s say you own a web design company specializing in creating templates allowing for further customization. Your latest batch of 75 templates each have 20 items which need to be checked prior to releasing for sale. Using Mechanical Turk you could break it down so a worker only checks one item at a time, i.e., ensuring all the template links function correctly. That worker can test that one template to see if the links function, and then go onto another template and checks the links of that template. Each time that worker tests a template they have the chance to earn a minimal amount, perhaps $.05 per template. (It’s quick work, so the rate is low.) Then to make sure you’re getting good results, the exact same task and template also gets presented to more workers. When you get the results, if you see everyone agrees the links for that one template work, you can have confidence they actually function properly. If two workers think the links are fine and three think the links don’t function properly, there are likely some bad links in the template. It’s a red flag that and the template needs to be examined further.

2. UTest, www.Utest.com, is a crowdsourcing company providing QA testing for companies. Similar to MTurk they have a field of on-demand workers and are a great resource for a scalable workforce. They offer 3 kinds of testing:
– Functional Testing – Catches bugs, typos, etc
– Load Testing – A combination of real users and/or simulated browser sessions to determine if a site is ready for peak demands
– Usability Testing – a focus group which can be assembled using specified criteria, who can give valuable feedback
The same web template testing task could be completed by the Utest workers. Utest has over 30,000 registered testers in their community. The differences with Utest are that they have an entire platform specifically designed for testing and finding bugs. Their platform allows the testers to upload attachments, including screenshots and even videos capturing screen activity to demonstrate the bugs which were encountered. Utest also assigns companies a project manager who can help with selecting testers, writing test requirements, bug approval and paying the testers. As noted earlier the can provide an array of different testing. Naturally, a more advanced platform and a higher level of service will result in higher rates. Utest doesn’t advertise their rates, but they will provide quotes on a project basis. It’s hinted that they are roughly 50% the cost of a typical offshore outsourcing firm.

So which site is better? It really depends on the size, scope and nature of work you need completed. Mturk is a better choice for easier tasks and getting things done on the cheap. Utest can handle advanced tasks and will cost you more, but they offer a lot more options for larger projects.

Leslie Handmaker runs a site offering gifts for friends as well as provides freelance marketing and project management work for small businesses.