Crowdsourcing - the Soylent Green of Tech: It's People!

Regular readers will know that the blogster is not above exploiting a cheap trope, but some dystopian images are better and more fun than others.

Let's hope the headline of this post is one of them.

Wikipedia has a well written article about the origins or crowdsourcing, including what they call historical examples that "in retrospect" could be called crowdsourcing.  Standard definitions still emphasize the nature of the work as "a type of participative online activity". The blogster, on the other hand, sides with those who consider the internet a tool and a means of communication, hugely important and far reaching, but a tool nonetheless.

The advantage of this is that we can focus on the people engaged in this activity and are less likely to fall for the glossy PR and the paid enthusiasts promoting crowdsourcing.

The difficulties associated with crowdsourcing are well known, and the Wikipedia article describes issues with quality, finding the right contributors, sorting through completed tasks, as well as ethical implications of low pay.

Since crowdsourcing is being used in a wide variety of sectors, results are also tied to a sector, even to specific projects within a sector, making general assessments a foolish undertaking.

Therefore, the blogster limits its* observations to paid online work, the kind of work you would do mainly to make a few extra dollars, and a few of the main issues for the workers.

1. Unrealistic compensation estimates
Not a single one of the piece work compensation projections encountered did prove realistic. They all were too high, even after allowing for a learning curve and adjusting for a "less than perfect" worker. True, sometimes lower productivity is caused by a single factor, such as a website that has a habit of failing, not saving, or the like. But unless fixed, that is what you will live with.
Coming up with realistic hourly compensation based on per task work is not trivial, but the fact that not a single one out of about 20 was anywhere near accurate gave the blogster pause.

If you decide to take per task work, cut the compensation dangled in front of your eyes in half.

2. Communication
That's the aspect which justifies the "It's People!" part. Communication is hard at the best of times, but when it comes to crowdsourcing, it often becomes very painful and costly to the workers.

If you doubt that it is a daunting effort to tell workers what the goals of a project are, to describe each task in a manner that is complete and reduces human error to the minimum, and to do so using language which often accommodates non-native speakers, hey, give it a try.

Just like on social media and in other encounters lacking visual and vocal cues, communication in a crowdsourcing setting can have lots of unintended effects. It is no coincidence that supervising personnel are constantly told to be polite and helpful.

When you write instructions for an online crowdsourcing project, it is wise to look at each step from two distinct angles. The first and obvious one is whether a step or detail wis correct.
The other is less clear: try to understand how a step can fail, how a statement can be misunderstood.

One recent example the blogster saw concerned two online bug testing projects. In the first project, the instructions to the "crowd" of about 45 people said "3 bugs only, no reproductions".

The second project said: "20 bugs, no more than 2 reproductions".

You can see where the fail was, right?

The first number, 3, meant 3 bugs per tester. The second, 20, meant a combined total of 20 bugs for all testers.

Needless to say, the second project saw twice the number of bugs filed, and this despite an intervention by a supervisor when the limit was reached. Of course, you could implement a technical solution for the second case, a maximum limit after which the system locks the project and prevents additional submissions.

But such a technical hard stop did not exist in the system.

It is anybody's guess, how many frustrated participants will not come back for another project like this one.

Coming back to the Soylent Green image, crowdsourcing can feel like the movie. There is an opaque, shadowy power structure insisting everything is fine. People disappear for no apparent reason.

And even if you get on top of the garbage truck, you are not safe.

* Gender neutrality is important to us.