by Steven-Clarke on 7/14/19, 12:05 PM with 141 comments
by ptero on 7/15/19, 6:46 AM
by Phemist on 7/15/19, 9:58 AM
7 or 8 years ago, my then supervisor and I experimented with running psychological (behavioural) experiments on MTurk. We had created a method that would run in browsers in native javascript and were looking to validate it. Coming from a testing-1st-year-psych-students-in-the-basement-of-the-faculty-building kind of thing, we naively took what the going wage was for that (something like 8 euros/hour), and put an experiment online that would take around 20 min to complete, and paid 2 euros.
My supervisor used his own credit card and set the spend limit to around 1000 euros; thinking that we'd never hit it. Boy how wrong were we. Apparently this 6 euro/hour wage was _much_ _much_ higher than the going rate, and we hit the spend limit in around 2 hours. Even though we had to throw out around 70% of the completions, we ended up with usable data from around 150 participants.
We went in expecting to run the experiment for a week or 2, and get maybe 50 participants, but came out with 500 in a span of 2 hours. Safe to say, we celebrated a job well done that night over one or two drinks. People were commenting on how nice of a change of pace it was compared to the then-usual MTurk tasks and that they would have done it for free. Some even left their e-mail addresses should we run another online experiment. I've since been out of the field of online behavioural experimentation, but it seems to have taken off quite a bit.
by neilv on 7/15/19, 3:30 PM
Since she could write, she mostly did writing assignments (which I suspected were for SEO Web sites), in which they tell you a topic and how many words to write, you research and write the article, and you get paid a pittance. But $2 will buy steel-cut oats for the day.
Her time should've been worth more than $1-$2/hour, and I don't like the idea of companies arguably exploiting desperate people this way. Though it's not just companies: university researchers sometimes use Mechanical Turk workers to process data, and as research subjects.
by dgivney on 7/15/19, 10:07 AM
I wonder just how does he manage a new baby, a side hustle that fills every minute with productive money-making and writing a book!.. The author really should be writing a book about his exceptional time management skills instead.
by blintz on 7/15/19, 10:06 AM
Should universities require that studies using MTurk pay some minimum pro-rated hourly wage? That is, if a Stanford HCI study (which seem to use MTurk often) wants workers to fill out a 15-minute survey, they would need to pay at least $8 an hour (so $2).
I’ve been to thesis defenses at Stanford where researchers have explicitly stated that they chose to use MTurk because they can pay workers far less than locally recruited participants.
by probably_wrong on 7/15/19, 6:22 AM
Our group made the calculations to ensure that our task would pay at least minimum wage. As a result, we had to throttle our participants because they would hit our servers pretty hard.
That's not to say that the experience was easy - we had to implement every possible sanity check, and even then we ended up throwing away half our data. For us it was still worth it, since twice the rate was still cheap and we knew not to trust the internet. But a less internet-savy researcher blindly trusting their data would have gotten some surprising results.
by dsissitka on 7/15/19, 5:49 AM
https://www.youtube.com/watch?v=h9Qt6bVc5mY&list=PLuh2SVJZ17...
The hourly rate will flash every few seconds.
by TrackerFF on 7/15/19, 6:25 AM
I've been doing side-hustles since I was a teen, and continue today, even though I have a well-paying professional job. My observations are that domain knowledge / specialized expertise always pays better.
You're much better off learning some hobby, almost any hobby, and monetizing it. How do you monetize it? Do content writing. Start flipping on Ebay / Craigslist / etc. Perform / create if you can. etc.
Now - I understand that these (MTurk) kinds of jobs are for desperate people, that want/need the money now - not 4 years down the road. Building expert knowledge will take time, and you will more likely than not get burnt a few times in the start - but the long-term payoff is much better.
Last month, I made $800 in selling two musical instruments. Time invested in it all was around 2-3 hours, mostly on picking 'em up, and shipping them.
Maybe not the answer for everyone, but it's extremely hard for me to imagine any other way now. I currently have three different hobbies which I enjoy very much (and have for years), and I'm lucky enough to have so much knowledge in that I also make a decent side-income off.
Best of all? It doesn't feel like you're working - though you need to treat it like work, if you're gonna make money.
by closetOperator on 7/15/19, 6:25 AM
Unless you have a clear
strategy, MTurk work is a
complete waste of time.
It would be a request like:
Gather phone numbers from
this queue of web pages.
You get into the queue, and start pulling up each URL in series, and they're all these expired domains with parked registrar pages for GoDaddy and Tucows or whatever, Along with some Geocities, Angelfile, Tripod and AOL home pages thrown in. Quickly, you get a sense that some fool of data science masters program enrollee paid good money for a dusty, mouldering text file, didn't even look at it, and dropped it right the fuck into a template for a Mechanical Turk task.
Now, there are three immediately obvious courses of action. One, abort and never again consider Mechanical Turk as a useful platform for operating an exchange of effort for rewards. Two, plead with the task owner by reporting feedback to them and ask them to stop for a moment and consider the flaws inherent to this framing of a human activity deemed worthy of compensation. Three, obey the letter of the law, and not the spirit, hold your nose, bellow the words "you asked for it!" and proceed to fill the task with the tech support numbers for all of the domain registrars, hoping that you'll not only get paid for grifting on the task owner, but also possibly inundate all these domain squatting registrars with robocalls trawling for psyche student surveys and questionnaires that will attempt to publish similarly terrible research papers designed with the intent to ostensibly "prove" a flawed hypothesis of human behavior with results that couldn't possibly be replicated because the hypothesis itself begs its own question.
Valuing my time, I just logged out, and haven't looked back since.
They need moderators to mechanical turk the quality of each task, because it benefits no one and wastes people's time, to even propose fruitless, unredeeming tasks.
Unless things have changed since winter, from what I witnessed, there is perhaps zero review of tasks to assess whether a request fits the profile of anything even remotely possible or worth trying.
by s_Hogg on 7/15/19, 6:34 AM
by gitgud on 7/15/19, 5:47 AM
It seems the Mechanical Turk ecconomy could be dwindling due to the accessiblity of AI these days.
by Mediterraneo10 on 7/15/19, 6:39 AM
by PaulHoule on 7/15/19, 1:35 PM
For instance there used to be a lot of HITs that involves transcribing printed receipts.
I wouldn't mind doing this so much if the receipts weren't smudged and creased or otherwise illegible, but I think these people have an OCR that can handle the easy ones and you're left with a residuum of hard cases which sometimes can't be transcribed at all, or for which you'll probably make enough mistakes to get in trouble. I find that mentally fatiguing on top of the extra time.
by anm89 on 7/15/19, 3:07 PM
by simonebrunozzi on 7/15/19, 7:06 PM
In my view, this is why you can get paid less than what you deserve, but that option is still attractive to you.
In other words, if you have 34 minutes between things, one of the few paying things you can do is MTurk, even if it doesn't pay well.
by mhh__ on 7/15/19, 1:25 PM
I've seen some people on Reddit who are effectively gaming it well enough to actually make consistent money, but that's clearly the exception not the rule.
by Havoc on 7/15/19, 7:20 AM
Else pretty much anything else will be more profitable
by sct202 on 7/15/19, 3:09 PM
by Ajs1 on 7/14/19, 11:30 PM
by lordnacho on 7/15/19, 8:36 AM
by sschueller on 7/15/19, 8:13 AM
by RocketSyntax on 7/15/19, 2:36 PM
by Yuval_Halevi on 7/15/19, 8:16 AM
by senectus1 on 7/15/19, 5:41 AM
I wonder what Nationality these "Turker" are from...