Mechanical Turk generally sucks as a substitute for a regular job, but can make sense as supplemental income, even at nominally very low hourly pay. If you work 8 hours at $10/hour but have a 30 minute commute, you are really getting $80 for 9 hours of your time. That actually puts you below $9/hour to think of it that way.
If you do freelance work, iirc, freelancers chase their pay about 40% of the time. That also drives their real wages down. Plus, there is time involved in getting each assignment.
I have read that you can expect to do one unbillable hour for every billable hour, so you need to charge at least twice as much to make the same wage. Plus, for jobs with benefits, roughly half your compensation can be in the form of benefits.
So freelancers should charge four times as much as the hourly rate they would accept at a salaried job with benefits.
But if you already have benefits and just want to supplement your income, you can accept half as much. If you can eliminate some of the time burden of freelancing, you can halve it again.
Looked at that way, $2/hour is potentially the equivalent of $8/hour as supplemental income. And you can fit it in to otherwise useless scraps of time.
And for a long game one will likely get much further by investing in primary job or developing new skills (teaching math to kids, home improvements, whatever) that gets them out of competing with millions of unskilled, low wage participants in MTurk.
I view MTurk as temporary fix for desperate times only. My 2c.
No. $2/hr is potentially the same as freelancing at $8/hr under your comparison.
That's completely different. Because no one freelances at $8/hr.
I could see how this could be different at lower hourly rates though. Clients are less likely to pay, they are less price sensitive to 16-32$/hr.
Surely a better way would be to stop smoking?
If this was generally perceived as fun, then Amazon would charge money for you to "play" mturk. The fact mturk work pays tells me that mturk is pretty much nobody's hobby.
Not sure about other parents, but when we had our baby, I was less ready for additional responsibilities than at any other point in my life.
A couple of low-effort, no-brain mturk forms to make extra diaper money? Great! I'll bust out the laptop while killing time at a kid's soccer practice.
I view this as passive time where spending a few hours while watching tv, or rocking the baby, or whatnot seems acceptable to some people.
Many free to play games have the same attention span requirements and they don’t pay anything.
But I dunno, I feel OK with the premise that this guy reduced his expenses and optimized his career as much as he could.
Say you did all that and had a few hours a week you wanted to use to make your household budget a bit healthier...getting paid for some kind of remote work via an automated system that'll accommodate your inconsistent hours has an appeal!
If he does that, I could see it being competitive on an hourly basis to a regular job.
Ooh, the idea of mTurk is "human powered." OTOH, a market for tasks that can be do e via a combination of automations and humans could be powerful.
That might be a little harsh, and perhaps there is no ill-intent here, but the majority of the second half of the article reads like a low-pressure sales pitch to me.
Isn't the entire point of the article that he thought it sounded like a good idea, and turned out that it wasn't? In principle, it's easier than some of the options you list, since any corporate job ("analyst") involves a lot of downtime at your computer.
7 or 8 years ago, my then supervisor and I experimented with running psychological (behavioural) experiments on MTurk. We had created a method that would run in browsers in native javascript and were looking to validate it. Coming from a testing-1st-year-psych-students-in-the-basement-of-the-faculty-building kind of thing, we naively took what the going wage was for that (something like 8 euros/hour), and put an experiment online that would take around 20 min to complete, and paid 2 euros.
My supervisor used his own credit card and set the spend limit to around 1000 euros; thinking that we'd never hit it. Boy how wrong were we. Apparently this 6 euro/hour wage was _much_ _much_ higher than the going rate, and we hit the spend limit in around 2 hours. Even though we had to throw out around 70% of the completions, we ended up with usable data from around 150 participants.
We went in expecting to run the experiment for a week or 2, and get maybe 50 participants, but came out with 500 in a span of 2 hours. Safe to say, we celebrated a job well done that night over one or two drinks. People were commenting on how nice of a change of pace it was compared to the then-usual MTurk tasks and that they would have done it for free. Some even left their e-mail addresses should we run another online experiment. I've since been out of the field of online behavioural experimentation, but it seems to have taken off quite a bit.
The biggest problem we had was with people with high latency connections. The effects we were looking for were measured in 10s of milliseconds. In order to tease out these effects, we had to be very particular about the timing of when certain stimuli were presented to the participant. High latency page reloads (which were unavoidable in the system we built our method on) would mess with this high-precision presentation requirement. We measured the latency, but did not pre-emptively exclude anyone based on their latency, hence the high % of people with unusable data in our initial validation experiment.
For subsequent experiments I built a "loader" screen that would pretend to be loading the experiment. What it in fact was doing was refreshing the webpage several times (ofcourse, while progressing a progress bar) to measure the latency of the connection of the participant. High average latency + high variance latency connections were excluded. The tresholds were based on what we found in the early validation study.
Surprisingly, after throwing out the high-latency data, there were no other exclusions necessary. It seemed that for our validation study, all participants were very attentive during the experiment.
In "in-person" cases, researchers would add attention checks to their experiments with the logic that failing these attention checks by itself is no indication of "spamming", but seeing weird quirks in the data + failed attention checks would be. One that my supervisor was fond of was throwing in instruction screens, in the middle of runs of trials that required you to press a very specific button to continue. E.g., the experiment has you press 'A' and 'J' constantly, and to continue you have to press 'N'. Secretly though, A and J were also valid ways to continue with the experiment. The thought being that if you were hammering one of those buttons to quickly get through the experiment, you would also skip past the instruction screen very quickly with it.
Since she could write, she mostly did writing assignments (which I suspected were for SEO Web sites), in which they tell you a topic and how many words to write, you research and write the article, and you get paid a pittance. But $2 will buy steel-cut oats for the day.
Her time should've been worth more than $1-$2/hour, and I don't like the idea of companies arguably exploiting desperate people this way. Though it's not just companies: university researchers sometimes use Mechanical Turk workers to process data, and as research subjects.
I don't think the IRB actually enforces a minimum wage, but the feeling of oversight seems to discourage super exploitative behavior. The downside is that high payments means higher requirements - something like 98% acceptance, 500 HITs, Master's qualification, and US location is what I generally see used, and we still get junk answers sometimes (in my last study one person somehow submitted a bunch of HTML forms with empty answers despite required fields, still not sure how they pulled that one off without intentionally doing some sketchy stuff...). This makes the fairly paid tasks inaccessible to most new Turkers, leaving them to pick up the scraps of the $1/hour HITs.
Let's say I live in Toronto (an expensive city to live in) and make 10$/h after taxes working at Starbucks.
40h/week, 4.5 weeks/mo = 1800$
Bedroom in apartment = 700$
Metropass = 150$
Phone = 50$
Internet = 50$
Food = 350$
Total = 1300$
That's 500$ left for things like going out, clothes, etc.
Nobody is exploiting your friends - it sounds like she complained to you, you took her words at face value and concluded that those other people are at fault.
I've done the math. I've lived here. I've financed years of study in the city on loans and been on Ontario Works at times at the start of my career. Your model is poorly constructed.
You're contradicting me by stating (false) assumptions about my friend, and stating (false) conclusions predicated on those.
Your analysis also neglects American Millennial debt service and healthcare costs.
I used a personal example of my friend because it was close to home for me, and I thought that by mentioning it (as connected to an HNer), along with an age range, it would be somewhat closer to home to some other HN people. When people mention a potentially upsetting personal example, we should try to be delicate in wording things.
I wonder just how does he manage a new baby, a side hustle that fills every minute with productive money-making and writing a book!.. The author really should be writing a book about his exceptional time management skills instead.
Want to make money online playing poker? Pay $800 for Upswing Poker's masterclass!
Want to make money online trading stocks? Check out the ultimate Penny Stock Playbook for only $50!
Want to make money online drop shipping? For only $1600 you can buy the ProfitableOnlineStore course and have all your questions answered!
So it makes you wonder doesn't it: If people are so good at making millions making money online, why are they offering to sell you the secret rather than actually making the millions themselves. The answer is: It's easier to sell gullible people tutorials than it is to actually make a living exploiting arbitrage opportunities in one of the most efficient markets ever known to man.
"Yo dawg, I heard you're thinking of a new way to make money...so I wrote a book about thinking of a new way to make money so you can think about a new way to make money while I make money from you thinking about my new way to think about making money."
I halfway expected to see a secondary market emerge where you “complete” a top expensive task like “translate this text” by bundling it with five other translation tasks and posting it for three times the price.
Should universities require that studies using MTurk pay some minimum pro-rated hourly wage? That is, if a Stanford HCI study (which seem to use MTurk often) wants workers to fill out a 15-minute survey, they would need to pay at least $8 an hour (so $2).
I’ve been to thesis defenses at Stanford where researchers have explicitly stated that they chose to use MTurk because they can pay workers far less than locally recruited participants.
I don't see an ethical problem in how much you pay participants (after all, how much it pays or if it pays at all will be one of the variables the study needs to account for). The ethical problem I see is not caring if the results are meaningful, just publishable.
Why do you say that? I imagine the MTurk demographic is much more diverse than "college freshman at X university"
I bet they do. One thing not really touched on in the article is that one of the task qualifiers is “acceptance rate”, i.e., how many of your task submissions were accepted by the person that created the tasks. And that number is usually 99.9% acceptance (of _all_ results you’ve ever submitted) or higher. There is a very strong incentive to not get any kind of rejected submission, because it can quickly tank your MTurk profile and shut off lots of opportunities.
Our group made the calculations to ensure that our task would pay at least minimum wage. As a result, we had to throttle our participants because they would hit our servers pretty hard.
That's not to say that the experience was easy - we had to implement every possible sanity check, and even then we ended up throwing away half our data. For us it was still worth it, since twice the rate was still cheap and we knew not to trust the internet. But a less internet-savy researcher blindly trusting their data would have gotten some surprising results.
* Control words: our server gave them two (unique) control words (one for joining, one for winning), and we asked for them in the post-task questionnaire. Some of these words ended up being reused among several participants, even from participants that never even started the task.
* IP checks: we had an experiment that you could "win" (and earn a bonus), but you only were allowed to play once. Some people restarted the task several times, so we only used the first attempt in a sequence (as reported by their IP and timestamp).
* Data thrown away: we further removed data where we had more than one player per IP (to control for both multiple accounts per person and use of proxies), experiments that were way too fast, and experiments with unsupported browsers (which we explictly mentioned in the description).
Regarding money, we were not allowed by the TOS to withold payment to anyone that filled the questionnaire, even if we knew they did it in bad faith. We therefore implemented the "winning" bonus, and also gave bonus to people who lost but really tried.
I want to point out that a LARGE percentage of participants played honestly, and some of their data was thrown away only out of an abundance of caution. Once you keep the first bad apples out, they simply move to other, easily-exploitable tasks.
What do you mean by this? Do you mean purely requests per second or what?
https://www.youtube.com/watch?v=h9Qt6bVc5mY&list=PLuh2SVJZ17...
The hourly rate will flash every few seconds.
I've been doing side-hustles since I was a teen, and continue today, even though I have a well-paying professional job. My observations are that domain knowledge / specialized expertise always pays better.
You're much better off learning some hobby, almost any hobby, and monetizing it. How do you monetize it? Do content writing. Start flipping on Ebay / Craigslist / etc. Perform / create if you can. etc.
Now - I understand that these (MTurk) kinds of jobs are for desperate people, that want/need the money now - not 4 years down the road. Building expert knowledge will take time, and you will more likely than not get burnt a few times in the start - but the long-term payoff is much better.
Last month, I made $800 in selling two musical instruments. Time invested in it all was around 2-3 hours, mostly on picking 'em up, and shipping them.
Maybe not the answer for everyone, but it's extremely hard for me to imagine any other way now. I currently have three different hobbies which I enjoy very much (and have for years), and I'm lucky enough to have so much knowledge in that I also make a decent side-income off.
Best of all? It doesn't feel like you're working - though you need to treat it like work, if you're gonna make money.
Unless you have a clear
strategy, MTurk work is a
complete waste of time.
It would be a request like:
Gather phone numbers from
this queue of web pages.
You get into the queue, and start pulling up each URL in series, and they're all these expired domains with parked registrar pages for GoDaddy and Tucows or whatever, Along with some Geocities, Angelfile, Tripod and AOL home pages thrown in. Quickly, you get a sense that some fool of data science masters program enrollee paid good money for a dusty, mouldering text file, didn't even look at it, and dropped it right the fuck into a template for a Mechanical Turk task.
Now, there are three immediately obvious courses of action. One, abort and never again consider Mechanical Turk as a useful platform for operating an exchange of effort for rewards. Two, plead with the task owner by reporting feedback to them and ask them to stop for a moment and consider the flaws inherent to this framing of a human activity deemed worthy of compensation. Three, obey the letter of the law, and not the spirit, hold your nose, bellow the words "you asked for it!" and proceed to fill the task with the tech support numbers for all of the domain registrars, hoping that you'll not only get paid for grifting on the task owner, but also possibly inundate all these domain squatting registrars with robocalls trawling for psyche student surveys and questionnaires that will attempt to publish similarly terrible research papers designed with the intent to ostensibly "prove" a flawed hypothesis of human behavior with results that couldn't possibly be replicated because the hypothesis itself begs its own question.
Valuing my time, I just logged out, and haven't looked back since.
They need moderators to mechanical turk the quality of each task, because it benefits no one and wastes people's time, to even propose fruitless, unredeeming tasks.
Unless things have changed since winter, from what I witnessed, there is perhaps zero review of tasks to assess whether a request fits the profile of anything even remotely possible or worth trying.
I have an even cleverer point suggesting that I might not be able to trust workers' self reports. Might my clever rebuttal be right? Yes. But you are using this tool because you want to help ensure fair treatment to workers, right? Fair treatment starts with trusting workers. They will make good faith estimates.
It seems the Mechanical Turk ecconomy could be dwindling due to the accessiblity of AI these days.
[1] https://arstechnica.com/information-technology/2017/11/expen...
>“Most people’s first mistake is they go on there and accept any task they see,” says Naab. “They’ll do these transcriptions — $0.01 to transcribe a whole receipt for Expensify. That’s a terrible ROI.”
We do language related things on MTurk (crowdsourced translation, for example). It's very easy to spot those who have used something like google translate or other artificial means - it's quite rare for a sentence to be translated exactly the same way twice, for starters.
For instance there used to be a lot of HITs that involves transcribing printed receipts.
I wouldn't mind doing this so much if the receipts weren't smudged and creased or otherwise illegible, but I think these people have an OCR that can handle the easy ones and you're left with a residuum of hard cases which sometimes can't be transcribed at all, or for which you'll probably make enough mistakes to get in trouble. I find that mentally fatiguing on top of the extra time.
In my view, this is why you can get paid less than what you deserve, but that option is still attractive to you.
In other words, if you have 34 minutes between things, one of the few paying things you can do is MTurk, even if it doesn't pay well.
I've seen some people on Reddit who are effectively gaming it well enough to actually make consistent money, but that's clearly the exception not the rule.
Else pretty much anything else will be more profitable
Of course he wouldn't be doing literally nothing. Instead he'd be just mindlessly wasting time browsing Facebook, checking out the latest clickbait, posting on forums, and engaging in the sort of activities that studies are increasingly showing are destroying our collective mental health and social cohesion. Instead, he's using that time to make an extra thousand a month.
Been tempted myself. But then I crunch the numbers vs my actual job and go clean my apartment instead.
That plus a very real fear of RSI from all the mouse clicking. I mean I already click at work all day then on hn/reddit in afternoons...
I guess XKCD said it before: https://xkcd.com/974/
I wonder what Nationality these "Turker" are from...