Have you ever wondered how search engines deliver relevant results to your questions? Or, how do search engines avoid websites that don’t work or host malware? While a lot of people might think that search algorithms or artificial intelligence are responsible, that is only half the picture. In fact, there are actual humans at work too, because even the most sophisticated technology needs a helping human hand.
In the case of search engines, humans develop guidelines to rank the success of a search. Then they hire people to test the search engine and determine how well those guidelines are being met. The job title of those hired to do the testing is variously called remote internet judge, search engine evaluator, web evaluator, or content assessor, among others.
What is a Remote Internet Judge or Search Engine Evaluator?
Remote internet judges review websites, landing pages, and advertisements to determine if the content is high quality, accurate, and likely to satisfy the needs of the searcher. Following a stringent set of guidelines, they rate and record their impressions in an online information management system.
These positions are usually fully remote, so you can work from anywhere. Most of the time, they are contractor positions which means you must handle paying income and payroll taxes on your own. However, there are some W-2 positions available. They are also almost always part-time, requiring a commitment of between 10 – 30 hours per week. When you complete the work is up to you, making it a good job for a side hustle.
What is the Job Description of a Remote Internet Judge or Search Engine Evaluator?
The daily tasks of a remote internet judge or search engine evaluator involve rating search results, evaluating queries, visiting web pages, and answering questions based on a set of guidelines. You will need to know how to navigate websites, search engines, and to find things online. You may have to answer questions that require you to be up-to-date on news, current events, and pop culture.
Guidelines vary by project, and detailed training is usually provided. The training consists of examples of what you will see while working on the project. Besides, you can expect to complete a test on your ability to apply the guidelines before getting access to the project work. Although tasks to be completed don’t usually take long, a few seconds to a minute or so; they require high focus. This can be a challenge because the tasks are usually repetitive and become tedious over time.
One important reason for testing search results in this way is to ensure the localization of results. Because of this, positions are available in a wide variety of languages and locations. Fluency in the language and culture of your location is mandatory for every position.
How Do I Become a Remote Internet Judge or Search Engine Evaluator?
Experience and education requirements for remote internet judges are not usually very stringent. This is because candidates must pass a qualification exam to demonstrate their skill at applying the project’s established guidelines before starting work. As for study aids, employers provide project guidelines and examples; and the tests are comprehensive and rigorous.
Each employer has its own unique qualifications for internet judges; however, they commonly request some qualifications, such as:
- Native level language proficiency
- Reside and be eligible to work in the location advertised in the position
- Strong computer skills and be comfortable performing web research
- Ability to work independently and manage your time efficiently
- Critical thinking and problem-solving skills
- Stable, high-speed Internet connection. Most companies only accept broadband Internet access, not satellite Internet.
- Newer Computer. You must have a relatively new computer, and in some cases, you may need to have certain technology and software products such as a Windows or iOS operating system, Android or Apple smartphone, or others depending on the project.
How Much Does a Remote Internet Judge or Search Engine Evaluator Make?
Pay and work hours vary by company, but you can usually expect to earn anywhere between $11-$15 per hour. Some employers may bump you up to a bonus rate if you meet their quality, throughput, and productivity goals.
Which Companies Hire Remote Internet Judges and Search Engine Evaluators?
Technology giants like Bing, Google, Amazon, and Facebook are a few of the companies that use the data provided by remote internet judges. However, they usually contract with third-party service providers to complete the testing. These third-party companies provide a range of services focused on artificial intelligence and localization in addition to search engine, web, and ad evaluations. Related positions available include transcribers, translators, audio data collectors, video game evaluators, voice recognition testing, and consumer surveys. A few companies that frequently hire for these positions include:
Appen
Appen is probably the largest employer of internet judges following their acquisition of other large employers such as Butler Hill, Leapforce, and Raterlabs. The company specializes in developing artificial intelligence and machine learning training data for eCommerce, software, translation, social media, and annotation companies. With a presence in 170+ countries, Appen’s AI/machine learning model uses human input to improve the quality of its client’s data, products, and technologies. Appen offers a variety of remote positions including project-based, micro-tasks, surveys, and data collection.
Lionbridge
Lionbridge works with companies to help them connect with global customers and employees by delivering translation and localization solutions in 350+ languages. Using a cloud-based technology platform, they apply human and machine intelligence to understand their customers’ customers. Based in Waltham, Mass., Lionbridge has operations in 23 countries. If a position isn’t available in an area you’re interested in, check back in a few days or weeks.
Welocalize
Welocalize works with companies to engage with their international audiences. Their services include multilingual content translation, localization, and adaptation in over 250 languages using their network of over 77,000 in-country contractors. The company blends technology and human input to collect, annotate, and evaluate content types and deliver high-quality training data transformation solutions for language processing machine learning.
While Google usually hires third-party companies to evaluate their web searches, they occasionally hire directly. Referred to as “ads quality raters,” this rare position involves providing feedback on image and text search results, web pages, and other information as required. To find these positions, you can search for “ads quality rater” on Google’s career page.
Teemwork.ai
Teemwork.ai is an IT jobs company offering crowdsourced tasks. They hire people worldwide who have skills in languages and technology to provide market insights. They offer a variety of jobs in addition to search engine evaluators, including hardware testing, data entry, localization, and translation. The company adds new projects periodically, and new opportunities are frequently posted on their jobs page.
KarmaHub
KarmaHub uses its “Karma Crowd” of employees located around the world to collect and analyze data for its clients. They specialize in providing solutions involving artificial intelligence, cloud-first development, and the internet of things. You can find remote internet judge positions or internet analyst positions on their career portal. If there are no current openings, you can email your resume to leads@mykarmahub.com for consideration.
mTurk
Amazon Mechanical Turk, or mTurk, is different from the others in this list because it is a database of micro jobs from many different companies. Most jobs take seconds to complete and pay only a few cents. Nevertheless, if you get good at choosing the best jobs you can make up to $15 an hour. At any one time, there are hundreds of search evaluator tasks available. When you get tired of those jobs, you can switch things up by checking out the thousands of other tasks to choose from.
Pros and Cons of Remote Internet Judge and Search Engine Evaluator Jobs
Before undertaking a position as a remote internet judge or search engine evaluator, it may help to consider the pros and cons.
Pros
- Flexible hours. There are no requirements for what time of day you need to work. This means you can work late at night or first thing in the morning. Whenever you can fit it in.
- Flexible days. Similarly, there is no requirement for what days you work. You can get all your hours in on the weekend or spread them out throughout the week.
- Fully remote. All of your work is completed online, and you will rarely if ever meet your supervisor or colleagues in person.
- Decent pay. Compared to some other internet-based employment opportunities, the pay is pretty good and there are opportunities to increase your rate over time with some companies.
Cons
- Unpaid training. The training is extensive and can take up to two weeks. Getting the hang of the guidelines takes time and passing the qualification exam is not guaranteed. This can mean a lot of wasted time, not to mention you’re not paid for any of it.
- Lack of comradery. You communicate almost entirely in their online system once you establish an account. If you like developing relationships with your colleagues, this is not the job for you.
- No job security. Once the project is complete, there may not be more work. While some projects are long term, some are much shorter. In addition, the company can choose not to renew your contract with little explanation, if any.
- Tedious work. While it takes some time to understand and apply the guidelines correctly in the beginning, it gets easier with time. This can lead to the work feeling repetitive and boring.
Is This the Job for You?
If, after considering the pros and cons, you think you’d like to become a remote internet judge, start by visiting the company websites mentioned above or search the Virtual Vocations jobs database. Keep in mind that many of these companies hire in cycles based on the projects they are working on. As a result, you may need to wait a bit before you can find a suitable position. While you’re waiting, you can check out the other project or micro jobs that might be on offer. Be part of the crowd!
What are the pros and cons of remote internet judge jobs we didn’t mention? Connect with Virtual Vocations on Facebook, Twitter, LinkedIn, Instagram, and YouTube to share your thoughts and tips. We’d love to hear from you!
Join Virtual Vocations
Joining Virtual Vocations grants you access to our hand-picked remote jobs database. Learn how our service works, browse job leads by location and career category, or search hundreds of hand-screened remote jobs to find legitimate work-at-home job leads that match your skills and background. Register for free or contact us for more information on our service guarantee.
Check out our menu of Career Services provided by our team of certified professionals, including resume and career coaching services for remote jobseekers. Resume assessments and writing, LinkedIn profile enhancement, and cover letter writing are available to maximize the success of your remote job applications. Discounts on all services available to subscription members, become one now.
