Scribd
Upload
676 views3 pages

Search Satisfaction Guidelines - 210820

This document provides guidelines for evaluating search result pages based on how well they meet a user's expectations for their query. It defines key terms like query, keyword, and satisfaction. The task is to understand the user's query, location, and expectations and then evaluate how well the result page could satisfy them. Questions assess if the page fully, somewhat, or does not meet expectations based on addressing keywords and scope. Examples illustrate labeling pages as meets, somewhat meets, or does not meet for sample queries and results. Situations where a page cannot be judged are also outlined.

Uploaded by

Ryan Silva
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
676 views3 pages

Search Satisfaction Guidelines - 210820

This document provides guidelines for evaluating search result pages based on how well they meet a user's expectations for their query. It defines key terms like query, keyword, and satisfaction. The task is to understand the user's query, location, and expectations and then evaluate how well the result page could satisfy them. Questions assess if the page fully, somewhat, or does not meet expectations based on addressing keywords and scope. Examples illustrate labeling pages as meets, somewhat meets, or does not meet for sample queries and results. Situations where a page cannot be judged are also outlined.

Uploaded by

Ryan Silva
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

Search Satisfactions Guidelines

Terms to Understand

Query – a question, especially one addressed to a search engine.

Keyword – a word or concept of significance.

Label – the answer selected for each question within the Hitapp.

Result Page – the webpage given in the hitapp to be evaluated.

Aspect – a particular part or feature of something.

Satisfaction – fulfillment of one’s wishes, expectations, or needs.

Expectations – the belief about, or mental picture of, the outcome.

Task Overview

When a user issues a query on a search engine, their goal is to find a webpage that satisfies their needs.
While we cannot know exactly what the user was looking for when they made the query, we can still
evaluate each possible page and determine if how well it could satisfy the user. Your task will be to
consider whether the user had a specific webpage in mind and whether they would be satisfied by the
result page.

Steps for Judging

Understand the query – Read the query. Use the Query Intent Description, Bing Results and Google
results to ensure you understand what the user is searching for. Think about what the keywords of the
query are.

Understand the User Location – Understand the user’s location and how it could affect the results the
user would want.

Understand the Result Page – Read through the main content of the Result Page and determine what
and how the different aspects of the page could affect the user’s satisfaction.

Understanding the Questions:

1. How well does the result page meet the user’s expectations for the query?
a. Understand the user’s expectations based on the query and the location of the user.
The rate the page based on how well it would meet their expectations.

Understanding the Labels:


1. How well does the result page meet the user’s expectations for the query?
a. Meets – The result page fully or largely meets the user’s expectations.
b. Somewhat Meets – The result page would meet the expectations of only a few
users. Either not many users would look for this information, or the page has several
issues or flaws.
c. Does not Meet – The result page does not meet the expectations of even a few
users. The result page may be unrelated to the query.

Labeling the Hitapp – select the label that best answer the hitapp questions.

Questions to consider:

Does the Result Page address all of the keywords of the query?

Does the Result Page focus only on the query? Is it also about other topics as well?

Does the scope of the Result Page match the scope of the query? Is one more narrow than the
other? Is one more broad?

Examples

Query Result Page Label Explanation


Iflagler https://iflagler.org Meets The result page
virtual meets the user’s
school expectations as it is
the homepage of
the Iflagler School.

New York https://www.usnews.com/topics/organizations/new_york_times Somewhat The results page


Times Meets somewhat meets
the user’s
expectations as it
provides headlines
from the New York
Times, however the
user would prefer
the official website
instead of an
aggregator site.
Craigslist https://orangecounty.craigslist.org/ Does not The result page
orange Meet does not meet the
county fl user’s expectations
as the user is
looking for
Craigslist in Orange
County, FL while
the result page is
Craigslist for
Orange County, CA.

Can’t judge the hit? – some hits can’t be judged for reasons that are outside of your control. If that
occurs, please select the appropriate reason and submit.

• In Wrong Language – if the query or result page is in a different language from the one
marked in the title (Search Intent [EN]). English queries with English Result Pages should
always be judged normally.
• Does Not Load – If the page doesn’t load.
• Adult Content – If the page has adult content.
• Requires Login – If you cannot view the content of the page without logging in to the
site.
• Harmful Content – If you suspect the page might download something harmful to your
computer (spyware or malware).

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy