In this task, you will be shown pairs of phrases referred to as Phrase A and Phrase B. Assume that these were spoken to the Google Assistant. Your job is to determine whether Phrase A and Phrase B are similar in the sense both sentences seek similar information.
A few guidelines:
- Certain words might have the same meaning to the Google Assistant, even if they don't mean the same thing in everyday language. In such cases the sentences should be considered similar.
- The tense of the phrase(e.g., past vs. present tense) is important and different.
- Some phrases may differ in politeness level. As long as they both have a similar semantic meaning, the phrases should be considered similar.
- Some phrases may include specific trigger words directed to the Assistant, such as "hey google", "ok google", "hey Siri", "hey Alexa", etc. You should ignore trigger words when rating.
- Action (e.g., "Set a timer") and help-seeking (e.g., "How to set a timer") sentences should be considered different as they seek different Assistant responses.
Below is a list of rating examples.
Examples
|
Sentence A |
Sentence B |
Semantically Similar |
Comments |
|
who taught you |
who created you |
Yes |
"Taught" and "created" mean the same to the Assistant. |
|
how are you google |
how are you doing |
Yes |
You can assume that these sentences are spoken to the Google Assistant. Both sentences are asking about the state of the Google Assistant. |
|
ok google can you help me |
could you please help me |
Yes |
Trigger words ("ok google", "hey google") should be ignored, and politeness level is not important. |
|
take screenshot |
how do I take a screenshot |
No |
Action and help-seeking sentences are different. |
|
what did you do yesterday |
what did you do last week |
No |
Seeking different information. |
|
What did you eat for breakfast |
What did you eat for dessert |
No |
Seeking different information. |
|
who's your favorite Superstar |
who's your favorite hero |
No |
Seeking different information. |
|
what is the craziest thing you have ever done |
what is the best thing you have ever done |
No |
Seeking different information. |
For more information on how Google shares and manages information, see Google’s Terms of Service and Privacy Policy.