0% found this document useful (0 votes)
97 views55 pages

Structure QA QC Guidelines - v1.3

Uploaded by

chegg7138
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
97 views55 pages

Structure QA QC Guidelines - v1.3

Uploaded by

chegg7138
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 55

Structured Q&A QC Guidelines

Version 1.3
June 2023

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

1
Table of Contents

1. What is a Structured Q&A?


2. Overview of Structured Q&A
3. Various sections in Structured Q&A
3.1: Course & Topic tagging
3.2: Step by step
3.3: Tools / Editors
3.4: Explanation
3.5: Final solution
4. What is a Structured Q&A Quality Check?
5. Quality Components for Structured Q&A QC
5.1: Primary check for a question
5.2: Primary check for solution
5.3: Taxonomy review
5.4: Solution review
5.5: Definitions of parameters
6. Skip question and Exit
7. Notable Points

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

2
1. What is Structured Q&A?
Structured Q&A is a product that provides consistent, high quality, and legible solutions in a
structured manner to students through the existing Q&A authoring platform. Structured Q&A
makes use of suitable authoring tools that help to enhance the quality of a solution.

2. Overview of Structured Q&A


In Structured Q&A authoring, the system allows one to update the below Metadata fields:
• Sub-subject
• Topic
The step-by-step structured authoring experience allows experts to create a solution in a
structured format as shown below:

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

3
3. Various sections in Structured Q&A
Below are the various sections/tools/ features in Structured Q&A authoring:
3.1: Course & Topic tagging
The first step in authoring a question is tagging. The sub-subject and topic can be selected via the
dropdowns provided on the page. Once they are selected, click Next to move on to the Step-by-
Step solution.

3.2: Step-by-Step
To create better solutions for students, the editor supports the step-by-step functionality. This
functionality allows users to create solutions with steps and required explanations, which are
separated into parts that are easy to follow for the student. Each question begins with one default
step and to add additional steps, click the Add another step button at the bottom of the editor.

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

4
If a solution has multiple steps, each step can be accessed by clicking the appropriate button in the
step title. This button will expand or collapse the step.

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

5
3.3: Tools / Editors
In SQnA authoring, for each content piece, the author can use the required tool support to provide
the solution without uploading an image / handwritten solution.
Given below is a list of tools/editors that we have in the SQnA authoring editor:
• Text Editor
• Table Editor
• T-Accounting templates
• Math-in-Text
• Insert an image
• Code Snippet Inserter
• Math Equation Renderer
• Inline Equation Renderer
• Drawing Tool (Illustration tool)
• Electric Circuit Tool
• Mathway
• Chem draw tool (Chemistry block tool)
• Chem Inline Equation tool
• Graphing tool
• Explanation field

Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

6
3.4: Explanation
To further clarify the work in each step, an optional explanation can be added to the step by
selecting the “Add Explanation” option from the tools bar.

Whenever the Explanation is added to the step, the block will appear in the following format:

The explanation can be added anywhere in the step. Here is a sample format after writing the
content in the explanation field and before submitting the explanation block content.

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

7
3.5: Final Solution
After completing the step-by-step solution, a final solution should be added. The final solution
should be a conclusion to the question in clear and direct language. In case the final solution
encapsulates content that has already been added to the solution and would be laborious to
duplicate (such as a table, or journal entries), a simple sentence pointing to the Step can be added
to this section.

Preview. Once the final solution has been added, the author can see a preview of the completed
solution by clicking the Preview your solution button. This view shows the compiled solution as it
will appear to the student.

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

8
Submitting the solution. Once the solution is completed, the user submits the solution by clicking
the Submit your solution button.

4. What is Structured Q&A Quality Check?


Structured Q&A Quality Review for Experts is a quality audit process for a pre-defined set of
solutions that are picked by the system from each expert’s account for a specific period. An SQnA
Quality Check expert reviews the individual samples to provide detailed feedback for each solution
separately.
The purpose of a quality review is to check plagiarism, copyright issues, honor code violations,
spam, incomplete solutions, irrelevant solutions and adopt structured Q&A to enhance the quality
of the solution.
Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

9
Click on ‘Review SQNA’ button on expert.chegg.com to start SQnA QC:

URL: https://expert.chegg.com/qna/qc
After clicking ‘Review SQNA’ button, the web page shown below will appear:

5. Quality Components for a Structured Q&A QC


The Quality Check expert will share the feedback on Structured Q&A solutions by considering the
five quality components, each with a star rating.
Before clicking on “start review” to award rating for the quality components in the SQnA QC review,
there is a preliminary check for both questions and solutions to report the issues.

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

10
Structured Q&A QC flow chart:

5.1: Preliminary Checklist for Question: The QC expert should crosscheck the question against the
below mentioned reason codes before moving on to the QnA quality review tab.
a) Honor Code Violation
b) Copyright Issue
c) Spam and Solicitation
d) Third Party URL
e) Data Insufficient

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

11
a) Honor Code Violation
Honor code violations are a major offense that can occur both from the student and expert’s side.
Moderation of these situations is critical as we seek to assist our students with learning, not
cheating.
What constitutes an honor code violation?
Questions involving tests, quizzes, or exams:
Any question mentioning an assessment that is undated, marked confidential or within the current
or future academic period, or explicitly prohibited is a violation of our Honor Code.
Below are the details of prohibitions and exceptions:
Prohibited Exceptions
• Questions mentioning any type of undated • Questions mentioning any type of assessment
assessments like “Test”, “Quiz”, “Midterm” from a past academic period are permitted
and “Exam/Examination”. (Example, Quiz March 1990)
• Questions with points/marks mentioned. • Homework/Assignment/Experiment/Lab
• Homework/Assignment/Experiment/Lab without points/marks.
with points/marks mentioned. • Questions mentioning any type of practice
• Questions mentioning any type of assessment quiz, practice test, practice midterm, or
within the current or future academic period practice exam are permitted unless the
with or without points/marks (Examples, question states otherwise.
Midterm Fall’23 (current) and Final Test • Questions mentioning study guides are
Spring 2024 (future)) permitted unless the question states
• Questions from any time that explicitly otherwise.
prohibit the student from seeking help • Graded assignment/assessment questions
(Examples, Test #2 DO NOT SEEK OUTSIDE with past date
HELP and Questions with watermarks
referring to tests, quizzes, or exams.
• Graded assignment/assessment questions
with due date

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

12
Questions involving unpermitted assignments:
Any assignment regardless of type that explicitly specifies no outside help is permitted is a violation
of our Honor Code. This can include any kind of graded or ungraded assignment any time an
instructor specifies this restriction. There are no exceptions to this rule. These assignments include
instructions such as the following:
• “All assignments are individual work, and sharing work is considered cheating.”
• “This assignment must be completed on your own.”
• “No outside help”
Example 1:

Examples of HCV cases:


Example 1:
The following is a screenshot of a question related to ‘Mid-term Exam’ (Future academic year,
2024), and such questions come under honor code violation:

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

13
Example 2:
The following is a screenshot of a question related to ‘Exam’ (current academic year, 2023), and
such questions come under honor code violation:

Example 3:
The following is a screenshot of a question related to ‘Quiz’ (Undated); we assume it to be a current
academic question and such questions, come under honor code violation:

Example 4:
The following is a screenshot of a question related to ‘Test’ (Undated); we assume it to be a current
academic question, and such questions come under honor code violation:

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

14
Example 5:
The following is a screenshot of a question related to ‘ACS Exam’ (Undated); we assume it to be a
current academic question, and such questions come under honor code violation:

Example 6:
The following is a screenshot of a question related to ‘Final Exam’ (Undated); we assume it to be a
current academic question, and such questions come under honor code violation:

Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

15
Example 7:
The following is a screenshot of a ‘ACS DIV CHED Exam’ (Undated); we assume it to be a current
academic question, and such questions come under honor code violation:

Example 8:
The following is a screenshot showing the term “Points” (10p and (15p)); Here, ‘10p’ stands for ‘10
points’ and ‘15p’ stands for ‘15points’. Such questions come under honor code violation:

Note: “(2M)” stands for “(2 Marks)”, “(25P)” or “(25Pts)” stands for “(25 Points)”.
Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

16
Example 9:
The following is a screenshot showing the term “CONFIDENTIAL” on the left top of the second
page. Such questions come under honor code violation:

Example 10:
Questions with watermarks messages like “Do not share this document”, “Do not share”, “Institute
names”, “Confidential”, etc. come under HCV.
Screenshot 1:

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

17
Screenshot 2:

Screenshot 3:

Example 11:
If the professor’s name is showing on the question window, then it comes under HCV.
Example screenshot:

Note: For further queries, please reach out to your manager.


Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

18
Examples of non-HCV cases:
Example 1:
The following is a screenshot of a question related to a Practice ‘Test’. Such questions do not come
under honor code violation:

Example 2:
The following is a screenshot of a question related to ‘Semester Exam’ conducted in 2022 (Previous
year), and such questions do not come under honor code violation:

Example 3:
The following is a screenshot showing the specific term “30 marks” accompanied by the previous
year ‘Oct/Nov 2022’. Such questions do not come under honor code violation:

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

19
Example 4:
The following is a screenshot showing the term “Spring” accompanied by the term ‘Quiz’ and it is
related to the completed year 2018 (4/4/18). Such questions do not come under honor code
violation:

b) Copyright.
Chegg has developed the Chegg Intellectual Property Rights Owner ("CHIPRO") program to protect
the rights of intellectual property owners, and consumers. Once we are alerted to infringing
content posted by our third-party users, we expeditiously remove the content and issue warnings
to any involved parties. These warnings are logged and tracked to ensure that when we find that
there is severe or repeated behavior, we can immediately ban those users from our platform.
Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

20
What qualifies as infringing content?

Any content that may be risky for Chegg to host for ownership reasons is prohibited.

Prohibited Exceptions
▪ Third party URLs ▪ References to public figures or
▪ Logos or brands that imply HCV. institutions
▪ Lecture materials ▪ Images of logos or brands that do
→ Post is from lectures or not imply HCV

other academic ▪ Lists of procedures, methods, or

presentations that are systems


▪ Watermarks that do not imply HCV
copyrighted or claimed by
→ Mobile camera watermarks
an instructor.
are permitted.
→ Materials are explicitly
▪ Free academic resources and open-
specified as not to be
source content references
reproduced or shared in any
way, shape, or form.
▪ Pirated books and /or literature
→ Content indicates that it is
copied directly from a book.
▪ Graphics, diagrams, photographs,
maps

Examples of logos or brands that imply Copyright:


Example 1:
The following are screenshots showing the other ‘online education website names’ (Like,
WileyPLUS, Sapling Learning, WeBWorK, etc.) and such questions come under copyright.
Screenshot 1: “Sapling Learning”

Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

21
Screenshot 2: “WileyPLUS”

Screenshot 3: “WeBWorK”

Screenshot 4: “MasteringChemistry”

Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

22
Example 2: The following are examples of questions having the message of ‘Copyright’.
Screenshot 1:

Screenshot 2:

Screenshot 3:

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

23
Example of Logos or brands that do not come under Copyright:
Monitor logo(s) are acceptable from a violation perspective. To be clearer, do not mark the
question image containing the ‘monitor logo’ as abusive.

Any company or brand names used in the part of the question text does not come under abusive.
QC experts should bring to the notice of their respective managers if they experience any edge
case(s). Make sure that QC expert should get clarity on edge cases before moving the cases to
abusive. This should apply to any company or brand names if they are used in the part of the
question text and the question satisfies our other criteria (Not honor code violation, not invalid,
not abusive, etc.)

Example:

Watermark of printer of the notebook page on which the expert has written his/her solution
(Examples: “Future team”, “Green House”, “Bic” or “Staples” watermark on one of Green
House/Bic/Staples notebooks, “Classmate” watermark on Classmate notebooks) are non-abusive.
Any such similar cases also come under non-abusive.
Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

24
Mobile camera watermarks that do not come under Copyright:
Examples:

Examples of questions having references to institutions:


In the below question images, the name of the data source (ALEKS and "Barley stripe mosaic
virus:Structure and relationship to the tobamoviruses") is provided in the bottom of the question.
Such cases do not come under copyright issues.

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

25
• If the software name is visible along with the software output, then it does not come under
Copyright. It is like mobile watermarks.
c) Spam and Solicitation
Offsite offers and advertisements may occur from both the student and expert sides. Offsite
requests can be more minor unless repeated, but spam is strictly prohibited and will be removed
immediately.
Example given below: In the below example, author shared his/her personal id (Original id hidden)

Spam/solicitation can occur throughout Q&A: in questions and solutions.


Prohibited Exceptions
• Spam advertisement questions or comments • There are no exceptions. All
o 1-800 numbers interactions between students and
o Weight loss supplements or Viagra experts must occur on our platform,
o Chain letters whether in questions or solutions.
o Commercial activities and / or sales,
i.e. contests, sweepstakes,
bartering, advertising, pyramid
schemes
o Unsolicited posts looking to drive
students to contact a specific person
or go to a non-Chegg website.
• Offsite requests from the student or offers from
the expert.
o Help requested or offered offsite via
email, Whatsapp, mobile, or other.
o Can also be a PII violation

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

26
Bad conduct can occur throughout the site in a variety of ways. We often come across behavior
that needs to be cleaned up for the sake of ensuring students a good, safe experience on our site.
How do we identify bad behavior?
Profanity, vulgar language, and sensitive topics most often indicate inappropriate conduct.
Prohibited Exceptions
• Profanity • Misspellings that result in a prohibited word
• Obscene and socially offensive language o “help” mistyped as “hell”.
• Vulgar languages: explicit or offensive references to sex • Abbreviations that result in a prohibited
or bodily function word
• Harassment o “assume” abbreviated as “ass.”
o Material that harasses or advocates
harassment of another, including sexual or
• Biologically relevant words in the correct
violent exploitation
context
• Sensitive topics
o i.e., terms related to sex and
o Violence, death, trauma, or grief
reproduction
o Sex/sexuality, pornography, indecency
o Illegal activities and substances
o Racism, discrimination, immigration
o Bigotry
o Sexism, ageism
o Disability/impairment
o Other ethically questionable behaviour

d) Third-party URLs
An external link is observed in the below question image. Such questions come under abuse, so
flag them with “Third party URL”.

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

27
• If apart of the URL indicates that it is related to a specific website, then it should be flagged.
• If the names of websites are part of the question, it is neither copyright nor HCV. The expert
should take enough care while authoring such questions. Do not use the website names as it
is in the solution but use them without violating guidelines. For example, website name
"Myspace.com" can be used as "Myspace".

e) Data insufficient
Any question that doesn’t have a meaningful question or incomplete question with missing
parameter or conditions, should be flagged with “Data insufficient.”
• Lab-related, project-based, and activity-based questions should be treated as non-abusive if
enough data is available in the question, and the QC expert should review the solution as per
the quality of the solution.
Note: If a question has multiple issues, multiple flags can be selected.
5.2: Primary Checklist for Solution: The QC expert should crosscheck the solution against the below
mentioned reason codes before moving on to QnA quality review tab.
• Answered invalid question.
• Honor Code Violation.
• Copyright Issue.
• Spam and Solicitation.
• Third Party URL.
• Plagiarised answer.
• Direct answer.
• Legibility issue .
• Irrelevant answer. Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

28
• Answered invalid question.
If a question is flagged with any reason code specified in section 5.1, such solutions should be
flagged with the reason code ‘Answered invalid question’.
• Honor Code Violation.
Providing the solution for a question that violates the Honor Code should be flagged with the
reason code “Honor Code Violation”.
o Essays
Do not attempt questions that are explicitly asking to ‘write an essay’ on a specific topic/point.
The QC expert should flag such question(s) and solution(s) with the reason code ‘Honor Code
Violation’.

o Questions including a standalone prompt that requires a written analysis or opinion,


often specifying a required length is prohibited.
o Questions that include explicit requests for the expert to write a fully or partially
written essay are prohibited.
Examples shown below are comes under HCV:

Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

29
• Copyright Issue
If any copyright violation cases specified in section 5.1 (b) are present in the solution, then such
solution should be flagged using the reason code ‘Copyright Issue’.
• Spam and Solicitation
If any spam and solicitation violation cases specified in section 5.1 (c) are present in the solution,
then such solutions should be flagged using the reason code ‘Spam and Solicitation’. Experts should
not provide their personal details like name, email Ids, contact numbers, blog URLs, identity
credentials, confidential correspondences, etc., in the solution space. If such abusive cases are
identified, then such solutions should be flagged with the reason code ‘Spam and Solicitation’.
• Third Party URL
Appearance of any external links in any step/section of the solution should be flagged using the
reason code ‘Third Party URL’.
The expert should not provide other websites link(s) in the solution window. The QC expert should
flag such solutions. Local folder drive address visible in the browser is not abusive.
• Plagiarized answer
Plagiarism is the act of fraudulently representing someone else’s work as new or original content.
Stealing the content produced by someone else is a serious offense of academic dishonesty, and it
is our responsibility to enforce honest work to the best of our ability throughout our expert
network.
What constitutes plagiarism?
Plagiarism is presented in two common forms: direct plagiarism and self-plagiarism.
▪ Direct plagiarism occurs purposefully and dishonestly. Copying/pasting or screenshotting
exact work from another source is direct plagiarism. This includes,
o Solutions/portions of solutions, written passages, excerpts.
o Diagrams or images
▪ Self-plagiarism occurs when an expert has submitted a solution that they have written as
original content more than once. Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

30
o It is also a plagiarism. Full duplication is not permitted; experts are welcome to
revise and improve their previous solution.
▪ A solution may also be considered plagiarized if the QC expert has determined that its
similarity (regardless of %) does not originate from a standard definition, standard theorem,
repeated technical terms, etc.
▪ Copying tables, diagrams, etc., from any source in the solutions comes under plagiarism.
▪ Copying the content/images from any source and modifying it is considered as plagiarism.
▪ Modifying the copied content using paraphrase tools is also treated as plagiarism.
▪ Directly copying software generated output and pasting as it as one’s own work is abusive
(Example: Output of Wolfram alfa). QC expert should flag such solutions.
The QC expert should provide the copied source link(s) in addition to the comments in case of a plagiarized
solution.

Note: In the case of plagiarism check, QC experts can use the available feature to copy the
solution.

• Direct answer
Any solution that performs all the work for a student without any explanation of the steps involved
or attempt at facilitating the learning process is a violation. Solutions must be outlined in a step-
by-step structure; the direct provision of any solution is considered as violation. This type of
violation often occurs in response to MCQ (multiple choice questions) and FITB (Fill in the blanks).
There are no exceptions to this rule.
o Violation: “Answer is (c)”
o Permitted: “Answer is (c) because abc results in xyz, resulting in an answer of (c).”

Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

31
• Examples of direct answers:

• Legibility issue
The solution content that is unclear or blurred or unreadable comes under Legibility issue, such
solutions should be flagged with the reason code ‘Legibility Issue’.
Legibility issue example for unclear: Major portion of the solution contains equations of below type
are difficulty to understand and review.

Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

32
Legibility issue example for blur: Major portion of the solution is blurred as shown below. It is
difficult to understand and review.

Legibility issue example for unreadable: Major portion of the solution is unreadable as shown
below. It is difficult to understand and review.

• Irrelevant answer
If the posted solution is completely unrelated to the given question, then it should be flagged with
the reason code ‘Irrelevant answer’.
Note 1: If incomplete solutions are posted, don’t flag them under irrelevant answer.
Note 2: If a solution has multiple issues, multiple flags can be selected.
Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

33
5.3: Taxonomy Review
In case a sample is reported with any one of the issues listed in 5.1 and 5.2, then there is no need
of further review for the ‘taxonomy review’ and ‘quality components.
In the ‘Taxonomy review’, the reviewer can select the decision ‘Invalid’ to tag the correct
course/topic. If the author-tagged course & topic is correct for the respective question, then mark
it as ‘Valid’ and move to the component-wise quality review.
Dropdowns to select the correct course & topic:

In this dropdown, there will be three system-recommended tags and an overall board-specific
course list for tagging. For the ‘course’ selected by the QC expert, if there is a standard topics list,
then it will be available in the respective dropdown.
If there are no topics under the selected ‘course’, then we will get a message stating ‘No topic
found’.

Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

34
5.4: Solution review
The ‘Solution review’ is to evaluate the subject-specific quality and structure components. At the
backend, for each component, there is a standard weightage to calculate the individual
component-wise score & overall score.
• Concept (25%)
• Explanation (25%)
• Accuracy (25%)
• Structure (15%)
• Answering Guidelines (10%)

Once the QC expert selects the suitable ratings for each component, there is a ‘Add comments’
field to update the overall feedback/comments.
For each quality component, the reviewer can select the suitable sub-component(s) or reason
codes based on the error/issue observed in SQnA response.

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

35
• Accuracy (Star Rating: 1 to 5; Weightage: 25%)
o Incomplete answer
o Incorrect answer
o Typo errors
o Formula error
o Incorrect graph/diagram/tables/charts
o Programming errors
o Grammatical errors
o Units / Notations / Significant figures errors
o Calculation errors.
• Explanation (Star Rating: 1 to 5; Weightage: 25%)
o Missing formulas
o Missing supporting statements
o Table/Plot missing
o Cohesiveness missing
o Diagram missing
o Missing Steps/calculations.
• Concept (Star Rating: 1 to 5; Weightage: 25%)
o Incorrect method
o Conceptual errors.
• Structure (Star Rating: 1 to 5; Weightage: 15%)
o Missed Step-by-Step approach
o Editor tools not used
o Inappropriate explanation
o Inappropriate final answer
o Skipped explanation.
Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

36
• Answering Guidelines (Star Rating: 1 to 5; Weightage: 10%)
o Formatting / Alignment issue
o Math-in-Text not used
o Equation Renderer not used
o Uploaded Improper Image
o Code Editor not used
o T accounting not used
o Table templates not used
o Multipart questions rule not followed
o Text Editor not used
o Chem Inline Equation tool not used
o Chemistry block tool not used
o Drawing tool not used
o Circuit Editor not used.
5.5: Definitions of Parameters

• Accuracy (Weightage: 25%): The key areas to evaluate are, ‘Calculations’, ‘Formula’, ‘Units’,
‘Notations’, ‘Significant figures’, ‘Language’, and ‘Accuracy of the final solution’.
o Incomplete answer
A complete solution starts with the problem's objective or introduces available data/information
or the right approach and then adds all necessary steps/supporting explanations until the solution
is complete.
Select the reason code “Incomplete answer” if the expert does not create a solution for all the
subparts/objectives of the question or does not provide necessary requirements of the solution for
better understanding to the student.
o Incorrect answer
The correct solution to a problem is one that is error-free in conceptual statements, calculations,
graphs, diagrams, tables, formulas, etc., whatever it takes to accomplish the goal.
Select this reason code if the solution has wrong calculations/wrong final solution/wrong content.

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

37
o Typo errors
A meaningful solution is possible if there are no typos in
statements/expressions/substitutions/diagrams/tables/graphs. Select the reason code “Typo
errors” if the solution has typo errors.
o Formula error
The correct formula conceptually drives the major chunks of the problem to interlink with variables
or steps. This will enable them to begin substituting, to assist in the analytical portion, etc.
Select the reason code “Formula error” if the solution uses a wrong formula or a formula that does
not apply to the given conditions.
o Incorrect graph/diagram/tables/charts
As per the requirements of the problem, adding objects such as graphs, diagrams, tables, and
charts to a solution will enhance its value and improve students' understanding. If the solution has
errors in graphs/diagrams/tables/charts, select the reason code.
o Programming errors.
Programming consists of understanding a need and constructing a set of instructions to execute
and to obtain the desired result. There are various programming languages to use based on the
question's requirements.
If the program provided in the solution has syntax/logical errors, select the reason code
‘programming errors.’
o Grammatical errors
Meaningful sentences and conceptually correct statements are possible if the solution does not
contain any grammatical errors.
Select the reason code ‘Grammatical errors’ if the solution has English language grammar or
spelling errors.
o Units / Notations / Significant figures errors
Often, when a problem is being solved, there will be standard units, notations, and significant
figures that should be used as per the problem. However, this should not alter the conceptual
meaning and desired value.
Select the reason code “Units/Notations/Significant figures errors if the solution contains errors
related to unit conversions, units, notations of parameters, significant figures or rounding off
values. Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

38
o Calculation errors
A ‘calculation’ refers to the process of determining a value or expression through logical or
mathematical steps. Correct calculation plays a vital role in finding intermediate values and final
conclusions.
Select the reason code “Calculation errors” if the solution has error/s in calculation, either in steps
or in recalling the correct values from the question/previous steps.
• Explanation (Weightage: 25%): The key areas to evaluate are, ‘Cohesiveness’, ‘Necessary
Explanation’, ‘Diagrams’, ‘Tables’, ‘Plot’, and ‘Steps used in the solution’.
o Missing formulae.
Formulas conceptually drive a major part of the problem to interlink with variables or steps. This
will enable authors to begin substituting, to assist in the analytical portion, etc.
Select this reason code if the solution does not show the formulae/equations used to solve the
steps.
o Missing supporting statements
Conceptual statements and sentences are needed to sustain the richness of explanation and
maintain logical flow throughout the entire process. Authors should place reasoning or supporting
statements wherever required in the solution.
Select this reason code if the solution hasn't provided the necessary statements to explain the steps
followed to solve the question.
o Table/Plot missing
For certain problems, tables and plots are mandated to accomplish the objective and can be
created in a step based on the previous steps / descriptions in the solution or as a step to initiate
follow-up steps within the solution.
Select this reason code if the solution has not provided the requisite tables or plots required to
solve the question.
o Cohesiveness missing
To show cohesion, the solution must be organized by maintaining logical connections between
every line and adding all necessary explanations.
Select this reason code if the solution has no logical flow/relevance between the steps/statements.
Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

39
o Diagram missing
Diagrams can be part of an explanation if the question asks for them, as they also contribute to the
completeness of the solution.
Select this reason code if the solution hasn't provided the diagrams required to solve question or
explain the solution adequately.
o Missing Steps/calculations
Calculations / steps are crucial to understand how intermediate conclusions are placed in the
solution. Authors should demonstrate how a calculation or work was performed to produce a
solution.
Select the reason code “Missing Steps/calculations” if the solution doesn't show the requisite steps
or calculations that are necessary for the students to understand the solution better.
• Concept (Weightage: 25%): The key areas to evaluate are, ‘Conceptual approach’, ‘Method’,
and ‘Accuracy of the Concept’.
o Incorrect method
For each problem, there is a standard procedure to follow throughout the solution. This is based
on the objective of the problem, depth of concept, need for explanation, tables, graphs, diagrams,
and complexity involved in problem solving and so many other factors. A deviation or omission
from this method should be marked as an 'incorrect method' according to the concept needed for
the problem.
o Conceptual errors
Using the correct approach, explanations, formulas, graphs, tables, diagrams, and calculations, etc.,
with all the necessary steps will lead to a conceptually correct solution. Any error that contradicts
the concepts, logic, or rules that must be followed in solving a problem should be considered a
conceptual error.
Select the reason code “Conceptual errors” if the solution has an incorrect application of the
concept or incorrect concept chosen to create the solution for the question.
• Structure (Weightage: 15%): The key areas to evaluate are, ‘Usage of the Explanation field’,
‘Usage of Step-by-Step process’, ‘Math expressions’, and ‘Elaboration of solution by using
Tools/Editors/Templates’.
Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

40
o Missed Step-by-Step approach
By grouping logically connected conceptual parts, calculations, statements, and content pieces and
writing them in each step, the Step-by-Step blocks make it easy to break down the solution into
logical steps. In case the author is unable to organize the solution into multiple steps by using step-
by-step block, the QC expert could label them as ‘Missed Step-by-Step approach' in terms of
structure expectations.
o Editor tools not used
To provide structured Q&A solutions by all subject specific experts, the expert 2.0 platform
provides various tools / editors, such as math-in-text, math equation renderer, math inline
equation renderer, table editor, journal entry table, t-accounting table, code snippet inserter,
chemistry block, circuit editor, and drawing or illustration tool. Authors should use the appropriate
editor based on the nature of the content in the solution; otherwise, the QC expert could label the
solution as "Editor tools not used."
o Inappropriate explanation
Explanation blocks are placed anywhere in steps to provide an overview or description, or to ensure
clarity with supporting statements / formulas. In case the author has provided irrelevant
information about the step or irrelevant / vague comments in the explanation field, it is called
“Inappropriate explanation” and the QC expert could label them as 'Inappropriate explanation.'

Below are few examples of inappropriate explanation:

Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

41
o Inappropriate final solution
At the end of the solution, Expert 2.0 platform has enabled a separate section to provide the
conclusion or final solution. It is advisable to write only the final solution or conclusion based on
the objective of the problem. Also, for multiple subpart questions, the author must provide
solutions for all subparts at one go. This block should not include any other details, steps,
calculations, or comments.
Select the reason code “Inappropriate final solution” for the following cases:
1. the solution has irrelevant statements/comments in the final solution block, below are few
such examples:

2. a part of the solution is included in the final solution block, below are few such examples:

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

42
3. all the final solutions of the subparts are not included in the final solution block, below is
one such example: (Four questions authored but only two final solutions given in the final
solution block)

More examples of inappropriate final solution are given below:

Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

43
o Skipped explanation
The Expert 2.0 answering editor has a separate block to provide additional explanations. Here, it is
recommended to write an overview or description or to ensure clarity through supporting
statements / formulas for the step. 'Explanation block' is optional if the author has included all the
necessary explanations in the step.
• Answering Guidelines (Weightage: 10%): The key areas to evaluate are, ‘Usage of Pre-made-
tables/Templates’, ‘Usage of Math-in-Text’, and ‘Formatting & Alignments’.
o Formatting / Alignment issue.
The solution may contain a variety of content objects. Each of them may expect a required format
or style, font size, alignment, margin, etc., to create an appealing presentation and make the
solution as understandable as possible.
Select the reason code “Formatting/Alignment issue” if the solution either does not have proper
alignment or has provided equations in the incorrect format (Example: Miss alignment of equations
or expressions).
o Math-in-Text not used
Math-in-Text is a tool utilized for 2 line or 3-line calculations where the expression starts with a
formula followed by substitutions and then the results are auto-calculated. Generally, this tool is
used for Accounting, Finance, and other business subjects.
Select the reason code “Missed to use Math-in-Text” if the solution has not used Math-in-Text tool
for providing structured 2- and 3-line math equation calculation, wherein the outcome is auto-
calculated.

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

44
o Uploaded Improper Image
Expert 2.0 platform enabled various editors to insert tables, diagrams, math expressions, chemical
equations, chemical block diagrams, and electric circuits, among others. Instead of using the
current tool, if an author uploads content in image format, select the reason code ‘Uploaded
Improper Image' in the solution.
Below are few examples of high impact & low impact issues on uploading of improper image:

Example of major solution uploaded as image.

Example of minor part of the solution uploaded as image.

Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

45
o Equation Renderer not used.
Inline equation renderer and Math Equation renderer are part of the category Math Equation
Renderer, and they are valuable tools for inserting math expressions & equations in multi-line
calculations and inline within a sentence.
Select the reason code “Equation Render not used” if the solutions that require equations/symbols
are provided without using the equation renderer tool.
Major solution submitted in text instead of tools / editor usage.

Minor part of the solution submitted in text instead of tools / editor usage.

o Code Editor not used


'Code editor inserter' enables editors to write programs in various programming languages. In
addition, it provides indentation to the program, giving the student a similar experience to that of
the respective programming language editor.
Select the reason code “Code Editor not used” if the code in the solution is provided without using
the corresponding code editor.
Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

46
o T-accounting table not used
‘T-accounting table’ is available as template to update entries based on the requirement of the
question. The template facilitates auto-generated results. Authors are expected to use in-built
templates where required to maintain accuracy, uniformity, and a standard structure. Select the
reason code “T accounting not used” if the expert not used T-accounting table in the solution.
o Table templates not used
There are a lot of accounting specific table formats available in Table templates. These include
budget tables, Income statements, Profit/loss statements, balance sheets, etc., that allow users to
insert the required table formats based on question requirements. These templates generate auto-
generated results. Authors are supposed to use in-built templates to maintain accuracy, uniformity,
and a standard structure. Select the reason code “Table templates not used” if the author has not
used table templates for tables in the solution.
o Multipart questions rule not followed
According to the Multi-part question rule, if there are multiple questions in a post, either the first
question in the post or the requested question by the student should be authored. If the author
has authored a different question other than the first question or a question requested by the
student, then QC reviewer should select the reason code “Multipart questions rule not followed”.
o Text Editor not used
Text parts/sentences must be typed in a text editor. The QC expert should select the reason code
“Text Editor not used” if the author uploaded handwritten or typed images.
o Chem Inline Equation tool not used
Chemistry equations / expressions should be written using the Chem Inline Equation tool. The QC
expert should select the reason code “Chem Inline Equation tool not used” if the author uploaded
handwritten or typed images or typed normal text instead of using this tool.
o Chemistry block tool not used
Chemistry structures / blocks / organic compounds must be created using the Chemistry block tool.
Select the reason code “Chemistry block tool not used” if an author uploaded images instead of
using this tool.
o Drawing tool not used or Illustration tool not used
Diagrams should be created using the drawing tool. If the author fails to use this, and uploads
images, select the reason code “Diagram tool not used”. Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

47
o Circuit Editor not used
'Circuit editor' is required to insert electrical circuits. If the author has uploaded handwritten or
image content for any kind of circuit, select the reason code “Circuit Editor not used”.
6. ‘Skip a question’ and ‘Exit’:
The QC expert can use the button ‘Skip question’ to skip a question with the following reasons,

• Don’t have sufficient knowledge.


• Not from my subject group
The QC expert can use the button ‘exit’ when they want to stop this activity.

Note: However, the first skip option (Don’t have sufficient knowledge) should not be used.
Instead, the QC expert is expected to complete the QC review with the help of other experts from
the relevant subject.
7. Notable Points:
• Experts have been instructed to avoid using AI Tools/AI text generative engines to create
solutions and copy/paste from such sources. The review team needs to identify such poor-
quality solutions with concrete evidence and continue to flag with the reason code “Spam
and Solicitation”.
• In any suspected case without proper evidence, the reviewer needs to audit the content as
per the quality. Content leads will guide the QC experts to place specific comments for such
cases to measure them over time.
• In case of multiple questions, experts have been instructed to solve the first question,
unless the student has asked for a specific question to be solved, subject to a minimum of
one question.
Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

48
• In case of multi sub-part questions, experts have been instructed to solve all parts (instead
of 4). However, in case the student has explicitly requested for solutions to specific parts in
the question itself, then Expert would be expected to author only those sub parts and not
all the sub-parts of the question.
• Experts are instructed not to solve questions that explicitly ask for the author’s opinions or
personal inputs. The QC expert should flag such question(s) and solution(s) with the reason
code ‘Honor Code Violation’.
• If self-plagiarism cases are observed, then write a comment "Full duplication is not
permitted; experts are welcome to revise and improve their previous solution. The QC
expert should treat it as plagiarism and proceed accordingly.
• Authors have been instructed not to solve questions that contain images/terms from
proctoring software indicating that the question is an assessment conducted via a
proctoring software, such as “Proctorio is sharing your screen”, “Honorlock is sharing your
screen”, “Requires Respondus LockDown Browser”, “ProctorU is sharing your screen”.
Experts should skip such questions under “spam”.

The QC expert should flag such question(s) and solution(s) with the reason code ‘Honor Code
Violation’.

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

49
HonorLock and Proctorio Exam questions:

• Homework/Assignment/Experiment/Lab with points/marks mentioned are considered as


HCV. The QC expert should flag such question(s) and solution(s) with the reason code ‘Honor
Code Violation’.
• Questions with points/marks mentioned are considered as HCV. The QC expert should flag
such question(s) and solution(s) with the reason code ‘Honor Code Violation’.
• Questions related to previous years with points/marks mentioned are not considered as
HCV. In the below screenshots ‘marked out of 30.00’ and ‘marked out of 1.00’ are given and
marks are not explicitly mentioned in the question. Such questions not considered as
HCV.

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

50
• The presence of specific terms like Final, Midterm, and Assessment in any question
indicates that the question belongs to an exam, test, or quiz. The QC expert should flag
such question(s) and solution(s) with the reason code ‘Honor Code Violation’. The
screenshots below are examples of such questions.

• ‘Graded assignment or Homework’ questions, that is questions having marks, grades, or


points violate Chegg’s Honor Code. Such questions are meant to assess students on their
subjective knowledge and solving them would mean aiding students earn high grades
unethically. Hence expert should skip questions belongings to a graded assignment or
homework under “spam”. However, ‘ungraded assignment and homework’ questions are
okay to solve as these questions are used by students to practice and learn.
Examples are given below:

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

51
• Graded assignment/Assessment questions with due date or undated are considered as
HCV. The QC expert should flag such question(s) and solution(s) with the reason code
‘Honor Code Violation’. Graded assignment/assessment questions with past date are not
HCV.
• Mathway has been integrated in the SQnA editor. Experts are instructed to use Mathway
for suitable problems. In the review process, Mathway generated solutions should not be
considered as plagiarism/abuse.
• As we launched automated solutions in SQnA, experts can accept, reject, or edit the
automated solution before submitting. Automated solution would obviously be part of the
existing SQnA solutions. For this reason, the QC expert should ignore self plagiarism (lifted
from Chegg) for this case and review as per the quality in QC review.
Example of automated solution by “content_automation_prod_user@chegg.com” in SQnA
is given below:

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

52
• As quality components are interdependent, a low score may have an impact on other
components as well. Here, reviewers need to identify the most relevant parameters for
each component and award ratings accordingly.
• Except for the parameter ‘Multiple Question rule not followed’, the component ‘Answering
guidelines’ is dependent on structure expectations like use of editors / tools.
• If the quality or structure issue relates to multiple parameters in any component, then
select those parameters and award the appropriate star rating.
• Experts’ solution should be relevant to the given question and experts should author only
valid questions. The QC expert should flag if the solution is irrelevant to the question details
or invalid.
• Lab-related, project-based, and activity-based questions should be treated as non-abusive
if enough data is available to solve the problem and the QC expert should review the
solution as per the quality of the solution.
• Graded/Non-graded Assignments and Homework Questions with past date are not abusive.
• Any assessment related to previous year is non-abusive.
• A local folder drive address visible in the browser is not abusive.
• There could be multiple questions, out of which the first question is fine as per our
guidelines, but the other questions contain external references. All questions that infringe
our guidelines are abusive, irrespective of which part of the question they are in.
• Experts have been instructed to skip questions posted in a language other than English if
they are unable to understand or translate the language. If any question with a language
other than English is identified authored in SQnA QC reviews, then the QC expert should
review the solution using an online translator.
• “Spring” / “Fall” or “F” with or without points/marks for the current or future year or
undated are considered as HCV. The QC expert should flag such question(s) and solution(s)
with the reason code ‘Honor Code Violation’.
• Take Home Exam, Take Home Test and Take Home Quiz are considered as HCV. The QC
expert should flag such question(s) and solution(s) with the reason code ‘Honor Code
Violation’.

Back
Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

53
• Test/quiz/exam review questions, practice test/quiz/exam questions, sample
test/quiz/exam questions, chapter test preparation, and questions from ‘test bank’ are not
to be considered as HCV.
• If the name of the institution or institution logo is part of the question, then it is not HCV.
• If the institution name or institution logo are explicitly showing in the question window and
the content belongs to the same, then it is HCV. For example, content with ‘Pearson’ logo
or ‘McGraw Hill’ logo, etc. are HCV.
• References to free academic resources is allowed but the content must not be copied. If
reference to textbook name(s) or textbook author(s) are provided in the solution, then it is
not abusive. If third party link is provided as reference, then it is considered as HCV.
• If the programming language is not specified in the question, then the QC expert should not
penalize the author, but review the solution as per the quality.
Some examples of opinion-based solutions are given below, and such questions considered as HCV:

Some examples of valid final solutions are given below:

(1). Example of ‘final solution’ having a numerical value.

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

54
(2). Example of ‘final solution’ having software generated output:

(3). Example of ‘final solution’ having multiple final solutions:

(4). Example of ‘final solution’ having concluding type of final solution:

Back

Confidential Material / Chegg, Inc. © 2005-2023 / All Rights Reserved

55

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy