Scribd
Upload
38 views14 pages

Onboarding Session 04-06-2025

Uploaded by

ludiseriously
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
38 views14 pages

Onboarding Session 04-06-2025

Uploaded by

ludiseriously
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 14

Onboarding

Onboarding

Welcome to Expert
Sturgeon
Useful Documents When Tasking

● Keep this presentation close at hand


● Instructions for Expert Sturgeon
● Coding languages sheet

🟣 Important: Bookmark the instructions


so you can reference when you start
tasking.
Rules & Work Ethics

❌ Donʼt use AI. You will get banned

✅ Both the code in the prompt and the responses need to be tested
thoroughly

✅ The goal of RLHF is human feedback - if you rely on AI you will not
succeed in this project
Why work for Outlier AI?
● Owned By Scale AI - One of the fastest growing startups in
Silicon Valley's history

● LLMs need more data - A platform that allows anyone with


expertise to contribute where needed

● New freelance format - As all new things, leads to scepticism


amongst some freelancers and leads to many not going through
with onboarding

● But due to this scepticism there is opportunity - Clear path to


more responsibilities and all that more responsibility entitles

● Get better at coding, especially TESTING code - and


understanding what LLMs are bad and good at
Current problems on Outlier
● Many scammers, cheaters or people not taking
it seriously in these coding projects 80%

● Banned contributors are very loud on social


media

● In general, you wonʼt be disabled from projects


unless you cheat or donʼt take it seriously

● Since the format is new, many donʼt take it


seriously and/or cheat

● Ask you colleagues, as in team members, pod


leads and QMs for help
Important aspects to keep in mind

1. The instructions on the platform versus the instruction


documents can be confusing. Projects within every genre.
Prioritize the official instructions.

2. Rely on your gut feeling. If you canʼt find the answer in the
instruction documents

3. Talk to your colleagues. Your squad lead and QM are there


to help!

4. Again: you should make an effort to test your code. Itʼs not
sufficient to only press the play button in the code example.

5. The linter errors ARE annoying. Ignore when youʼre certain.


QMs host Office Hours for contributors
Go to your Outlier dashboard and sign up for your projectʼs office hours.
Task Overview

● A  Check/Rewrite the prompt


● B  Prompt quality assessment
● C  Model failure evaluation
● D  Per-model response ratings & justifications
Step A: Check/Rewrite the prompt
1. Review prompt and identify:
2. Domain categories L0, L1, L2
3. Task intent
4. Verify prompt meets all requirements:
5. ✓ Correct language tag (e.g., ```python)
6. ✓ Complete task coverage
7. ✓ Medium or Hard complexity
8. ✓ Self-contained and executable
9. ✓ Original content (no copied code)
Step B: Double check prompt
● Re-read the prompt you just wrote/revised
● Verify against all criterias
● Edit the prompt until at least one of the responses contains an error
● If the prompt isnʼt good enough itʼs impossible to get a good grade. Th
means that it needs to be a prompt that can be measured and
evaluated.
Step C: Model Failure Evaluation
● Generate responses from all four models
● Read each model's response explanation
● Decide if at least one model failed to meet task requirements
● Record as Yes/No
Step D: Per-Model Response Ratings & Justifications
For each of the four model responses:

1. Rate every rubric dimension 14 or N/A


2. Write justification 80+ characters) for each rating
3. If rating is N/A, provide justification
Important reminders:

● Save task ID immediately to prevent loss of work


● Skip if coding language is unfamiliar/low skill
● Fill out language preference sheet Slide 2
● Time commitment: up to 3 hours per task
● Requires full attention
● Challenging for inexperienced coders
Thank you for
listening:
Any questions?
Ask in chat or raise your hand.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy