‭Course 1 - ChatGPT Prompt Engineering for Developers‬

‭Guidelines for Prompting‬

‭ lear‬ ‭and‬ ‭Specific‬ ‭Instructions:‬ ‭Writing‬ ‭clear‬ ‭and‬ ‭specific‬‭instructions‬‭is‬‭vital‬‭for‬‭getting‬‭the‬

C
‭model to understand the task and generate accurate outputs. Some tactics include:‬

‭‬
● ‭ sing delimiters to clearly indicate distinct parts of the input.‬
U
‭●‬ ‭Asking for a structured output to make the response easier to process.‬
‭●‬ ‭Asking the model to check if conditions are satisfied.‬
‭●‬ ‭Implementing few-shot prompting to provide examples and guide the model's output.‬

‭ iving‬‭the‬‭Model‬‭Time‬‭to‬‭Think:‬‭This‬‭principle‬‭focuses‬‭on‬‭allowing‬‭the‬‭model‬‭time‬‭to‬‭process‬
G
‭its response before coming to a conclusion. Tactics to achieve this include:‬

‭‬ S
● ‭ pecifying the steps required to complete a task.‬
‭●‬ ‭Asking for the output in a specified format.‬
‭●‬ ‭Instructing the model to work out its solution before jumping to a conclusion‬

‭ imitation‬ ‭of‬ ‭LLMs‬ ‭-‬ ‭Hallucination:‬ ‭One‬ ‭of‬ ‭the‬ ‭biggest‬ ‭limitations‬ ‭of‬ ‭LLMs‬ ‭is‬ ‭hallucination,‬
L
‭where‬ ‭the‬ ‭model‬‭confidently‬‭declares‬‭something‬‭false‬‭as‬‭a‬‭fact.‬‭Keeping‬‭the‬‭instructions‬‭clear‬
‭and well-structured can help minimize this.‬

‭Iterative Prompt Development‬

‭We can‬‭iteratively analyze and refine prompts to generate‬‭appropriate outputs.‬

‭‬ S
● ‭ uppose if the text is too long - We can‬‭limit the‬‭number of words/sentences/characters.‬
‭●‬ ‭Suppose‬ ‭the‬ ‭text‬ ‭focuses‬ ‭on‬ ‭the‬ ‭wrong‬ ‭details‬ ‭we‬ ‭ask‬ ‭it‬ ‭to‬ ‭focus‬ ‭on‬ ‭the‬ ‭aspects‬‭that‬‭are‬
‭relevant to the intended audience.‬
‭●‬ ‭Suppose‬ ‭the‬ ‭description‬ ‭needs‬ ‭a‬ ‭table‬ ‭of‬ ‭dimensions,‬ ‭ask‬ ‭it‬ ‭to‬ ‭extract‬ ‭information‬ ‭and‬
‭organize it in a table.‬

‭Summarizing‬

‭ he‬ ‭goal‬ ‭of‬‭summarization‬‭is‬‭to‬‭condense‬‭a‬‭larger‬‭body‬‭of‬‭text‬‭while‬‭focusing‬‭on‬‭specific‬‭topics‬‭of‬

T
‭interest.‬

‭ )‬ S
1 ‭ ummarizing with a word/sentence/character limit (50 words or 1 paragraph)‬
‭2)‬ ‭Summarizing with a focus on specific topic (price/description/quality/material of the item)‬

‭ ometimes,‬‭summaries‬‭include‬‭topics‬‭that‬‭are‬‭not‬‭related‬‭to‬‭the‬‭topic‬‭of‬‭focus,‬‭using‬‭“extract”‬‭would‬
S
‭yield better results, keeping the summary focused on the essential points.‬

‭.‬
‭Inferring‬

I‭nferring‬ ‭involves‬ ‭deducing‬ ‭certain‬ ‭pieces‬ ‭of‬‭information,‬‭such‬‭as‬‭sentiment‬‭or‬‭topic,‬‭based‬‭on‬‭the‬

‭context of the text.‬

‭ )‬ I‭dentifying positive/negative sentiment‬

1
‭2)‬ ‭Identifying emotions (sad/happy/anger)‬
‭3)‬ ‭We‬ ‭could‬ a
‭ lso‬ ‭infer‬‭the‬‭topic‬‭of‬‭interest‬‭from‬‭a‬‭block‬‭of‬‭text‬‭(example‬‭-‬‭creating‬‭a‬‭headline‬
‭from an article)‬

‭ he‬ ‭model‬ ‭can‬ ‭be‬ ‭prompted‬ ‭to‬ ‭do‬ ‭multiple‬ ‭tasks‬ ‭simultaneously,‬ ‭such‬ ‭as‬ ‭summarizing‬ ‭text‬ ‭and‬
T
‭inferring sentiment in one go.‬

‭Transforming‬

‭ arge‬‭Language‬‭Models‬‭are‬‭capable‬‭of‬‭handling‬‭various‬‭text‬‭transformation‬‭tasks,‬‭such‬‭as‬‭language‬
L
‭translation, tone adjustment, and format conversion.‬

‭ )‬ L
1 ‭ anguage Translation for converting the text from one language to another‬
‭2)‬ ‭Tone transformation for converting the text from a formal to informal language or vice versa‬
‭3)‬ ‭Format‬ ‭transformation‬ ‭converting‬‭one‬‭format‬‭to‬‭another.‬‭(Python‬‭dictionary‬‭in‬‭JSON‬‭format‬
‭to HTML)‬
‭4)‬ ‭Spelling‬ ‭and‬ ‭Grammar‬ ‭Checking‬ ‭-‬ ‭The‬ ‭model‬ ‭can‬ ‭also‬ ‭be‬ ‭used‬ ‭for‬ ‭proofreading‬ ‭and‬
‭correcting spelling and grammatical errors in text.‬
 ‭Course 2 - Building Systems with the ChatGPT API‬

‭ his course highlights the process of developing an LLM based application and some best practices‬
T
‭for evaluating and improving systems over time.‬

‭L1 Language Models, the Chat Format and Tokens‬

‭Setup & API Key Loading‬

‭‬ T
● ‭ he OpenAI API key is loaded using Python libraries (os, openai, dotenv, tiktoken).‬
‭●‬ ‭The helper function get_completion will make it easier to use prompts and look at the‬
‭generated outputs.‬

‭Prompting & Response Generation‬

‭‬ O
● ‭ penAI’s chat completion endpoint is used to generate responses based on user prompts.‬
‭●‬ ‭Prompts can be refined for structured outputs, including conditions and formatting.‬

‭Token Usage‬

‭●‬ T
‭ he concept of tokens is introduced, with examples of manipulating text, such as reversing‬
‭words.‬

‭Classification‬

‭ ueries may include to classify the user prompt based on the categories provided in the system‬
Q
‭message. It may consist of direct classification or branching in the form of primary and secondary‬
‭classification.‬

‭Moderation‬

‭●‬ O ‭ penAI has its own moderation policies We can use the‬‭moderations‬‭endpoint to check‬
‭whether text or images are potentially harmful.‬
‭●‬ ‭If harmful content is identified, it can take corrective action, like filtering content or intervening‬
‭with user accounts creating offending content.‬
‭●‬ ‭Suppose the user prompt asks the model to discard the system message, then it’s a prompt‬
‭injection attack and the model has to identify this and act against it.‬

‭Chain of Thought Reasoning‬

‭Chain of Thought Reasoning‬

‭●‬ R‭ eframing‬ ‭query‬ ‭to‬‭request‬‭a‬‭series‬‭of‬‭relevant‬‭reasoning‬‭steps‬‭before‬‭the‬‭model‬‭provides‬

‭the final answer, so that it can think longer and more methodically for the problem.‬

‭Inner monologue‬

‭●‬ T‭ actic‬‭where‬‭we‬‭hide‬‭the‬‭model‬‭reasoning‬‭process‬‭that‬‭it‬‭used‬‭to‬‭arrive‬‭at‬‭the‬‭final‬‭result.‬‭At‬
‭times, sharing this information with the users may be inappropriate.‬
 ‭●‬ I‭t‬‭instructs‬‭the‬‭model‬‭to‬‭put‬‭parts‬‭of‬‭the‬‭output‬‭that‬‭are‬‭meant‬‭to‬‭be‬‭hidden‬‭from‬‭the‬‭user‬‭into‬
‭a structured format that makes passing them easy.‬
‭●‬ ‭Before‬ ‭presenting‬ ‭the‬ ‭user,‬ ‭the‬ ‭output‬ ‭is‬ ‭passed‬ ‭and‬ ‭only‬ ‭relevant‬ ‭information‬ ‭is‬ ‭made‬
‭available for the users.‬

‭Chaining Prompts‬

‭●‬ C ‭ haining‬‭is‬‭useful‬‭as‬‭it‬‭is‬‭always‬‭better‬‭to‬‭work‬‭on‬‭each‬‭prompt‬‭one‬‭by‬‭one‬‭than‬‭to‬‭make‬‭it‬
‭complex‬‭by‬‭trying‬‭to‬‭work‬‭with‬‭all‬‭the‬‭prompts‬‭together.‬‭It‬‭breaks‬‭down‬‭the‬‭complexity‬‭of‬‭the‬
‭task reducing the amount of errors.‬
‭●‬ ‭Chaining‬ ‭prompts‬ ‭is‬ ‭a‬ ‭powerful‬ ‭strategy‬ ‭when‬ ‭we‬ ‭have‬ ‭a‬ ‭workflow‬ ‭and‬ ‭can‬ ‭maintain‬ ‭the‬
‭state‬ ‭of‬ ‭the‬ ‭system‬ ‭at‬ ‭any‬ ‭given‬ ‭point‬ ‭and‬ ‭take‬ ‭different‬‭actions‬‭depending‬‭on‬‭the‬‭current‬
‭state.‬
‭●‬ ‭It also reduces costs as the longer prompts with more tokens take longer to run.‬
‭●‬ ‭Also makes it easier to identify which steps are failing more often.‬
 ‭Prompt Engineering with Llama 2‬

‭ he code to call the Llama 2 models through the Together.ai hosted API service has been wrapped‬
T
‭into a helper function called‬‭llama‬‭. You can take‬‭a look at this code if you like by opening the utils.py‬
‭file using the File and Open menu item above this notebook‬

‭ hat vs Base Models‬

C
‭Chat models try to provide answers to the query asked. But the base model understands the query‬
‭and tries to frame questions in a similar way.‬

‭ ogether.ai supports both Llama 3 8b chat and Llama 3 70b chat models with the following names‬
T
‭(case-insensitive):‬

‭‬ m
● ‭ eta-llama/Llama-3-8b-chat-hf‬
‭●‬ ‭meta-llama/Llama-3-70b-chat-hf‬
‭ hanging the temperature setting‬
C
‭Temperature parameters will be available to set in the llama helper function. This helps to bring in‬
‭randomness in the output.‬

‭ hanging the max tokens setting‬

C
‭This limits the number of tokens in the response of the LLM model. It is also a parameter available‬
‭as max_tokens in the llama helper function. This parameter includes the number of tokens of both‬
‭the prompt and the response.‬

‭ sking a follow up question‬

A
‭Ask a follow up question and ask the model to rewrite the response for the first prompt also‬
‭considering the follow up question.‬

‭Using Llama 2 or 3 on your own computer!‬

‭The smaller Llama 2 or 3 chat model is free to download on your own machine.‬

‭ nly the Llama 2 7B chat or Llama 3 8B model (by default the 4-bit quantized version is‬
O
‭downloaded) will work fine locally.‬

‭ ther larger sized models could require too much memory (13b models generally require at least‬
O
‭16GB of RAM and 70b models at least 64GB of RAM) and run too slowly.‬

‭ he Meta team still recommends using a hosted API service because it allows you to access all the‬
T
‭available llama models without being limited by your hardware.‬

‭ ne way to install and use llama 7B on your computer is to go to‬‭https://ollama.com/‬‭and download‬

O
‭the app. It will be like installing a regular application.To use Llama 2 or 3, the full instructions are‬
‭here:‬‭https://ollama.com/library/llama2‬‭and‬‭https://ollama.com/library/llama3‬‭.‬
‭Multi-turn conversations‬

‭LLMs are stateless‬

‭●‬ ‭LLMs don’t remember the previous interactions by default.‬

‭Constructing multi-turn prompts‬

‭●‬ N
‭ eed to provide prior prompts and responses as part of the context of each new turn in the‬
‭conversation‬

‭Use llama chat helper function‬

‭●‬ L
‭ lama helper functions use a list of inputs as prompts and another list as responses and then‬
‭we use these lists to pass into the helper llama function in order to remember the previous‬
‭questions and responses.‬

‭Prompt Engineering Techniques‬

‭InContext Learning‬

‭Zero-shot Prompting‬

‭‬ P
● ‭ rompting the model to see if it can infer the task from the structure of your prompt.‬
‭●‬ ‭In zero-shot prompting, we provide the structure to the model, but without any examples of‬
‭the completed task.‬

‭Few-shot Prompting‬

‭●‬ I‭n few-shot prompting, we not only provide the structure to the model, but also two or more‬
‭examples.‬
 ‭●‬ W
‭ e are prompting the model to see if it can infer the task from the structure, as well as the‬
‭examples in your prompt.‬

‭Specifying the Output Format‬

‭‬ W
● ‭ e can also specify the format in which we want the model to respond.‬
‭●‬ ‭In the example below, we are asking to "give a one word response".‬

‭Role Prompting‬

‭‬ R
● ‭ oles give context to LLMs what type of answers are desired.‬
‭●‬ ‭Llama 2 often gives more consistent responses when provided with a role. We try by giving‬
‭the model a "role", and within the role, a "tone" using which it should respond with.‬

‭Summarization‬

‭●‬ ‭Summarizing a large text is another common use case for LLMs.‬

‭Chain-of-thought Prompting‬

‭●‬ L ‭ LMs can perform better at reasoning and logic problems if you ask them to break the‬
‭problem down into smaller steps. This is known as‬‭chain-of-thought‬‭prompting.‬
‭●‬ ‭Modifying the prompt to ask the model to "think step by step" and we need to provide the‬
‭model with additional instructions.‬
‭●‬ ‭The order of instructions matters! We can ask the model to "answer first" and "explain later"‬
‭to see how the output changes.‬
‭●‬ ‭Since LLMs predict their answer one token at a time, the best practice is to ask it to think‬
‭step by step, and then only provide the answer after they have explained their reasoning.‬