Overcoming Content Filtering in GitHub Copilot: Tips for Better Prompts

Introduction

This post aims to address potential issues with content filtering (briefly) and some strategies to work within the ethos of the policy while still being productive with GitHub Copilot. The content filter is switched on by Organization admins often for good reasons and we need to help find ways to still proceed when we are blocked, without breaking the rules. By addressing this and adapting some of the provided examples to your own prompting skillset, you will be able to maintain flow when working with GitHub Copilot.

Understanding Content Filtering

GitHub Copilot has several built-in safety mechanisms for responses, these include things like inappropriate language but also tools to ensure code suggestions are unique. I can’t do the definition of the suggested code matching policy justice, so I have quoted it here and linked to the documentation.

This is a direct quote from the “Policies for suggestion matching” GitHub Documentation.
Organization settings include an option to either allow or block code suggestions that match publicly available code. If you choose to block suggestions matching public code, GitHub Copilot will check potential code suggestions and the surrounding code of about 150 characters against public code on GitHub. If there is a match, or a near match, the suggestion is not shown.

In my experience code suggestions being blocked are less than 1% of my chat conversations with GitHub Copilot. When it does get triggered it is usually obvious why, I am asking for something really generic. However, every now and then it throws even the best of us to understand what is causing the suggestion to be blocked and that is why I wanted to write this document.

Recognizing a Blocked Response

You ask a question or are asking for an edit to your code and you see a message like this in your chat window.

Or you might see a simpler message like this

In InteliJ it might look like this

Strategies to Overcome Content Filtering

Let’s now examine some strategies and examples you have asked for a Write Unit tests for my app and got the message it has been blocked. This might get blocked because the unit tests would be generic and likely to have been written verbatim in many public projects.

1) Try appending uniqueness

Make your prompt more specific to your codebase or use case; this could be as simple as appending Please make the tests unique to this codebase or asking Write unit tests for this specific #selection of code

2) Try asking for a smaller response

Break down large requests into smaller, more focused prompts for example append Write the descriptions of the tests and plan so we can implement one at a time later. or dig into a specific route. Write a single unit test for this endpoint only
In a similar way, you can ask GitHub Copilot to break the response into smaller blocks, for example Write unit tests for this #selection of code, in your output code only include one assertion per test and add a comment describing each test from here you can build further prompts to include the missing features.

3) Only show changed lines of code

If you ask for only the changed lines of code in GitHub Copilot the output will be smaller and interspersed with your existing code. This will be less likely to match in the public code filter. An example would be Write Unit tests for my app; but only show the changed lines please

4) Only show pseudocode as the output

If the code GitHub Copilot responds with could be considered public code then ask about the algorithm and use that to break down the next prompts (or even work inline) through the problem at hand. An example might be along the lines of Help me write the pseudocode for the Unit tests that test the happy path for this #selection of code

5) Try including more specific instructions

Add details about what you want, including edge cases or formatting like this example Explicitly test for nulls in strings as this is a failure mode I want to protect against.
Another example might be to explicitly include the libraries and version you want to use as part of your testing Write unit tests for this #Selection of code; Explicitly test this code using JUnit test 5.12.2 and ensure there are mocks for the interactions with x and y functions

6) Try Including Example Output Unique to Your Codebase

Provide a sample of your existing code or test format. Example: Here is an example of a current unit test: [code block]. Please replicate this format and create additional tests for the rest of the code in this function

Tips for writing effective prompts that don’t trigger the content filter

Generally when we are prompting we should always use clear, concise language. It must also be specific in what you want GitHub Copilot to do. I have seen the content filter block responses when prompts have simply been an error code copy and pasted. This would be because plenty of other people have seen exactly the same error before and GitHub Copilot is suggesting something extremely common. Try asking the question in terms of your codebase and the error message to either help you fix it or highlight where the problem might be.

Ensure you are referencing files, functions, or formats specific to your project. By giving GitHub Copilot more context of your unique code, it is less likely to suggest code that is generic and more inline with your desired outcome.

Context is key, consider not only the code you have selected, the files you have open, or the files the IDE is included in the context, but also the chat window history. If you have asked GitHub Copilot to describe the applications’ function, it will consider that as well as part of the answer and in doing so you may get a better answer in the next prompt than if you started a new prompt. It is like building layers of knowledge together to get a better answer out the end.

Consider capturing the best prompts within your business and sharing them with other developers.

Finally, iterate and refine your prompts based on GitHub Copilot’s responses. It is infinitely patient and will always let you go back and try again or roll on and amend additional context.

Conclusion

Prompt engineering is crucial when using GitHub Copilot, especially when dealing with content filtering. Carefully crafting your requests can significantly impact the quality and relevance of the suggestions you receive. By being specific, providing context, and structuring your prompts effectively, you can avoid generic or blocked responses and get more tailored and useful code completions.

Furthermore, it's highly beneficial to experiment with different prompting techniques and share your successful prompt patterns with others. This collaborative approach can help everyone in the team or organization leverage GitHub Copilot more effectively. By exchanging insights and best practices, you can collectively discover ways to overcome content filtering and optimize your workflows. Please consider sharing your own tips and tricks in the discussion on this post!

Resources

• Copilot policy for matching code

Finally

I want to thank Matthew, Saurav, and Tuvy for helping me review and refine this post, the GitHub Copilot team for an amazing product and the Community team for this forum to share with you all.

Remember if you have any tips or comments feel free to join the conversation!