Skip to content

Merge main into releases/v2 #1246

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 58 commits into from
Sep 15, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
c7bb894
Update changelog
aeisenberg Aug 29, 2022
0e98efa
Add support for downloading packs from GHES
aeisenberg Aug 29, 2022
1d92118
Add integration test for using `registries-auth-tokens`
aeisenberg Aug 30, 2022
215c3cb
Autobuild Go in analyze step
angelapwen Aug 26, 2022
8dc4685
Use a more explicit name for checking Go extraction output
henrymercer Aug 30, 2022
fff56ee
Add environment variable for enabling Go extraction reconcilation
henrymercer Aug 30, 2022
e466e75
Simplify `doesGoExtractionOutputExist` implementation
henrymercer Aug 30, 2022
5b4b44c
Add PR check for reconciled tracing with legacy workflow
henrymercer Aug 30, 2022
b42a495
Fix TRAP directory location
henrymercer Aug 31, 2022
14d7039
Add logging for determining whether to run the Go autobuilder
henrymercer Aug 31, 2022
9552903
Fix language inclusion test
henrymercer Aug 31, 2022
8e0846c
Check TRAP directory exists first
henrymercer Aug 31, 2022
a3607ca
Update changelog and version after v2.1.22
invalid-email-address Sep 1, 2022
62fab91
Update checked-in dependencies
invalid-email-address Sep 1, 2022
fe1bd9a
Improve clarity of logging
henrymercer Sep 1, 2022
d05538f
Merge pull request #1227 from github/mergeback/v2.1.22-to-main-b398f525
edoardopirovano Sep 1, 2022
cf5d465
Trace Go when Go extraction reconciliation is enabled
henrymercer Sep 1, 2022
6d34731
Make `createdDBForScannedLanguages` test robust to new async calls
henrymercer Sep 1, 2022
e460fa2
Tidy up `createdDBForScannedLanguages`
henrymercer Sep 1, 2022
e303e2c
Update legacy workflow PR check
henrymercer Sep 1, 2022
1d311fe
Add PR checks for reconciled tracing with autobuilder and build steps
henrymercer Sep 1, 2022
6699d47
Work around `test -v` not being available in Mac Bash
henrymercer Sep 1, 2022
ab396da
Run Lua tracing for Go on Windows in CLI 2.10.4+
henrymercer Sep 1, 2022
70426bb
Merge pull request #1229 from github/henrymercer/reenable-lua-tracing…
henrymercer Sep 1, 2022
e9b47b1
Change to using a single input
aeisenberg Sep 1, 2022
21530f5
Merge pull request #1219 from github/angelapwen/autobuild-in-analyze
henrymercer Sep 2, 2022
62b4f23
Merge remote-tracking branch 'origin/main' into henrymercer/start-go-…
henrymercer Sep 2, 2022
4b5dea8
Address review comments
henrymercer Sep 2, 2022
bea5e4b
Update ava to 4.3.3
henrymercer Sep 2, 2022
b044362
Merge branch 'main' into aeisenberg/ghes-pack-download
aeisenberg Sep 2, 2022
85ba42a
Merge pull request #1231 from github/henrymercer/fix-running-tests-on…
henrymercer Sep 2, 2022
fc2f344
Reuse `getApiDetails` code
henrymercer Sep 2, 2022
ab4197c
Merge remote-tracking branch 'origin/main' into henrymercer/start-go-…
henrymercer Sep 2, 2022
03e3453
Merge pull request #1228 from github/henrymercer/start-go-tracing-in-…
henrymercer Sep 2, 2022
abdf26c
Convert from json to yaml for registries input
aeisenberg Sep 2, 2022
299b774
TRAP Caching: Be tolerant to not finding the extractor
edoardopirovano Sep 5, 2022
8cdb7d8
Merge pull request #1234 from github/edoardo/more-tolerant
edoardopirovano Sep 5, 2022
7e086b2
Merge remote-tracking branch 'upstream/main' into aeisenberg/ghes-pac…
aeisenberg Sep 6, 2022
bf97a6d
Apply suggestions from code review
aeisenberg Sep 6, 2022
376fea6
Clarify description of `registries` input
aeisenberg Sep 6, 2022
add1437
Add workflowEventName
Daverlo Sep 6, 2022
335aa5e
Add test for workflowEventName
Daverlo Sep 7, 2022
a03f3bd
Build js
Daverlo Sep 7, 2022
063e083
Fix linting
Daverlo Sep 7, 2022
4fa3e8b
Gate the new `registries` input behind version constraints
aeisenberg Sep 7, 2022
5dcb9de
Update changelog and input descriptions
aeisenberg Sep 7, 2022
aa434aa
Only run the init-with-registries check on nightly
aeisenberg Sep 7, 2022
5974446
Fix unit tests
aeisenberg Sep 7, 2022
21ae865
Merge pull request #1238 from github/daverlo/workflow-event-name
Daverlo Sep 8, 2022
6085805
Append `/` to end of registries url
aeisenberg Sep 8, 2022
9ba4d50
Merge pull request #1221 from github/aeisenberg/ghes-pack-download
aeisenberg Sep 8, 2022
4a8d26e
Bump CodeQL version to 2.10.5
henrymercer Sep 8, 2022
ff575f2
Add changelog note
henrymercer Sep 8, 2022
3038e97
Remove the lua tracer feature flag check from the codeql-action.
criemen Sep 13, 2022
0a38743
Merge pull request #1240 from github/henrymercer/2.10.5-bump
henrymercer Sep 13, 2022
c256033
Fix runner integration test.
criemen Sep 13, 2022
fd4dc5b
Merge pull request #1244 from github/criemen/remove-lua-tracer-ff
criemen Sep 13, 2022
99d0a6b
Update changelog for v2.1.23
invalid-email-address Sep 14, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Apply suggestions from code review
  • Loading branch information
aeisenberg committed Sep 6, 2022
commit bf97a6da5b294d8318869f0268956978fc78680d
4 changes: 2 additions & 2 deletions init/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ inputs:
required: false
registries:
description: |
A YAML string that defines the list of GitHub container registries to use for downloading packs. The string is in the following forma (the | is required on the first line):
A YAML string that defines the list of GitHub container registries to use for downloading packs. The string is in the following form (the | is required on the first line):

registries: |
- url: https://containers.GHEHOSTNAME1/v2/
Expand All @@ -28,7 +28,7 @@ inputs:
packages: */*
token: ${{ secrets.GHCR_TOKEN }}

The url property contains the url to the container registry you want to connect to.
The url property contains the URL to the container registry you want to connect to.

The packages property contains a single entry or a list of globs specifying packages that can be found in the container registry. Order is important. Earlier entries will match before later entries.

Expand Down
18 changes: 16 additions & 2 deletions lib/config-utils.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion lib/config-utils.js.map

Large diffs are not rendered by default.

8 changes: 5 additions & 3 deletions lib/config-utils.test.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion lib/config-utils.test.js.map

Large diffs are not rendered by default.

9 changes: 5 additions & 4 deletions src/config-utils.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2253,12 +2253,13 @@ test("downloadPacks-no-registries", async (t) => {
go: ["c", "d"],
python: ["e", "f"],
},
undefined,
undefined, // registries
sampleApiDetails,
tmpDir,
logger
);

// Expecting packs to be downloaded once for java and once for python
t.deepEqual(packDownloadStub.callCount, 2);
// no config file was created, so pass `undefined` as the config file path
t.deepEqual(packDownloadStub.firstCall.args, [["a", "b"], undefined]);
Expand All @@ -2283,7 +2284,7 @@ test("downloadPacks-with-registries", async (t) => {
{
url: "https://containers.GHEHOSTNAME1/v2/",
packages: "semmle/*",
token: "still-a-token",
token: "still-not-a-token",
},
];

Expand All @@ -2292,15 +2293,15 @@ test("downloadPacks-with-registries", async (t) => {
packDownloadStub.callsFake((packs, configFile) => {
t.deepEqual(configFile, expectedConfigFile);
// verify the env vars were set correctly
t.deepEqual(process.env.GITHUB_TOKEN, "token");
t.deepEqual(process.env.GITHUB_TOKEN, sampleApiDetails.auth);
t.deepEqual(
process.env.CODEQL_REGISTRIES_AUTH,
"http://ghcr.io=not-a-token,https://containers.GHEHOSTNAME1/v2/=still-a-token"
);

// verify the config file contents were set correctly
const config = yaml.load(fs.readFileSync(configFile, "utf8")) as {
registries: configUtils.SafeRegistryConfig[];
registries: configUtils.RegistryConfigNoCredentials[];
};
t.deepEqual(
config.registries,
Expand Down
36 changes: 25 additions & 11 deletions src/config-utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ export interface UserConfig {

export type QueryFilter = ExcludeQueryFilter | IncludeQueryFilter;

export type RegistryConfig = SafeRegistryConfig & {
export type RegistryConfigWithCredentials = RegistryConfigNoCredentials & {
// Token to use when downloading packs from this registry.
token: string;
};
Expand All @@ -70,7 +70,7 @@ export type RegistryConfig = SafeRegistryConfig & {
* The list of registries and the associated pack globs that determine where each
* pack can be downloaded from.
*/
export interface SafeRegistryConfig {
export interface RegistryConfigNoCredentials {
// URL of a package registry, eg- https://ghcr.io/v2/
url: string;

Expand Down Expand Up @@ -1721,15 +1721,15 @@ export async function initConfig(
return config;
}

function parseRegistries(registriesInput: string | undefined) {
function parseRegistries(
registriesInput: string | undefined
): RegistryConfigWithCredentials[] | undefined {
try {
return registriesInput ? yaml.l(registriesInput) : undefined;
return registriesInput
? (yaml.load(registriesInput) as RegistryConfigWithCredentials[])
: undefined;
} catch (e) {
throw new Error(
`Invalid registries input. Must be a JSON string, but got: ${
e instanceof Error ? e.message : String(e)
}`
);
throw new Error("Invalid registries input. Must be a YAML string.");
}
}

Expand Down Expand Up @@ -1834,7 +1834,7 @@ export async function downloadPacks(
codeQL: CodeQL,
languages: Language[],
packs: Packs,
registries: RegistryConfig[] | undefined,
registries: RegistryConfigWithCredentials[] | undefined,
apiDetails: api.GitHubApiDetails,
tmpDir: string,
logger: Logger
Expand Down Expand Up @@ -1888,7 +1888,9 @@ export async function downloadPacks(
);
}

function createRegistriesBlock(registries: RegistryConfig[]) {
function createRegistriesBlock(registries: RegistryConfigWithCredentials[]): {
registries: RegistryConfigNoCredentials[];
} {
// be sure to remove the `token` field from the registry before writing it to disk.
const safeRegistries = registries.map((registry) => ({
url: registry.url,
Expand All @@ -1900,6 +1902,18 @@ function createRegistriesBlock(registries: RegistryConfig[]) {
return qlconfig;
}

/**
* Create a temporary environment based on the existing environment and overridden
* by the given environment variables that are passed in as arguments.
*
* Use this new environment in the context of the given operation. After completing
* the operation, restore the original environment.
*
* This function does not support un-setting environment variables.
*
* @param env
* @param operation
*/
async function wrapEnvironment(
env: Record<string, string | undefined>,
operation: Function
Expand Down
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy