Skip to content

download from node-versions and fallback to node dist #147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 32 commits into from
May 19, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
ecf0ce6
wip
bryanmacfarlane Apr 24, 2020
8885b71
updated tests
bryanmacfarlane May 2, 2020
889def3
update ts-jest
bryanmacfarlane May 2, 2020
ec6092f
dbg
bryanmacfarlane May 2, 2020
1c2f59f
format
bryanmacfarlane May 2, 2020
22c0aea
cya snapshot
bryanmacfarlane May 2, 2020
8a1d983
format
bryanmacfarlane May 2, 2020
efab84a
dist
bryanmacfarlane May 2, 2020
ed3a918
Merge pull request #153 from actions/start-v2
bryanmacfarlane May 2, 2020
2bdb2ab
strip 1 on fallback extract
bryanmacfarlane May 2, 2020
501fd29
Merge branch 'master' into node-versions
bryanmacfarlane May 2, 2020
e3ad114
dist
bryanmacfarlane May 2, 2020
2fb08c4
update workflows
bryanmacfarlane May 3, 2020
5e5ef8f
update
bryanmacfarlane May 3, 2020
1b2d431
win change
bryanmacfarlane May 3, 2020
c8617ac
win change
bryanmacfarlane May 3, 2020
42746a4
dbg
bryanmacfarlane May 3, 2020
a414557
dbg
bryanmacfarlane May 3, 2020
f7c5caf
dbg
bryanmacfarlane May 3, 2020
dd6f5ab
only node binary issue
bryanmacfarlane May 3, 2020
11f9205
no proxy issue
bryanmacfarlane May 3, 2020
beb2155
testing 7z alt
bryanmacfarlane May 3, 2020
9fcef3f
testing 7z alt
bryanmacfarlane May 3, 2020
6a1b66a
dbg
bryanmacfarlane May 3, 2020
8e22109
dbg
bryanmacfarlane May 3, 2020
5e7b076
dbg
bryanmacfarlane May 3, 2020
a80a170
no proxy fix
bryanmacfarlane May 3, 2020
a875da2
output info
bryanmacfarlane May 3, 2020
0a9e8b1
output info
bryanmacfarlane May 3, 2020
2a0fbec
output info
bryanmacfarlane May 3, 2020
2ae9264
back to master
bryanmacfarlane May 4, 2020
e73ffbc
support ghes (#157)
ericsciple May 18, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
wip
  • Loading branch information
bryanmacfarlane committed Apr 24, 2020
commit ecf0ce62f9fc44aa8a7c932625b9335ed461e377
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ node_modules/
lib/
__tests__/runner/*

validate/temp
validate/node
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created a bash script to automate interactively validating without queuing a full run


# Rest of the file pulled from https://github.com/github/gitignore/blob/master/Node.gitignore
# Logs
logs
Expand Down
10 changes: 7 additions & 3 deletions action.yml
Original file line number Diff line number Diff line change
@@ -1,17 +1,21 @@
name: 'Setup Node.js environment'
description: 'Setup a Node.js environment and add it to the PATH, additionally providing proxy support'
description: 'Setup a Node.js environment by adding problem matchers and optionally downloading and adding it to the PATH'
author: 'GitHub'
inputs:
always-auth:
description: 'Set always-auth in npmrc'
default: 'false'
node-version:
description: 'Version Spec of the version to use. Examples: 10.x, 10.15.1, >=10.15.0'
default: '10.x'
description: 'Version Spec of the version to use. Examples: 12.x, 10.15.1, >=10.15.0'
registry-url:
description: 'Optional registry to set up for auth. Will set the registry in a project level .npmrc and .yarnrc file, and set up auth to read in from env.NODE_AUTH_TOKEN'
scope:
description: 'Optional scope for authenticating against scoped registries'
token:
description: Used to pull node distributions from node-versions. Since there's a default, this is typically not supplied by the user.
default: ${{ github.token }}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to make sure that querying api.github.com for the manifest and release data doesn't get throttled. Authenticated is per user and much higher (similar to our runner pulling actions) while anon would be throttled much lower and per IP

# TODO: add input to control forcing to pull from cloud or dist.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to do this as a separate PR while in v2-beta

# escape valve for someone having issues or needing the absolute latest which isn't cached yet
# Deprecated option, do not use. Will not be supported after October 1, 2019
version:
description: 'Deprecated. Use node-version instead. Will not be supported after October 1, 2019'
Expand Down
1,218 changes: 1,013 additions & 205 deletions dist/index.js

Large diffs are not rendered by default.

20 changes: 15 additions & 5 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "setup-node",
"version": "1.0.0",
"version": "2.0.0",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't influence anything but conveys the lineage.

"private": true,
"description": "setup node action",
"main": "lib/setup-node.js",
Expand All @@ -27,7 +27,7 @@
"@actions/github": "^1.1.0",
"@actions/http-client": "^1.0.6",
"@actions/io": "^1.0.2",
"@actions/tool-cache": "^1.3.3",
"@actions/tool-cache": "^1.5.2",
"semver": "^6.1.1"
},
"devDependencies": {
Expand Down
216 changes: 127 additions & 89 deletions src/installer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import * as tc from '@actions/tool-cache';
import * as os from 'os';
import * as path from 'path';
import * as semver from 'semver';
import { Url } from 'url';

let osPlat: string = os.platform();
let osArch: string = translateArchToDistUrl(os.arch());
Expand All @@ -19,36 +20,64 @@ interface INodeVersion {
files: string[];
}

export async function getNode(versionSpec: string) {
interface INodeVersionInfo {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These interfaces normalize the resolution from manifest and fallback to node dist so common code downstream can be agnostic.

downloadUrl: string;
token: string | null;
resolvedVersion: string;
fileName: string;
}

export async function getNode(versionSpec: string, stable: boolean, token: string) {
// check cache
let info: INodeVersionInfo | null = null;
let toolPath: string;
toolPath = tc.find('node', versionSpec);

// If not found in cache, download
if (!toolPath) {
let version: string;
const c = semver.clean(versionSpec) || '';
// If explicit version
if (semver.valid(c) != null) {
// version to download
version = versionSpec;
} else {
// query nodejs.org for a matching version
version = await queryLatestMatch(versionSpec);
if (!version) {
throw new Error(
`Unable to find Node version '${versionSpec}' for platform ${osPlat} and architecture ${osArch}.`
);
if (toolPath) {
console.log(`Found in cache @ ${toolPath}`);
} else {
console.log(`Attempting to download ${versionSpec}...`)
let info = await getInfoFromManifest(versionSpec, stable, token);
if (!info) {
console.log('Not found in manifest. Falling back to download directly from Node')
info = await getInfoFromDist(versionSpec);
}

if (!info) {
throw new Error(
`Unable to find Node version '${versionSpec}' for platform ${osPlat} and architecture ${osArch}.`
);
}

console.log(`Acquiring ${info.resolvedVersion} from ${info.downloadUrl}`);

let downloadPath = ""
try {
downloadPath = await tc.downloadTool(info.downloadUrl, undefined, token);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldnt this use info.token?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when downloading from nodejs.org, shouldnt use a token

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or would it better to check the hostname of the download url to determine whether to pass token?

} catch (err) {
if (err instanceof tc.HTTPError && err.httpStatusCode == 404) {
return await acquireNodeFromFallbackLocation(info.resolvedVersion);
}

// check cache
toolPath = tc.find('node', version);
throw err;
}

if (!toolPath) {
// download, extract, cache
toolPath = await acquireNode(version);
//
// Extract
//
let extPath: string;
if (osPlat == 'win32') {
let _7zPath = path.join(__dirname, '..', 'externals', '7zr.exe');
extPath = await tc.extract7z(downloadPath, undefined, _7zPath);
} else {
extPath = await tc.extractTar(downloadPath);
}

//
// Install into the local tool cache - node extracts with a root folder that matches the fileName downloaded
//
toolPath = await tc.cacheDir(extPath, 'node', info.resolvedVersion);
}

//
Expand All @@ -65,41 +94,56 @@ export async function getNode(versionSpec: string) {
core.addPath(toolPath);
}

async function queryLatestMatch(versionSpec: string): Promise<string> {
// node offers a json list of versions
let dataFileName: string;
switch (osPlat) {
case 'linux':
dataFileName = `linux-${osArch}`;
break;
case 'darwin':
dataFileName = `osx-${osArch}-tar`;
break;
case 'win32':
dataFileName = `win-${osArch}-exe`;
break;
default:
throw new Error(`Unexpected OS '${osPlat}'`);
async function getInfoFromManifest(versionSpec: string, stable: boolean, token: string): Promise<INodeVersionInfo | null> {
let info: INodeVersionInfo | null = null;
const releases = await tc.getManifestFromRepo("actions", "node-versions", token)
console.log(`matching ${versionSpec}...`)
const rel = await tc.findFromManifest(versionSpec, stable, releases);

if (rel && rel.files.length > 0) {
info = <INodeVersionInfo>{};
info.resolvedVersion = rel.version;
info.downloadUrl = rel.files[0].download_url;
info.fileName = rel.files[0].filename;
info.token = token;
}

let versions: string[] = [];
let dataUrl = 'https://nodejs.org/dist/index.json';
let httpClient = new hc.HttpClient('setup-node', [], {
allowRetries: true,
maxRetries: 3
});
let response = await httpClient.getJson<INodeVersion[]>(dataUrl);
let nodeVersions = response.result || [];
nodeVersions.forEach((nodeVersion: INodeVersion) => {
// ensure this version supports your os and platform
if (nodeVersion.files.indexOf(dataFileName) >= 0) {
versions.push(nodeVersion.version);
return info;
}

async function getInfoFromDist(versionSpec: string): Promise<INodeVersionInfo | null> {
let info: INodeVersionInfo | null = null;
let version: string;

// If explicit version don't query
if (semver.clean(versionSpec) != null) {
// version to download
version = versionSpec;
} else {
// query nodejs.org for a matching version
version = await queryDistForMatch(versionSpec);
if (!version) {
return null;
}
});
}

// get the latest version that matches the version spec
let version: string = evaluateVersions(versions, versionSpec);
return version;
//
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all mostly the same code - just moved into another function for fallback.

// Download - a tool installer intimately knows how to get the tool (and construct urls)
//
version = semver.clean(version) || '';
let fileName: string =
osPlat == 'win32'
? `node-v${version}-win-${osArch}`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If version is an empty string, won't this file name be invalid?

: `node-v${version}-${osPlat}-${osArch}`;
let urlFileName: string =
osPlat == 'win32' ? `${fileName}.7z` : `${fileName}.tar.gz`;
let url = `https://nodejs.org/dist/v${version}/${urlFileName}`;

return <INodeVersionInfo>{
downloadUrl: url,
resolvedVersion: version,
fileName: fileName
}
}

// TODO - should we just export this from @actions/tool-cache? Lifted directly from there
Expand Down Expand Up @@ -130,47 +174,41 @@ function evaluateVersions(versions: string[], versionSpec: string): string {
return version;
}

async function acquireNode(version: string): Promise<string> {
//
// Download - a tool installer intimately knows how to get the tool (and construct urls)
//
version = semver.clean(version) || '';
let fileName: string =
osPlat == 'win32'
? `node-v${version}-win-${osArch}`
: `node-v${version}-${osPlat}-${osArch}`;
let urlFileName: string =
osPlat == 'win32' ? `${fileName}.7z` : `${fileName}.tar.gz`;
let downloadUrl = `https://nodejs.org/dist/v${version}/${urlFileName}`;

let downloadPath: string;

try {
downloadPath = await tc.downloadTool(downloadUrl);
} catch (err) {
if (err instanceof tc.HTTPError && err.httpStatusCode == 404) {
return await acquireNodeFromFallbackLocation(version);
}

throw err;
async function queryDistForMatch(versionSpec: string): Promise<string> {
// node offers a json list of versions
let dataFileName: string;
switch (osPlat) {
case 'linux':
dataFileName = `linux-${osArch}`;
break;
case 'darwin':
dataFileName = `osx-${osArch}-tar`;
break;
case 'win32':
dataFileName = `win-${osArch}-exe`;
break;
default:
throw new Error(`Unexpected OS '${osPlat}'`);
}

//
// Extract
//
let extPath: string;
if (osPlat == 'win32') {
let _7zPath = path.join(__dirname, '..', 'externals', '7zr.exe');
extPath = await tc.extract7z(downloadPath, undefined, _7zPath);
} else {
extPath = await tc.extractTar(downloadPath);
}
let versions: string[] = [];
let dataUrl = 'https://nodejs.org/dist/index.json';
let httpClient = new hc.HttpClient('setup-node', [], {
allowRetries: true,
maxRetries: 3
});
let response = await httpClient.getJson<INodeVersion[]>(dataUrl);
let nodeVersions = response.result || [];
nodeVersions.forEach((nodeVersion: INodeVersion) => {
// ensure this version supports your os and platform
if (nodeVersion.files.indexOf(dataFileName) >= 0) {
versions.push(nodeVersion.version);
}
});

//
// Install into the local tool cache - node extracts with a root folder that matches the fileName downloaded
//
let toolRoot = path.join(extPath, fileName);
return await tc.cacheDir(toolRoot, 'node', version);
// get the latest version that matches the version spec
let version: string = evaluateVersions(versions, versionSpec);
return version;
}

// For non LTS versions of Node, the files we need (for Windows) are sometimes located
Expand Down
10 changes: 7 additions & 3 deletions src/setup-node.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,16 @@ async function run() {
// Version is optional. If supplied, install / use from the tool cache
// If not supplied then task is still used to setup proxy, auth, etc...
//
let version = core.getInput('version');
let version = core.getInput('node-version');
if (!version) {
version = core.getInput('node-version');
version = core.getInput('version');
}

console.log(`version: ${version}`);
if (version) {
await installer.getNode(version);
let token = core.getInput('token');
let stable = (core.getInput('stable') || 'true').toUpperCase() === 'TRUE';
await installer.getNode(version, stable, token);
}

const registryUrl: string = core.getInput('registry-url');
Expand Down
22 changes: 22 additions & 0 deletions validate/test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@

#/bin/bash

set -e

rm -rf ./temp
rm -rf ./node

# uncomment to use charles proxy or other debugging proxy
# export NODE_TLS_REJECT_UNAUTHORIZED=0
# export https_proxy=http://127.0.0.1:8888

export RUNNER_TOOL_CACHE=$(pwd)
export RUNNER_TEMP="${RUNNER_TOOL_CACHE}/temp"
export INPUT_STABLE=true
export INPUT_VERSION="12.x"
# export your PAT with repo scope before running
export INPUT_TOKEN=$GITHUB_TOKEN

echo "Getting ${INPUT_VERSION} ($INPUT_STABLE) with ${INPUT_TOKEN}..."

node ../dist/index.js
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy