The committers listed above are authorized under a signed CLA.

• ✅ login: LiviaMedeiros / name: Livia Medeiros (72bf938, aca961e, df082d6)

✅ Deploy Preview for docs-eslint canceled.

Would it fix the problem, and would the performance be better if we just do return results.flatMap(result => result.value); instead of pushing to filePaths?

Do you mean checking non-fulfilled results in separate loop? I don't think we can await throwErrorForUnmatchedPatterns inside flatMap.
Without awaiting, it would fix the problem, the performance is within error margin (for single directory with 65535 files I'm getting 4150~4270ms in both versions; with 262143 files flatMap was slower by about 100ms out of 16 seconds).

throwErrorForUnmatchedPatterns always throws so we don't have to worry about it? I believe the loop checks if the promise is rejected and in that case throws an error, otherwise pushes the values to filePaths.

I was thinking about something like this:

- return filePaths;
+ return results.flatMap(result => result.value);

And then we could remove the filePaths intermediary variable?

Like this? It looks cleaner and wouldn't be significantly slower, but we effectively iterate over results twice.

but we effectively iterate over results twice.

I think that isn't a concern as results is typically a very small array. Its length is less than or equal to the number of globs and directories passed on the command line.

I can reproduce the error locally: I changed DEFAULT_FILE_COUNT in the emfile check to 150_000, ran npm run test:emfile and got the following.

$ npm run test:emfile

> eslint@9.13.0 test:emfile
> node tools/check-emfile-handling.js

Generating 150000 files in tmp/emfile-check...
Running ESLint...
(node:10236) ESLintIgnoreWarning: The ".eslintignore" file is no longer supported. Switch to using the "ignores" property in "eslint.config.js": https://eslint.org/docs/latest/use/configure/migration-guide#ignoring-files
(Use `node --trace-warnings ...` to show where the warning was created)

Oops! Something went wrong! :(

ESLint: 9.13.0

RangeError: Maximum call stack size exceeded
    at globMultiSearch (C:\projects\eslint\lib\eslint\eslint-helpers.js:435:27)
    at async findFiles (C:\projects\eslint\lib\eslint\eslint-helpers.js:591:27)
    at async ESLint.lintFiles (C:\projects\eslint\lib\eslint\eslint.js:740:27)
    at async Object.execute (C:\projects\eslint\lib\cli.js:498:23)
    at async main (C:\projects\eslint\bin\eslint.js:158:22)
node:child_process:965
    throw err;
    ^

Error: Command failed: node bin/eslint.js tmp/emfile-check -c tests/fixtures/emfile/eslint.config.js
    at genericNodeError (node:internal/errors:984:15)
    at wrappedFn (node:internal/errors:538:14)
    at checkExecSyncError (node:child_process:890:11)
    at execSync (node:child_process:962:15)
    at Object.<anonymous> (C:\projects\eslint\tools\check-emfile-handling.js:92:1)
    at Module._compile (node:internal/modules/cjs/loader:1369:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1427:10)
    at Module.load (node:internal/modules/cjs/loader:1206:32)
    at Module._load (node:internal/modules/cjs/loader:1022:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:135:12) {
  status: 2,
  signal: null,
  output: [ null, null, null ],
  pid: 3348,
  stdout: null,
  stderr: null
}

eslint-helpers.js:435:27 indeed points to the filePaths.push(...result.value); line.

So, marking as accepted.

I can confirm that the changes from this PR fix the "Maximum call stack size exceeded" problem in my test setup. Though, the test eventually fails with an EMFILE error, but we'll take another look at the EMFILE problem in #18977.

Speaking of tools/check-emfile-handling.js, i suspect that

Ah okay, it fails with FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory rather than call stack error. Reverting.

Speaking of tools/check-emfile-handling.js, i suspect that

eslint/tools/check-emfile-handling.js

Lines 43 to 46 in 50f03a1

	// if we're on a Mac, make sure the limit isn't high enough to cause a call stack error
	if (os.platform() === "darwin") {
	FILE_COUNT = Math.min(FILE_COUNT, 100000);
	}

is directly related to this issue. If removing it won't break CI (it totes can, I also can confirm that the EMFILE still occurs at least after exceeding hard limit), perhaps it should be done in scope of this PR.

Yeah, it's quite possible that the comment refers to this particular call stack error.

I'm not sure if it's possible to determine the max number of elements that can be spread into a function call, to ensure we'll not use spread arguments with the list of files anywhere, and then test with that number. And it seems that the test would eventually run into either an EMFILE error or a heap out of memory error. So, I'm inclined to merging this without any particular tests and consider increasing limits in the EMFILE check in a PR that will fix #18977. @nzakas what do you think?

I'm not sure if it's possible to determine the max number of elements that can be spread into a function call, to ensure we'll not use spread arguments with the list of files anywhere, and then test with that number.

I don't know that we can determine the max number of spread arguments a function may have. I'm guessing it's close to the max number of array elements (2^32 - 1), so I hope we wont' have to worry about it.

Determining the real stack limit here is quite simple: we can just run something like [].push(...Array.from({ length })) and catch RangeError. I'm getting something around 122825 in ideal conditions on x86_64. This value is never constant though, i'd expect any test involving exceeding stack to be flaky.

I think the tools/check-emfile-handling.js, once the EMFILE error is fixed, would be sufficient as a test for this.
Right now the test passes despite error still not fixed, so perhaps it should test with bigger amounts of files, maybe something like Math.max(100_000, ulimit * 2) or more.
Determining the limit might also be more complicated... On my system, ulimit -n outputs 1024, ulimit -Hn outputs 4096, but ${child_process.execSync('ulimit -n')} in Node.js outputs 4096. Meanwhile /proc/sys/fs/file-max tells that max amount of FDs is 8192. And the worst thing is, running test that actually pushes the limits renders OS unusable, not allowing to run even something simple like ls or ps.

If we want to test specifically for spread operator issue and similar stuff (something.apply(something, beeeegArray)), i'd recommend to lower the stack size in the test(s), e.g. by adding // Flags: --stack-size=256.

I'm okay merging without a test, but would like @mdjermanovic to confirm.