Content-Length: 603523 | pFad | http://github.com/typescript-eslint/typescript-eslint/pull/11323

22 feat(eslint-plugin-internal): [no-dynamic-tests] new internal Lint rule to ban dynamic syntax in generating tests by nayounsang · Pull Request #11323 · typescript-eslint/typescript-eslint · GitHub
Skip to content

feat(eslint-plugin-internal): [no-dynamic-tests] new internal Lint rule to ban dynamic syntax in generating tests #11323

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 24 commits into
base: main
Choose a base branch
from

Conversation

nayounsang
Copy link
Contributor

@nayounsang nayounsang commented Jun 20, 2025

PR Checklist

Overview

no-dynamic-tests

Ban these:

{
  invalid: {...spread},
  valid: identifier,
  invalid: bi + nary,
  valid: callExp(),
  invalid: `${templateLiteralWithDynamic()}`,
  valid: obj.member,
  invalid: condition? foo : bar,
}

For test case object, It validate these keys recursively: code, error.
No checks are performed on other keys.

Applied the rules to the test file in eslint-plugin & eslint-plugin-internal. Bulk suppression feature was applied.
Create the script to update suppression.

@typescript-eslint
Copy link
Contributor

Thanks for the PR, @nayounsang!

typescript-eslint is a 100% community driven project, and we are incredibly grateful that you are contributing to that community.

The core maintainers work on this in their personal time, so please understand that it may not be possible for them to review your work immediately.

Thanks again!


🙏 Please, if you or your company is finding typescript-eslint valuable, help us sustain the project by sponsoring it transparently on https://opencollective.com/typescript-eslint.

Copy link

netlify bot commented Jun 20, 2025

Deploy Preview for typescript-eslint ready!

Name Link
🔨 Latest commit e668354
🔍 Latest deploy log https://app.netlify.com/projects/typescript-eslint/deploys/6877560a7e4c200008ab8622
😎 Deploy Preview https://deploy-preview-11323--typescript-eslint.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
Lighthouse
Lighthouse
1 paths audited
Performance: 95 (🟢 up 1 from production)
Accessibility: 97 (no change from production)
Best Practices: 100 (no change from production)
SEO: 92 (no change from production)
PWA: 80 (no change from production)
View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify project configuration.

Copy link

nx-cloud bot commented Jun 20, 2025

View your CI Pipeline Execution ↗ for commit e668354

Command Status Duration Result
nx run-many -t lint ✅ Succeeded 3m 20s View ↗
nx run-many -t typecheck ✅ Succeeded 2m 37s View ↗
nx test eslint-plugin-internal --coverage=false ✅ Succeeded 4s View ↗
nx run integration-tests:test ✅ Succeeded 5s View ↗
nx run types:build ✅ Succeeded 5s View ↗
nx test eslint-plugin --coverage=false ✅ Succeeded 3s View ↗
nx test typescript-estree --coverage=false ✅ Succeeded 1s View ↗
nx run generate-configs ✅ Succeeded 6s View ↗
Additional runs (27) ✅ Succeeded ... View ↗

☁️ Nx Cloud last updated this comment at 2025-07-16 07:41:46 UTC

@nayounsang nayounsang changed the title feat(eslint-plugin-internal) : Lint rule to ban dynamic syntax in generating tests feat(eslint-plugin-internal): Lint rule to ban dynamic syntax in generating tests Jun 20, 2025
Copy link

codecov bot commented Jun 20, 2025

Codecov Report

Attention: Patch coverage is 93.60000% with 8 lines in your changes missing coverage. Please review.

Project coverage is 90.86%. Comparing base (3f58afe) to head (e668354).

Files with missing lines Patch % Lines
...lint-plugin-internal/src/rules/no-dynamic-tests.ts 93.60% 8 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #11323      +/-   ##
==========================================
- Coverage   90.86%   90.86%   -0.01%     
==========================================
  Files         503      504       +1     
  Lines       51036    51161     +125     
  Branches     8424     8456      +32     
==========================================
+ Hits        46373    46485     +112     
- Misses       4648     4661      +13     
  Partials       15       15              
Flag Coverage Δ
unittest 90.86% <93.60%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...lint-plugin-internal/src/rules/no-dynamic-tests.ts 93.60% <93.60%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@nayounsang nayounsang changed the title feat(eslint-plugin-internal): Lint rule to ban dynamic syntax in generating tests feat(eslint-plugin-internal): [no-dynamic-tests] new internal Lint rule to ban dynamic syntax in generating tests Jun 20, 2025
Copy link
Member

@JoshuaKGoldberg JoshuaKGoldberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks! Only the one request for an added test is blocking IMO. Everything else is nitpicks that can be ignored if you don't like them. 🙂

@JoshuaKGoldberg JoshuaKGoldberg added the awaiting response Issues waiting for a reply from the OP or another party label Jun 23, 2025
Copy link
Member

@JoshuaKGoldberg JoshuaKGoldberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥 thanks!

@JoshuaKGoldberg JoshuaKGoldberg added 1 approval >=1 team member has approved this PR; we're now leaving it open for more reviews before we merge and removed awaiting response Issues waiting for a reply from the OP or another party labels Jun 24, 2025
Copy link
Member

@kirkwaiblinger kirkwaiblinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that if we intend to use this ever, we should enable it already now for all the files that don't currently have violations. (Maybe even this is a good chance for us to try out the eslint bulk suppressions feature)?

Also, please do enable the rule on the codebase and have a look at the reports to see whether they make sense and would make sense to a contributor.

Thanks!

prop.value.elements.forEach(element => {
if (element && isDynamicExpression(element)) {
context.report({
node: element,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than reporting on the whole object expression, let's report on the specific offending node(s). The closer the report is to the issue, the the easier it is for a user to understand what's wrong and how to fix it.

For example, in no-relative-paths-to-internal-packages, we should report like this

    {
      code: "import packageJson from '../../package.json' with { type: 'json' };",
      filename: path.resolve(PACKAGES_DIR, 'ast-spec/vitest.config.mts'),
//              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    },

rather than the entire object

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice review. I committed. I made it so that recursive functions report on elements rather than returning whether they are dynamic.

@kirkwaiblinger kirkwaiblinger added awaiting response Issues waiting for a reply from the OP or another party and removed 1 approval >=1 team member has approved this PR; we're now leaving it open for more reviews before we merge labels Jul 6, 2025
@nayounsang
Copy link
Contributor Author

nayounsang commented Jul 9, 2025

Change

As In the interest of the current issue and in my opinion, I enable dynamic object value.

// ex
{
  output: getOutput()
}
  1. From the perspective of repo maintenance, this case is better. I think that writing values ​​by hand increases the inconvenience and does not significantly harm readability.
  2. I don't think this change causes the following problems:
They tended to exponentially grow in size as more cases get grown
This is especially bad for slow typed tests
They make it hard to see what cases are or aren't being tested
They're not very statically analyzable

However, the code and error properties are excluded as an exception. These two properties often make analysis difficult when dynamically allocated.

+1
I forgot this case:

invalid: foo

So, I fix and test it.

@nayounsang
Copy link
Contributor Author

Probably if CI is run again, there will be no prob: Docusaurus can't find @typescript-eslint/eslint-plugin. This PR has nothing to do with this failure.

This is my first time using bulk suppressions, but it's amazing. Let's try it out.
However, this requires guidance and a process for disposal when issues about hand written test cases are resolved.
If only that could be maintained, management costs would be greatly reduced.


This is perf comparison for bulk suppressions feature. There is no difference between the two. Because size of eslint-suppressions.json is small.

# with suppressions 
=== result ===
total time: 55.0sec
max CPU: 109.1%
avg CPU: 1.37%
max Memory: 1.0%
avg Memory: .30%
max RSS: 168256KB
avg RSS: 49736.64KB
=== result ===
total time: 52.0sec
max CPU: 90.4%
avg CPU: 1.44%
max Memory: 1.0%
avg Memory: .28%
max RSS: 167744KB
avg RSS: 46318.16KB

# without suppressions
=== result ===
total time: 57.0sec
max CPU: 114.0%
avg CPU: 1.50%
max Memory: 1.0%
avg Memory: .26%
max RSS: 167952KB
avg RSS: 43129.38KB
=== result ===
total time: 52.0sec
max CPU: 85.1%
avg CPU: 1.29%
max Memory: 1.0%
avg Memory: .31%
max RSS: 171712KB
avg RSS: 52057.46KB

Monitor with this sh:

#!/bin/bash

start=$(date +%s.%N)

# RUN
yarn nx lint eslint-plugin --skip-nx-cache > /dev/null 2>&1 &
pid=$!

# Variables
max_cpu=0; max_mem=0; max_rss=0
total_cpu=0; total_mem=0; total_rss=0
sample_count=0

# Monitor
while true; do

    if ! kill -0 $pid 2>/dev/null; then
        break
    fi

    ps_info=$(ps -p $pid -o %cpu,%mem,rss 2>/dev/null | tail -1)
    if [ ! -z "$ps_info" ] && [ "$ps_info" != "%CPU" ]; then
        cpu=$(echo "$ps_info" | awk '{print $1}')
        mem=$(echo "$ps_info" | awk '{print $2}')
        rss=$(echo "$ps_info" | awk '{print $3}')

        # isNumber?
        if [[ "$cpu" =~ ^[0-9]+\.?[0-9]*$ ]] && [[ "$mem" =~ ^[0-9]+\.?[0-9]*$ ]] && [[ "$rss" =~ ^[0-9]+$ ]]; then
            [ $(echo "$cpu > $max_cpu" | bc -l) -eq 1 ] && max_cpu=$cpu
            [ $(echo "$mem > $max_mem" | bc -l) -eq 1 ] && max_mem=$mem
            [ $(echo "$rss > $max_rss" | bc -l) -eq 1 ] && max_rss=$rss
            total_cpu=$(echo "$total_cpu + $cpu" | bc -l)
            total_mem=$(echo "$total_mem + $mem" | bc -l)
            total_rss=$(echo "$total_rss + $rss" | bc -l)
            sample_count=$((sample_count + 1))
        fi
    fi
    sleep 0.5
done

wait $pid
end=$(date +%s.%N)
time=$(echo "$end - $start" | bc -l)

if [ $sample_count -gt 0 ]; then
    avg_cpu=$(echo "scale=2; $total_cpu / $sample_count" | bc -l)
    avg_mem=$(echo "scale=2; $total_mem / $sample_count" | bc -l)
    avg_rss=$(echo "scale=2; $total_rss / $sample_count" | bc -l)
else
    avg_cpu=0
    avg_mem=0
    avg_rss=0
fi

echo ""
echo ""
echo "=== result ==="
echo "total time: ${time}sec"
echo "max CPU: ${max_cpu}%"
echo "avg CPU: ${avg_cpu}%"
echo "max Memory: ${max_mem}%"
echo "avg Memory: ${avg_mem}%"
echo "max RSS: ${max_rss}KB"
echo "avg RSS: ${avg_rss}KB"

@github-actions github-actions bot removed the awaiting response Issues waiting for a reply from the OP or another party label Jul 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Repo: Lint rule to ban dynamic syntax in generating tests
3 participants








ApplySandwichStrip

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier!      Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

Fetched URL: http://github.com/typescript-eslint/typescript-eslint/pull/11323

Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy