JS: QL-side type/name resolution for TypeScript and JSDoc #19078

asgerf · 2025-03-20T13:49:55Z

Prepares for disabling TypeScript type extraction by recovering the relevant type information on the QL side.

Actually disabling type extraction will happen in a separate PR so it's easier to validate in isolation and rollback if needed.

The code is divided into three main components:

Name resolution: resolve a type name or variable to its definition; looking through imports and (re-)exports.
Type resolution: determine the type of an expression. Here we use a TypeExpr as a placeholder for an actual type which is good enough for our needs. Does not support generics.
Underlying types: Determine the 'underlying types' of a type. The concept of "underlying types" is meant to provide library models a simple way of reasoning about named types without having to deal with unions, intersections, and subtyping relationship. For example (Request & { x: T }) | null has Request as an underlying type.

One of the complexities of TypeScript is the fact that there are three "declaration spaces" in which variables can exist: value, type, and namespace. For example, for a declaration like class C {}, the value C refers to the class itself (i.e. its constructor), whereas the type C describes an instance of the class (not the class itself). In practice, it seems that values and namespaces can be merged without problems so I decided to simplify things by doing that.

Effects on call graph

Disabling type extraction on main would lose us 40k call edges on the default benchmark suite.
The new QL-based implementation recovers 98% of those edges, only 746 edges were not recovered.
On top of that, it discovers 30k new call edges that were missed in the original extractor-based implementation.
- One of reasons for this is better support for JSDoc type annotations in .js files. The new solution has unified support for the two kinds of type annotations, which was not possible previously since we didn't want to run the TypeScript compiler on .js files.
- Another reason is better tolerance for missing dependencies, as something like KnownClass | UnresolvedClass will always propagate information about KnownClass even if UnresolvedClass could not be resolved.

javascript/ql/lib/semmle/javascript/internal/TypeResolution.qll

javascript/ql/lib/semmle/javascript/internal/NameResolution.qll

+    }
+
+    /** Helps track flow from a particular set of source nodes. */
+    module Track<nodeSig/1 isSource> {


javascript/ql/lib/semmle/javascript/internal/TypeResolution.qll

javascript/ql/lib/semmle/javascript/JSDoc.qll

…ed-value

Overload resolution has little impact on data flow analysis, because there we care about the concrete implementation of the function, which is the same for all overloads. It can affect the return type, which in turn can affect the call graph we generate, but we'll just have to accept this as overload resolution is too hard without negative recursion.

…nt declarations This test enforced the opinion that ambient declarations should have no impact on data flow, which is no longer the case. For now I'm just updating the test output.

Napalys

Extraordinary... 👏

Napalys · 2025-06-03T16:11:05Z

javascript/ql/lib/semmle/javascript/internal/NameResolution.qll

+    then result = a
+    else (
+      (if a = "" or b = "" then result = a + b else result = a + "." + b) and
+      result.length() < 100


Why a 100 limit?

It's to avoid infinite recursion on pathological examples. In practice these access paths aren't very long.

javascript/ql/lib/semmle/javascript/internal/TypeResolution.qll

Co-authored-by: Napalys Klicius <napalys@github.com>

erik-krogh

I still haven't looked closely at the three big commits, but here are some small comments.

erik-krogh · 2025-06-04T12:49:21Z

javascript/ql/lib/semmle/javascript/internal/TypeResolution.qll

+        or
+        method instanceof GetterMethodDeclaration and
+        memberType = method.getBody().getReturnTypeAnnotation()


This change to have some special handling of getter methods seems fine, but it also seems disconnected from the rest of the commit? (And from the commit-message "Hide shadowed inherited members").

You're right, I seem to have omitted this in the commit message. I was fixing some issues found in a DCA run and it seems two independent fixes ended up in the same commit. I'm not sure it makes sense to go back and change the commit history though.

erik-krogh · 2025-06-04T13:07:06Z

javascript/ql/lib/semmle/javascript/dataflow/Sources.qll

@@ -333,7 +333,13 @@ module SourceNode {
        astNode instanceof TaggedTemplateExpr or
        astNode instanceof Templating::PipeRefExpr or
        astNode instanceof Templating::TemplateVarRefExpr or
-        astNode instanceof StringLiteral
+        astNode instanceof StringLiteral or
+        astNode instanceof TypeAssertion


What about a SatisfiesExpr?

Also I don't see a test of TypeAssertion as part of this commit?

Added a test and a case for SatisfiesExpr 👍

erik-krogh · 2025-06-04T13:14:39Z

javascript/ql/lib/definitions.qll

+  exists(MethodDeclaration member |
+    not member instanceof ConstructorDeclaration and
+    node = member.getBody() and
+    result = member.getNameExpr()
+  )


Should we have a case for FunctionDeclStmt?

Makes sense

github-actions bot added the JS label Mar 20, 2025

github-advanced-security bot found potential problems Mar 20, 2025

View reviewed changes

javascript/ql/lib/semmle/javascript/internal/TypeResolution.qll Fixed Show fixed Hide fixed

javascript/ql/lib/semmle/javascript/internal/TypeResolution.qll Fixed Show fixed Hide fixed

asgerf force-pushed the js/name-resolution branch from cee75ae to fa3e5ed Compare March 20, 2025 15:31

github-advanced-security bot found potential problems Mar 20, 2025

View reviewed changes

asgerf force-pushed the js/name-resolution branch 2 times, most recently from fe9b23d to c07cc6e Compare March 27, 2025 20:20

github-advanced-security bot found potential problems Apr 1, 2025

View reviewed changes

javascript/ql/lib/semmle/javascript/JSDoc.qll Fixed Show fixed Hide fixed

asgerf force-pushed the js/name-resolution branch from 2d928f2 to 3b395af Compare April 3, 2025 09:22

asgerf force-pushed the js/name-resolution branch from 3b395af to d92247c Compare April 11, 2025 11:37

asgerf force-pushed the js/name-resolution branch from 87454f7 to f289592 Compare May 2, 2025 11:43

asgerf force-pushed the js/name-resolution branch 4 times, most recently from 45b09df to ae0aeb9 Compare May 19, 2025 10:00

asgerf added 16 commits May 20, 2025 13:19

JS: Exclude externs from CallGraph meta-query

5064cd5

JS: Add ImportSpecifier.getImportDeclaration()

9fc0b8c

JS: Do not ignore variables from ambient declarations

50e4ac8

JS: Make Closure concepts based on AST instead

b5a4fc0

JS: Avoid accidental recursion with API graphs

4cd6f45

JS: Add helper for getting local type names

9566265

JS: Resolve JSDocLocalTypeAccess to a variable in scope

4bfb048

JS: Add test

1051136

JS: Add NameResolution.qll

1533e13

JS: Add UnderlyingTypes.qll

d61f576

JS: Add TypeResolution.qll

fc580a5

JS: Use underlying types in DataFlow::Node

b923eac

JS: Use in TypeAnnotation.getClass and hasUnderlyingType predicates

cca48c0

JS: Update jQuery model

9fd85c9

JS: Use sanitizing primitive types in ViewComponentInput

2d21074

JS: Use sanitizing primitive type in Nest model

6fdd7fe

asgerf added 18 commits May 20, 2025 13:20

JS: Remove some dependencies on type extraction

989402d

JS: Some test updates

57811ed

JS: Use type resolution for CG augmentation

307715a

JS: Add call graph test with types

f06b9a9

JS: Hide shadowed inherited members

500291d

JS: Also propagate through promise types

167f752

JS: Add failing test for assigning a non-SourceNode to a type annotat…

6e82b6e

…ed-value

JS: Mark type-annotated nodes as SourceNode

e07a036

JS: Update to avoid deprecations after import resolution change

fbafd6f

JS: Remove redundant casts

b8dc1b3

JS: Make jump-to-def behave nicer

bba872a

JS: Update output of old HasUnderlyingType test

de7d851

JS: Accept change in handling of variable resolution in face of ambie…

b610e10

…nt declarations This test enforced the opinion that ambient declarations should have no impact on data flow, which is no longer the case. For now I'm just updating the test output.

JS: Add regression tests for declared globals

27979c6

JS: Fix regression from global declare vars

9bcc620

JS: Update TRAP after extractor change

11607e5

JS: Add test for missing type flow through generics

b698b4e

asgerf force-pushed the js/name-resolution branch from aab67e7 to 2b208d6 Compare May 20, 2025 13:57

JS: Remove obsolete meta query

d644f80

asgerf force-pushed the js/name-resolution branch from 2b208d6 to d644f80 Compare May 20, 2025 14:21

asgerf added the no-change-note-required This PR does not need a change note label May 22, 2025

asgerf marked this pull request as ready for review May 22, 2025 09:54

asgerf requested a review from a team as a code owner May 22, 2025 09:54

Napalys reviewed Jun 3, 2025

View reviewed changes

Update javascript/ql/lib/semmle/javascript/internal/TypeResolution.qll

853ba49

Co-authored-by: Napalys Klicius <napalys@github.com>

erik-krogh reviewed Jun 4, 2025

View reviewed changes

asgerf added 3 commits June 4, 2025 22:17

JS: Add test with type casts

79101fd

JS: Add SatisfiesExpr

57fad7e

JS: Nicer jump-to-def for function declarations

691fdb1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

JS: QL-side type/name resolution for TypeScript and JSDoc #19078

JS: QL-side type/name resolution for TypeScript and JSDoc #19078

Uh oh!

asgerf commented Mar 20, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Check warning

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Napalys left a comment

Uh oh!

Napalys Jun 3, 2025

Uh oh!

asgerf Jun 4, 2025

Uh oh!

Uh oh!

erik-krogh left a comment

Uh oh!

erik-krogh Jun 4, 2025

Uh oh!

asgerf Jun 4, 2025

Uh oh!

erik-krogh Jun 4, 2025

Uh oh!

asgerf Jun 4, 2025

Uh oh!

erik-krogh Jun 4, 2025

Uh oh!

asgerf Jun 4, 2025

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

JS: QL-side type/name resolution for TypeScript and JSDoc #19078

Are you sure you want to change the base?

JS: QL-side type/name resolution for TypeScript and JSDoc #19078

Uh oh!

Conversation

asgerf commented Mar 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Effects on call graph

Uh oh!

Uh oh!

Uh oh!

Check warning

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Napalys left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

erik-krogh left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

asgerf commented Mar 20, 2025 •

edited

Loading