Skip to content

Performance opportunity for yarn 3 cache #325

Closed
@belgattitude

Description

@belgattitude

Thanks for the new cache feature. Much easier.

After few weeks, I realized that while it supports yarn, there's some improvements that can be made.

Yarn 3 (probably yarn 2+ too) manages downloaded archives pretty well (.yarn/cache/*.zip) and invalidating on yarn.lock changes does not take that into account and I saw a lot of cache misses.

As an example I converted back to action-cache to illustrate and test.

I'm wondering if a similar approach could be done with setup-node ?

Updated example with action cache

Updated on Nov 23th, taking into account @merceyz comment.

Setup action

      # Get the yarn cache path.
      - name: Get yarn cache directory path
        id: yarn-cache-dir-path
        run: echo "::set-output name=dir::$(yarn config get cacheFolder)"
    
      - name: Restore yarn cache
        uses: actions/cache@v2
        id: yarn-cache # use this to check for `cache-hit` (`steps.yarn-cache.outputs.cache-hit != 'true'`)
        with:
          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
          key: yarn-cache-folder-${{ hashFiles('**/yarn.lock', '.yarnrc.yml') }}
          restore-keys: |
            yarn-cache-folder-

Testing a cache hit after adding a dependency

With setup node cache, as the yarn.lock have changed all packages would be fetched again (1m 28s)
rather than (1s 124ms). Environmentally friendlier 🌳

yarn install --immutable
➤ YN0000: ┌ Fetch step
  ➤ YN0013: │ 1719 packages were already cached, one had to be fetched (superjson@npm:1.7.5)
➤ YN0000: └ Completed in 1s 124ms

Cache Size: ~127 MB (133261279 B)
Cache saved successfully
Cache saved with key: yarn-cache-folder-os-Linux-node--f118ea4bee07eada9df36ad2e83fd6febcbaf06b5b8962689c7650659e872ad3

PS: Key points

~Example with action cache~ (old version, before @merceyz improvements)
      - name: Get yarn cache directory path
        id: yarn-cache-dir-path
        run: echo "::set-output name=dir::$(yarn config get cacheFolder)"
      - name: Restore yarn cache
        uses: actions/cache@v2
        id: yarn-cache 
        with:
          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}       
          key: yarn-cache-folder-os-${{ runner.os }}-node-${{ env.node-version }}-${{ hashFiles('**/yarn.lock', '.yarnrc.yml') }}      
          restore-keys: |
            yarn-cache-folder-os-${{ runner.os }}-node-${{ env.node-version }}-
            yarn-cache-folder-os-${{ runner.os }}-

Note

Here's a gist with an optimized install example: https://gist.github.com/belgattitude/042f9caf10d029badbde6cf9d43e400a

Metadata

Metadata

Assignees

Labels

feature requestNew feature or request to improve the current logic

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    pFad - Phonifier reborn

    Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

    Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


    Alternative Proxies:

    Alternative Proxy

    pFad Proxy

    pFad v3 Proxy

    pFad v4 Proxy