Skip to content

Increase resilience to generating large workbooks #2319

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

hfhchan-plb
Copy link
Contributor

Summary

Updates the xml-stream.js to generate a single string instead of multiple strings for each attribute or record of attributes.

Closes: #2303

Increase resilience to generating large workbooks
@zurmokeeper
Copy link
Contributor

@hfhchan-plb Can you please tell me how to add test cases for this.

@hfhchan-plb
Copy link
Contributor Author

I'm not sure if adding any test case is realistic for this, because after the patch I can now generate a sheet with 18527 rows x 272 columns (with some cells with some formatting). Before this patch, I could only generate the sheet with 18527 rows x 31 columns. While generating the sheet the node process went over 4GB RAM.

@hfhchan-plb
Copy link
Contributor Author

OK it seems the limit for GitHub Actions is 7 GB (https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners), so it should be feasible to write a use case which generates workbooks of that size.

@hfhchan-plb
Copy link
Contributor Author

A brief simulation:

const rowCount = 18527;
const colCount = 272;

for (let i = 1; i <= rowCount; i++) {
  const row = [];
  for (let j = 1; j <= colCount; j++) {
    row.push(Math.random());
  }
  worksheet.addRow(row);

  if (i % 33 === 1) { // Make header every 33 rows
    for (let col = 1; col <= colCount; col += 1) {
      const cell = worksheet.getCell(i, col);
      cell.font = { color: { argb: "FFFFFFFF" }, bold: true };
      cell.fill = {
        type: "pattern",
        pattern: "solid",
        fgColor: { argb: "#FFD9D9D9" },
      };
    }
  } else {
    for (let col = 1; col <= colCount; col += 1) {
      const cell = worksheet.getCell(i, col);
      cell.numFmt = '0.00;[Red]-0.00;-'
    }
  }
}

@hfhchan-plb
Copy link
Contributor Author

I renamed my branch to hfhchan-plb:fix-large-file-export and GitHub automatically closed this PR. Please see the new PR #2320.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Out-of-memory with lots of cells
2 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy