[data grid] Excel export of a big amount of rows hangs up the page, any suggestions?

[jbaccarelli-equinix]

Duplicates

• I have searched the existing issues

Latest version

• I have tested the latest version

Summary 💡

I'm trying to handle downloading all the records from a table, it works like a charm for CSV files, but when I use the downloadAsExcel function, if the rows are more than let's say 50k, the page hangs up until the file is compiled.

I was trying to figure out a way with a web-worker but couldn't get it to work properly (might be due to my repo using yarn 2 + craco)

Examples 🌈

I'm downloading all records using a call that get's me everything and then calling this function

const download = useCallback(
    async ({ exportAs, rows }: DownloadProps) => {
      const exportMethod = {
        CSV: apiRef.current.exportDataAsCsv,
        EXCEL: apiRef.current.exportDataAsExcel,
      }
      const prevRows = Array.from(apiRef.current.getRowModels().values())

      apiRef.current.setRows(rows)

      updateProgress(1, 'Compiling file, this might take a while...') 

      setTimeout(async () => {
        await exportMethod[exportAs]({
          getRowsToExport: () => rows.map(({ id }) => id) as GridRowId[], // when I call the method here for excel with more than 50k ~ 100k rows it dies for up to 30s (I have a really nice MacBook pro m1 so it shouldn't be my setup)
        })

        apiRef.current.setRows(prevRows)

        doneLoading()
      }, 300)
    },
    [apiRef, doneLoading, updateProgress],
  )

It would be awesome if mui-x already supported compiling the file using a web-worker but I would appreciate any ideas on how to make this work better

Motivation 🔦

No response

Order ID 💳 (optional)

No response

[m4theushw]

The page hang up is probably because a Excel file has a complex internal structure and with 50k rows it needs too much processing resources. The problem with built-in support for web workers is that we'll to duplicate the logic to export to Excel. We can't pass apiRef to the worker because workers clone the data and the API is not cloneable.

I added the "waiting for upvotes" label to see if we should prioritize this or not.

[m4theushw]

In #7770, we've been working to support delegating part of the Excel export to a web worker. It reduced a lot the amount of the time that the main thread is frozen. It will still appear locked for some moment because we need to serialize the data to pass to the worker, however, once this task is completed all processing is done off the main thread. It would be nice to have your feedback. Here's the preview: https://deploy-preview-7770--material-ui-x.netlify.app/x/react-data-grid/export/#using-a-web-worker I also tested exporting 100k rows and it worked, but the main thread remained frozen for more time. If this is not enough, we could also delegate the serialization to the worker but it will sacrifice the developer experience, because the columns definition will also have to be added inside the worker.