Best Practices for Safely Using db.getRange

[damartripamungkas]

Hello! Are there any specific guidelines for retrieving a large dataset—such as 5 million records—using getRange? I am already utilizing the "start" and "end" options, but I need advice on other optional parameters to ensure safe and efficient data handling. Any guidance would be appreciated!

[kriszyp]

getRange should be quite efficient by default, as it will retrieve records on-demand for each iteration (doesn't hold the 5 million records in memory). However, if the query will continue for a long period of time and there are writes taking place and you aren't concerned about viewing an isolated snapshot of the data, you may want to consider disabling the snapshot flag (snapshot: false). This allows the read transaction to be continuously reset during iteration and avoids the cost of a long-lived read transactions (which causes database growth since it prevents in use old pages from being reclaimed/reused).