Skip to content

Commit

Permalink
Set DelimitedFiles.readdlm use_mmap default to false for all OSes (Ju…
Browse files Browse the repository at this point in the history
…liaLang#40415)

Increased resilience to edge cases where OS is reported Unix-like but Filesystem is abnormal

Fixes JuliaLang#8891
Fixes JuliaLang#40352
  • Loading branch information
quildtide authored and ElOceanografo committed May 4, 2021
1 parent 6c06f05 commit cd5eb55
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 3 deletions.
3 changes: 3 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,9 @@ Standard library changes

* `mmap` is now exported ([#39816]).

#### DelimitedFiles

* `readdlm` now defaults to `use_mmap=false` on all OSes for consistent reliability in abnormal filesystem situations ([#40415]).

Deprecated or removed
---------------------
Expand Down
7 changes: 4 additions & 3 deletions stdlib/DelimitedFiles/src/DelimitedFiles.jl
Original file line number Diff line number Diff line change
Expand Up @@ -190,8 +190,9 @@ Specifying `skipstart` will ignore the corresponding number of initial lines fro
If `skipblanks` is `true`, blank lines in the input will be ignored.
If `use_mmap` is `true`, the file specified by `source` is memory mapped for potential
speedups. Default is `true` except on Windows. On Windows, you may want to specify `true` if
the file is large, and is only read once and not written to.
speedups if the file is large. Default is `false'. On a Windows filesystem, `use_mmap` should not be set
to `true` unless the file is only read once and is also not written to.
Some edge cases exist where an OS is Unix-like but the filesystem is Windows-like.
If `quotes` is `true`, columns enclosed within double-quote (\") characters are allowed to
contain new lines and column delimiters. Double-quote characters within a quoted field must
Expand Down Expand Up @@ -232,7 +233,7 @@ readdlm_auto(input::IO, dlm::AbstractChar, T::Type, eol::AbstractChar, auto::Boo
function readdlm_auto(input::AbstractString, dlm::AbstractChar, T::Type, eol::AbstractChar, auto::Bool; opts...)
isfile(input) || throw(ArgumentError("Cannot open \'$input\': not a file"))
optsd = val_opts(opts)
use_mmap = get(optsd, :use_mmap, Sys.iswindows() ? false : true)
use_mmap = get(optsd, :use_mmap, false)
fsz = filesize(input)
if use_mmap && fsz > 0 && fsz < typemax(Int)
a = open(input, "r") do f
Expand Down

0 comments on commit cd5eb55

Please sign in to comment.