Skip to content

fix(subst): skip large files on 32-bit #9811

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 30, 2024
Merged

fix(subst): skip large files on 32-bit #9811

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b3b1b7f
fix(subst): ignore large files
emillon Jan 23, 2024
b8af2ca
Merge branch 'main' into fix-9538
emillon Jan 30, 2024
3c0a7b8
Merge branch 'main' into fix-9538
emillon Jan 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 21 additions & 9 deletions bin/subst.ml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,15 +120,27 @@ let subst_string s path ~map =
;;

let subst_file path ~map =
let s = Io.read_file path in
let s =
if Path.is_root (Path.parent_exn path) && Package.is_opam_file path
then "version: \"%%" ^ "VERSION_NUM" ^ "%%\"\n" ^ s
else s
in
match subst_string s ~map path with
| None -> ()
| Some s -> Io.write_file path s
match Io.with_file_in path ~f:Io.read_all_unless_large with
| Error () ->
let hints =
if Sys.word_size = 32
then
[ Pp.textf
"Dune has been built as a 32-bit binary so the maximum size \"dune subst\" \
can operate on is 16MiB."
]
else []
in
User_warning.emit ~hints [ Pp.textf "Ignoring large file: %s" (Path.to_string path) ]
| Ok s ->
let s =
if Path.is_root (Path.parent_exn path) && Package.is_opam_file path
then "version: \"%%" ^ "VERSION_NUM" ^ "%%\"\n" ^ s
else s
in
(match subst_string s ~map path with
| None -> ()
| Some s -> Io.write_file path s)
;;

(* Extending the Dune_project APIs, but adding capability to modify *)
Expand Down
1 change: 1 addition & 0 deletions doc/changes/9811.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- subst: do not fail on 32-bit systems when large files are encountered. Just log a warning in this case. (#9811, fixes #9538, @emillon)
11 changes: 4 additions & 7 deletions test/blackbox-tests/test-cases/subst/32bit.t
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,7 @@ This test uses subst, which needs a git repository:
create mode 100644 dune-project
create mode 100644 large.dat

$ dune subst 2>&1 | head -n 6
Error: exception Invalid_argument("Bytes.create")
Raised by primitive operation at Stdune__Io.Make.eagerly_input_string in file
"otherlibs/stdune/src/io.ml", line 273, characters 14-30
Called from Stdune__Io.Make.read_all.(fun) in file
"otherlibs/stdune/src/io.ml", line 308, characters 16-40
Called from Stdune__Exn.protectx in file "otherlibs/stdune/src/exn.ml", line
$ dune subst
Warning: Ignoring large file: large.dat
Hint: Dune has been built as a 32-bit binary so the maximum size "dune subst"
can operate on is 16MiB.