Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Messed up output when using -d:useMalloc on program with block statement, options, and json-parsing #21540

Closed
tsoj opened this issue Mar 18, 2023 · 5 comments · Fixed by #21688

Comments

@tsoj
Copy link

tsoj commented Mar 18, 2023

Description

When running the following program with nim r -d:useMalloc main.nim it produces unexpected results with the newest version, but it works fine with Nim 1.6.12.

import std/[
    json,
    options
]

let initialFEN = block:
    let initialFEN = some("""{"initialFen": "startpos"}""".parseJson{"initialFen"}.getStr)
    echo "<", initialFEN.get, ">"
    initialFEN.get

echo "<", initialFEN, ">"

Nim Version

Nim Compiler Version 1.9.1 [Linux: amd64]
Compiled at 2023-03-17
Copyright (c) 2006-2023 by Andreas Rumpf

git hash: b5ee81f
active boot switches: -d:release

Current Output

<startpos>
<�§�â*>

Expected Output

<startpos>
<startpos>

Possible Solution

No response

Additional Information

Workaround:

import std/[
    json,
    options
]

let initialFEN = block:
    let initialFEN = some("""{"initialFen": "startpos"}""".parseJson{"initialFen"}.getStr)
    echo "<", initialFEN.get, ">"
    let tmp = initialFEN.get # this temporary variable seems to mitigate the issue
    tmp

echo "<", initialFEN, ">"
@tsoj tsoj changed the title Think about the title, twice. Garbled output when using -d:useMalloc on program with block statement, options, and json-parsing Mar 18, 2023
@tsoj tsoj changed the title Garbled output when using -d:useMalloc on program with block statement, options, and json-parsing Messed up output when using -d:useMalloc on program with block statement, options, and json-parsing Mar 18, 2023
@ghost
Copy link

ghost commented Mar 18, 2023

Simplified a bit (x2, edited), seems to be related to get of Option returning a lent + copyMem

type
  Option = object
    val: string
    has: bool

proc some(val: string): Option =
  result.has = true
  result.val = val

# Remove lent and it works
proc get(self: Option): lent string =
  result = self.val

type
  StringStream = ref object
    data: string
    pos: int

proc readAll(s: StringStream): string =
  result = newString(s.data.len)
  copyMem(addr(result[0]), addr(s.data[0]), s.data.len)

proc newStringStream(s: string = ""): StringStream =
  new(result)
  result.data = s

proc parseJson(s: string): string =
  let stream = newStringStream(s)
  result = stream.readAll()

proc main = 
  let initialFEN = block:
    let initialFEN = some parseJson("startpos")
    initialFEN.get

  echo initialFEN

main()

@ringabout
Copy link
Member

Here is what the valgrind says

==19365== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0)
==19365== 
==19365== 1 errors in context 1 of 3:
==19365== Invalid read of size 1
==19365==    at 0x4852A10: memmove (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==19365==    by 0x10F7F0: memcpy (string_fortified.h:29)
==19365==    by 0x10F7F0: nimCopyMem (memory.nim:10)
==19365==    by 0x10F7F0: copyMem__system_1752 (system.nim:1966)
==19365==    by 0x10F7F0: nimAsgnStrV2 (strs_v2.nim:156)
==19365==    by 0x10F8AB: eqcopy___stdZassertions_33 (assertions.nim:25)
==19365==    by 0x1111DA: main__test50_59 (assertions.nim:25)
==19365==    by 0x11131E: NimMainModule (test2.nim:38)
==19365==    by 0x11136C: NimMainInner (assertions.nim:49)
==19365==    by 0x111383: NimMain (assertions.nim:60)
==19365==    by 0x1113A9: main (assertions.nim:68)
==19365==  Address 0x4a92110 is 16 bytes inside a block of size 17 free'd
==19365==    at 0x484B27F: free (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==19365==    by 0x10AB3D: deallocImpl__system_1772 (malloc.nim:28)
==19365==    by 0x10AB4F: deallocSharedImpl__system_1785 (malloc.nim:45)
==19365==    by 0x10AB61: deallocShared (memalloc.nim:308)
==19365==    by 0x10AB8C: eqdestroy___stdZassertions_30 (assertions.nim:25)
==19365==    by 0x110FE9: eqdestroy___test50_65 (test2.nim:33)
==19365==    by 0x1111BB: main__test50_59 (test2.nim:33)
==19365==    by 0x11131E: NimMainModule (test2.nim:38)
==19365==    by 0x11136C: NimMainInner (assertions.nim:49)
==19365==    by 0x111383: NimMain (assertions.nim:60)
==19365==    by 0x1113A9: main (assertions.nim:68)
==19365==  Block was alloc'd at
==19365==    at 0x484DA83: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==19365==    by 0x10AB19: alloc0Impl__system_1770 (malloc.nim:11)
==19365==    by 0x10AB2B: allocShared0Impl__system_1783 (malloc.nim:37)
==19365==    by 0x10F7B0: nimAsgnStrV2 (strs_v2.nim:151)
==19365==    by 0x10F8AB: eqcopy___stdZassertions_33 (assertions.nim:25)
==19365==    by 0x11085C: some__test50_4 (assertions.nim:25)
==19365==    by 0x11112C: main__test50_59 (test2.nim:33)
==19365==    by 0x11131E: NimMainModule (test2.nim:38)
==19365==    by 0x11136C: NimMainInner (assertions.nim:49)
==19365==    by 0x111383: NimMain (assertions.nim:60)
==19365==    by 0x1113A9: main (assertions.nim:68)
==19365== 
==19365== 
==19365== 1 errors in context 2 of 3:
==19365== Invalid read of size 8
==19365==    at 0x485298D: memmove (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==19365==    by 0x10F7F0: memcpy (string_fortified.h:29)
==19365==    by 0x10F7F0: nimCopyMem (memory.nim:10)
==19365==    by 0x10F7F0: copyMem__system_1752 (system.nim:1966)
==19365==    by 0x10F7F0: nimAsgnStrV2 (strs_v2.nim:156)
==19365==    by 0x10F8AB: eqcopy___stdZassertions_33 (assertions.nim:25)
==19365==    by 0x1111DA: main__test50_59 (assertions.nim:25)
==19365==    by 0x11131E: NimMainModule (test2.nim:38)
==19365==    by 0x11136C: NimMainInner (assertions.nim:49)
==19365==    by 0x111383: NimMain (assertions.nim:60)
==19365==    by 0x1113A9: main (assertions.nim:68)
==19365==  Address 0x4a92108 is 8 bytes inside a block of size 17 free'd
==19365==    at 0x484B27F: free (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==19365==    by 0x10AB3D: deallocImpl__system_1772 (malloc.nim:28)
==19365==    by 0x10AB4F: deallocSharedImpl__system_1785 (malloc.nim:45)
==19365==    by 0x10AB61: deallocShared (memalloc.nim:308)
==19365==    by 0x10AB8C: eqdestroy___stdZassertions_30 (assertions.nim:25)
==19365==    by 0x110FE9: eqdestroy___test50_65 (test2.nim:33)
==19365==    by 0x1111BB: main__test50_59 (test2.nim:33)
==19365==    by 0x11131E: NimMainModule (test2.nim:38)
==19365==    by 0x11136C: NimMainInner (assertions.nim:49)
==19365==    by 0x111383: NimMain (assertions.nim:60)
==19365==    by 0x1113A9: main (assertions.nim:68)
==19365==  Block was alloc'd at
==19365==    at 0x484DA83: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==19365==    by 0x10AB19: alloc0Impl__system_1770 (malloc.nim:11)
==19365==    by 0x10AB2B: allocShared0Impl__system_1783 (malloc.nim:37)
==19365==    by 0x10F7B0: nimAsgnStrV2 (strs_v2.nim:151)
==19365==    by 0x10F8AB: eqcopy___stdZassertions_33 (assertions.nim:25)
==19365==    by 0x11085C: some__test50_4 (assertions.nim:25)
==19365==    by 0x11112C: main__test50_59 (test2.nim:33)
==19365==    by 0x11131E: NimMainModule (test2.nim:38)
==19365==    by 0x11136C: NimMainInner (assertions.nim:49)
==19365==    by 0x111383: NimMain (assertions.nim:60)
==19365==    by 0x1113A9: main (assertions.nim:68)
==19365== 
==19365== 
==19365== 1 errors in context 3 of 3:
==19365== Invalid read of size 8
==19365==    at 0x10F723: nimAsgnStrV2 (strs_v2.nim:26)
==19365==    by 0x10F8AB: eqcopy___stdZassertions_33 (assertions.nim:25)
==19365==    by 0x1111DA: main__test50_59 (assertions.nim:25)
==19365==    by 0x11131E: NimMainModule (test2.nim:38)
==19365==    by 0x11136C: NimMainInner (assertions.nim:49)
==19365==    by 0x111383: NimMain (assertions.nim:60)
==19365==    by 0x1113A9: main (assertions.nim:68)
==19365==  Address 0x4a92100 is 0 bytes inside a block of size 17 free'd
==19365==    at 0x484B27F: free (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==19365==    by 0x10AB3D: deallocImpl__system_1772 (malloc.nim:28)
==19365==    by 0x10AB4F: deallocSharedImpl__system_1785 (malloc.nim:45)
==19365==    by 0x10AB61: deallocShared (memalloc.nim:308)
==19365==    by 0x10AB8C: eqdestroy___stdZassertions_30 (assertions.nim:25)
==19365==    by 0x110FE9: eqdestroy___test50_65 (test2.nim:33)
==19365==    by 0x1111BB: main__test50_59 (test2.nim:33)
==19365==    by 0x11131E: NimMainModule (test2.nim:38)
==19365==    by 0x11136C: NimMainInner (assertions.nim:49)
==19365==    by 0x111383: NimMain (assertions.nim:60)
==19365==    by 0x1113A9: main (assertions.nim:68)
==19365==  Block was alloc'd at
==19365==    at 0x484DA83: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==19365==    by 0x10AB19: alloc0Impl__system_1770 (malloc.nim:11)
==19365==    by 0x10AB2B: allocShared0Impl__system_1783 (malloc.nim:37)
==19365==    by 0x10F7B0: nimAsgnStrV2 (strs_v2.nim:151)
==19365==    by 0x10F8AB: eqcopy___stdZassertions_33 (assertions.nim:25)
==19365==    by 0x11085C: some__test50_4 (assertions.nim:25)
==19365==    by 0x11112C: main__test50_59 (test2.nim:33)
==19365==    by 0x11131E: NimMainModule (test2.nim:38)
==19365==    by 0x11136C: NimMainInner (assertions.nim:49)
==19365==    by 0x111383: NimMain (assertions.nim:60)
==19365==    by 0x1113A9: main (assertions.nim:68)
==19365== 
==19365== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0)

@ringabout ringabout self-assigned this Apr 12, 2023
@ringabout
Copy link
Member

related: #20107

@ringabout
Copy link
Member

var
  initialFEN
  :tmpD
try:
  `=copy`(initialFEN, block :tmp:
    var
      seed
      :tmpD_1
    :tmpD =
      seed = some do:
        :tmpD_1 = parseJson("startpos")
        :tmpD_1
      get(seed)
    `=destroy`(:tmpD_1)
    `=destroy_1`(seed)
    :tmpD)
  echo [initialFEN]
finally:
  `=destroy`(initialFEN)

The problem is that :tmpD borrows from seed, which is destroyed before:tmpD is assigned to variables outside the block.

@ringabout
Copy link
Member

ringabout commented Apr 18, 2023

A possible soultion I have in mind is somehow to deref the block at the sempass/transf, not in the codegen phase so that the tmp will get the derefed value without worrying about the borrowed objects.

ringabout added a commit that referenced this issue Apr 19, 2023
Araq pushed a commit that referenced this issue Apr 19, 2023
…unction properly (#21688)

* fixes #21540; deref block at transf phase to make injectdestructors function properly

* add a test case

* add one more test

* fixes the type of block

* transform block
capocasa pushed a commit to capocasa/Nim that referenced this issue May 15, 2023
…uctors function properly (nim-lang#21688)

* fixes nim-lang#21540; deref block at transf phase to make injectdestructors function properly

* add a test case

* add one more test

* fixes the type of block

* transform block
capocasa pushed a commit to capocasa/Nim that referenced this issue May 16, 2023
…uctors function properly (nim-lang#21688)

* fixes nim-lang#21540; deref block at transf phase to make injectdestructors function properly

* add a test case

* add one more test

* fixes the type of block

* transform block
narimiran pushed a commit that referenced this issue Jun 19, 2023
…unction properly (#21688)

* fixes #21540; deref block at transf phase to make injectdestructors function properly

* add a test case

* add one more test

* fixes the type of block

* transform block

(cherry picked from commit 9cb06d3)
narimiran pushed a commit that referenced this issue Jun 19, 2023
…unction properly (#21688)

* fixes #21540; deref block at transf phase to make injectdestructors function properly

* add a test case

* add one more test

* fixes the type of block

* transform block

(cherry picked from commit 9cb06d3)
bung87 pushed a commit to bung87/Nim that referenced this issue Jul 29, 2023
…uctors function properly (nim-lang#21688)

* fixes nim-lang#21540; deref block at transf phase to make injectdestructors function properly

* add a test case

* add one more test

* fixes the type of block

* transform block
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants