Add toBytes proc #128

ehmry · 2023-06-19T10:14:23Z

Export isNegative and isZero because it is easier to export here than to reimplement downstream.
Add toBytes proc for serializing. This is used by the CBOR package. Also easier to implement here than downstream.
Rename the tests/tliterals.nim to comply with Nimble standards. Not all tools are using nimble test for testing, some rely only on Nimble standards.

dlesnoff · 2023-06-19T11:32:38Z

Thanks for your contribution! The algorithm looks fine to me.
– Could you add tests for the toBytes procedure?
– Could you add the corresponding fromBytes procedure?
I believe that the tliterals was renamed literals as to not run these tests. You might want to check the log for this specific file.

dlesnoff · 2023-06-19T11:40:00Z

src/bigints.nim

+  ## Convert a `BigInt` to a byte-sequence.
+  ## The byte-sequence *does not* include a sign-bit.


Why this "limbs-only" bytes serialization?

Suggested change

## Convert a `BigInt` to a byte-sequence.

## The byte-sequence *does not* include a sign-bit.

## Convert the limbs of a `BigInt` to a byte-sequence.

## Sign-bit must be stored separately.

I think sign-bits only belong in hardware-registers and it's more common for arbitrary width integers in binary serializations to be tagged some other way if they are negative. If a sign bit really needs to be in there then it can be inserted in the buffer after toBytes.

ehmry · 2023-06-21T09:11:02Z

I pushed a test for toBytes. I will push fromBytes soon.

ehmry · 2023-06-21T10:17:24Z

Now with fromBytes and a test of random serializations.

konsumlamm · 2023-07-22T18:51:18Z

tests/tbigints.nim

-      include tliterals
+      include ./literals


Why rename this specific file and not the others? Since we overwrite the default nimble test task, we shouldn't need the t prefix anymore. Either way, this is unrelated to the rest of the PR, so please make a new PR for this.

konsumlamm · 2023-07-22T18:58:02Z

src/bigints.nim

+    doAssert buf == @[0x1'u8,0x2,0x3,0x4,0x5,0x6,0x7,0x8,0xA]
+  if not a.isZero:
+    result = newSeq[byte](a.limbs.len shl 2)
+    var i: int


Suggested change

var i: int

var i = 0

Nim initializes integers to zero.

I know, but imo it's clearer to explicitly initialize them, rather than relying on implicit initialization.

konsumlamm · 2023-07-22T19:00:58Z

tests/trandom.nim

+  var a, b: Bigint
+  for x in 0..trials:
+    a = rand(0.initBigInt..pow(2.initBigInt, x))
+    for endian in [bigEndian, littleEndian]:


Suggested change

for endian in [bigEndian, littleEndian]:

for endian in [bigEndian, littleEndian]:

bigEndian and littleEndian aren't in scope here. I wonder why the CI didn't run...

konsumlamm · 2023-07-22T19:14:34Z

tests/tbigints.nim

@@ -880,6 +880,22 @@ proc main() =
    doAssert pred(a, 3) == initBigInt(4)
    doAssert succ(a, 3) == initBigInt(10)

+  when nimvm: discard


Why doesn't this work in a static context?

Endianness.

You mean std/endians is not available? I see.

konsumlamm · 2023-07-22T19:15:53Z

src/bigints.nim

+      for s in [24, 16, 8, 0]:
+        result[i] = uint8(a.limbs[a.limbs.high] shr s)
+        if result[0] != 0x00: inc(i)
+      for l in countdown(a.limbs.high.pred, a.limbs.low):


Suggested change

for l in countdown(a.limbs.high.pred, a.limbs.low):

for l in countdown(a.limbs.high.pred, 0):

low on a seq is always 0. You also used 0 below, so that's consistent.

A low on an array is always 0, a low on an empty seq can be -1.

We already checked that a is not zero (so a.limbs is not empty), but I don't really care, we can keep it that way.

konsumlamm · 2023-07-22T19:31:42Z

src/bigints.nim

+    var
+      li = result.limbs.low
+      bi = buf.low
+    while (bi + 4) < buf.len:


Suggested change

while (bi + 4) < buf.len:

while (bi + 3) < buf.len:

littleEndian32 uses buf[bi], buf[bi + 1], buf[bi + 2] and buf[bi + 3], not buf[bi + 4].

konsumlamm · 2023-07-22T19:38:47Z

src/bigints.nim

+      inc(li)
+    if bi < buf.len:
+      var
+        limb: uint32


Suggested change

limb: uint32

limb = 0'u32

Nim initializes uint32 to zero.

Explicit is better than implicit. For integers, I may understand since we implicitly rely on the litteral 0 being an integer.

konsumlamm · 2023-07-22T19:40:36Z

src/bigints.nim

+      bi = buf.low
+    block:
+      var
+        limb: uint32


Suggested change

limb: uint32

limb = 0'u32

Nim initializes uint32 to zero.

konsumlamm · 2023-07-22T19:46:07Z

src/bigints.nim

+      for s in [24, 16, 8, 0]:
+        result[i] = uint8(a.limbs[a.limbs.high] shr s)
+        if result[0] != 0x00: inc(i)


Suggested change

for s in [24, 16, 8, 0]:

result[i] = uint8(a.limbs[a.limbs.high] shr s)

if result[0] != 0x00: inc(i)

# convert a.limbs[a.limbs.high]

let highLimb = a.limbs[a.limbs.high]

for s in [24, 16, 8, 0]:

result[i] = uint8(highLimb shr s)

if result[0] != 0x00: inc(i)

konsumlamm · 2023-07-22T20:00:51Z

src/bigints.nim

+      var
+        limb: uint32
+        j = 4 - (buf.len and 3)
+      while j < 4:
+        limb = (limb shl 8) or buf[bi].uint32
+        inc(bi)
+        inc(j)


Suggested change

var

limb: uint32

j = 4 - (buf.len and 3)

while j < 4:

limb = (limb shl 8) or buf[bi].uint32

inc(bi)

inc(j)

var limb = 0'u32

for _ in 1..(buf.len and 0b11):

limb = (limb shl 8) or buf[bi].uint32

inc(bi)

(buf.len and 3) should be (buf.len and 0b11).

Right, edited.

konsumlamm · 2023-07-22T20:01:28Z

I'm against exporting isNegative and isZero. I think the better solution would be to support comparing with ints, then we can just use < 0 and == 0 (see #132).

ehmry · 2023-07-23T07:25:09Z

I'm against exporting isNegative and isZero. I think the better solution would be to support comparing with ints, then we can just use < 0 and == 0 (see #132).

That would be adding more code to do something that is already implemented.

konsumlamm · 2023-07-23T11:39:22Z

I'm against exporting isNegative and isZero. I think the better solution would be to support comparing with ints, then we can just use < 0 and == 0 (see #132).

That would be adding more code to do something that is already implemented.

Yes, but it's also uglier imo.

dlesnoff · 2023-07-24T20:51:40Z

I am for exporting isNegative only and implement equality with integers.
Comparing with integers and fetching a boolean flag are two different things.

konsumlamm · 2023-07-24T20:53:27Z

Note that isNegative really means "is less than or equal to zero", as 0 can also have isNegative set.

dlesnoff · 2023-07-24T21:11:49Z

Thanks for the reminder.

ehmry force-pushed the bytes branch from 3363d52 to eca7a52 Compare June 19, 2023 10:23

dlesnoff mentioned this pull request Jun 19, 2023

Rename tests/tliterals for automated testing #127

Closed

dlesnoff reviewed Jun 19, 2023

View reviewed changes

ehmry force-pushed the bytes branch from eca7a52 to 8c78aa2 Compare June 21, 2023 08:59

ehmry force-pushed the bytes branch from 8c78aa2 to 6d6a077 Compare June 21, 2023 10:16

ehmry force-pushed the bytes branch from 6d6a077 to 6d77ea7 Compare June 21, 2023 10:27

konsumlamm reviewed Jul 22, 2023

View reviewed changes

ehmry added 4 commits October 6, 2023 09:29

Rename tests/tliterals according to Nimble standards

bf0efb4

Export isNegative and isZero

1a150ec

Add toBytes proc

c132edd

Add fromBytes proc

0f0284f

ehmry force-pushed the bytes branch from 6d77ea7 to 0f0284f Compare October 6, 2023 08:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add toBytes proc #128

Add toBytes proc #128

ehmry commented Jun 19, 2023 •

edited

Loading

dlesnoff commented Jun 19, 2023 •

edited

Loading

dlesnoff Jun 19, 2023 •

edited

Loading

ehmry Jun 21, 2023

ehmry commented Jun 21, 2023

ehmry commented Jun 21, 2023

konsumlamm Jul 22, 2023

ehmry Jul 23, 2023

konsumlamm Jul 23, 2023

konsumlamm Jul 22, 2023

ehmry Jul 23, 2023

konsumlamm Jul 23, 2023

konsumlamm Jul 22, 2023

konsumlamm Jul 22, 2023

ehmry Jul 23, 2023

konsumlamm Jul 23, 2023

konsumlamm Jul 22, 2023

ehmry Jul 23, 2023

konsumlamm Jul 23, 2023

konsumlamm Jul 22, 2023

konsumlamm Jul 22, 2023

ehmry Jul 23, 2023

dlesnoff Jul 24, 2023

konsumlamm Jul 22, 2023

ehmry Jul 23, 2023

konsumlamm Jul 22, 2023

konsumlamm Jul 22, 2023 •

edited

Loading

ehmry Jul 23, 2023

konsumlamm Jul 23, 2023

konsumlamm commented Jul 22, 2023

ehmry commented Jul 23, 2023

konsumlamm commented Jul 23, 2023

dlesnoff commented Jul 24, 2023

konsumlamm commented Jul 24, 2023

dlesnoff commented Jul 24, 2023

		## Convert a `BigInt` to a byte-sequence.
		## The byte-sequence does not include a sign-bit.

	for endian in [bigEndian, littleEndian]:
	for endian in [bigEndian, littleEndian]:

	for l in countdown(a.limbs.high.pred, a.limbs.low):
	for l in countdown(a.limbs.high.pred, 0):

Add toBytes proc #128

Are you sure you want to change the base?

Add toBytes proc #128

Conversation

ehmry commented Jun 19, 2023 • edited Loading

dlesnoff commented Jun 19, 2023 • edited Loading

dlesnoff Jun 19, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ehmry commented Jun 21, 2023

ehmry commented Jun 21, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

konsumlamm Jul 22, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

konsumlamm commented Jul 22, 2023

ehmry commented Jul 23, 2023

konsumlamm commented Jul 23, 2023

dlesnoff commented Jul 24, 2023

konsumlamm commented Jul 24, 2023

dlesnoff commented Jul 24, 2023

ehmry commented Jun 19, 2023 •

edited

Loading

dlesnoff commented Jun 19, 2023 •

edited

Loading

dlesnoff Jun 19, 2023 •

edited

Loading

konsumlamm Jul 22, 2023 •

edited

Loading