Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incremental md5 #10

Merged
merged 2 commits into from Jul 14, 2016
Merged

Incremental md5 #10

merged 2 commits into from Jul 14, 2016

Conversation

ghost
Copy link

@ghost ghost commented Jul 2, 2016

Implements #8.

Tested with a 47 Gb file, with LuaJIT only, using the following program:

local md5 = require 'md5'

local f,x = io.open(arg[1], 'rb')
print(f, x)

local state = md5.new()

repeat
  local s = f:read(math.random(1, 800))
  if not s then break end
  state:update(s)
until false
f:close()

print(md5.tohex(state:finish()))

(took 3+ hours; the result was compared to that of md5sum which took ~8 minutes). The updates were sent with random lengths to exercise different buffer fills.

Couldn't test with Lua as it refused to open the file.

It is based on the commits of PR #9, so they are included too. I can rebase if necessary.

Two new functions:

md5.new(): returns a new MD5 state object with two methods:
 - MD5State:update(s) - adds the string s to the calculation
 - MD5State:finish()  - returns the final MD5 as a binary string

md5.tohex(): converts to hexadecimal the binary string returned by md5.sum() and MD5State:finish()
@kikito
Copy link
Owner

kikito commented Jul 4, 2016

I have not forgotten about this (hectic weekend). I still have to give it a more thorough review.

@ghost
Copy link
Author

ghost commented Jul 4, 2016

Thanks for the heads-up. Take your time, there's no hurry. Meanwhile I'll push my rebased version.

@kikito kikito merged commit 46b5000 into kikito:master Jul 14, 2016
@kikito
Copy link
Owner

kikito commented Jul 14, 2016

I finally was able to review this (sorry about the delay).

I made a small change: update returns self. This way, you can chain commands:

local x = md5.new():update('foo'):update('bar'):finish()

I will now create a changelog and release md5 1.1.0

EDIT: updated and uploaded to Luarocks!

@ghost ghost deleted the incremental-md5 branch July 15, 2016 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant