v3.0.0

What's Changed

v3.0.0 is in principle a refactor of v2.0.1 with the goal of making the codebase maintainable. Hopefully some bugs have been fixed and not too many introduced.

Compatibility with version v2.0.1 is broken in the sense that some flags have changed name, some new flags have been introduced and the logging is not the same.

Other Changes

• Fix panic when using the skip-file feature of hooks by @johnerikhalse in #50
• ci: add dependabot configuration by @trym-b in #51
• build(deps): bump actions/checkout from 3 to 4 by @dependabot in #58
• ci: remove unnecessary GitHub Actions triggers by @trym-b in #52
• docs: change maintainer by @trym-b in #54
• build(deps): bump golangci/golangci-lint-action from 3 to 4 by @dependabot in #56
• build(deps): bump actions/setup-go from 3 to 5 by @dependabot in #59
• build(deps): bump github.com/dgraph-io/badger/v3 from 3.2103.4 to 3.2103.5 by @dependabot in #60
• build(deps): bump goreleaser/goreleaser-action from 4 to 5 by @dependabot in #57
• build(deps): bump github.com/shirou/gopsutil from 2.21.11+incompatible to 3.21.11+incompatible by @dependabot in #63
• build(deps): bump github.com/klauspost/compress from 1.17.0 to 1.17.7 by @dependabot in #61
• build(deps): bump github.com/sirupsen/logrus from 1.8.1 to 1.9.3 by @dependabot in #62
• docs: update required version of go by @trym-b in #55
• refactor: use longer argument names by @trym-b in #53
• ci: configure golangci-lint by @trym-b in #67
• refactor: format imports with goimports by @trym-b in #64
• ci: enable goimports linter by @trym-b in #74
• refactor: separation of arguments and logic by @trym-b in #71
• Fix/enable mirror linter by @trym-b in #73
• Fix/avoid shadowing reserved keywords by @trym-b in #75
• refactor: remove license headers from source files by @trym-b in #69
• CI: enable misspell linter by @trym-b in #72
• fix: replace deprecated goreleaser command by @trym-b in #77
• refactor: consolidate workflows by @trym-b in #78
• refactor: consolidate scripts location by @trym-b in #68
• CI: enable bodyclose linter by @trym-b in #81
• build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 by @dependabot in #82
• Refactor: cat by @trym-b in #80
• ci: store artifacts after build by @trym-b in #79
• fix: remove deprecated replacements by @trym-b in #83
• build: bump goreleaser version by @trym-b in #84
• fix: remove deprecated archives.rlcp by @trym-b in #85
• fix: silence output when running go test by @trym-b in #88
• CI: add benchmark workflow by @trym-b in #91
• ci: push benchmark results to dedicated branch by @trym-b in #92
• fix: use correct OS by @trym-b in #93
• build(deps): bump google.golang.org/protobuf from 1.31.0 to 1.33.0 by @dependabot in #95
• ci: compare with previous benchmarks by @trym-b in #96
• build(deps): bump actions/setup-go from 4 to 5 by @dependabot in #97
• ci: run tests on windows by @trym-b in #98
• ci: enable mac testing by @trym-b in #101
• ci: enable benchmarking for different OSs by @trym-b in #102
• Revert "ci: enable benchmarking for MacOS" by @trym-b in #103
• Revert "ci: enable benchmarking for windows" by @trym-b in #104
• refactor: workerpool by @trym-b in #105
• refactor: filewalker.go by @trym-b in #106
• fix: handle filewalker errors by @trym-b in #107
• fix: avoid shadowing err by @trym-b in #109
• test: add simplest test for ls by @trym-b in #111
• test: initial benchmark for ls by @trym-b in #112
• refactor: variables in validate.go by @trym-b in #114
• refactor: dedup by @trym-b in #116
• fix: error handling for ls.RecordWriter by @trym-b in #113
• refactor: ssurt.go by @trym-b in #122
• test: add initial test for validate.go by @trym-b in #115
• refactor: delete unused code by @trym-b in #123
• ci: enable windows benchmarks by @trym-b in #125
• refactor: console by @trym-b in #117
• refactor: arc.go by @trym-b in #118
• refactor: nedlib.go by @trym-b in #119
• refactor: warc.go by @trym-b in #120
• refactor: rename variables in nedlibreader.go by @trym-b in #121
• test: avoid leaking goroutines by @trym-b in #124
• fix: silence benchmark output by @trym-b in #126
• refactor: nedlib.go function by @trym-b in #127
• refactor: split functions in filewalker.go by @trym-b in #128
• test: add test for Result and Stats by @trym-b in #129
• build(deps): bump go.uber.org/goleak from 1.2.0 to 1.3.0 by @dependabot in #131
• build(deps): bump peaceiris/actions-hugo from 2 to 3 by @dependabot in #130
• build(deps): bump github.com/klauspost/compress from 1.17.7 to 1.17.8 by @dependabot in #137
• build(deps): bump peaceiris/actions-gh-pages from 3 to 4 by @dependabot in #138
• test: add initial test data for nedlib by @trym-b in #140
• fix: set hugo version to 1.24.1 by @trym-b in #142
• build(deps): bump golang.org/x/net from 0.19.0 to 0.23.0 by @dependabot in #143
• build(deps): bump golangci/golangci-lint-action from 4 to 5 by @dependabot in #144
• fix: use integer parameter by @trym-b in #133
• refactor: rename testdata dir by @trym-b in #145
• build(deps): bump github.com/nlnwa/gowarc from 1.5.0 to 1.6.0 by @dependabot in #147
• build: bump go version to latest by @trym-b in #146
• build(deps): bump golangci/golangci-lint-action from 5 to 6 by @dependabot in #148
• fix: validate panics if record with error by @maeb in #136
• fix: parse time function for nedlib by @trym-b in #139
• feat: support parsing all known time formats by @maeb in #150
• build(deps): bump github.com/nlnwa/whatwg-url from 0.4.0 to 0.4.1 by @dependabot in #149
• feat: WARC record iterator by @maeb in #134
• fix: parse broken rfc1123 format by @maeb in #152
• test: add test for nedlibreader by @maeb in #151
• Never fail on missing date by @maeb in #153
• build(deps): bump github.com/spf13/viper from 1.18.2 to 1.19.0 by @dependabot in #154
• build(deps): bump goreleaser/goreleaser-action from 5 to 6 by @dependabot in #155
• build(deps): bump github.com/spf13/cobra from 1.8.0 to 1.8.1 by @dependabot in https://gith...

v3.0.0-beta.4

What's Changed

Other Changes

• Fix iteration of warc records by @maeb in #169

Full Changelog: v3.0.0-beta.3...v3.0.0-beta.4

v3.0.0-beta.3

What's Changed

Other Changes

• Fix ARC reading/writing by @maeb in #168

Full Changelog: v3.0.0-beta.2...v3.0.0-beta.3

v3.0.0-beta.2

What's Changed

Other Changes

• Update _index.md by @white-gecko in #162
• build(deps): bump github.com/klauspost/compress from 1.17.9 to 1.17.10 by @dependabot in #164
• Fix one-to-one writing by @maeb in #166
• build(deps): bump github.com/klauspost/compress from 1.17.10 to 1.17.11 by @dependabot in #165
• build(deps): bump github.com/nlnwa/whatwg-url from 0.4.1 to 0.5.0 by @dependabot in #163

New Contributors

• @white-gecko made their first contribution in #162

Full Changelog: v3.0.0-beta.1...v3.0.0-beta.2

v3.0.0-beta.1

The codebase has been refactored and needs usability testing to get out of beta.

What's Changed

• Some flags in v3 have changed their short and/or longform
• Logging has been reworked (no progress)
• Configuration has been simplified (no hierarchical merging and no config-per-subcommand in the configuration file)
• Unified flags
• Unified record iteration

Other Changes

• Fix panic when using the skip-file feature of hooks by @johnerikhalse in #50
• ci: add dependabot configuration by @trym-b in #51
• build(deps): bump actions/checkout from 3 to 4 by @dependabot in #58
• ci: remove unnecessary GitHub Actions triggers by @trym-b in #52
• docs: change maintainer by @trym-b in #54
• build(deps): bump golangci/golangci-lint-action from 3 to 4 by @dependabot in #56
• build(deps): bump actions/setup-go from 3 to 5 by @dependabot in #59
• build(deps): bump github.com/dgraph-io/badger/v3 from 3.2103.4 to 3.2103.5 by @dependabot in #60
• build(deps): bump goreleaser/goreleaser-action from 4 to 5 by @dependabot in #57
• build(deps): bump github.com/shirou/gopsutil from 2.21.11+incompatible to 3.21.11+incompatible by @dependabot in #63
• build(deps): bump github.com/klauspost/compress from 1.17.0 to 1.17.7 by @dependabot in #61
• build(deps): bump github.com/sirupsen/logrus from 1.8.1 to 1.9.3 by @dependabot in #62
• docs: update required version of go by @trym-b in #55
• refactor: use longer argument names by @trym-b in #53
• ci: configure golangci-lint by @trym-b in #67
• refactor: format imports with goimports by @trym-b in #64
• ci: enable goimports linter by @trym-b in #74
• refactor: separation of arguments and logic by @trym-b in #71
• Fix/enable mirror linter by @trym-b in #73
• Fix/avoid shadowing reserved keywords by @trym-b in #75
• refactor: remove license headers from source files by @trym-b in #69
• CI: enable misspell linter by @trym-b in #72
• fix: replace deprecated goreleaser command by @trym-b in #77
• refactor: consolidate workflows by @trym-b in #78
• refactor: consolidate scripts location by @trym-b in #68
• CI: enable bodyclose linter by @trym-b in #81
• build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 by @dependabot in #82
• Refactor: cat by @trym-b in #80
• ci: store artifacts after build by @trym-b in #79
• fix: remove deprecated replacements by @trym-b in #83
• build: bump goreleaser version by @trym-b in #84
• fix: remove deprecated archives.rlcp by @trym-b in #85
• fix: silence output when running go test by @trym-b in #88
• CI: add benchmark workflow by @trym-b in #91
• ci: push benchmark results to dedicated branch by @trym-b in #92
• fix: use correct OS by @trym-b in #93
• build(deps): bump google.golang.org/protobuf from 1.31.0 to 1.33.0 by @dependabot in #95
• ci: compare with previous benchmarks by @trym-b in #96
• build(deps): bump actions/setup-go from 4 to 5 by @dependabot in #97
• ci: run tests on windows by @trym-b in #98
• ci: enable mac testing by @trym-b in #101
• ci: enable benchmarking for different OSs by @trym-b in #102
• Revert "ci: enable benchmarking for MacOS" by @trym-b in #103
• Revert "ci: enable benchmarking for windows" by @trym-b in #104
• refactor: workerpool by @trym-b in #105
• refactor: filewalker.go by @trym-b in #106
• fix: handle filewalker errors by @trym-b in #107
• fix: avoid shadowing err by @trym-b in #109
• test: add simplest test for ls by @trym-b in #111
• test: initial benchmark for ls by @trym-b in #112
• refactor: variables in validate.go by @trym-b in #114
• refactor: dedup by @trym-b in #116
• fix: error handling for ls.RecordWriter by @trym-b in #113
• refactor: ssurt.go by @trym-b in #122
• test: add initial test for validate.go by @trym-b in #115
• refactor: delete unused code by @trym-b in #123
• ci: enable windows benchmarks by @trym-b in #125
• refactor: console by @trym-b in #117
• refactor: arc.go by @trym-b in #118
• refactor: nedlib.go by @trym-b in #119
• refactor: warc.go by @trym-b in #120
• refactor: rename variables in nedlibreader.go by @trym-b in #121
• test: avoid leaking goroutines by @trym-b in #124
• fix: silence benchmark output by @trym-b in #126
• refactor: nedlib.go function by @trym-b in #127
• refactor: split functions in filewalker.go by @trym-b in #128
• test: add test for Result and Stats by @trym-b in #129
• build(deps): bump go.uber.org/goleak from 1.2.0 to 1.3.0 by @dependabot in #131
• build(deps): bump peaceiris/actions-hugo from 2 to 3 by @dependabot in #130
• build(deps): bump github.com/klauspost/compress from 1.17.7 to 1.17.8 by @dependabot in #137
• build(deps): bump peaceiris/actions-gh-pages from 3 to 4 by @dependabot in #138
• test: add initial test data for nedlib by @trym-b in #140
• fix: set hugo version to 1.24.1 by @trym-b in #142
• build(deps): bump golang.org/x/net from 0.19.0 to 0.23.0 by @dependabot in #143
• build(deps): bump golangci/golangci-lint-action from 4 to 5 by @dependabot in #144
• fix: use integer parameter by @trym-b in #133
• refactor: rename testdata dir by @trym-b in #145
• build(deps): bump github.com/nlnwa/gowarc from 1.5.0 to 1.6.0 by @dependabot in #147
• build: bump go version to latest by @trym-b in #146
• build(deps): bump golangci/golangci-lint-action from 5 to 6 by @dependabot in #148
• fix: validate panics if record with error by @maeb in #136
• fix: parse time function for nedlib by @trym-b in #139
• feat: support parsing all known time formats by @maeb in #150
• build(deps): bump github.com/nlnwa/whatwg-url from 0.4.0 to 0.4.1 by @dependabot in #149
• feat: WARC record iterator by @maeb in #134
• fix: parse broken rfc1123 format by @maeb in #152
• test: add test for nedlibreader by @maeb in #151
• Never fail on missing date by @maeb in #153
• build(deps): bump github.com/spf13/viper from 1.18.2 to 1.19.0 by @dependabot in #154
• build(deps): bump goreleaser/goreleaser-action from 5 to 6 by @dependabot in #155
• build(deps): bump github.com/spf13/cobra from 1.8.0 to 1.8.1 by @...

v2.0.1

What's Changed

Other Changes

• Fix panic when using the skip-file feature of hooks by @johnerikhalse in #50

Full Changelog: v2.0.0...v2.0.1

v2.0.0

What's Changed

Exciting New Features 🎉

• Support for reading source files from ftp by @johnerikhalse in #42
• Processing hooks by @johnerikhalse in #43
• Nedlib converter: Create WARC files with filename based on date of first record. by @johnerikhalse in #45
• Support tar and tar.gz files as filesystem for input files by @johnerikhalse in #46
• Support for exit codes in hooks by @johnerikhalse in #47

Other Changes

• Bump golang.org/x/net from 0.11.0 to 0.17.0 by @dependabot in #38
• Enhanced ARC to WARC conversion by @johnerikhalse in #39
• Fix panic when no hooks was defined by @johnerikhalse in #44
• Fix command flag names by @johnerikhalse in #48
• Add WARC_ERROR_COUNT env variable to close-output-file hook by @johnerikhalse in #49

New Contributors

• @dependabot made their first contribution in #38

Full Changelog: v1.1.0...v2.0.0

v1.1.0

What's Changed

Exciting New Features 🎉

• Bump github.com/nlnwa/gowarc dependency and make use of repair-warcfelds-block feature by @johnerikhalse in #37

Full Changelog: v1.0.1...v1.1.0

v1.0.1

What's Changed

Other Changes

• Create tiny top level test by @trym-b in #34
• Added automatic testing and linting by @johnerikhalse in #35
• Bump github.com/nlnwa/gowarc and github.com/nlnwa/whatwg-url dependencies by @johnerikhalse in #36

New Contributors

• @trym-b made their first contribution in #34

Full Changelog: v1.0.0...v1.0.1

v1.0.0

What's Changed

Breaking Changes 🛠

• Change boolean flags from toggle to default pflag behaviour by @johnerikhalse in #30
• Inheritance for config files by @johnerikhalse in #31
• Avoid naming conflicts for environment variables by @johnerikhalse in #32

Full Changelog: v0.4.1...v1.0.0