Add InMemoryFileIO as a test helper class #6538
Conversation
InMemoryInputFile as a test helper class stitching the two together and maintaining an in-memory listing of files. Add a dedicated unittest for the new test helper.
|
As suggested in #6428 (comment) this PR breaks out the addition of the InMemoryFileIO as a separate addition that can be generally useful in iceberg-core for unittesting purposes. |
nastra
left a comment
nastra
left a comment
There was a problem hiding this comment.
overall LGTM, just a few small suggestions
|
|
||
| public class InMemoryFileIO implements FileIO { | ||
|
|
||
| private Map<String, byte[]> inMemoryFiles = Maps.newHashMap(); |
There was a problem hiding this comment.
nit: maybe worth using Maps.newConcurrentMap() here?
| private Map<String, byte[]> inMemoryFiles = Maps.newHashMap(); | ||
| private boolean closed = false; | ||
|
|
||
| public void addFile(String path, byte[] contents) { |
There was a problem hiding this comment.
nit: in other FileIO implementations we use location rather than path
| public void deleteFile(String path) { | ||
| Preconditions.checkState(!closed, "Cannot call deleteFile after calling close()"); | ||
| if (!inMemoryFiles.containsKey(path)) { | ||
| throw new NotFoundException("No in-memory file found for path: %s", path); |
There was a problem hiding this comment.
nit: in other FileIO implementations we use location rather than path
| @Override | ||
| public void deleteFile(String path) { | ||
| Preconditions.checkState(!closed, "Cannot call deleteFile after calling close()"); | ||
| if (!inMemoryFiles.containsKey(path)) { |
There was a problem hiding this comment.
nit: what about using the result of the remove call here? if (!inMemoryFiles.remove(path)) { ... }
There was a problem hiding this comment.
Done, also updated the get in newInputFile to follow the same pattern.
|
Thanks! Suggestions applied. |
|
Thanks @dennishuo !! |
* Add InMemoryFileIO alongside existing InMemoryOutputFile and InMemoryInputFile as a test helper class stitching the two together and maintaining an in-memory listing of files. Add a dedicated unittest for the new test helper. * Update variable naming for consistency, refactor to avoid using containsKey * Use Maps.newConcurrentMap instead of Maps.newHashMap
4 participants
Add InMemoryFileIO alongside existing InMemoryOutputFile and InMemoryInputFile as a test helper class stitching the two together and maintaining an in-memory listing of files. Add a dedicated unittest for the new test helper.