Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Since it was impossible to load a Package stored on s3 this adds an adapter to the AWS Plugin that previously only had a loader.
It would be useful to merge #1632 first, as I was not able to run the tests before applying those changes.
Note that this allows loading a Package, but it is still not possible to load Resources contained in a datapackage file that are stored on s3. i.e. the following test would still fail:
The first issue for that is in the
remote
loader, that greedily captures anything thathelpers.is_remote_path
recognises as remote, including s3 URLs. This can easily be patched, but then I found out that for some reason those resources get assigned a scheme offile
rather than a scheme ofs3
, so they are not loaded by the AWS loader; I was not able to find why this happens, it seems to be some dynamically generated code.@roll you have marked that issue as "good first issue" but I ended up getting quite confused by the Plugin system when trying to solve it. Do you have any pointers that could be useful in finding what is happening to these Resource descriptors?