You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Events stream in source-amplitude is currently preventing the source from being fully migrated to manifest-only because it requires custom record parsing with the zipfile library. We should create a low-code component that can unzip a response and prepare it to be parsed by an underlying connector-dev-specified parser (e.g. JsonParser).
This will require creating a "decoder" class that unzips the requests.Response object and then leverages a specified parsers to parser the decompressed data and yields the data in the same format that the other decoders return data.
Proposed Solution
Create a ZipfileDecoder class
Class should inherit from the Decoder class and have is_stream_response and decoder
Class should have an underlying parser which defaults to a JsonParser, which is similar to the JsonDecoder but is agnostic to the requests library -> should accept a ByteArray (or sim.) and parse accordingly and return type Generator[MutableMapping[str, Any], None, None] per Decoder interface.
The text was updated successfully, but these errors were encountered:
pnilan
changed the title
feat(zip decoder) - Create new zip decoder that takes an underlying parser to decoder and parse a response
feat(ZipfileDecoder) - Create new zip decoder that takes an underlying parser to decoder and parse a response
Dec 10, 2024
Problem
The
Events
stream insource-amplitude
is currently preventing the source from being fully migrated to manifest-only because it requires custom record parsing with the zipfile library. We should create a low-code component that can unzip a response and prepare it to be parsed by an underlying connector-dev-specified parser (e.g. JsonParser).This will require creating a "decoder" class that unzips the
requests.Response
object and then leverages a specified parsers to parser the decompressed data and yields the data in the same format that the other decoders return data.Proposed Solution
ZipfileDecoder
classDecoder
class and haveis_stream_response
anddecoder
parser
which defaults to aJsonParser
, which is similar to theJsonDecoder
but is agnostic to therequests
library -> should accept a ByteArray (or sim.) and parse accordingly and return typeGenerator[MutableMapping[str, Any], None, None]
perDecoder
interface.The text was updated successfully, but these errors were encountered: