Message meta-data

[alexec]

Currently messages are just byte-arrays. This the lowest common-denominator, but means that dataflow can't infer useful information about messages.

Use case for meta-data include:

• Tracing messages through the system to diagnose problem
• Message de-duplication
• Grouping by time
• Verifying messages

What sources already provide:

• HTTP:
  • Etag
  • Content-length
• Kafka:
  • Timestamp
  • Topic
  • Partition
  • Offset
• S3:
  • Etag
  • Last-modified time
  • Size
  • Owner
• STAN:
  • Sequence
  • Timestamp
  • Redelivery
  • Subject

Cloud-events provides a formal specification:

• ID (UUID)
• Source (URI)
• Type
• Data-content-type
• Dataschema
• Subject
• Time

{
    "specversion" : "1.0",
    "type" : "com.github.pull_request.opened",
    "source" : "https://github.com/cloudevents/spec/pull",
    "subject" : "123",
    "id" : "A234-1234-1234",
    "time" : "2018-04-05T17:31:00Z",
    "comexampleextension1" : "value",
    "comexampleothervalue" : 5,
    "datacontenttype" : "text/xml",
    "data" : "<much wow=\"xml\"/>"
}