build: support --format (pb and json)

[AkihiroSuda]

This PR implements continuity build --format.
Fix #46

e.g.

$ continuity build --format application/vnd.continuity.manifest.v0+json .
$ continuity build --format application/vnd.continuity.manifest.v0+pb .

alias:

$ continuity build --format json .
$ continuity build --format pb .

The JSON format is implemented using jsonpb.Marshaller{EnumAsInts: false, EmitDefaults: false, OrigName: false}. https://godoc.org/github.com/golang/protobuf/jsonpb#Marshaler

JSON format would be useful when a continuity manifest is included in an OCI image.

Signed-off-by: Akihiro Suda [email protected]

[AkihiroSuda]

example output (after jq)

{
  "resource": [
    {
      "path": [
        "/a"
      ],
      "uid": "1001",
      "gid": "1001",
      "mode": 2147484157
    },
    {
      "path": [
        "/a/b"
      ],
      "uid": "1001",
      "gid": "1001",
      "mode": 436,
      "size": "476",
      "digest": [
        "sha256:ef290472c423c80ce142a1617ac556e5935f02b30db1c28c8ed38c15b62e9e3e"
      ]
    }
  ]
}

[dmcgowan]

I think this is a reasonable addition. I am wondering though if we want to keep the existing package interface the same for Marshal and just have a MarshalJSON and same for unmarshal. Seems weird to me to have the marshaling functions take in a media type.

[AkihiroSuda]

@dmcgowan thank you, updated PR

[AkihiroSuda]

@stevvooe PTAL if you have a time? 😃

[stevvooe]

@AkihiroSuda Could we not emit fields when they are empty? "xattr": {} is an example.

[AkihiroSuda]

@stevvooe

It should have been already omitted, but due to a bug in protobuf, it is not actually omitted 😭

I confirmed the following patch for protobuf successfully omits empty xattr map.

diff --git a/jsonpb/jsonpb.go b/jsonpb/jsonpb.go
index 82c6162..9999705 100644
--- a/jsonpb/jsonpb.go
+++ b/jsonpb/jsonpb.go
@@ -188,6 +188,10 @@ func (m *Marshaler) marshalObject(out *errWriter, v proto.Message, indent, typeU
 
                if !m.EmitDefaults {
                        switch value.Kind() {
+                       case reflect.Map:
+                               if len(value.MapKeys()) == 0 {
+                                       continue
+                               }
                        case reflect.Bool:
                                if !value.Bool() {
                                        continue

I'll open PR to protobuf later.
( EDIT: opened golang/protobuf#327 )

[AkihiroSuda]

protobuf maintainers seems unresponsive to golang/protobuf#327, do we need to wait for golang/protobuf#327 ?

[AkihiroSuda]

@stevvooe

protobuf maintainers are still unresponsive to my PR about omitempty (golang/protobuf#327), but I think lack of omitempty is not a critical issue.

Can you please consider merging this PR as-is?

[AkihiroSuda]

rebased, and omitempty issue has been resolved in golang/protobuf@7a211bc

[stevvooe]

@AkihiroSuda Has that change made it to gogo/protobuf?

[AkihiroSuda]

Seems not yet

[AkihiroSuda]

opened PR gogo/protobuf#296

[awalterschulze]

Its merged. Sorry for the delay.

[AkihiroSuda]

thank you

[stevvooe]

@awalterschulze As always, thanks for the great support!

[stevvooe]

@AkihiroSuda Could you update the example output?

[stevvooe]

@AkihiroSuda Also, should probably rename the path field to paths. ;)

[AkihiroSuda]

Also, should probably rename the path field to paths. ;)

let me do that in another PR

[AkihiroSuda]

update the example output.
Is this mergeable? 😃

[stevvooe]

@AkihiroSuda I think so, but we let's not build anything permanent on it yet.