proposal: spec: support nested types or name spaces

[ghost]

What version of Go are you using (go version)?

1.8

What operating system and processor architecture are you using (go env)?

darwin/amd64

What did you do?

In my project, we generate C++/python APIs from certain schema that model network configuration and operational data. We guarantee that each node in the data model has a corresponding type in the C++/python APIs. For example, APIs for python can be seen here.

So, if the data model looks like the below,

container employee {
  leaf employee-id {
    type string;
  }

  container employment-data { // data model is nested
    leaf date-of-joining {
      type string;
    }
  }
}

the corresponding python classes look like:

class Employee(object):
  def __init__(self):
    self.employee_id = None
    self.employment_data = EmploymentData()

  class EmploymentData(object): #type is nested
    def __init__(self):
      self.date_of_joining = None

Some of the data model schema happen to be deeply nested, which can be nicely represented as nested classes (aka inner classes) in C++/python. When it comes to generating code for Go, however, there is no way to have named nested types. The only approach is to have named types which are defined in a flat way and then the fields themselves can be nested. So, we are being forced to have long struct names due to concatenating the names of the "inner" types to guarantee unique names for all the types in a package.

type Employee struct {
    employee_id string
    employment_data EmployeeEmploymentData // data is nested
}

type EmployeeEmploymentData struct { // type is flat (not nested)
      date_of_joining string
}

The alternative would be to use nested anonymous structs, but it is a bit cumbersome for users to initialize such structs.

Can Go allow named nested structs like the below?

type Employee struct {
    EmployeeId string
    EmploymentData type EmploymentData struct { // type is named and nested
      DateOfJoining string
  }
}

Note that even C lets you have named inner types (although they are not of much practical use):

struct Employee {
    char* employee_id;
    struct EmploymentData { // type is named and nested
      char* date_of_joining;
  } employment_data'
}

However, in Go, it could perhaps let users initialize their objects like the below (EmploymentData only has scope as a nested type of Employee)

bob := Employee{EmployeeId:"1234", EmploymentData{DateOfJoining: "12-12-2010"}}

As a consequence, the below could also be valid for declaring orphan objects of the above nested type

data := Employee.EmploymentData{DateOfJoining: "12-12-2010"}

[ianlancetaylor]

This is a significant language change, so marking as Go2.

Personally I think this change is unlikely to be adopted. Note that although C permits a struct to be defined within another struct, the struct tags are all at top level. There is no nesting of names, which seems to be your concern. As far as I can see the only effect of nesting of names in Go would be to permit writing X.Y instead of XY. That seems like a minor benefit. And your example is about code generation; it is trivial for a code generator to generate long names, so even the benefit seems even less important in that few people will ever type that long name.

[ghost]

Thanks for your reply, @ianlancetaylor. I agree that it is trivial for code generators to generate the long names. But in our case, these generated APIs are ultimately consumed by developers. It would make it a bit cumbersome for such users to use the APIs with long names.

[ianlancetaylor]

@abhikeshav But just to clarify, we are talking about the difference between writing Employee.EmploymentData and writing EmployeeEmploymentData, right?

[ghost]

@ianlancetaylor Exactly.

To give a different example, if there are slices inside a struct like the below,

type Employee struct {
    employee_id string
    hello []EmployeeEmploymentDataInsideADeeplyNestedStructForExampleHello
    bye []EmployeeEmploymentDataInsideADeeplyNestedStructForExampleBye
}

type EmployeeEmploymentDataInsideADeeplyNestedStructForExampleHello struct {
      date_of_joining string
}

type EmployeeEmploymentDataInsideADeeplyNestedStructForExampleBye struct {
      date_of_joining string
}

users of this Go API will have to instantiate the inner objects and append them like below:

bob := Employee{employee_id:"1234"}

data1 := EmployeeEmploymentDataInsideADeeplyNestedStructForExampleHello{"12-12-2010"} // this is cumbersome since the name is super-long
data2 := EmployeeEmploymentDataInsideADeeplyNestedStructForExampleBye{"13-10-2012"}

bob.hello = append(bob.hello, data1)
bob.bye = append(bob.bye, data2)

https://play.golang.org/p/CVDAEaNOR2

The above could be made easier to read if '.' notation was allowed for the types, meaning the types are nested.

bob := Employee{employee_id:"1234"}

// nested types makes it a lot clearer than having a super-long name
data1 := Employee.EmploymentData.Inside.ADeeplyNestedStruct.ForExample.Hello{"12-12-2010"} 
data2 := Employee.EmploymentData.Inside.ADeeplyNestedStruct.ForExample.Bye{"13-10-2012"} 

bob.hello = append(bob.hello, data1)
bob.bye = append(bob.bye, data2)

[ibrasho]

Won't this have the same effect:

bob := Employee{employee_id:"1234"}

data1 := Employee_EmploymentData_Inside_ADeeplyNestedStruct_ForExample_Hello{"12-12-2010"} 
data2 := Employee_EmploymentData_Inside_ADeeplyNestedStruct_ForExample_Bye{"13-10-2012"} 

bob.hello = append(bob.hello, data1)
bob.bye = append(bob.bye, data2)

[ghost]

It is not the same thing. From a programmatic point of view, having nested types is a better representation of deeply nested yang data models.

As a side note, using underscores actually goes against the golang guidelines :-|

[dsnet]

(not that I'm supporting this). As a data point, protocol buffers allow for embedded messages, which get compiled by protoc-gen-go as a struct named GrandParentName_ParentName_ChildName.

[111pontes]

@ianlancetaylor, it's not just the difference between writing Employee.EmploymentData and writing EmployeeEmploymentData. The flat approach results in ambiguous output. Employee.Employment.Data, EmployeeEmployment.Data, Employee.EmploymentData and EmployeeEmploymentData would all result in the latter. We can't prevent a hierarchy form having two or more of those patterns. Clearly segmenting nodes in a hierarchy is important.

Furthermore, hierarchies can be 10+ or even 20+ deep. Here's a real life example:

NetworkInstances.NetworkInstance.Protocols.Protocol.Ospfv2.Areas.Area.Lsdb.LsaTypes.LsaType.Lsas.Lsa.OpaqueLsa.ExtendedPrefix.Tlvs.Tlv.SidLabelBinding.Tlvs.Tlv.EroPath.Segments.Segment.UnnumberedHop.State.InterfaceId

versus:

NetworkInstancesNetworkInstanceProtocolsProtocolOspfv2AreasAreaLsdbLsaTypesLsaTypeLsasLsaOpaqueLsaExtendedPrefixTlvsTlvSidLabelBindingTlvsTlvEroPathSegmentsSegmentUnnumberedHopStateInterfaceId

Think of trying to navigate a deep file system with flat paths instead of slash-separated directories. You'd have issues with ambiguity, plus you'd force all paths to be explicitly referenced from the root. You'd want to have the flexibility of relative paths.

It seems that protoc-gen-go could also benefit from this enhancement proposal.

[dsnet]

@111pontes, why would a file system be represented as having a nested type for each directory? How would that work? My understanding is that the structure of a filesystem is typically discovered at runtime, but in order to have a type structure like you're suggesting, it would have to be known at compile time. This seems like a odd example to me.

[111pontes]

@dsnet, I just used a file system as an analogy of a deeply nested hierarchy everybody is familiar with.

[ianlancetaylor]

Go in general decomposes concepts rather than nesting them. For instance, methods are not defined inside a type. That was in fact a key point in permitting any type to have a method; nested types as described here would only be permitted within a struct type, which is not very orthogonal.

It seems fine to use an underscore where this proposal would use a dot.

Closing.