Attributed Metadata allowing overriding of metadata creation via an IMetadataProvider interface on metadata attributes

[kcuzner]

I found in a project I was working on that I wanted to have sequences of groupings of data defined by metadata attributes.

For example, let's say we had an object which always represents a specific group of People. We could have the object manually say that it represents those people using a Property, but then each instance would essentially be providing the exact same data which says to me that it needs to use some sort of Metadata facility. I really like attributed metadata, so I make an attribute called PersonAttribute which has a Name and Age property and is allowed to be applied multiple times to a class. However, when I register this object, an ArgumentException results since two Name values and two Age values were defined and it fails when they are added to the Metadata IDictionary.

What I have done is modified the Autofac.Extras.Attributed library to do the following:

• If Metadata has been supplied with a specific Key via attributes multiple times (i.e. giving multiple values for the same key), the type stored in the Metadata IDictionary will be an T[] where T is the type of the Metadata Value. Values with the same Key but different Types will cause an ArgumentException to be thrown. Order is preserved among groups of attributes (as will be illustrated in my example).
• If Metadata has been supplied with a Key only once (giving just one value), the type stored in the Metadata IDictionary will be T where T is the type of the Metadata Value (just like it always has been).

Example to illustrate how this could be used

[MetadataAttribute]
[AttributeUsage(AttributeTargets.Class, AllowMultiple=true)]
public class PeopleAttribute : Attribute
{
    public PeopleAttribute(string name, int age)
    {
        Name = name;
        Age = age;
    }

    public string Name { get; private set; }

    public int Age { get; private set; }
}

class PeopleMetadata
{
    public string[] Name { get; set; }
    public int[] Age { get; set; }
}

[Person("Alice", 42)]
[Person("Bob", 27)]
class PeopleGroup { }

When PeopleGroup is resolved with its metadata (represented by PeopleMetadata here to illustrate the type that the metadata values are resolved to), it will place the values for Name and Age into the two arrays. So, Age would contain either [42,27] or [27,42] and Name would contain either ["Alice","Bob"] or ["Bob","Alice"]. Now, the problem is keeping the association between which Name goes with which Age. To solve this, the metadata values coming out of the attributes are kept in groups according to which attribute they came from until they are "consolidated" into either KeyValuePair<string, T> or KeyValuePair<string, T[]> which both are stored in the Metadata IDictionary as KeyValuePair<string, object> (as it always has been done). So long as the number of values in each key of a group is the same, when the groups of KeyValuePairs are consolidated into T[] the ordering will be preserved as the various T[]s are constructed. So, if Name[0] == "Bob" then Age[0] == 27. Similarly, if Name[0] == "Alice" then Age[0] == 42.

I use Autofac on a daily basis and I think something like this could be useful for people who use (or perhaps overuse 😛, like myself) Metadata via attributes. I know this new feature likely wouldn't be compatible with other libraries that use MetadataAttributeAttribute stuff to define their metadata since I don't know how they would handle multiple decorations with the same metadata attribute, but I wrote this mainly with Autofac in mind. It doesn't change any existing functionality as far as I can tell and those who need to have compatibility for other libraries can simply not use AllowMultiple-enabled metadata. I hope this will be considered for inclusion and all existing tests pass as well as some additional tests I added for this new functionality.

Suggestions for improvement and comments are appreciated

[tillig]

I'm sorry it's taken a bit to address your request. I've been trying to think about how this would be used "in the wild" and how supportable such a thing might be. Once we take something in, we pretty much commit to having to support it for a really long time so it's important to get it right.

Something I'm concerned about is the limited flexibility of the feature. The idea that it's limited to key/value pairs makes me think about the next logical step - supporting arbitrary objects with more hierarchical data.

I'm curious if you could get the feature to work and allow for arbitrary generation of metadata.

One idea I thought of is using a known interface to override the metadata scanning in attributes. This is just an idea. Might work, might not, but I figured I'd raise it.

For example, you could have an interface like this:

public interface IMetadataProvider
{
  IDictionary<string, object> GetMetadata(Type targetType);
}

You could use that to override the functionality of the MetadataAttributeAttribute implementation so if it implements that particular interface, instead of doing key/value pair generation on the fly, we could use that interface:

[MetadataAttribute]
public class DictionaryMetadataAttribute : Attribute, IMetadataProvider
{
  public IDictionary<string, object> GetMetadata(Type targetType)
  {
    return new Dictionary<string, object>
    {
      { "key", "value" }
    };
  }
}

The MetadataHelper could scan for metadata attributes and if they don't implement that interface, we do the same thing we do now... but if they do implement that interface, we run the interface method instead.

The dictionary it returns would be added directly to the set of metadata in the MetadataHelper. (MetadataHelper uses an IEnumerable<KeyValuePair<string, object>> but that's messy for consumers.)

The targetType passed in could be the type currently being scanned for metadata. Maybe the implementation needs it, maybe it doesn't. But it could use that information to key off other attributes like the PersonAttribute in your example.

public class Person
{
  public string Name { get; set; }
  public int Age { get; set; }
}

public class PeopleMetadataAttribute : Attribute, IMetadataProvider
{
  public IDictionary<string, object> GetMetadata(Type targetType)
  {
    var peopleAttribs = targetType.GetCustomAttributes(typeof(PersonAttribute), true);
    var people = new People[peopleAttribs.Length];
    for(var i = 0; i < peopleAttribs.Length; i++)
    {
      var attrib = peopleAttribs[i];
      people[i] = new Person { Name = attrib.Name, Age = attrib.Age };
    }

    return new Dictionary<string, object>
    {
      { "People", people }
    };
  }
}

You would have a slightly less clean set of attributes, but not by much:

[PeopleMetadata]
[Person("Alice", 42)]
[Person("Bob", 27)]
class PeopleGroup { }

The provider attribute could then scan the type using the other attributes and accomplish a slightly more flexible generation of metadata - you could include more than just name/value pairs; you could have a whole object hierarchy. You could even write hooks so existing attributes that aren't metadata attributes could still generate metadata - things like DataAnnotations attributes and such.

You'll probably notice I didn't put [MetadataAttribute] on that metadata attribute. I figure if we're already adding a custom interface to scan for, there's not a lot of point in including the redundant [MetadataAttribute]; however, it might make the scanning a more unified process, so I'm not against it, either.

Again, this is one idea - one way to be a little more flexible with the feature. There are probably other ways, too, and we're totally open to those.

Would you be interested in trying out a more flexible design?

[kcuzner]

I spent some time tonight trying to make this a little more flexible.

I liked the IMetadataProvider concept you mentioned, so I changed things to include that. A MetadataAttribute can now opt in to provide its own metadata description to autofac by implementing the IMetadataProvider interface. This could be used recursively since I put the test inside the public Autofac.Extras.Attributed.MetadataHelper class, although I haven't yet built a test for that to see what it will do. I did make the IMetadataProvider return an IEnumerable<KeyValuePair<string, object>> because I still wanted the "consolidation" ability I added earlier. However, that type does feel little strange for a public interface as you mentioned, so I really wouldn't mind changing it.

To get the effect that I wanted to achieve, I added a MetadataGroupAttribute which specifies a group key and an attribute type to use for populating the group. It is marked as a MetadataAttribute. The attribute type is the type it will scan the object for to get metadata. This type shouldn't be marked with MetadataAttribute since MetadataAttribute causes it to be scanned by the GetMetadata methods on the "root" metadata level for the object which would lead to double-counting the metadata. Its IMetadataProvider implementation returns an enumerable containing a single KeyValuePair<string, object> with the Key being the group name supplied at construction and the Value being an array constructed from the scanned properties (using MetadataHelper.GetProperties) of the attributes found on the target object that match the attribute type it was set to use.

The next change I made I feel like is a little more controversial and I could see it being a little less acceptable since it involved a change to the core library. As I was doing all the metadata grouping stuff, I found myself having to re-implement the MetadataViewProvider into the Autofac.Extras.Attributed library since it was internal to the Autofac library. To me, this violated the DRY principle and so I tried to find another solution. What I ended up doing was making the Autofac.Features.Metadata.MetadataViewProvider.GetMetadataViewProvider method recursive from inside the GetMetadataValue method. This has the following effect:

• If the metadata for TValue is IDictionary<string, object>, it gets a metadata view provider function for TValue and uses that on the metadata. Because of the checks already inside GetMetadataViewProvider, if TValue is actually an IDictionary<string, object>, it will simply be assigned the metadata value with no change. Otherwise, it goes through the rest of the sequence where it creates a TValue and populates it using the passed metadata IDictionary.
• If the metadata for TValue is IDictionary<string, object>[], it uses a metadata view provider for the element type of TValue repeatedly on each element of the metadata to construct an array which is the same type as TValue. For example, if TValue was MyObject[], it would get the metadata view provider for MyObject and run it for each element of the IDictionary<string, object>[] metadata and cast the resulting sequence into an array (which would have type MyObject[] which is TValue).
• In all other cases, the existing behavior was followed where it does the null check and either sets the default value or tries to cast the metadata to TValue before returning it.

Example

With this change, the metadata can be fully recursive, populating strongly typed objects, or appearing as a recursive IDictionary<string, object> where the object type is actually another IDictionary<string, object>. Previously, it would have required the metadata to be declared as the strong types on any level other than the root level.

So now, metadata which looks like this:

[AttributeUsage(AttributeTargets.Class, AllowMultiple=true)]
public class PersonAttribute : Attribute, IMultipleAttributedMetadata
{
    public PersonAttribute(string name, int age)
    {
        Name = name;
        Age = age;
    }

    public string Name { get; private set; }

    public int Age { get; private set; }
}

[MetadataGroup("People", typeof(PersonAttribute))]
[Person("Alice", 42)]
[Person("Bob", 27)]
class PersonGroup { }

class PeopleMetadata
{
    public Person[] People{ get; set; }

    public class Person
    {
        public string Name { get; set; }
        public int Age { get; set; }
    }
}

Can be accessed either like this:

var metadata = MetadataHelper.GetMetadata(typeof(PersonGroup ));
var nested = metadata.Where(p => p.Key == "People").Select(kv => kv.Value).FirstOrDefault() as IDictionary<string, object>[];
var names = nested.Select(n => n["Name"]);
var ages = nested.Select(n => n["Age"]);

Or like this:

...
var container = builder.Build();
var withMetaObject = container.Resolve<Meta<PersonGroup, PeopleMetadata>>();

var alice = withMetaObject.Metadata.People.Where(a => a.Name == "Alice").SingleOrDefault();
var bob = withMetaObject.Metadata.People.Where(a => a.Name == "Bob").SingleOrDefault();

I feel like this is more flexible than last week's iteration because:

• The metadata is grouped into concrete objects, eliminating the need to track which index goes to which data row. Also, the metadata declarations have no need to know about any strong metadata view type, just like metadata has always been in autofac.
• Metadata could possibly be built into a nested object hierarchy using only attributes. I haven't yet built an example for more than two levels, but its entirely possible now because of the MetadataViewProvider recursively providing metadata views from IDictionary-typed metadata.
• Nested metadata hierarchies no longer need to have specific concrete types at the time of metadata declaration.

Sorry that was a little lengthy. I wanted to try to fully explain what I did so that its a little easier to review the changes I made. I feel my latest changes, while certainly making this much more flexible, are a little more far-reaching since it affects the core metadata library. While all the tests still pass, I was wondering what the general guidelines for writing tests were. Identifying all of the test cases needed has never been my strong suit and I feel like I should have more than just one or two tests to test my change to the MetadataViewProvider.

Thanks for taking the time to look at this. Any further suggestions and comments are greatly appreciated.

[tillig]

I admit I got a little lost while looking at the changes in some cases. There's some complexity here that I'm curious if we could reduce.

Thinking about the grouping feature... I'm wondering if that's something that Autofac should provide as part of the framework, or if maybe that's something you might keep in your code. As a framework, it's good to provide the flexible building blocks to support people to create what they need to, but it's also good to find that balance between providing not enough (lack of flexibility) and providing too much (features that might be used by very few people but are things we have to commit to owning long-term).

You can find some examples of these sorts of things if you browse StackOverflow. There are some great questions about things like how to conditionally apply decorators based on configuration values and that's something we could offer in the framework... but it's a fairly complex feature that not a lot of people would use. Instead, we provide the ability for others to create it, and if we see there are a lot of people posting about how to do XYZ or asking about it in places, then it might be something to consider adding.

Thinking about the changes to Core... It'd be nice if we could keep the changes to the attribute metadata assembly. There's a whole metadata infrastructure in Core that supports metadata and metadata relationships in a certain way. The intent of the extras assembly is to enable interaction with that system via attributes. If we're doing something that requires the change to the core assembly (more than, say, a one-liner sort of thing) it might indicate the integration has gone a little far. In that case, it would be good to step back and figure out what the overall shortcomings of the metadata feature in Core Autofac are so we can figure out whether the changes are warranted.

(Changes in Core are sort of scary because things that seem tiny and still allow tests to pass can have really big effects. See Issue #397 for a great example of me making some seemingly innocuous changes and causing some pretty bad memory leaks.)

Thinking about the recursive scanning of nested classes... I think this is interesting, but I'm curious how often it'd be used. It appears that doing this is part of what causes the complexity, and I'm wondering if it wouldn't be better to potentially let the IMetadataProvider attribute do the scanning internally rather than doing it in the framework. If it needs to build up some sort of data about nested classes and things, the key building block is that we can provide that interface to let you do it.

[kcuzner]

I see your point. As I was making the changes to create the MetadataGroupingAttribute the thought did cross my mind that simply having the IMetadataProvider interface mechanism was enough and that everything else could be done outside the library. I guess I got a little caught up in messing with the metadata view provider 😛. To compound the problem, I know my use of Activator.CreateInstance in two places isn't the most performant compared to the compiled expressions that autofac normally uses and would probably slow down metadata generation slightly for larger models.

If I were to just include the IMetadataProvider & related changes, ditching the ConsidateKeys, MetadataGroupAttribute, and MetadataViewProvider changes, would that be more acceptable? That particular change was quite small comparatively and still offers the flexibility I wanted with attributed metadata when originally creating this last week. It would offer a "key building block" as you mentioned for this sort of behavior IMO.

If that is a better option and that seems like a more viable (and slimmer) candidate for inclusion, I can re-do the changes to just include that and force push to this pull request, or I could create a new pull request.

[tillig]

I think that'd probably be good. Small, simple, not too complex, and enough that you can get done what you want to get done while also providing something fairly flexible for other folks.

Thanks again for your work on this. Great stuff!

[kcuzner]

Ok, I have packaged up and squashed just the IMetadataProvider and the changes to the MetadataHelper along with some tests. I removed all of the extra MetadataGroupingAttribute and MetadataViewProvider changes along with the metadata "consolidate by key into array" stuff. Is there anything else I should add/remove/revise?

[tillig]

This is looking nice and clean. Sweet!

Two minor things and I'm ready to click the Merge button.

• Can the GetMetadata method return an IDictionary<string, object> instead of the IEnumerable<KeyValuePair<string, object>>? It appears IDictionary<K,V> implements IEnumerable<KeyValuePair<K,V>> so you shouldn't have to change anything but the return type and that would make the usage a little nicer for folks extending.
• Can you make the test fixtures and scenario classes public? Unless there's a reason they shouldn't be, the other fixtures and test scenarios are public so that'd be consistent.

Again, thanks for the great work on this. Very cool.

[kcuzner]

Alright, I believe that should do it. I changed the return type of GetMetadata and changed the tests to be public. Thank you very much for considering including this 😄

[tillig]

Looks great. Thanks again! I'll update the version of the assembly/package and see if I can get a new version pushed shortly.

[tillig]

This will go out in version 3.2.0 of Autofac.Extras.Attributed. It's running the build now.

[tillig]

3.2.0 just went live.