Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collection<T> and ObservableCollection<T> do not support ranges #18087

Open
robertmclaws opened this issue Aug 13, 2016 · 409 comments · Fixed by dotnet/corefx#35772
Open

Collection<T> and ObservableCollection<T> do not support ranges #18087

robertmclaws opened this issue Aug 13, 2016 · 409 comments · Fixed by dotnet/corefx#35772
Labels
api-needs-work API needs work before it is approved, it is NOT ready for implementation area-System.Collections in-pr There is an active PR which will close this issue when it is merged
Milestone

Comments

@robertmclaws
Copy link

Update 10/04/2018

@ianhays and I discussed this and we agree to add this 6 APIs for now:

    // Adds a range to the end of the collection.
    // Raises CollectionChanged (NotifyCollectionChangedAction.Add)
    public void AddRange(IEnumerable<T> collection) => InsertItemsRange(0, collection);

    // Inserts a range
    // Raises CollectionChanged (NotifyCollectionChangedAction.Add)
    public void InsertRange(int index, IEnumerable<T> collection) => InsertItemsRange(index, collection);

    // Removes a range.
    // Raises CollectionChanged (NotifyCollectionChangedAction.Remove)
    public void RemoveRange(int index, int count) => RemoveItemsRange(index, count);

    // Will allow to replace a range with fewer, equal, or more items.
    // Raises CollectionChanged (NotifyCollectionChangedAction.Replace)
    public void ReplaceRange(int index, int count, IEnumerable<T> collection)
    {
         RemoveItemsRange(index, count);
         InsertItemsRange(index, collection);
    }

    #region virtual methods
    protected virtual void InsertItemsRange(int index, IEnumerable<T> collection);
    protected virtual void RemoveItemsRange(int index, int count);
    #endregion

As those are the most commonly used across collection types and the Predicate ones can be achieved through Linq and seem like edge cases.

To answer @terrajobst questions:

Should the methods be virtual? If no, why not? If yes, how does eventing work and how do derived types work?

Yes, we would like to introduce 2 protected virtual methods to stick with the current pattern that we follow with other Insert/Remove apis to give people hability to add their custom removals (like filtering items on a certain condition).

Should some of these methods be pushed down to Collection?

Yes, and then ObservableCollection could just call the base implementation and then trigger the necessary events.

Let's keep the final speclet at the top for easier search

Speclet (Updated 9/23/2016)

Scope

Modernize Collection<T> and ObservableCollection<T> by allowing them to handle operations against multiple items simultaneously.

Rationale

The ObservableCollection is a critical collection when it comes to XAML-based development, though it can also be useful when building API client libraries as well. Because it implements INotifyPropertyChanged and INotifyCollectionChanged, nearly every XAML app in existence uses some form of this collection to bind a set of objects against UI.

However, this class has some shortcomings. Namely, it cannot currently handle adding or removing multiple objects in a single call. Because of that, it also cannot manipulate the collection in such a way that the PropertyChanged events are raised at the very end of the operation.

Consider the following situation:

  • You have a XAML app that accesses an API.
  • That API call returns 25 objects that need to be bound to the UI.
  • In order to get the data displayed into the UI, you likely have to cycle through the results, and add them one at a time to the ObservableCollection.
  • This has the side-effect of firing the CollectionChanged event 25 times. If you are also using that event to do other processing on incoming items, then those events are firing 25 times too. This can get very expensive, very quickly.
  • Additionally, that event will have ChangedItems Lists that will only ever have 0 or 1 objects in them. That is... not ideal.

This behavior is unnecessary, especially considering that NotifyCollectionChangedEventArgs already has the components necessary to handle firing the event once for multiple items, but that capability is presently not being used at all.

Implementing this properly would allow for better performance in these types of apps, and would negate the need for the plethora of replacements out there (here, here, and here, for example).

Usage

Given the above scenario as an example, usage would look like this pseudocode:

    var observable = new ObservableCollection<SomeObject>();
    var client = new HttpClient();
    var result = client.GetStringAsync("http://someapi.com/someobject");
    var results = JsonConvert.DeserializeObject<SomeObject>(result);
    observable.AddRange(results);

Implementation

This is not the complete implementation, because other *Range functionality would need to be implemented as well. You can see the start of this work in PR dotnet/corefx#10751

    // Adds a range to the end of the collection.
    // Raises CollectionChanged (NotifyCollectionChangedAction.Add)
    public void AddRange(IEnumerable<T> collection)

    // Inserts a range
    // Raises CollectionChanged (NotifyCollectionChangedAction.Add)
    public void InsertRange(int index, IEnumerable<T> collection);

    // Removes a range.
    // Raises CollectionChanged (NotifyCollectionChangedAction.Remove)
    public void RemoveRange(int index, int count);

    // Will allow to replace a range with fewer, equal, or more items.
    // Raises CollectionChanged (NotifyCollectionChangedAction.Replace)
    public void ReplaceRange(int index, int count, IEnumerable<T> collection);

    // Removes any item that matches the search criteria.
    // Raises CollectionChanged (NotifyCollectionChangedAction.Remove)
    // RWM: Excluded for now, will see if possible to add back in after implementation and testing.
    // public int RemoveAll(Predicate<T> match);

Obstacles

Doing this properly, and having the methods intuitively named, could potentially have the side effect of breaking existing classes that inherit from ObservableCollection to solve this problem. A good way to test this would be to make the change, compile something like Template10 against this new assembly, and see if it breaks.


So the ObservableCollection is one of the cornerstones of software development, not just in Windows, but on the web. One issue that comes up constantly is that, while the OnCollectionChanged event has a structure and constructors that support signaling the change for multiple items being added, the ObservableCollection does not have a method to support this.

If you look at the web as an example, Knockout has a way to be able to add multiple items to the collection, but not signal the change until the very end. The ObservableCollection needs the same functionality, but does not have it.

If you look at other extension methods to solve this problem, like the one in Template10, they let you add multiple items, but do not solve the signaling problem. That's because the ObservableCollection.InsertItem() method overrides Collection.InsertItem(), and all of the other methods are private. So the only way to fix this properly is in the ObservableCollection itself.

I'm proposing an "AddRange" function that accepts an existing collection as input, optionally clears the collection before adding, and then throws the OnCollectionChanging event AFTER all the objects have been added. I have already implemented this in a PR dotnet/corefx#10751 so you can see what the implementation would look like.

I look forward to your feedback. Thanks!

@robertmclaws
Copy link
Author

@joshfree @Priya91 Since I already have a PR that addresses this issue, is there any way this could be moved up to 1.1?

@LanceMcCarthy
Copy link

While you're in there adding an AddRange() method, can you throw an OnPropertyChanged() into the Count property's setter? Thanks :)

@thomaslevesque
Copy link
Member

thomaslevesque commented Sep 13, 2016

A long time ago I had implemented a RangeObservableCollection with AddRange, RemoveRange, InsertRange, ReplaceRange and RemoveAll. But it turned out that the WPF binding system didn't support CollectionChanged notifications with multiple items (I seem to remember it has been fixed since then, but I'm not sure).

@joshfree
Copy link
Member

@Priya91 can you help shepherd this through the API review process http://aka.ms/apireview with @robertmclaws ?

/cc @terrajobst

@Priya91
Copy link
Contributor

Priya91 commented Sep 13, 2016

@Priya91 can you help shepherd this through the API review process http://aka.ms/apireview with

Sure.

@Priya91
Copy link
Contributor

Priya91 commented Sep 13, 2016

@robertmclaws Can you create an api speclet on this issue, outling the api syntax, like this. Mainly interested in usage scenarios

@svick
Copy link
Contributor

svick commented Sep 14, 2016

@robertmclaws

Doing this properly, and having the methods intuitively named, could potentially have the side effect of breaking existing classes that inherit from ObservableCollection to solve this problem.

In what situation could it be a breaking change? The only issue I can think of is that it would cause a warning that tells you to use new if you meant to hide a base class member, which would be actually an error with warnings as errors enabled. Is this what you meant? Or is there another case I'm missing?

@robertmclaws
Copy link
Author

@svick Could possibly be a runtime problem. If you just upgraded the framework w/o recompiling, I'm not sure exactly how the runtime execution would react. We'd need to test it just to make sure.

@svick
Copy link
Contributor

svick commented Sep 14, 2016

@robertmclaws I think that could only be a problem if you don't recompile, but you do upgrade a library with the custom type inheriting from ObservableCollection<T>, which removed its version of AddRange() in the new version. But that would be the fault of that library.

Otherwise, adding a new member won't affect how old binaries behave.

@Priya91
Copy link
Contributor

Priya91 commented Sep 14, 2016

+1 The api sounds good to me. For manipulating multiple items , along with AddRange, does it provide value to add, InsertRange, RemoveRange, GetRange for the specified usage scenarios?

cc @terrajobst

@robertmclaws
Copy link
Author

@svick You are probably right. I personally would want to test the behavior just to be sure we're not breaking anyone... otherwise this would move to a 2.0 release item.

@Priya91 I'm not sure if a GetRange() would be necessary, but InsertRange() and RemoveRange() would be, along with ReplaceRange(), and possible a Clear() method if one is not currently available.

So if we're comfortable with the API, what's the next step? :)

@Priya91
Copy link
Contributor

Priya91 commented Sep 16, 2016

Clear is already available. We still haven't gotten the shape of apis to add, if RemoveRange and InsertRange are to be added, then we need these apis added to the speclet. And then we'll mark api-ready-for-review, to be discussed in the next api-review meeting either on tuesday or friday.

@robertmclaws
Copy link
Author

OK, I made changes to the speclet. Note that the parameters might change for the actual implementation, but those are what makes the most sense at this particular second. Please LMK if I need to do anything else. Thanks!

@Priya91
Copy link
Contributor

Priya91 commented Sep 16, 2016

RemoveRange(int index, int count) instead of RemoveRange(ICollection) ? How does RemoveRange behave when the ICollection elements are duplicated in ObservableCollection

@Priya91
Copy link
Contributor

Priya91 commented Sep 16, 2016

count instead of endIndex..

public void ReplaceRange(IEnumerable<T> collection, int startIndex, int count)

@Priya91
Copy link
Contributor

Priya91 commented Sep 16, 2016

public void AddRange(IEnumerable<T> collection, bool clearFirst = false) { }
public void InsertRange(IEnumerable<T> collection, int startIndex) { }
public void RemoveRange(int startIndex, int count) { }
public void ReplaceRange(IEnumerable<T> collection, int startIndex, int count) { }

@thomaslevesque
Copy link
Member

Basically the signatures should be the same as in List<T>.

I don't think the clearFirst parameter in AddRange is useful, and anyway optional parameters should be avoided in public APIs.

A RemoveAll method would be useful all well, for consistency with List<T>:

public int RemoveAll(Predicate<T> match)

@robertmclaws
Copy link
Author

I think RemoveRange(IEnumerable<T> collection) should remain. It would cycle through collection, call IndexOf(item) and then call RemoveAt(index). Duplicates of the same item would also be removed.

@thomaslevesque I have the clearFirst parameter in there specifically because it IS useful, as in I'm using it in production code right now. Consider in UWP apps when you are resetting a UI... if you call Clear() first, it will fire another CollectionChanged event, which is not always desirable.

I'm not against a RemoveAll function.

@thomaslevesque
Copy link
Member

Also, the index parameter usually comes first in existing APIs, so InsertRange, RemoveRange and ReplaceRange should be updated accordingly.

And I don't think ReplaceRange needs a count parameter; what should the method do if the count parameter doesn't much the number of items in the replacement collection?

Here's the API as I see it:

public void AddRange(IEnumerable<T> collection) { }
public void InsertRange(int index, IEnumerable<T> collection) { }
public void RemoveRange(int index, int count) { }
public void ReplaceRange(int index, IEnumerable<T> collection) { }
public int RemoveAll(Predicate<T> match)

@thomaslevesque
Copy link
Member

@thomaslevesque I have the clearFirst parameter in there specifically because it IS useful, as in I'm using it in production code right now. Consider in UWP apps when you are resetting a UI... if you call Clear() first, it will fire another CollectionChanged event, which is not always desirable.

I'm not sold on it, but hey, it's your proposal, not mine 😉. At the very least, I think it should a separate overload, rather than an optional parameter.

@thomaslevesque
Copy link
Member

@thomaslevesque I have the clearFirst parameter in there specifically because it IS useful, as in I'm using it in production code right now. Consider in UWP apps when you are resetting a UI... if you call Clear() first, it will fire another CollectionChanged event, which is not always desirable.

This makes me think... there are lots of possible combination of changes you might want to do on the collection without triggering events for each one. So instead of trying to think of each case and introduce a new method for each, perhaps we should lean toward a more generic solution. Something like this:

using (collection.DeferCollectionChangedNotifications())
{
    collection.Add(...);  // no event raised
    collection.Add(...); // no event raised
    // ...
} // event raised here for all changes

@robertmclaws
Copy link
Author

robertmclaws commented Sep 16, 2016

@thomaslevesque Overload vs optional parameter makes no practical difference to the end user. It's just splitting hairs. Having overloads just adds unnecessary lines of code.

ReplaceRange with a count would remove all items in the given range, and then insert the new items at that point. The counts not matching would be irrelevant.

If the index comes first in existing APIs, then I'm fine with this:

public void AddRange(IEnumerable<T> collection, clearFirst bool = false) { }
public void InsertRange(int index, IEnumerable<T> collection) { }
public void RemoveRange(int index, int count) { }
public void ReplaceRange(int index, int count, IEnumerable<T> collection) { }
public int RemoveAll(Predicate<T> match)

@thomaslevesque
Copy link
Member

@thomaslevesque Overload vs optional parameter makes no practical difference to the end user. It's just splitting hairs.

It's not. Optional parameter can cause very real issues when used in public APIs. Read this blog post by @haacked for details.

@shmuelie
Copy link
Contributor

I'm actually liking @thomaslevesque's idea about using a batching class. It's a common pattern, well understood, and makes complex workflows easier.

@thomaslevesque
Copy link
Member

ReplaceRange with a count would remove all items in the given range, and then insert the new items at that point. The counts not matching would be irrelevant.

That would be quite inefficient. Removing items would cause all following items to be moved backwards, and inserting new ones would cause them to be moved forward again. The implementation I have in mind would replace each item in-place, without moving anything.

@robertmclaws
Copy link
Author

So instead of trying to think of each case and introduce a new method for each, perhaps we should lean toward a more generic solution.

The point of this proposal was to fill in the gaps on the existing implementation, not coming up with a new pattern for people to deal with. I'm not against that proposal, but that's an entirely new piece of functionality that I don't believe should be a part of this discussion.

@shmuelie
Copy link
Contributor

@robertmclaws but since there is no way to currently do bulk operations there isn't a "new pattern"

@robertmclaws
Copy link
Author

That would be quite inefficient. Removing items would cause all following items to be moved backwards, and inserting new ones would cause them to be moved forward again. The implementation I have in mind would replace each item in-place, without moving anything.

Why does that matter? Is it a memory allocation issue?

@thomaslevesque
Copy link
Member

that's an entirely new piece of functionality

I agree that it should probably be a separate proposal, but it does solve the initial problem you were having.

@airbreather
Copy link

@robertmclaws

The fix is simple: delete the validation code and anything referencing it, then see what breaks and fix it. [...]

This was attempted in the PR that triggered the comment I linked in my previous reply on this thread, dotnet/wpf#6097.

[...] one on the WPF team with a solution-oriented mindset [...]

The first comment on that PR, dotnet/wpf#6097 (review), included a bulleted list of five items that need to change in order for it to be OK to remove the validation. Furthermore, as the discussion continued on that thread, someone else chimed in, dotnet/wpf#6097 (comment), pointing to a specific line of code. The link is stale now... here's a permalink: https://github.com/dotnet/wpf/blob/ae1790531c3b993b56eba8b1f0dd395a3ed7de75/src/Microsoft.DotNet.Wpf/src/PresentationFramework/System/Windows/Data/ListCollectionView.cs#L1748

Personally, I'm also a little perplexed as to why this is still considered to be a big enough blocker when the Reset workaround feels like a pretty good 80% solution, but:

  • I'm not willing to delve into so much of the internals of WPF in order to bring this to a conclusion that's satisfying to everybody (especially since I've mostly stopped using Windows outside of my full-time job), and
  • Maybe you will find this to be overly optimistic (I get that from people a lot), but I imagine that if a community member happened to step up with a PR in dotnet/wpf that addresses all of the issues that have been raised over the "just remove the validation" solution, then it might get reviewed in time for .NET 10.

@eiriktsarpalis
Copy link
Member

My personal opinion is that someone involved with this boneheaded decision is still on the WPF team and have dug their heels in because they don't want their "record" blemished with an expensive fix. Once this change broke their code, they threw up their hands and have been a never-ending roadblock.

This is not true and completely misrepresents how the .NET team(s) work (and the type of challenges they are facing). I don't think anybody ever expressed the opinion that this shouldn't be fixed, in fact consensus is very strongly in favor of this being addressed eventually. The simple reality is that resources are limited and priorities can be conflicting so as with any effort requiring substantial coordination the windows of opportunity are tiny and easy to miss.

@omariom
Copy link
Contributor

omariom commented Jul 4, 2024

WPF is open source.
Just send PRs fixing it.

@jl0pd
Copy link
Contributor

jl0pd commented Jul 4, 2024

Another solution to problem of ObservableCollection<T> is to replace it with new better type.

Current problems:

  1. It's widely used and any change to it is problematic.
  2. It doesn't support ranges.
  3. It doesn't send removed items on call to Clear().
  4. INotifyCollectionChanged isn't a generic type, which leads to boxing on value types and requires downcasting on everything.
  5. subjective. collection lives in System.Collections.ObjectModel and event-args lives in System.Collections.Specialized which are strange choice of namespaces.
  6. subjective. it's called ObserableCollection, but it doesn't relate to System.IObservable<T>

I propose to create MonitoringList<T> (use same monitoring prefix as MonitoringStream in Nerdbank.Streams which lives under dotnet organization). This collection will implement generic interface, support range updates, yield removed items on Clear().

Downside is that UI frameworks will need to adopt new type.

@jnm2
Copy link
Contributor

jnm2 commented Jul 4, 2024

but once the instance method is added, then that binding takes priority when the application is recompiled and triggers the "no bulk operations" exception in ListCollectionView where it wouldn't have done so before.

Since the error only strikes on recompile, why not have WPF ship an analyzer with a build error if any AddRange that now binds to the instance method could also find an AddRange extension method in scope that it would have previously been calling? If deleting or dereferencing the extension method is not an option, you can silence the error once for the project/solution.

Or, taking a page from the C# language's breaking changes strategy, start providing a warning on the extension methods today and leave it in place for at least one .NET cycle. The warning will tell you to future-proof your app to be ready for .NET 10/11, maybe with a light bulb fix having you rename the extension method so that you aren't broken on upgrade.

@koszeggy
Copy link

koszeggy commented Jul 5, 2024

WPF is open source. Just send PRs fixing it.

I wish it was that simple... But in the WPF repo approving even the smallest fixes with no known risk may take forever. In contrast, WinForms guys usually react super fast, or even encourage me to send a PR so it will go through quicker.

@ShawnTheBeachy
Copy link
Contributor

Ever since we got it over the finish line just to be rejected a few hours later, subsequent release cycles have just been met with the same excuse on an infinite loop that is now nearing an OutOfMemoryException.

"The change has to be made in coordination with downstream frameworks, which takes a long time."

It actually won't take that long at all. BUT as we've clearly seen, it takes a lot more time if nobody actually does it. 🤷🏻‍♂️

This is what I was trying to get at. As someone who's just been keeping an eye on this issue loosely, it seems to me like what keeps happening is that Microsoft says, "It's going to take time to coordinate", but nobody has actually even begun the process.

@eiriktsarpalis
Copy link
Member

but nobody has actually even begun the process.

Again, not accurate. The teams do make legitimate attempts to prioritize this issue every year (many of which are reflected in this discussion) however please consider that every engineer is accountable for literally hundreds of issues like this one and the fact that there sometimes exist conflicting business priorities.

@ShawnTheBeachy
Copy link
Contributor

@eiriktsarpalis I understand that, what I'm advocating for is clearer messaging. When you say, "The change has to be made in coordination with downstream frameworks, which takes a long time" IMHO that doesn't tell most of us much about what the actual state of the issue is. Is it in progress? Is communication and coordination happening? Or are you saying that nobody has been able to commit to it yet, and it hasn't moved at all? It can lead to a feeling that nothing is happening on the issue.

@robertmclaws
Copy link
Author

Whelp, Implicit Extension Types just got moved to .NET 10 (scroll to the bottom), so I guess I'm not solving this problem that way this year either.

@dotnet-policy-service dotnet-policy-service bot added the in-pr There is an active PR which will close this issue when it is merged label Jul 10, 2024
@LWChris
Copy link

LWChris commented Jul 16, 2024

however please consider that every engineer is accountable for literally hundreds of issues like this one

Yes and no. I can absolutely imagine that every engineer is accountable for hundreds of issues. But are those issues really "like this one"? That is, are they equally old, advanced, and perceived as important? Has each and every one of those issues invoked dozens of libraries and thousands of apps implementing workarounds, caused hundreds of questions leading to upvotes on this issue, got dozens of duplicates because they all want the same?

I find it hard to believe that for 8 years, every single time the threshold of "important enough" lies just above the importance of this one. I think the real issue that it is always easier to add new stuff even if only 5,000 people need that, than to fix old stuff 10,000,000 people need when it breaks something for 10,000 of them.

@eiriktsarpalis
Copy link
Member

But are those issues really "like this one"? That is, are they equally old, advanced, and perceived as important?

Yes, unequivocally. Issue age is not the only deciding factor.

@LWChris
Copy link

LWChris commented Jul 17, 2024

Having slept about it, I wonder: if the core reason this whole API is so hard to roll out to the public is "just" the batching of events breaking downstream libraries, would it be possible to add the new methods plus a switch ENABLE_OBSERVABLE_COLLECTION_NOTIFICATION_BATCHING that is by default set to false?

This way there would be some movement in the right direction, that at least adds the new methods so people can start to use them. And if their program is capable of handling the batched notifications, they can change the switch to true at their own risk and judgment.

And later on, when sufficient progress was made in downstream libraries, the default of that switch can be set to true, which will not break binaries because it is a compile time switch that will not change binaries, with the added benefit of still being available to be set to false as a quick emergency response, in case the default behavior of batching breaks something in your own code or some obscure library.

And even if in the very distant future the switch would eventually get removed altogether, it still doesn't break anything, because it's just a compiler switch and defining them without any place in the code using them doesn't break anything either.

I think it would be a good idea to get the API out there to be used, even if it doesn't provide performance benefits yet by default. But at least we would be able to start to do the right thing (use AddRange instead of foreach Add), and make the decision of "single vs batched notifications" one that can be internal to ObservableCollection with all the leverage of what events are raised inside the framework.

Start to do the right things now, benefit later.

@ChaseFlorell
Copy link

I support the implementation of a feature toggle but strongly oppose defaulting it to 'off'. This feature addresses a longstanding need within the community, bringing valuable new functionality. The real bottleneck lies with non-MS libraries and applications using custom extension methods, which impede progress.

As this change does not affect legacy applications unless they opt for a newer binary, any update should handle either resolving breaking changes or disabling the feature via the feature flag. Since this is entirely new additive code and not a true breaking change, it should be included by default in standard updates.

@vslee
Copy link

vslee commented Jul 21, 2024

strongly oppose defaulting it to 'off'

Defaulting to off, while debatable, would still be better than not shipping it at all in 9.0. Which is actually the most likely scenario at this point anyways.

@terrajobst
Copy link
Member

terrajobst commented Aug 2, 2024

Apologies that this got punted again from .NET 9. To a large extent this was my fault for not doing a better job driving an investigation with design options.

Since it's clear that this request isn't simple, I've started a design document that we can use at the beginning of .NET 10 to do a better job socializing the changes across the .NET teams and getting the necessary buy-in to complete it (or decide that we don't want to and close it).

dotnet/designs#320

Since there was some conversation around prioritization and how/why this hasn't happened, I thought it might be helpful to put into context how the .NET team works.

Generally speaking we work both "top down" and "bottom up", the direction indicating who in the org chart is in the driver seat.

  • Top-down. Sizable investments, such as .NET Aspire, AI, or ahead-of-item compilation, are usually done in a top-down fashion. That is, the .NET leadership, with input from our internal and external partners decides in every release what the big rocks are and coordinates the necessary work via a centralized review, where features are being approved/rejected, status is being reported, and breaking changes during development are communicated.

  • Bottom-up. Smaller changes, such as a perf fix or a new overload are usually done in a bottom-up fashion. That is, the PM/engineering owner of the component look at customer requests and decide which ones to take on for a given release.

Why do we do both?

  • Top-down is great to get traction on expensive requests because the priorities on those are shared across the team and the centralized steering ensures they stay on track and you get the necessary resources when you need them.

  • Bottom-up is great when you don't need help from anyone else and you can just do it. Smaller crews of a half a dozen people are self-organizing. It's very satisfactory because throughput can be quite high.

Now, in practice these two aren't binary but locations on a spectrum because the org chart has several levels. The resources you get to fund work increases as you go up in the org chart, at the expense of reduced throughput due to higher scrutiny and organizational overhead.

This brings us back to this item. It started as "let's just add some methods on a type to allow bulk operations". We do changes like this all the time and it usually fits into our bottom-up workflow.

Then we ran into issues. Given that this team is very close to the bottom of the stack, we can't just make changes we identified as disruptive and tell the upstream teams to just deal with it. That's irresponsible because it adds risk to the release.

Where we failed our customers here is that we didn't move this issue to a more top-down item in order to secure corresponding work from the WPF team.

Another way to think about this is that "Microsoft" isn't a centralized being. Rather, it's a decentralized set of people. If we fail to deliver something that requires multiple parties to coordinate, it's almost always a failure of communication and/or a problem of getting priorities aligned.

That's why it's not as simple as saying "this work costs Microsoft X, but it costs the ecosystem 10 * X, therefore it should be worth it" because there often is no single person that has both the power and the context to make that decision. It's an exercise of building consensus and at any given point in time there are only so many of those that people like me are able to pick up.

@terrajobst terrajobst added api-needs-work API needs work before it is approved, it is NOT ready for implementation and removed api-ready-for-review API is ready for review, it is NOT ready for implementation labels Aug 2, 2024
@terrajobst
Copy link
Member

I've marked the API as "needs work" because we don't have an API at this point. We need to figure out the interactions of the feature and how we handle the fact that WPF is broken. There are several options and the design document is the artifact driving those.

@dotMorten
Copy link

@terrajobst Nice write-up Immo.
I think one thing we can do as a community is start making PRs that are fixing the WPF controls that can't handle multi-item notifications one control at a time, by testing them with a custom implementation of INotifyCollectionChanged. That's pretty non-disruptive and allows us to start chipping away at the blockers before this becomes a thing in .NET 10. Who's with me? /cc @pchaurasia14

@robertmclaws
Copy link
Author

@terrajobst Thanks so much for taking the time to put everything together, including this response. You're a fantastic leader of .NET and I for one really appreciate you. 🤜🏻

@LWChris
Copy link

LWChris commented Aug 6, 2024

@dotMorten I've been looking for an opportunity to participate in .NET development, but I'm new to this, so I don't know yet if I have what it takes to make changes to core components. But I think we could try to compile a list of the controls that need fixing, which makes it easier for me and others to judge if there's some controls that are "on our level" of complexity. I have to admit I haven't looked through all 500 posts or linked/related items, so maybe this list already exists somewhere.

@ShawnTheBeachy
Copy link
Contributor

@terrajobst Thanks for the fantastic explanation. For me, just having an idea what's going on like that really does help!

@fubar-coder
Copy link
Contributor

To me, it seems that the main problem is the wrong handling of the CollectionChanged event. The examples out there almost always use the first element of the list of changed/updated/removed/moved elements, which is - in most cases - wrong.

Here's my suggestion for an analyzer:

  • Analyze all CollectionChanged event handlers
    • maybe even every function, which takes a NotifyCollectionChangedEventArgs argument
  • Ensure that OldItems and NewItems are used correctly
    • Only allow the usage of OldItems[0], and NewItems[0], if test of Count == 1 was successful
    • Otherwise always require access to OldItems and NewItems to...
      • ...use a variable indexer (e.g. OldItems[i]
      • ...be passed to a function accepting IEnnumerable, e.g. OldItems.Cast<object>().Single()

This analyzer should produce a build warning. Fixing this warning should ensure that range operations work as expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-needs-work API needs work before it is approved, it is NOT ready for implementation area-System.Collections in-pr There is an active PR which will close this issue when it is merged
Projects
None yet