Add explanation/concept for extension maturity model

[holly-cummins]

This is part of the bigger plan for extension documentation: https://github.com/quarkusio/quarkus/issues/37288


Obviously, there’s a gap for dev services documentation, which I’ll move on to next. Once I've got something linkable for dev services documentation, I will come back and link here.

One of the links needs quarkiverse/quarkiverse#229 to merge for it to resolve. I decided to be optimistic about the order things would happen and put the link in anyway.

The preview comments aren't working, but this is the link to the new page: https://quarkus-pr-main-42446-preview.surge.sh/version/main/guides/extension-maturity-model

Question for reviewers: Should the list at the top be numbered? Or perhaps a graphic, so that it's more distinct from the table of contents lower down? https://en.m.wikipedia.org/wiki/File:Characteristics_of_Capability_Maturity_Model.svg is a somewhat standardised visualisation of maturity models, but it's harder to edit than raw text.

[quarkus-bot]

Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 16e3f8e.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

Warning

There are other workflow runs running, you probably need to wait for their status before merging.

You can consult the Develocity build scans.

[maxandersen]

the list is good - but afaics the only thing that "fits" a maturity model (as in something that is progressively more) is "works in jvm" and "works in native" (too). The rest a checkbox features.

i.e. does it help to call it a extension maturity model or does it confuse more ?

[holly-cummins]

I wondered the same thing, because you're right that the progression is not a strict one. But I think it does make sense to talk about the maturity of an extension. In this context it's a synonym for completeness, but completeness wouldn't be helpful term to use, because it makes it sound like people aren't 'allowed' to do extensions unless they do everything. And 'checklist' has a similar problem - it doesn't have enough progression.

Another way of thinking about it is that it's a model to distinguish a really good extension from an average one, but I don't think any words like 'quality' would be helpful, because there's too much judgement. This should be an aspiration list, not a shame list.

[maxandersen]

yeah, I don't have too strong opinion ....just that in past we kept getting stuck trying to "rank" it .

Alternative suggestion to maturity model I can think of is Capability Model: Emphasizes the capabilities or features that are completed or available.

another is "Implementation Levels: Describes the levels at which different parts of the project have been implemented." but that also "ranks"

[holly-cummins]

The good thing about "maturity model" is that it will be really obvious to people what it means. "This page tells me what my extension will be like when it starts, and where it could eventually end up if I continue working on it." "Capability model" or "Implementation levels" would need some words around it to try to explain to people what the purpose of the page is, because they're not familiar concepts in the same way.

The bad thing about maturity model is that this isn't a maturity model in the strictest sense, because the progression/dependencies on lower levels isn't strict enough. And there definitely could be some debates about the relative position of a few things.

We could change the page title to something generic like extension-capability-model but leave the in-page title as "maturity model". That way if we decide to change the the title later on we don't have a problem with the url?

[maxandersen]

+1 on using capability model for the URL.

I've looked at the staircase multiple times and it keeps rubbing me the wrong way that the two cornerstone capabilities Quarkus pushes/priorities is the last two steps before last.

It would be more coherent and signal of focus if we move devservice/devui higher up maturity scale as we really want programming model and performance to be early priority.

With that its:

Works In JVM -> Works in devmode -> works in native mode -> configuration -> joyful programming model -> supersonic performance -> provide a dev service -> dev UI -> codestart

Yes, we could debate additional ordering but with that at least config/joyful/perf is better balanced with respect to what implicit message the stair case inherently brings.

[cescoffier]

Ordering will be very different from one extension to another. We should not crystalize on that.

The only ordering we have is: works in JVM -> works in native (as the opposite is rarely true)

I would use "capabilities", or "traits" as I mentioned in another comment.

In another comment, Max shows some radar graph. While tricky to set it right, I think it is slightly better:

• no ordering
• extensions can decide to focus on different aspects (some dev only extension do not care or performance or cloud readiness, or even native (mailpit for example))

[maxandersen]

good point but wouldn't it be better fitting on dev joy?

[holly-cummins]

See the comment above - I think if we're suggesting an implementation order, we can't group too much. So let's get rid of 'Developer Joy' altogether from that list, so there's no question of things being sub-points under it or not.

Developer Joy should be a theme, not an implementation order.

[maxandersen]

i see the graph now have "joy full programming model" is that a missing removal or intentional?

[holly-cummins]

The graphic has "joyful programming model", which is intentional. "Developer Joy" includes a whole bunch of things, which would be implemented at different times - so what the graphic is talking about is specifically the boilerplate reduction, and not things like support for live reload or unified configuration.

[holly-cummins]

i see the graph now have "joy full programming model" is that a missing removal or intentional?

Just to add a bit of context, the graphic switched to its current, non-nested, form on August 12th (#42446 (comment)). That was just the visual version of this change in the list, from August 10th:
#42446 (comment)

I wanted to pull it out to the top level, because it's so important.

For consistency I could also pull config out to the top level, and leave developer joy to be the more amorphous 'take advantage of quarkus magic to revolutionise your programming model.'

... and also this comment, also August 10th:

I think if we group too much under developer joy it starts to break the ranking. I'd say that tolerating dev mode is table stakes for an extension, but 'redefine the programming model' is a pretty advanced feature.

What about pulling developer joy some sort of tag or 'key'. So Config and Boilerplate reduction become top level items, dev mode goes back to being in front of native mode, and we put the joy icon at the beginning or end of those lines. Or if we do boxes, the joy icon tags the box.

[maxandersen]

ack - I like the use of the developer joy icon. its a good fit.

staircase vs graph vs something else is for sure tricky.

Its fine and at least sends a message.

Only crazy thought I had is to make it a circle as in iteratively keep improving them but it probably gets too big :)

I'm not an artist so got some help from chatgpt:

I kind a like this and maybe that kind of diagram could be used as a way to express what is explained in the paragraph that "This is a suggested pathway, not a minimum feature set."

Just a thought that came to me re-reading it. Does not invalidate the simplicity of the staircase.

if it resonate we can consider it in an update.

[cescoffier]

I like the radar idea. I would have used different axis to not invent something else (and makes it less intimidating).

Also some of them are boolean (codestart, you have it or not, how can you have half of a codestart (I have one but it does not compile?) - these should be incorporated into the "dev joy" axis.

[maxandersen]

good stuff! just suggestion on cleaner ordering

[gsmet]

I like the general idea.

I added some comments here and there.

[gsmet]

I think it needs to be clear that you don't need to implement all of this before releasing your extension. It's more or less mentioned but people have a tendency to go over the top sometimes when developing in the open and we need to clarify it's OK to publish a first version that doesn't handle everything.

[holly-cummins]

Yes, agree 100%. I was trying to convey that (and I think it's kind of implied by the maturity model), but I will be more explicit about that.

[holly-cummins]

The order in this model isn't exact. Different developers will have different views on what capabilities are most important. You may wish to (say) prioritise performance over enhancing your extension's Dev UI tile. That's fine!
Also, not every step will apply to every extension. For example, you don't need a dev service if your extension doesn't depend on external services.
It's completely OK to publish a first version of an extension that doesn't handle everything. In fact, it's ok if your extension never gets to the more advanced features. This is a suggested pathway, not a minimum feature set.

?

[gsmet]

Perfect! (sorry forgot about it!)

[gsmet]

I think we should try to add pointers to extensions that are good inspiration. Does it make sense?

If so, I suppose your next question will be for me to point them to you? :)

[holly-cummins]

Yes, and no. And yes, but no. But yes. :) For dev services in particular, I’m planning to get out a basic dev service documentation this week, and then cross-link to it from here. So ‘do it this this week’ can sometimes be a fiction, so there’s no harm having links to examples in the interim, but we also don’t need to spend too much time on stuff that I’m hoping to remove in a day or two. Those links to good examples would be helpful to include in the dev services doc, though. 
More generally, I really like the idea of providing pointers to example extensions. I’ve got another doc in draft that’s an an extension FAQ with questions like “I need to remove a method” and so on, and there I 100% want to link to examples to follow. So if you have any … :)

[holly-cummins]

I’ve changed the list to a graphic. This has the benefit of adding something beyond what’s in the ToC below and the base writing-extensions guide. It has the disadvantage it’s hard to maintain. I don’t have the tools to do it, but @insectengine might be able to convert it to an svg, which would still render nicely, but would have a slightly higher bus number than my png.

[github-actions]

🎊 PR Preview 45dbff7 has been successfully built and deployed to https://quarkus-pr-main-42446-preview.surge.sh/version/main/guides/

• Images of blog posts older than 3 months are not available.
• Newsletters older than 3 months are not available.

[insectengine]

Created a slightly different view of the graphic (PNG and .SVG)

[holly-cummins]

Created a slightly different view of the graphic (PNG and .SVG)

Thanks! Visually, that looks way better than mine, but I'm not sure about the shape. I think people's expectations of a maturity model would be a bit more stair-case-y. There's some variation in what google shows, including one pyramid, but almost all of them have a linear progression from bottom to top and left to right:

I had in mind something like this:

Because we have more stages than that example it does make it harder to fit everything on the page, but I think it does make it easier to understand the intent.

[maxandersen]

I'm a bit puzzled with your comments where, if I understand correctly, you say different things:

* You like the matrix, but a different "scale" for each category. IMHO, that can be confusing and tricky. First, for all categories, we define a precise order (which we don't have) or a checklist of things to achieve. A checklist would be perceived as something you need to implement, which is not necessarily the case. It is trying to classify precisely all the features an extension can implement. I would not over-complexify this, but provide examples (in the text) of features for each category (like it is more or less in the PR).

Yeah so I'm not sure how you say that is not a checklist of things - the extension either done it or it haven't.

* You said that the radar is nice because it can show "progress/completion," - but this is a bit of the opposite of the previous point, no? Also, it is not the goal for every extension to be "complete". I don't even know what it means for an extension to be "complete", especially when we have things like "performance".

What I was referring to that since we do not have an order of each of the axes the matrix can't be rendered with the blue/pink "progress" markers, but a radar view could - as it just shows the number of things done.

* The problem with the radar for outliers is the negative perception of these "outliers." They look "incomplete" (reusing your wording), while they are perfectly fine.

Sure, but the same applies for the Matrix - I was just saying that any way you do this and visualize it things will be "missing" from extensions that are totally fine.

We could say that this only applies to "normal" extensions—but what's a normal extension? Or are the others second-class citizens? I don't think we should go there.

I'm not going there - I'm saying that its perfectly fine that some extensions haven't filled out the full matrix/radar/whatever - that's the whole point; we are not trying to have a precise capture of the "usefulness" of an extension here - we are trying to signal/educate which things an extension dev should focus on and what things to consider/do when making an extension.

[maxandersen]

Ohh, I hadn't thought about generating a version for each extension, but once #40336 is done, that would be pretty possible. Even for the matrix, we could turn it into an svg and show it on a tab on quarkus.io/extensions. I've raised quarkusio/extensions#1208.

I think for extensions.quarkus.io and similar the "radar" is more achievable and

I don't think we should worry too much about the implementation complexity of rendering a grid vs a table at this stage.

I'm not talking about implementation complexity - I'm talking that the radar visualization doesn't require perfect ordering thus more achievable.

Is that what you're thinking about, in terms of 'achievable'? There's a lot more complex things we'd need to sort out first, which would need to be done in both cases, like getting consensus on #40336 (metadata generation in the deployment module), and then implementing the code introspections to discover the features and put it into metadata.

well, I would simply just let a human record a value for each "dimension" and render it. if we over time makes it possible to automatically extract thats great - but I doubt it will be possible for all of them.

faster to read

It's faster to read, but less informative, because it doesn't have any information about what extension authors should be doing to go higher up the axes. For people who're not extension authors, it doesn't convey much information about what an extension offers, other than an (implied) numeric score. So it's a tradeoff.

So just to be clear - the radar visualization is for me an additonal way to show how "much" an extension does from the staircase or now in the discussed matrix without requiring a full order.

I'm not pushing for the radar to be the primary visual to put on this page.

Just to make sure I'm understanding your suggestion right, for each axis, we'd have an un-ordered collection of features, with no notion of order or staircase or difficulty level? An extension's position on the radar axis would just be an unweighted 'score', for how many of them it does?

and can express "progress"/"completion" without having to have a perfect sequenced ordered axises.

My suggestion is that we accept imperfect ordering, so that we can make progress. :)

Yes - but a matrix is pushing for even more order if intention that it would have that blue/pink "progress" rendering.

Thats why I was saying we can't put in there as its more a list of things developers can do - not a strict order.

For most of the axes, there is a fairly natural order. You cannot make your extension work in native if it it doesn't work in JVM. You cannot do a dev service unless you have unified config. Adding CDI bean support is a prerequisite for most higher level developer joy features. You're unlikely to be doing a whole rewrite of the programming model if you haven't got the basics right. Even where there's not a technical reason that things have to be implemented in a particular order, there's usually an implicit maturity hierarchy of 'this is an advanced feature, only really sophisticated extensions would be doing this, this is table stakes and most extensions should be doing this.' I think this is an area where we're allowed to be opinionated ("opinionated" is a Quarkus principle 😄 ) ). Extension features do differ in difficulty, so I think it's helpful to encode some information about difficulty by using an ordering.

You are giving examples of orders between two items, not the lists of multiple items actually in the proposed axises you made. Thus I'm going to assume I'm not understanding how you and @cescoffier propose the matrix to look like.

So I'll just wait to see the updated PR :)

[cescoffier]

(On the run so keeping this short)

I think we are conflating 2 things: extension developers and extension users.

To who is that document really intended to?

For an extension developer (as I think it was the initial idea), having a checklist per category would be helpful and provide guidance. But it should not be too intimidating. It's not necessary a level of maturity - mostly the kind of things you can do to improve your extension.

For an extension user, it's slightly different. Of course there is overlap, but in this case, the user wants to know the capabilities of the extension and not the underlying details (a user should not care about synthetic beans, or if it reuses Netty event loops but could be interested to know if it works in native or support virtual threads).

Now, the question is: is there a representation working for both? I'm not sure. I think we should focus on one of the persona for now (extension developers probably because it was the initial target).

[holly-cummins]

I think for extensions.quarkus.io and similar the "radar" is more achievable and

I don't think we should worry too much about the implementation complexity of rendering a grid vs a table at this stage.

I'm not talking about implementation complexity - I'm talking that the radar visualization doesn't require perfect ordering thus more achievable.

Ah, I see. It was the reference to achieving the visualisation on extensions.quarkus.io that confused me. In terms of achievability, I already have a draft of the matrix above.

Is that what you're thinking about, in terms of 'achievable'? There's a lot more complex things we'd need to sort out first, which would need to be done in both cases, like getting consensus on #40336 (metadata generation in the deployment module), and then implementing the code introspections to discover the features and put it into metadata.

well, I would simply just let a human record a value for each "dimension" and render it. if we over time makes it possible to automatically extract thats great - but I doubt it will be possible for all of them.

I'm -1 on anything that creates more work for extension authors. Manually maintaining such a list is likely to be extremely error-prone, and inaccurate information is worse than no information. But lets move that discussion to quarkusio/extensions#1208. This PR is about publishing a document to help extension authors.

It's faster to read, but less informative, because it doesn't have any information about what extension authors should be doing to go higher up the axes. For people who're not extension authors, it doesn't convey much information about what an extension offers, other than an (implied) numeric score. So it's a tradeoff.

So just to be clear - the radar visualization is for me an additonal way to show how "much" an extension does from the staircase or now in the discussed matrix without requiring a full order.

Ah, I see. The suggestion is I do the matrix, as discussed above, and we also have the option in the future of the radar as a 'compressed' view?

I'm not pushing for the radar to be the primary visual to put on this page.

Got it. Sorry, I'd thought you were suggesting we do the radar rather the matrix. Ok, I'll proceed with a draft for the matrix.

[maxandersen]

Ah, I see. The suggestion is I do the matrix, as discussed above, and we also have the option in the future of the radar as a 'compressed' view?

yes. I'm saying that the matrix without order instead of the staircase I think is a good option for this PR/page targeting quarkiverse extension writers.

the "radar" is as you say - more for a compressed view/illustration that would be good to (optionally) have for extensions where its possible to do (automatically or manually)

[quarkus-bot]

Status for workflow Quarkus Documentation CI

This is the status report for running Quarkus Documentation CI on commit 508f8c7.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.