Design an initial usage page for Cloud.gov accounts

[chizo-gov]

We’re creating an design for an organization’s “usage” page, which will show various pieces of data relevant to an organization.

This will be building off the work done in #502.

Can this task be done in one increment?

Yes.

Acceptance Criteria

• Designs are accepted by cloud.gov
• Designs have been reviewed for technical feasibility, and handed off for implementation

[beepdotgov]

Added a new title + description for this ticket to track the design work for this increment; @chizo-gov, does this sound okay to you?

[beepdotgov]

Hi @hursey013, @sknep — thought I’d check in with the latest version of the org usage page.

(Note: this is still in progress, but I wanted to get the latest version in front of you before I’m OOO tomorrow/Friday.)

Before I dive in: visual feedback is welcome at this stage! At the same time, I think it’d be most helpful to keep discussing whether we’re including the right information here. Put another way: what does a Dashboard user need to see or do on this page?

Okay, enough preamble! Here’s the design (also in Figma):

Here are some highlights:

• After Friday’s chat, I broke out memory allocation vs usage, and tried to make them a bit more distinct from each other.
• I also made a number of small visual refinements to the Memory section, like tightening up the graph, and making the visual hierarchy a little clearer.
• I spent some time thinking about ways to link off to more detailed pages for (e.g.) logging information. That’s one of the biggest small changes here, with CTAs for individual spaces, and links for more info under the big memory graph.
• Added some timestamps to show the, well, timeliness of some of the data we’re displaying: if we’re showing information that might not be generated in real-time, we should make sure that’s evident to the user. (This came up in conversation with @echappen, and I think it’s a great addition — kudos to her for the idea.)

Phew, that was a lot. I’d love to hear any thoughts you have, and figure out what else we need to include here.

(And feedback’s always welcome here, but I can find time for a sync discussion when I’m back next week. Let me know.)

[sknep]

some notes from chatting w/ @jameshochadel about this and looking for opportunities:

• we might want to show maximums on the graphs as well (remember that quotas can be bumped up/down)
• There's all usage data, and then there's the billable usage.... eyeball what we offer under "Additional charges" here: https://cloud.gov/pricing/
• unsure where the historic data/averages are coming from, perhaps the events api? maybe @hursey013 knows
• some things might not be quota out of a max. Might wanna show the number of instances for each plan within each brokered service (e.g., Elasticsearch usage begins with 6 nodes of "es-medium", but can be augmented with, for example, 5-7 nodes of "es-large" at an additional fee). The App Autoscaling option is paid for with a single price, and they're either using it or not, for a given app (or a number of them). Some things are just "how much you're using" without a max (denominator). Routes, for example, we tally up, but don't enforce a max.
• could we show the # of support hours available / quota? we start with 10 hours.
• for CTAs, I wish the view logs was more findable (like down at the bottom of the card, near the date?), it feels buried. I also imagine people wanting to be able to request a quota increase/decrease, we could do that with a prefilled mailto link addressed to inquiries@ with a subject naming convention until we have real business logic for that. Someday, we'd love to add CTAs or ways to self-service manage their quotas as long as they can afford it. But that won't be til late FY25 at best
• I also found out that text-based "logs" aren't available for these items right now, but "time series metrics" are. They are more structured and might have dimensions like org, space, app, and instance, which is probably how we'd make the graph points. OpenSearch, Prometheus, and Grafana (stuff we're working with or working on) can be configured to expose that data, so in that case, we could link the user to a filtered view of a given dimension/usage metric in one of those tools, where they can customize the query and nerd out. So that's the "dig deeper" option rather than "view logs".
• If the current user isn't a billing manager, should we show them who their org's billing manager is?
• what about showing the IAA/contract expiration/renewal date so they know when it's coming?
• maybe a summary of alerts near the top for being at or within 10% of your quota max either presently or within the last.... week?? I'm not sure how many of these cards might be on the page, and in what order. Wouldn't want them to miss an important one way down at the bottom.
• as background, here's a list of the auditable events that could bubble up into analytics sometime, though i'm not sure which are usefully related to usage: https://docs.cloudfoundry.org/running/managing-cf/audit-events.html

down for a chat! i just took rapid notes while noodling ^^

[beepdotgov]

Thanks, @sknep and @jameshochadel! This is fantastically helpful. Just confirming receipt, but on first read I have a couple quick notes/questions:

unsure where the historic data/averages are coming from, perhaps the events api?

That’s something we’re investigating for technical feasibility. (Sorry, I should have noted that — thanks so much for flagging.)

some things might not be quota out of a max

Just to make sure I’m following: does this apply to everything that’s currently under the “Addons” section up above?

(Related: if there’s a list of services/features that are simply “how much you’re using”, that’d be really helpful.)

---

I think that’s all for now. Everything else looks stellar, but I’ll follow up with questions. (And thanks for the offer of a chat; I think that’ll be a likely next step.)

(@hursey013, holler if you have any thoughts on the above!)

[jameshochadel]

@beepdotgov I can clarify the point about quotas. All of the metrics shown in the design have soft quotas in the IAA, but can't be shown as quotas in the web UI because the values in the IAA are not stored in a machine-readable format anywhere. (IAAs are ~PDFs.) Showing them as you did in the design would be ideal because, as I understand it, our target user wants to know how their usage compares to their agreement. But unfortunately the data isn't there (yet).

Why "soft" quotas? In practice, sometimes customers temporarily use more resources on some dimension than is specified in the IAA. As long as they have enough funding on the IAA to support this — or as long as they plan to add more to cover the difference — that's ok. The most common dimension for which this happens is memory. It's a manual process involving emails to our business & support teams.

[beepdotgov]

@jameshochadel, thank you very much for the insight. I’ve been wondering when/if we’d run up against which IAA values were programmatically accessible, so I’m really glad you weighed in. Given that, I might recommend we refocus this design on what is currently available to us in a machine-readable format.

If that sounds like the right approach — and I’d love to hear others’ thoughts on this! — I have two questions:

1. Is it safe to say that everything under “Addons” in the current mockup can’t currently be shown in a web UI?
2. Are there any other pieces of information on the current mockup that can’t currently be shown in a web UI? (I tried to work off Eleni’s work in Define what usage metrics are available from the Cloud API and what requires an additional data source #497, but I should double-check I didn’t introduce any errors.)

Let me know what you folks think. And thank you again for the clear direction here, it’s been incredibly helpful!

[jameshochadel]

@beepdotgov Of course!

1. My two cents: We can display the numerator but not the denominator. I.e. we can show the X from each X/Y. That still seems useful to me; we're giving the billing persona one place to see all of their billable usage. To enumerate each metric in the mockup:
   1. S3 storage: This is a CloudWatch metric we'd have to expose to the UI via a backend somehow. Not available via CAPI.
   2. RDS disk usage: Also a CloudWatch metric, or something similar from AWS. Not available via CAPI.
   3. Number of redis nodes: Available in CAPI. This also applies to ElasticSearch nodes. Note that we will need to report the plan type as well. 5 nodes of the small plan != 5 nodes of the medium plan. So the hierarchy is 1 service offering -- many service plans, and each service plan corresponds to a number of nodes. (Details on https://cloud.gov/pricing.)
2. The historical data is likely not available. I think we'll have to scrape and store it ourselves if we want to show that. Memory usage (not quota) is available as a Prometheus metric, but similar to CloudWatch metrics, a backend service will have to aggregate that, since Prometheus is not exposed to users. Current memory quota is available, based on the audit events docs, we don't think a history is.

[beepdotgov]

James, thank you again. This is such a huge help.

Questions/comments inline below:

We can display the numerator but not the denominator. I.e. we can show the X from each X/Y.

Just to clarify: this applies to just the items under Addons, correct? Or do we need to drop denominators elsewhere on the page?

To enumerate each metric in the mockup: [snip]

Thanks so much for walking me through this. For S3 and RDS, I think there’s an engineering/LoE discussion to be had here, at least in terms of what we want to invest in for a v1 of this page. I’ll sync up with @echappen to see how best to manage that.

(@hursey013, if you have any thoughts on priorities here, let me know!)

Note that we will need to report the plan type as well.

If you happen to know: is the plan type accessible via an API?

The historical data is likely not available. I think we'll have to scrape and store it ourselves if we want to show that.

That was our sense too — thanks for confirming it. (And again, apologies for not flagging that earlier!)

[jameshochadel]

Happy to help.

Just to clarify: this applies to just the items under Addons, correct?

Correct. The memory quota is in CAPI, as are the other quotas listed in the box on the right-hand side. The full list is here.

For S3 and RDS, I think there’s an engineering/LoE discussion to be had here

Agreed — happy to weigh in on that if you'd like.

is the plan type accessible via an API?

Yep! I should've used more precise language, but the relevant endpoints are service offerings, service plans, and service instances.

As an example of each: Cloud.gov might provide a service offering called "RDS (Relational Database Service)". It could have plans like postgres-small, postgres-large, mysql-small, mysql-large. A customer creates one or more service instances called whatever they want (my-database, my-other-db), each of which is associated with exactly one plan.

Service instances can be further configured with individual parameters. Typically, configuration that results in differences in billing gets broken out into different plans, and non-billing configuration is handled as parameters. (In our case, we don't currently bill for RDS instances — only their storage — but that's the general pattern.)

[beepdotgov]

Okay! Thanks to @jameshochadel’s help, I got a lot more clarity on some of the API nuances at play here, which helped unlock me on the design revisions. Which are: right here! (Also in Figma, always in Figma)

A couple questions as we talk through this second round:

1. First and foremost: does this feel like it better represents information that’s programmatically accessible to us?

• (As discussed, the obvious exceptions here are S3 storage, RDS disk usage, and the “memory allocation over time” chart, all of which would need some feasibility review before we can proceed. But if anything else feels off, API-wise, please let me know!)

2. I really want to make sure the underlying content/data is right, but: please keep the visual feedback coming! Does the hierarchy feel right? Any elements you’d like weighted differently?
3. What other information would we need on this page for a v1?

Thanks again for all the input, all. I’m really looking forward to hearing your feedback on this latest version.

[sknep]

Can I ask for one alternate, which takes the top left box and makes it larger/gives more weight? It feels very orienting, like an overview. I'd be curious what bumping up the size/impact would do, leaving the usage graphs/stats in the regular sized card/islands.

[beepdotgov]

Oh, great call @sknep — thank you!

Here’s a (very) quick riff where I tweaked the type and content to make it stand out a bit more; if you’d like the entire card wider / more prominent, I’ll play around with some layout options. Let me know!

[beepdotgov]

Okay! How’s this feel to you, @sknep? I quite like the new placement; I think that orienting block is a really great idea. Would love to hear what you think, though:

[sknep]

Oh I think you know the answer. ✨🎉

[beepdotgov]

Marvelous news, thanks @sknep! So glad it’s shaping up nicely.

One thing came up in discussion with @echappen and @hursey013 today: the treatment of allocated memory could use some refinement. (The progress bar treatment’s a little unclear, and also presents things as an error.) We wondered if a pie chart might be a better approach, like so:

(That’s just the revised card; the entire thing’s viewable in context in Figma.)

How does that feel to folks?

[sknep]

You might consider some pattern backgrounds to differentiate beyond just the colors. They all seem of a similar amount of light and dark, so people who can't differentiate reds or blues and greens might struggle to match the segments to the legend.

[beepdotgov]

So glad you flagged that, @sknep — thank you! I was trying to treat the chart’s as an enhancement to the table, which has all of the info the user’ll need. But I’ll do another pass on the chart, and play with a few other values.

(Also! I did run the chart through a few different (simulated!) models for color vision deficiency, and things hold up fairly well. But there are definitely a few areas where the contrast needs tweaking; thanks again for calling this out.)

Screenshots

[beepdotgov]

Okay! Here’s a tweaked pie chart, with some updated colors:

I think these colors are much better, contrast-wise, and they seem to perform a lot better in these simulations:

Screenshots

(Of course, these are just simulations; I’d love to see how well these perform for our users. And once we get to implementation, I’ll make sure we’ve got a few other colors to cycle through for the chart.)

But! How do these feel to you, @sknep? And thanks again for raising this!

[sknep]

Thank you! Appreciate you bringing receipts 😁

[beepdotgov]

Hey, I gotta show my work! Thanks for moving this forward, @sknep 🚀

@hursey013 @jameshochadel, would love any thoughts you might have too! Here’s the latest mockup (also in Figma):

Visual feedback’s very welcome, but/and if there are any product/data/API considerations we should talk through, I’d love to hear them.

Thanks all!

[hursey013]

First off, let me just say how wonderful this looks! The information being presented here will allow folks to quickly assess the status of their org in one place, something that they cannot easily do today. I especially love the inline calls to action for support for things like increasing memory, it's simple enough but allows customers an immediate prompt for where they need to go if they need help.

It's already been covered above, but the pie chart looks great and adds a great visual representation of memory, one of the primary levers customers can adjust within their org. Thank you @sknep for your feedback on colors, I think where we landed looks great. Similarly, the ability to see both allocation and current usage within the same page provides some instant feedback on how close customers may be to hitting their threshold and where they might be able to make some changes.

During the feasibility review I think we can evaluate the sources from which we may be able to pull: historical data, IAA expiration, and support hour information. Off the top of my head I'm not sure if that information is readily available, but it warrants a deeper dive based on the value that information provides.

I think my only actionable feedback on the design based above would be in reference to the Additional Services. Right now in the mockup there is one of each type of service represented, but in practice an org my have dozens or more of each of those services. There could be 10 S3 buckets in multiple spaces like dev, staging, and prod. Would the expectation be that these values (current usage) would be the sum of all of those S3 buckets or might there be a way to give a bit more insight into how that storage is allocated across all of the buckets? Perhaps using a similar approach to the At a Glance section where we specify how many of each of those services exist? Something like, "Currently there is 4TB of usage across 12 S3 buckets, within 3 spaces"? I defer to you on the best way to present that information, but I do want to make sure we have the ability to represent the usage/status of what could be an unlimited number of brokered services such as s3 storage, various types of databases, and external domains.

Again, this looks beautiful @beepdotgov, amazing work!

[beepdotgov]

@hursey013, thank you for the feedback. This is massively helpful! And I’m thrilled you’re happy with the progress.

(Responses inline below.)

During the feasibility review I think we can evaluate the sources from which we may be able to pull: historical data, IAA expiration, and support hour information. Off the top of my head I'm not sure if that information is readily available, but it warrants a deeper dive based on the value that information provides.

Thanks so much for noting that, and absolutely agreed. If any of those items aren’t programmatically available today, it might make sense to parcel out “now” vs “later” versions of the design, like we did for the welcome mat.

(I was already thinking the “View more detailed information »” link would be on that “later” version, as it’d require a page we’re not building yet.)

I think my only actionable feedback on the design based above would be in reference to the Additional Services. Right now in the mockup there is one of each type of service represented, but in practice an org my have dozens or more of each of those services. There could be 10 S3 buckets in multiple spaces like dev, staging, and prod. Would the expectation be that these values (current usage) would be the sum of all of those S3 buckets or might there be a way to give a bit more insight into how that storage is allocated across all of the buckets?

I’m very glad you called this out! I was thinking of it as the former: that this would be the sum of all (e.g.) S3 buckets. But if we need a little more detail at this level, we can work on adding it here.

Perhaps using a similar approach to the At a Glance section where we specify how many of each of those services exist? Something like, "Currently there is 4TB of usage across 12 S3 buckets, within 3 spaces"?

I like that suggestion. I’ll start tinkering with that, and try to get some ideas in front of you tomorrow.

[beepdotgov]

Okay! I’ve spent the morning tinkering, but ultimately landed on a fairly straightforward approach: namely, adding @hursey013’s suggested text to each of the brokered service cards at the bottom of the page. And I think it works well!

(Also viewable in Figma.)

Per James’ notes, some of these cards will need a technical feasability review to decide if they’re ready for version 1 of the usage page, or a later version. (But again, I’d be happy to prep “now” and “future” mockups for this page.)

Let me know how this looks to you, @hursey013!

[beepdotgov]

@echappen @hursey013 @jameshochadel Thanks so much for your time today!

I’ve got a revised version of the :cadillac: design below, and I’d love to hear how it looks to you all. I’m specifically curious to know if the rolled-up Elasticache / Elasticsearch sections accurately reflect what we discussed today; feedback there would be really appreciated.

Once we’ve got the “cadillac” version looking as we need it to, I’ll prep the “what we can implement today” version for everyone’s review.

Thanks again, all!

[jameshochadel]

@beepdotgov Amazing, great to see these updates so quickly. I think the ES/redis updates look great. The only change I'd make is "This organization has X Elasticache plans" -> "This organization has X Elasticache instances". An org may have zero, one, or multiple instances on a given plan, for a given offering. The IAA will enumerate how many instances on each plan they are paying for. So the list on the right of each block could work more like an aggregation / "group by": The number of instances per plan, and the total cost per plan type. E.g.

Plan	Instances	Monthly Cost
es-small	2	$400

[beepdotgov]

@jameshochadel Thank you for weighing in, that’s perfect! I love the rollup idea; I’ll get those consolidated tables in now.

One small question:

The only change I'd make is "This organization has X Elasticache plans" -> "This organization has X Elasticache instances".

Since you explicitly called out Elasticache, I just want to confirm: should both Elasticache and Elasticsearch mention instances? Or just Elasticache?

[jameshochadel]

Splendid. Yep, I meant both, thanks!

[beepdotgov]

Fantastic, thank you!

Okay, the updated design’s below. I really like this rollup approach, but/and @jameshochadel @hursey013 I’d love to hear if this matches what you’d expect.

Thanks again for all the input today!

[jameshochadel]

Looks great to me 🙌

[hursey013]

Yep, love this! New rollup approach provides a lot of valuable information - let's move forward with it!

[beepdotgov]

_Wonderful_, thank you both! Pulling together the non-cadillac version of the page now; I’ll have that ready for you two shortly.

[beepdotgov]

Okay! I’ve got a “today” screen that’s ready for review. The major changes from the “cadillac” version are:

• Removed the IAA expiry note
• Removed the “support hours remaining” block
• Removed the historical memory usage line graph
• Removed the link to view more detailed logging information
• Removed the S3 and RDS modules

Here are the mockups:

Organization usage, ”today“ edition — widescreen

Organization usage, ”today“ edition — small screen

(Both of these are available in Figma, if you’d like to review them next to the “cadillac” version.)

Would love to hear what you all think! Once this looks good, we can set it on the path to implementation.

[hursey013]

This looks great @beepdotgov, let's move forward!

Small note: I think, if we wanted to, we could leave the view more detailed logging link in, and we can actually populate a link to ElasticSearch for the specific org which will give those log based metrics that we have on hand (example below). I think this could be a nice opportunity to make people aware of our existing logging solutions in case they don't know about them, or if they don't know how to access them.

[beepdotgov]

Sounds great, Brian! I dropped that link back into the designs for now (attached); let’s discuss how best to proceed during feasibility review.

Since we’re coming to the end of the engagement, should we sync up as a team next week to figure out how to move this forward? (cc @echappen)

[echappen]

Yep, let's chat about it at increment planning on Monday

[chizo-tts]

Designed but not yet implemented.