Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/hami-webui/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ The command removes all the Kubernetes components associated with the chart and
| fullnameOverride | string | `""` | |
| hamiServiceMonitor.additionalLabels.jobRelease | string | `"hami-webui-prometheus"` | |
| hamiServiceMonitor.enabled | bool | `true` | |
| hamiServiceMonitor.honorLabels | bool | `false` | |
| hamiServiceMonitor.honorLabels | bool | `true` | Keep HAMi's workload namespace/pod/container labels when they collide with Prometheus scrape-target labels. |
| hamiServiceMonitor.interval | string | `"15s"` | |
| hamiServiceMonitor.relabelings | list | `[]` | |
| hamiServiceMonitor.svcNamespace | string | "kube-system" | Default is "kube-system", but it should be set according to the namespace where the HAMi components are installed. || image.backend.pullPolicy | string | `"IfNotPresent"` | |
Expand Down
6 changes: 4 additions & 2 deletions charts/hami-webui/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,9 @@ serviceMonitor:
hamiServiceMonitor:
enabled: true
interval: 15s
honorLabels: false
# Preserve HAMi's namespace/pod/container workload labels when they collide
# with Prometheus scrape-target labels; required by the WebUI queries.
honorLabels: true
additionalLabels:
jobRelease: hami-webui-prometheus
svcNamespace: kube-system
Expand Down Expand Up @@ -202,4 +204,4 @@ metricsExporter:
# evaluates process.env at build, not at runtime. To change it for a deployed
# cluster you need to rebuild packages/web with a different VUE_APP_REQUEST_TIMEOUT
# (set in packages/web/.env.production or as a build-arg). It cannot be tuned
# through this values.yaml.
# through this values.yaml.
29 changes: 28 additions & 1 deletion docs/installation/helm/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,33 @@ To set up the HAMi-WebUI Helm repository so that you download the correct HAMi-W

You should get the expected both 'hami-webui' and 'hami-webui-dcgm-exporter' in running state if installation is successful.

### Prometheus scrape labels

HAMi's vgpu-monitor exposes workload identity as `namespace`, `pod`, and
`container`. When Prometheus adds scrape-target labels with the same names and
source labels are not honored, it renames the workload labels to `exported_*`.
HAMi-WebUI's queries then return no series, and task utilization is reported as
`0`.

The included ServiceMonitor sets `hamiServiceMonitor.honorLabels: true`. If an
external Prometheus discovers this ServiceMonitor, no additional scrape job is
needed. For a separately managed scrape, set `honorLabels: true` on its
ServiceMonitor or use the equivalent raw Prometheus setting:

```yaml
scrape_configs:
- job_name: hami-device-plugin-monitor
honor_labels: true
# ...
```

`Prometheus.spec.overrideHonorLabels: true` forces
`honorLabels` off for every ServiceMonitor, including this one.

If the same Prometheus selects both this chart's ServiceMonitor and HAMi's
built-in device-plugin ServiceMonitor (`prometheus.enabled` in the HAMi chart),
the endpoint is scraped twice. Configure that Prometheus to select only one.

### Access HAMi-WebUI

1. Configure ~/.kube/config in your localhost to be able to connect your cluster.
Expand Down Expand Up @@ -110,4 +137,4 @@ If you want to delete the namespace `hami`, then run the command:

```bash
kubectl delete namespace hami
```
```
Loading