Skip to content

feat: add device custom fields#6248

Merged
gustavosbarreto merged 11 commits into
shellhub-io:masterfrom
danielgatis:feat/device-custom-fields
Apr 30, 2026
Merged

feat: add device custom fields#6248
gustavosbarreto merged 11 commits into
shellhub-io:masterfrom
danielgatis:feat/device-custom-fields

Conversation

@danielgatis

@danielgatis danielgatis commented Apr 28, 2026

Copy link
Copy Markdown
Contributor

Motivation

As device fleets grow, operators often need to attach context-specific metadata to individual devices — things like physical location, environment tags, asset identifiers, or team ownership. Without a dedicated mechanism, this information ends up scattered across external spreadsheets or encoded into hostnames, making it hard to query and maintain.

This feature introduces custom fields: a simple key-value map attached to each device. Operators can freely define any metadata they need, and that metadata becomes a first-class citizen in the dashboard — visible in the device list, searchable alongside the hostname, and fully manageable from the device detail page without touching the API directly.

Screen.Recording.2026-04-28.at.8.17.06.AM.mov

Summary

  • Add custom_fields map[string]string to the Device model, stored as JSONB in PostgreSQL and an embedded document in MongoDB
  • Expose via the existing PUT /devices/{uid} endpoint — nil means no change, empty map clears all fields
  • Search in the device list also queries custom field values (OR with hostname search)
  • Display and manage custom fields in the device detail page (add, delete with confirmation)
  • Add custom fields column in the devices list table

Changes

Scope Description
pkg Add CustomFields to Device model and DeviceUpdate request DTO
api Service logic, MongoDB $objectToArray filter, PostgreSQL JSONB filter, migration 002
openapi Add custom_fields to device schema and PUT request body
docker Add binutils-gold to Alpine images for ARM64 linker support (see note below)
ui (Vue) Display custom fields on device detail page
ui-react Custom fields column in list, management UI in detail page, search integration
test(api) Unit tests for PG/Mongo filters and service UpdateDevice cases
test(ui-react) Tests for devices list and device details custom fields behavior

Note: binutils-gold in Alpine Dockerfiles

During local development on an ARM64 machine (Apple Silicon), the Docker builds were failing with:

/usr/lib/gcc/x86_64-alpine-linux-musl/14.2.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find 'ld'

The root cause is that golangci-lint is compiled with -fuse-ld=gold, which requires the gold linker (binutils-gold package) to be present on the system. Alpine's default binutils does not include it. Adding binutils-gold to the Alpine images that install golangci-lint resolves the build failure on ARM64 hosts.

Test plan

  • go test ./api/services/... ./api/store/pg/internal/... ./api/store/mongo/internal/...
  • cd ui-react/apps/console && npx vitest run src/pages/devices/__tests__
  • PUT /devices/{uid} with {"name": "x", "custom_fields": {"env": "prod"}} → 200
  • GET /devices/{uid} → response includes custom_fields
  • Search on /devices by custom field value returns matching devices
  • Device detail page shows custom fields with add/delete UI

Add `CustomFields map[string]string` to the Device model (BSON/JSON tagged)
and a `*map[string]string` pointer field to DeviceUpdate so that omitting the
field in a PUT payload leaves existing values untouched.
- Service: apply CustomFields from request when non-nil; register
  "custom_fields" as a valid filter field with "contains" operator
- MongoDB: route custom_fields filter via ParseCustomFieldsFilter using
  \$objectToArray + \$regexMatch to search across all map values
- PostgreSQL entity: add JSONB column mapping to Device entity
- PostgreSQL filters: add fromCustomFieldsFilter that generates an
  EXISTS (jsonb_each_text) subquery for ILIKE value matching
- Migration 002: add custom_fields jsonb column with default '{}'
golangci-lint uses -fuse-ld=gold which requires binutils-gold on Alpine
ARM64. Without it the build fails with "cannot find 'ld'".
- Device list: new Custom Fields column showing key/value badges with
  pill styling; search also queries custom field values
- Device detail: CustomFieldsSection component with add form, inline
  delete confirmation, and useUpdateDeviceCustomFields mutation
- useDevices: extend buildFilter to OR-search across custom_fields values
- useDeviceMutations: export useUpdateDeviceCustomFields hook
@danielgatis danielgatis requested review from a team as code owners April 28, 2026 11:39
- pg/internal: TestFromCustomFieldsFilter and
  TestParseFilterProperty_CustomFields
- mongo/internal: TestParseCustomFieldsFilter covering contains,
  unsupported operators, and non-string value error
- services: extend TestDeviceUpdate with cases for setting, clearing,
  and nil (no-op) custom fields
…fields

- Devices list: rendering, loading, empty state, row navigation, custom
  fields column (empty, single field, multiple fields), error state
- DeviceDetails: loading spinner, device data rendering, custom fields
  section display, add form, delete confirmation flow, mutation calls,
  duplicate key validation
The search filter now ORs across both name and custom_fields. Update
existing test expectations accordingly.
@danielgatis danielgatis force-pushed the feat/device-custom-fields branch from d361b29 to 8473bdf Compare April 28, 2026 15:43
@gustavosbarreto

Copy link
Copy Markdown
Member

/review

@github-actions

Copy link
Copy Markdown

Claude Code is working…

I'll analyze this and get back to you.

View job run

@gustavosbarreto

Copy link
Copy Markdown
Member

/review

@github-actions

github-actions Bot commented Apr 28, 2026

Copy link
Copy Markdown

Claude encountered an error —— View job


I'll analyze this and get back to you.

@github-actions

github-actions Bot commented Apr 28, 2026

Copy link
Copy Markdown

Claude finished @gustavosbarreto's task in 5m 29s —— View job


Code Review Complete

Reviewed 30 files across code quality, security, testing, language patterns, and architecture. 3 inline issues posted (1 critical, 1 high, 1 medium).

Severity File Issue
Critical api/store/pg/entity/device.go:34 nil CustomFields map sends SQL NULL on INSERT, violating the NOT NULL constraint — breaks device creation
High pkg/models/device.go:52 bson:"omitempty" silently prevents clearing custom fields via empty map on MongoDB
Medium pkg/api/requests/device.go:20 No validation limits on custom_fields (key count, key/value length) — unlike tags which have strict bounds

Additional notes

  • Cross-repo impact (cloud/): Clean — DeviceToModel and DeviceFromModel changes are backward-compatible. No cloud code changes needed.
  • Pre-existing patterns preserved: The unescaped regex in ParseCustomFieldsFilter and the hardcoded "device" table alias in fromCustomFieldsFilter follow existing conventions in fromContains and fromTagsFilter respectively — not flagged since they're consistent with the codebase.

If you've addressed the feedback and want a new review, tag @shellhub-io/admin and a team member can trigger it.

Comment thread api/store/pg/entity/device.go Outdated
Comment thread pkg/models/device.go Outdated
Comment thread pkg/api/requests/device.go Outdated
- pg entity: add nullzero,default:'{}' to CustomFields bun tag so that
  nil maps are omitted from INSERT (letting the column DEFAULT apply)
  instead of sending SQL NULL and violating the NOT NULL constraint
- model: remove omitempty from bson tag on CustomFields so that an
  empty map is included in MongoDB \$set operations, allowing callers
  to clear all fields by passing an empty object
- request: add validation limits to CustomFields (max 20 entries, keys
  1-64 chars, values max 256 chars) to prevent storage abuse
@otavio

otavio commented Apr 29, 2026

Copy link
Copy Markdown
Member

/review

@github-actions

github-actions Bot commented Apr 29, 2026

Copy link
Copy Markdown

Claude finished @otavio's task in 9m 34s —— View job


Code Review Complete

Reviewed 30 files across code quality, security, testing, language patterns, and architecture — no new issues found.

Previous review issues — all addressed

The 3 issues from the prior review have been fixed in commit 3785db0:

Issue Status
Critical — nil CustomFields map sends SQL NULL on INSERT Fixed: bun:"custom_fields,type:jsonb,nullzero,default:'{}'"
Highbson:"omitempty" prevents clearing custom fields via empty map Fixed: bson:"custom_fields" (omitempty removed)
Medium — No validation limits on custom_fields size Fixed: validate:"omitempty,max=20,dive,keys,min=1,max=64,endkeys,max=256"

Verification notes

  • Cross-repo impact (cloud/): Clean — DeviceFromModel/DeviceToModel changes are backward-compatible; cloud/ uses named-field struct literals, so the new field defaults to nil. No build or runtime impact.
  • API contract: Additive-only change to PUT /devices/{uid}. The *map[string]string pointer type with nil-check in the service layer ensures existing clients omitting custom_fields are unaffected.
  • Migration: 002_device_custom_fields is the correct sequence number following 001_initial_schema.
  • Security: PG filter uses parameterized ? placeholders (no SQL injection). MongoDB regex pattern in ParseCustomFieldsFilter follows the pre-existing convention from fromContains. UI templates use text interpolation (no dangerouslySetInnerHTML or v-html).
  • Filter OR semantics: The buildFilter function's leading OR operators are consistent with the pre-existing default (currentOperator := "OR" at pg/query-options.go:86). The tag filter was already OR'd before this change.

If you've addressed the feedback and want a new review, tag @shellhub-io/admin and a team member can trigger it.

@danielgatis

danielgatis commented Apr 29, 2026

Copy link
Copy Markdown
Contributor Author

@gustavosbarreto

When you have some free time, please consider doing another review.

@gustavosbarreto

Copy link
Copy Markdown
Member

/review

@github-actions

github-actions Bot commented Apr 29, 2026

Copy link
Copy Markdown

Claude finished @gustavosbarreto's task in 13m 12s —— View job


Code Review Complete

Reviewed 30 files across code quality, security, testing, language patterns, and architecture. 2 inline issues posted (2 high).

Severity File Issue
High ui-react/apps/console/src/pages/DeviceDetails.tsx:276 CustomFieldsSection renders add/remove controls for all users — no useHasPermission gate (unlike TagsSection and RenameSection)
High ui-react/apps/console/src/pages/DeviceDetails.tsx:303 Custom fields mutation includes cached name in body, silently reverting external device renames

Previous review issues — all addressed

The 3 issues from the prior review remain fixed in commit 3785db0:

Issue Status
Critical — nil CustomFields map sends SQL NULL on INSERT Fixed: nullzero,default:'{}'
Highbson:"omitempty" prevents clearing custom fields Fixed: bson:"custom_fields"
Medium — No validation limits on custom_fields Fixed: max=20,dive,keys,min=1,max=64,endkeys,max=256

Verification notes

  • Cross-repo impact (cloud/): Clean — all models.Device{} and coreentity.Device{} literals in cloud/ use named fields. No build or runtime impact.
  • Security: PG filter uses parameterized ? placeholders (no SQL injection). MongoDB $regexMatch follows the pre-existing fromContains convention. UI uses JSX/Vue text interpolation (no XSS). Input validation enforces size limits.
  • API contract: Additive-only change to PUT /devices/{uid}. The *map[string]string pointer type with nil-check ensures existing clients are unaffected.
  • Migration: 002_device_custom_fields is correctly sequenced after 001_initial_schema.

If you've addressed the feedback and want a new review, tag @shellhub-io/admin and a team member can trigger it.

Comment thread ui-react/apps/console/src/pages/DeviceDetails.tsx
Comment thread ui-react/apps/console/src/pages/DeviceDetails.tsx
- Remove name prop from CustomFieldsSection (was causing silent device
  rename when custom fields were mutated due to stale closure value)
- Send name: "" in mutation body so only custom_fields are changed
- Guard remove buttons and add form behind useHasPermission("device:rename")
@gustavosbarreto

Copy link
Copy Markdown
Member

/review

@github-actions

github-actions Bot commented Apr 29, 2026

Copy link
Copy Markdown

Claude finished @gustavosbarreto's task in 1m 22s —— View job


Code Review Complete

Reviewed 30 files across code quality, security, testing, language patterns, and architecture — no new issues found.

Previous review issues — all addressed

The 5 issues from prior reviews have been fixed in commits 3785db0 and 0175231:

Issue Status
Critical — nil CustomFields map sends SQL NULL on INSERT Fixed: nullzero,default:'{}'
Highbson:"omitempty" prevents clearing custom fields Fixed: bson:"custom_fields"
Medium — No validation limits on custom_fields Fixed: max=20,dive,keys,min=1,max=64,endkeys,max=256
HighCustomFieldsSection missing permission gate Fixed: useHasPermission("device:rename")
High — Custom fields mutation silently reverts external renames Fixed: sends name: ""

Verification notes

  • Cross-repo impact (cloud/): Clean — all models.Device{} and entity.Device{} literals in cloud/ use named fields. No build or runtime impact.
  • Security: PG filter uses parameterized ? placeholders (no SQL injection). MongoDB $regexMatch follows the pre-existing fromContains convention. UI uses JSX text interpolation (no XSS). Input validation enforces size limits (max 20 fields, key 1–64 chars, value max 256 chars).
  • API contract: Additive-only change to PUT /devices/{uid}. The *map[string]string pointer type with nil-check ensures existing clients are unaffected.
  • Migration: 002_device_custom_fields is correctly sequenced after 001_initial_schema, uses IF NOT EXISTS/IF EXISTS for idempotency.
  • Filter integration: custom_fields filter is properly isolated with its own dispatch before the generic operator switch, cannot interfere with existing filter behavior.
  • Tests: Service tests cover set/clear/nil paths. PG and Mongo filter tests cover supported and unsupported operators, type validation, and structural verification. UI tests cover rendering, add/remove with confirmation, duplicate key detection, and column display.

If you've addressed the feedback and want a new review, tag @shellhub-io/admin and a team member can trigger it.

@danielgatis

Copy link
Copy Markdown
Contributor Author

@gustavosbarreto

🎉 looks like the PR is good now

Any chance to get it released in earlier future?

@gustavosbarreto

Copy link
Copy Markdown
Member

Thanks! It'll go out in the next release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants