Skip to content

feat: add support for extra_files configuration in ethereum-package#11

Merged
mattevans merged 5 commits intomasterfrom
feat/extra_files
Aug 14, 2025
Merged

feat: add support for extra_files configuration in ethereum-package#11
mattevans merged 5 commits intomasterfrom
feat/extra_files

Conversation

@mattevans
Copy link
Member

@mattevans mattevans commented Aug 14, 2025

Depends on: ethpandaops/ethereum-package#1144 (tests will fail until then). Now merged.


This PR adds support for the extra_files feature, allowing users to programmatically define file contents that will be mounted into containers at runtime.

Example

  // Create helper for convenient file management
  helper := config.NewExtraFilesHelper()

  // Add inline configuration
  helper.AddFile("beacon-config.yaml", `
  metrics:
    enabled: true
    port: 8080
  `)

  // Add from filesystem
  helper.AddFileFromPath("validator.yaml", "./configs/validator.yaml")

  // Add JSON configuration
  helper.AddJSON("features.json", map[string]interface{}{
      "metrics": true,
      "tracing": false,
  })

  // Create participant that references these files
  participant := config.NewParticipantBuilder().
      WithEL(client.Geth).
      WithCL(client.Lighthouse).
      WithCLExtraMounts(map[string]string{
          "/configs/beacon.yaml": "beacon-config.yaml",  // References extra_files entry
          "/configs/features.json": "features.json",
      }).
      Build()

  // Build configuration with extra files
  networkConfig, err := config.NewConfigBuilder().
      WithParticipant(participant).
      WithExtraFiles(helper.Build()).  // Extra files at root level
      Build()

Generated YAML

  extra_files:
    beacon-config.yaml: |
      metrics:
        enabled: true
        port: 8080
    features.json: |
      {"metrics": true, "tracing": false}

  participants:
    - el_type: geth
      cl_type: lighthouse
      cl_extra_mounts:
        /configs/beacon.yaml: beacon-config.yaml
        /configs/features.json: features.json

- Change DefaultPackageVersion to "feat/extra_files" to test new extra_files
 feature branch
- Introduce ExtraFiles field in EthereumPackageConfig for mounting additional
 configuration files into containers
- Add ExtraFilesHelper utility for convenient file management including
 inline content, JSON marshalling, and directory imports
- Provide comprehensive example in examples/extra_files demonstrating
 how to use extra_files for beacon and validator configuration
- Update ConfigBuilder with WithExtraFile(s) methods for fluent configuration
- Add validation to ensure extra file names are valid and don't contain
 path separators
Switch from temporary feat/extra_files branch to the official
release-please--branches--main--components--ethereum-package tag to
ensure reproducible and stable deployments.
@mattevans mattevans requested a review from samcm August 14, 2025 02:35
@mattevans mattevans self-assigned this Aug 14, 2025
Uncomment previously disabled configuration lines to allow
custom beacon, features and validator files to be injected into
the Lighthouse consensus client and validator client containers.
…of release-please branch

The release-please branch no longer exists, causing package pulls to fail.
Using "main" ensures we always track the latest stable commit.
* master:
  fix
  remove omit empty
  fix lint
  fix tests
  cleanup
  change order. rpc should appear on the end, since it will match quickly with engine-rpc
  add ws-rpc as fallback (used on erigon)
  fix bug where rpc-engine would appear as rpc endpoint
  add rpc fallback
  fixup tests
  go mod tidy
  wip
  wip
  wip
  wip
@mattevans mattevans merged commit f9d3030 into master Aug 14, 2025
3 checks passed
@mattevans mattevans deleted the feat/extra_files branch August 14, 2025 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants