Skip to content

Commit 81d1441

Browse files
ruflinruflin
committed
feat: add custom Kibana configuration support
- Add support for custom kibana-custom.yml configuration files - Custom configs are appended to base kibana.yml configuration - Enable via stack.kibana_custom_config_enabled profile setting - Support template variables in custom configurations - Add comprehensive documentation and examples - Maintain backward compatibility with existing setups Resolves: Custom Kibana configuration enhancement
1 parent d2229e4 commit 81d1441

File tree

5 files changed

+336
-8
lines changed

5 files changed

+336
-8
lines changed

‎README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -689,6 +689,10 @@ The following settings are available per profile:
689689
* `stack.elastic_subscription` allows to select the Elastic subscription type to be used in the stack.
690690
Currently, it is supported "basic" and "[trial](https://www.elastic.co/guide/en/elasticsearch/reference/current/start-trial.html)",
691691
which enables all subscription features for 30 days. Defaults to "trial".
692+
* `stack.kibana_custom_config_enabled` can be set to true to enable custom Kibana configuration.
693+
When enabled, you can create a `kibana-custom.yml` file in your profile directory with additional
694+
Kibana settings that will be appended to the base configuration. Defaults to false.
695+
See [Custom Kibana Configuration](docs/howto/custom_kibana_config.md) for details.
692696

693697
## Useful environment variables
694698

‎docs/howto/custom_kibana_config.md‎

Lines changed: 191 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,191 @@
1+
# Custom Kibana Configuration
2+
3+
This document explains how to add custom configuration to Kibana when using elastic-package.
4+
5+
## Overview
6+
7+
You can provide additional Kibana configuration that will be appended to the base configuration generated by elastic-package. This allows you to:
8+
9+
- Override default settings
10+
- Add custom plugins configuration
11+
- Set up custom security settings
12+
- Configure additional features
13+
14+
## Setup
15+
16+
1. Enable custom configuration in your profile:
17+
```bash
18+
# Edit ~/.elastic-package/profiles/default/config.yml
19+
stack.kibana_custom_config_enabled: true
20+
```
21+
22+
2. Create your custom configuration file:
23+
```bash
24+
# Create ~/.elastic-package/profiles/default/kibana-custom.yml
25+
touch ~/.elastic-package/profiles/default/kibana-custom.yml
26+
```
27+
28+
3. Add your custom configuration:
29+
```yaml
30+
# Example custom configuration
31+
logging.loggers:
32+
- name: plugins.security
33+
level: debug
34+
35+
server.customResponseHeaders:
36+
X-Custom-Header: "MyValue"
37+
38+
# Template variables are supported
39+
xpack.security.enabled: {{ if eq .security_enabled "true" }}true{{ else }}false{{ end }}
40+
```
41+
42+
## Template Support
43+
44+
Your custom configuration supports the same templating as the base configuration:
45+
46+
- `{{ fact "kibana_version" }}` - Kibana version
47+
- `{{ fact "username" }}` - Elasticsearch username
48+
- `{{ fact "password" }}` - Elasticsearch password
49+
- `{{ fact "apm_enabled" }}` - APM enabled flag
50+
- `{{ fact "self_monitor_enabled" }}` - Self monitoring enabled flag
51+
- All other facts available in the base template
52+
53+
### Available Template Variables
54+
55+
The following template variables are available in your custom configuration:
56+
57+
| Variable | Description | Example |
58+
|----------|-------------|---------|
59+
| `kibana_version` | Version of Kibana | `8.11.0` |
60+
| `elasticsearch_version` | Version of Elasticsearch | `8.11.0` |
61+
| `agent_version` | Version of Elastic Agent | `8.11.0` |
62+
| `username` | Elasticsearch username | `elastic` |
63+
| `password` | Elasticsearch password | `changeme` |
64+
| `kibana_host` | Kibana host URL | `https://kibana:5601` |
65+
| `elasticsearch_host` | Elasticsearch host URL | `https://elasticsearch:9200` |
66+
| `fleet_url` | Fleet server URL | `https://fleet-server:8220` |
67+
| `apm_enabled` | APM enabled flag | `true`/`false` |
68+
| `logstash_enabled` | Logstash enabled flag | `true`/`false` |
69+
| `self_monitor_enabled` | Self monitoring flag | `true`/`false` |
70+
71+
## Configuration Precedence
72+
73+
1. Base elastic-package configuration (from template)
74+
2. Your custom configuration (appended)
75+
76+
Since YAML allows duplicate keys and Kibana processes them in order, your custom settings will override base settings with the same key.
77+
78+
## Examples
79+
80+
### Enable Debug Logging
81+
```yaml
82+
logging.loggers:
83+
- name: root
84+
level: debug
85+
- name: plugins.fleet
86+
level: trace
87+
```
88+
89+
### Custom Security Settings
90+
```yaml
91+
xpack.security.session.idleTimeout: "8h"
92+
xpack.security.session.lifespan: "24h"
93+
xpack.security.authc.providers:
94+
basic.basic1:
95+
order: 0
96+
```
97+
98+
### Development Features
99+
```yaml
100+
server.rewriteBasePath: false
101+
server.dev.basePathProxyTarget: 3000
102+
103+
# Enable experimental features
104+
xpack.fleet.enableExperimental:
105+
- customIntegrations
106+
- agentTamperProtectionEnabled
107+
```
108+
109+
### Custom UI Settings
110+
```yaml
111+
uiSettings:
112+
overrides:
113+
"theme:darkMode": true
114+
"dateFormat": "YYYY-MM-DD HH:mm:ss.SSS"
115+
"discover:sampleSize": 1000
116+
```
117+
118+
### Template Usage Example
119+
```yaml
120+
# Use template variables in your custom config
121+
server.name: kibana-{{ fact "kibana_version" }}
122+
server.customResponseHeaders:
123+
X-Kibana-Version: "{{ fact "kibana_version" }}"
124+
125+
# Conditional configuration based on features
126+
{{ if eq (fact "apm_enabled") "true" }}
127+
elastic.apm.active: true
128+
elastic.apm.serverUrl: "http://fleet-server:8200"
129+
elastic.apm.environment: "development"
130+
{{ end }}
131+
132+
# Version-specific configuration
133+
{{ if semverLessThan (fact "kibana_version") "8.0.0" }}
134+
# Legacy configuration for older versions
135+
xpack.monitoring.ui.container.elasticsearch.enabled: true
136+
{{ else }}
137+
# Modern configuration for newer versions
138+
monitoring.ui.container.elasticsearch.enabled: true
139+
{{ end }}
140+
```
141+
142+
## Troubleshooting
143+
144+
### Configuration Not Applied
145+
- Ensure `stack.kibana_custom_config_enabled: true` is set in your profile's `config.yml`
146+
- Verify the `kibana-custom.yml` file exists in your profile directory
147+
- Check that the YAML syntax is valid
148+
149+
### Template Errors
150+
- Use `{{ fact "variable_name" }}` syntax for template variables
151+
- Ensure template functions like `semverLessThan` are used correctly
152+
- Check the elastic-package logs for template parsing errors
153+
154+
### Kibana Startup Issues
155+
- Validate your custom configuration against Kibana documentation
156+
- Start with minimal changes and add complexity gradually
157+
- Check Kibana logs for configuration validation errors
158+
159+
## Profile-Specific Configurations
160+
161+
You can have different custom configurations for different profiles:
162+
163+
```bash
164+
# Development profile with debug settings
165+
~/.elastic-package/profiles/development/kibana-custom.yml
166+
167+
# Production profile with optimized settings
168+
~/.elastic-package/profiles/production/kibana-custom.yml
169+
```
170+
171+
Each profile can enable or disable custom configuration independently.
172+
173+
## Best Practices
174+
175+
1. **Start Simple**: Begin with basic configuration changes and add complexity gradually
176+
2. **Use Comments**: Document your custom configurations for future reference
177+
3. **Test Changes**: Verify that Kibana starts successfully after configuration changes
178+
4. **Version Compatibility**: Use template conditions for version-specific settings
179+
5. **Backup Configurations**: Keep copies of working configurations before making changes
180+
181+
## Limitations
182+
183+
- Custom configuration is appended to the base configuration (not merged at the YAML structure level)
184+
- Template processing errors will prevent stack startup
185+
- Some Kibana settings may require specific ordering or dependencies
186+
187+
## Related Documentation
188+
189+
- [Kibana Configuration Settings](https://www.elastic.co/guide/en/kibana/current/settings.html)
190+
- [elastic-package Profiles](../README.md#profiles)
191+
- [Stack Configuration](../README.md#stack-configuration)

‎internal/profile/_static/config.yml.example‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,3 +29,7 @@
2929

3030
## Set license subscription
3131
# stack.elastic_subscription: "basic"
32+
33+
## Custom Kibana Configuration
34+
# Enable custom kibana.yml configuration file
35+
# stack.kibana_custom_config_enabled: true

‎internal/stack/kibana_config.go‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
// Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
2+
// or more contributor license agreements. Licensed under the Elastic License;
3+
// you may not use this file except in compliance with the Elastic License.
4+
5+
package stack
6+
7+
import (
8+
"bytes"
9+
"fmt"
10+
"html/template"
11+
"io"
12+
"os"
13+
14+
"github.com/elastic/go-resource"
15+
16+
"github.com/elastic/elastic-package/internal/profile"
17+
)
18+
19+
// kibanaConfigWithCustomContent generates kibana.yml with custom config appended
20+
func kibanaConfigWithCustomContent(profile *profile.Profile) func(resource.Context, io.Writer) error {
21+
return func(ctx resource.Context, w io.Writer) error {
22+
// First, generate the base kibana.yml from template
23+
var baseConfig bytes.Buffer
24+
baseTemplate := staticSource.Template("_static/kibana.yml.tmpl")
25+
err := baseTemplate(ctx, &baseConfig)
26+
if err != nil {
27+
return fmt.Errorf("failed to generate base kibana config: %w", err)
28+
}
29+
30+
// Write base config to output
31+
_, err = w.Write(baseConfig.Bytes())
32+
if err != nil {
33+
return fmt.Errorf("failed to write base kibana config: %w", err)
34+
}
35+
36+
// Check if custom config is enabled and exists
37+
if profile.Config(configKibanaCustomConfigEnabled, "false") == "false" {
38+
return nil // No custom config needed
39+
}
40+
41+
customConfigPath := profile.Path(KibanaCustomConfigFile)
42+
customConfigData, err := os.ReadFile(customConfigPath)
43+
if os.IsNotExist(err) {
44+
return nil // No custom config file, that's fine
45+
}
46+
if err != nil {
47+
return fmt.Errorf("failed to read custom kibana config: %w", err)
48+
}
49+
50+
// Add separator comment
51+
_, err = w.Write([]byte("\n\n# Custom Kibana Configuration\n"))
52+
if err != nil {
53+
return fmt.Errorf("failed to write custom config separator: %w", err)
54+
}
55+
56+
// Process custom config as template
57+
customTemplate, err := template.New("kibana-custom").
58+
Funcs(templateFuncs).
59+
Parse(string(customConfigData))
60+
if err != nil {
61+
return fmt.Errorf("failed to parse custom kibana config template: %w", err)
62+
}
63+
64+
// Create template data from resource context facts
65+
templateData := createTemplateDataFromContext(ctx)
66+
67+
err = customTemplate.Execute(w, templateData)
68+
if err != nil {
69+
return fmt.Errorf("failed to execute custom kibana config template: %w", err)
70+
}
71+
72+
return nil
73+
}
74+
}
75+
76+
// createTemplateDataFromContext creates template data from resource context
77+
// This function extracts commonly used facts and makes them available to templates
78+
func createTemplateDataFromContext(ctx resource.Context) map[string]interface{} {
79+
data := make(map[string]interface{})
80+
81+
// List of facts that should be available in custom templates
82+
factNames := []string{
83+
"kibana_version",
84+
"elasticsearch_version",
85+
"agent_version",
86+
"username",
87+
"password",
88+
"kibana_host",
89+
"elasticsearch_host",
90+
"fleet_url",
91+
"apm_enabled",
92+
"logstash_enabled",
93+
"self_monitor_enabled",
94+
"kibana_http2_enabled",
95+
"logsdb_enabled",
96+
"elastic_subscription",
97+
"geoip_dir",
98+
"agent_publish_ports",
99+
"api_key",
100+
"enrollment_token",
101+
}
102+
103+
// Extract facts from context
104+
for _, factName := range factNames {
105+
if value, found := ctx.Fact(factName); found {
106+
data[factName] = value
107+
}
108+
}
109+
110+
return data
111+
}

‎internal/stack/resources.go‎

Lines changed: 26 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,9 @@ const (
3434
// KibanaConfigFile is the kibana config file.
3535
KibanaConfigFile = "kibana.yml"
3636

37+
// KibanaCustomConfigFile is the custom kibana config file.
38+
KibanaCustomConfigFile = "kibana-custom.yml"
39+
3740
// LogstashConfigFile is the logstash config file.
3841
LogstashConfigFile = "logstash.conf"
3942

@@ -58,13 +61,14 @@ const (
5861
elasticsearchUsername = "elastic"
5962
elasticsearchPassword = "changeme"
6063

61-
configAPMEnabled = "stack.apm_enabled"
62-
configGeoIPDir = "stack.geoip_dir"
63-
configKibanaHTTP2Enabled = "stack.kibana_http2_enabled"
64-
configLogsDBEnabled = "stack.logsdb_enabled"
65-
configLogstashEnabled = "stack.logstash_enabled"
66-
configSelfMonitorEnabled = "stack.self_monitor_enabled"
67-
configElasticSubscription = "stack.elastic_subscription"
64+
configAPMEnabled = "stack.apm_enabled"
65+
configGeoIPDir = "stack.geoip_dir"
66+
configKibanaHTTP2Enabled = "stack.kibana_http2_enabled"
67+
configKibanaCustomConfigEnabled = "stack.kibana_custom_config_enabled"
68+
configLogsDBEnabled = "stack.logsdb_enabled"
69+
configLogstashEnabled = "stack.logstash_enabled"
70+
configSelfMonitorEnabled = "stack.self_monitor_enabled"
71+
configElasticSubscription = "stack.elastic_subscription"
6872
)
6973

7074
var (
@@ -189,7 +193,21 @@ func applyResources(profile *profile.Profile, stackVersion string) error {
189193
resourceManager.RegisterProvider("file", &resource.FileProvider{
190194
Prefix: stackDir,
191195
})
192-
resources := append([]resource.Resource{}, stackResources...)
196+
// Create kibana resource with custom config support
197+
kibanaResource := &resource.File{
198+
Path: KibanaConfigFile,
199+
Content: kibanaConfigWithCustomContent(profile),
200+
}
201+
202+
// Replace the kibana resource in stackResources with our custom one
203+
resources := make([]resource.Resource, 0, len(stackResources))
204+
for _, res := range stackResources {
205+
if file, ok := res.(*resource.File); ok && file.Path == KibanaConfigFile {
206+
resources = append(resources, kibanaResource)
207+
} else {
208+
resources = append(resources, res)
209+
}
210+
}
193211

194212
// Keeping certificates in the profile directory for backwards compatibility reasons.
195213
resourceManager.RegisterProvider(CertsFolder, &resource.FileProvider{

0 commit comments

Comments
 (0)