diff --git a/docs/ingest/telemetry/index.md b/docs/ingest/telemetry/index.md index eb9e1aff..18682b34 100644 --- a/docs/ingest/telemetry/index.md +++ b/docs/ingest/telemetry/index.md @@ -2,7 +2,7 @@ (metrics-store)= (telemetry)= (integrate-metrics)= -# Metrics and telemetry data +# Metrics, telemetry, and logs :::::{grid} :padding: 0 @@ -59,6 +59,12 @@ Prometheus is an open-source systems monitoring and alerting toolkit for collecting metrics data from applications and infrastructures. :::: +::::{grid-item-card} rsyslog +:link: rsyslog +:link-type: ref +Send logs with rsyslog, a rocket‑fast system for log processing. +:::: + ::::{grid-item-card} Telegraf :link: telegraf :link-type: ref diff --git a/docs/integrate/index.md b/docs/integrate/index.md index 7c11fc4c..b649aab4 100644 --- a/docs/integrate/index.md +++ b/docs/integrate/index.md @@ -66,6 +66,7 @@ queryzen/index r/index rill/index risingwave/index +rsyslog/index scikit-learn/index sql-server/index streamlit/index diff --git a/docs/integrate/rsyslog/index.md b/docs/integrate/rsyslog/index.md new file mode 100644 index 00000000..76fea489 --- /dev/null +++ b/docs/integrate/rsyslog/index.md @@ -0,0 +1,45 @@ +(rsyslog)= +# rsyslog + +```{div} .float-right +[![rsyslog logo](https://www.rsyslog.com/files/2019/01/logo_neu_cropped.png){height=60px loading=lazy}][rsyslog] +``` +```{div} .clearfix +``` + + +:::{rubric} About +::: + +[Rsyslog] is a rocket-fast system for log processing. + +It offers high performance, advanced security features, and a modular design. +Originally a regular syslogd, rsyslog has evolved into a highly versatile +logging solution capable of ingesting data from numerous sources, +transforming it, and outputting it to a wide variety of destinations. + +Rsyslog can deliver over one million messages per second to local +destinations under minimal processing load. Even with complex routing +and remote forwarding, performance remains excellent. + +:::{rubric} Learn +::: + +::::{grid} 2 + +:::{grid-item-card} Tutorial: Store server logs in CrateDB using rsyslog +:link: rsyslog-tutorial +:link-type: ref +Storing server logs in CrateDB delivers fast search and aggregations on them. +::: + +:::: + +:::{toctree} +:maxdepth: 1 +:hidden: +Tutorial +::: + + +[rsyslog]: https://www.rsyslog.com/ diff --git a/docs/integrate/rsyslog/tutorial.md b/docs/integrate/rsyslog/tutorial.md new file mode 100644 index 00000000..f9382d64 --- /dev/null +++ b/docs/integrate/rsyslog/tutorial.md @@ -0,0 +1,145 @@ +(rsyslog-tutorial)= +# Store server logs on CrateDB for fast search and aggregations + +## Introduction + +CrateDB stores server logs efficiently and makes them easy to query. + +Common pain points with traditional log stacks and SIEMs include: + +* timeouts when searching across long time ranges +* proprietary, complex query syntaxes +* awkward integrations with application monitoring dashboards + +CrateDB addresses these issues: query logs with standard SQL from any +PostgreSQL‑compatible tool, and use full‑text search and aggregations +backed by efficient indexes. The sections below walk through a minimal +setup. + +## Setup + +### CrateDB + +First, start CrateDB. For production, use a dedicated cluster. For this demo, run a single‑node container: + +```bash +sudo docker run -d --name cratedb \ + -p 4200:4200 -p 5432:5432 \ + -e CRATE_HEAP_SIZE=1g \ + crate:latest -Cdiscovery.type=single-node +``` + +Next, create a table for logs. Open `http://localhost:4200/#!/console` or invoke `crash` and run: + +```sql +CREATE TABLE doc.systemevents ( + message TEXT, + INDEX message_ft USING FULLTEXT(message) WITH (analyzer = 'english'), + facility INTEGER, + fromhost TEXT, + priority INTEGER, + DeviceReportedTime TIMESTAMP, + ReceivedAt TIMESTAMP, + InfoUnitID INTEGER, + SysLogTag TEXT +); +``` +Tip: On headless systems, run queries with the {ref}`command-line tools `. + +Then we need an account for the logging system: + +```sql +-- Use a strong secret; e.g. from a secret manager or env var. +CREATE USER rsyslog WITH (PASSWORD='pwd123'); +``` + +and we need to grant permissions on the table above: + +```sql +GRANT DML ON TABLE doc.systemevents TO rsyslog; +``` + +### rsyslog + +We will use [rsyslog](https://github.com/rsyslog/rsyslog) to send the logs to CrateDB, for this setup we need `rsyslog` v8.2202 or higher and the `ompgsql` module: + +```bash +sudo DEBIAN_FRONTEND=noninteractive apt install --yes software-properties-common +sudo add-apt-repository -y ppa:adiscon/v8-stable +sudo apt update --yes +sudo debconf-set-selections <<< 'rsyslog-pgsql rsyslog-pgsql/dbconfig-install string false' +sudo apt install --yes rsyslog rsyslog-pgsql +``` + +Let's now configure it to use the account we created earlier: + +```bash +echo 'module(load="ompgsql")' | sudo tee /etc/rsyslog.d/pgsql.conf +echo '*.* action(type="ompgsql" conninfo="postgresql://rsyslog:pwd123@localhost/doc")' | sudo tee -a /etc/rsyslog.d/pgsql.conf +sudo chmod 640 /etc/rsyslog.d/pgsql.conf +sudo systemctl restart rsyslog +``` + +If you are interested in more advanced setups involving queuing for additional reliability in production scenarios, you can read more about available settings in the [rsyslog documentation](https://www.rsyslog.com/doc/v8-stable/tutorials/high_database_rate.html). + +### MediaWiki + +To generate logs, run a [MediaWiki](https://www.mediawiki.org/wiki/MediaWiki) container and forward its logs to rsyslog: + +```bash +sudo docker run --name mediawiki \ + -p 80:80 -d \ + --log-driver syslog \ + --log-opt syslog-address=unixgram:///dev/log \ + mediawiki +``` + +Open `http://localhost/` to see the MediaWiki setup page. +Click “set up the wiki”, then “Continue” to generate log entries. +CrateDB now stores new rows in `doc.systemevents`, with `syslogtag` matching the container ID. + + +## Explore + +Use {ref}`crate-reference:predicates_match` to find specific error messages: + +```sql +SELECT devicereportedtime,message +FROM doc.systemevents +WHERE MATCH(message_ft, 'Could not reliably determine') USING PHRASE +ORDER BY 1 DESC; +``` + +```text ++--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| devicereportedtime | message | ++--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 1691510710000 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.3. Set the 'ServerName' directive globally to suppress this message | +| 1691510710000 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.3. Set the 'ServerName' directive globally to suppress this message | ++--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +``` + +Show the top log sources by event count: + +```sql +SELECT syslogtag,count(*) +FROM doc.systemevents +GROUP BY 1 +ORDER BY 2 DESC +LIMIT 5; +``` + +```text ++----------------------+----------+ +| syslogtag | count(*) | ++----------------------+----------+ +| kernel: | 23 | +| 083053ae8ea3[52134]: | 20 | +| systemd[1]: | 15 | +| sudo: | 10 | +| rsyslogd: | 5 | ++----------------------+----------+ +``` + +We hope this was useful. Share feedback and questions in the +[CrateDB Community](https://community.cratedb.com/).