Skip to content

Commit 9961a2d

Browse files
author
henryzhx
committed
update docs related to V2 config
1 parent c88668c commit 9961a2d

File tree

111 files changed

+1096
-974
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

111 files changed

+1096
-974
lines changed

.markdownlint.json

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"MD013": false,
3+
"MD033": false
4+
}

README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ Our official **User Manual** is located here:
7171

7272
[Configuration](https://ilogtail.gitbook.io/ilogtail-docs/configuration/collection-config)
7373

74-
[All Plugins](https://ilogtail.gitbook.io/ilogtail-docs/data-pipeline/overview)
74+
[All Plugins](https://ilogtail.gitbook.io/ilogtail-docs/plugins/overview)
7575

7676
[Getting Started](https://ilogtail.gitbook.io/ilogtail-docs/awesome-ilogtail/getting-started)
7777

docs/cn/SUMMARY.md

+66-66
Original file line numberDiff line numberDiff line change
@@ -26,79 +26,79 @@
2626
## 概念 <a href="#concepts" id="concepts"></a>
2727

2828
* [关键概念](concepts/key-concepts.md)
29-
* [数据流水线](concepts/data-pipeline.md)
3029

3130
## 配置 <a href="#configuration" id="configuration"></a>
3231

3332
* [采集配置](configuration/collection-config.md)
3433
* [系统参数](configuration/system-config.md)
3534
* [日志](configuration/logging.md)
3635

37-
## 数据流水线 <a href="#data-pipeline" id="data-pipeline"></a>
38-
39-
* [概览](data-pipeline/overview.md)
40-
* [插件版本管理](data-pipeline/stability-level.md)
41-
* [输入](data-pipeline/input/README.md)
42-
* [文本日志](data-pipeline/input/file-log.md)
43-
* [脚本执行数据](data-pipeline/input/input-command.md)
44-
* [容器标准输出](data-pipeline/input/service-docker-stdout.md)
45-
* [文本日志(debug)](data-pipeline/input/metric-debug-file.md)
46-
* [MetricInput示例插件](data-pipeline/input/metric-input-example.md)
47-
* [主机Meta数据](data-pipeline/input/metric-meta-host.md)
48-
* [Mock数据-Metric](data-pipeline/input/metric-mock.md)
49-
* [eBPF网络调用数据](data-pipeline/input/metric-observer.md)
50-
* [主机监控数据](data-pipeline/input/metric-system.md)
51-
* [MySQL Binlog](data-pipeline/input/service-canal.md)
52-
* [GO Profile](data-pipeline/input/service-goprofile.md)
53-
* [GPU数据](data-pipeline/input/service-gpu.md)
54-
* [HTTP数据](data-pipeline/input/service-http-server.md)
55-
* [ServiceInput示例插件](data-pipeline/input/service-input-example.md)
56-
* [Journal数据](data-pipeline/input/service-journal.md)
57-
* [Kafka](data-pipeline/input/service-kafka.md)
58-
* [Mock数据-Service](data-pipeline/input/service-mock.md)
59-
* [SqlServer 查询数据](data-pipeline/input/service-mssql.md)
60-
* [OTLP数据](data-pipeline/input/service-otlp.md)
61-
* [PostgreSQL 查询数据](data-pipeline/input/service-pgsql.md)
62-
* [Syslog数据](data-pipeline/input/service-syslog.md)
63-
* [处理](data-pipeline/processor/README.md)
64-
* [添加字段](data-pipeline/processor/processor-add-fields.md)
65-
* [添加云资产信息](data-pipeline/processor/processor-cloudmeta.md)
66-
* [原始数据](data-pipeline/processor/processor-default.md)
67-
* [数据脱敏](data-pipeline/processor/processor-desensitize.md)
68-
* [丢弃字段](data-pipeline/processor/processor-drop.md)
69-
* [字段加密](data-pipeline/processor/processor-encrypy.md)
70-
* [条件字段处理](data-pipeline/processor/processor-fields-with-condition.md)
71-
* [日志过滤](data-pipeline/processor/processor-filter-regex.md)
72-
* [Go时间格式解析](data-pipeline/processor/processor-gotime.md)
73-
* [Grok](data-pipeline/processor/processor-grok.md)
74-
* [Json](data-pipeline/processor/processor-json.md)
75-
* [日志转SLS Metric](data-pipeline/processor/processor-log-to-sls-metric.md)
76-
* [正则](data-pipeline/processor/processor-regex.md)
77-
* [重命名字段](data-pipeline/processor/processor-rename.md)
78-
* [分隔符](data-pipeline/processor/processor-delimiter.md)
79-
* [键值对](data-pipeline/processor/processor-split-key-value.md)
80-
* [多行切分](data-pipeline/processor/processor-split-log-regex.md)
81-
* [字符串替换](data-pipeline/processor/processor-string-replace.md)
82-
* [聚合](data-pipeline/aggregator/README.md)
83-
* [基础](data-pipeline/aggregator/aggregator-base.md)
84-
* [上下文](data-pipeline/aggregator/aggregator-context.md)
85-
* [按Key分组](data-pipeline/aggregator/aggregator-content-value-group.md)
86-
* [按GroupMetadata分组](data-pipeline/aggregator/aggregator-metadata-group.md)
87-
* [输出](data-pipeline/flusher/README.md)
88-
* [Kafka(Deprecated)](data-pipeline/flusher/flusher-kafka.md)
89-
* [kafkaV2](data-pipeline/flusher/flusher-kafka_v2.md)
90-
* [ClickHouse](data-pipeline/flusher/flusher-clickhouse.md)
91-
* [ElasticSearch](data-pipeline/flusher/flusher-elasticsearch.md)
92-
* [SLS](data-pipeline/flusher/flusher-sls.md)
93-
* [标准输出/文件](data-pipeline/flusher/flusher-stdout.md)
94-
* [OTLP日志](data-pipeline/flusher/flusher-otlp.md)
95-
* [Pulsar](data-pipeline/flusher/flusher-pulsar.md)
96-
* [HTTP](data-pipeline/flusher/flusher-http.md)
97-
* [Loki](data-pipeline/flusher/loki.md)
98-
* [加速](data-pipeline/accelerator/README.md)
99-
* [分隔符加速](data-pipeline/accelerator/delimiter-accelerate.md)
100-
* [Json加速](data-pipeline/accelerator/json-accelerate.md)
101-
* [正则加速](data-pipeline/accelerator/regex-accelerate.md)
36+
## 插件 <a href="#plugins" id="plugins"></a>
37+
38+
* [概览](plugins/overview.md)
39+
* [版本管理](plugins/stability-level.md)
40+
* [输入](plugins/input/README.md)
41+
* [文本日志](plugins/input/file-log.md)
42+
* [脚本执行数据](plugins/input/input-command.md)
43+
* [容器标准输出](plugins/input/service-docker-stdout.md)
44+
* [文本日志(debug)](plugins/input/metric-debug-file.md)
45+
* [MetricInput示例插件](plugins/input/metric-input-example.md)
46+
* [主机Meta数据](plugins/input/metric-meta-host.md)
47+
* [主机监控数据](plugins/input/metric-system.md)
48+
* [Mock数据-Metric](plugins/input/metric-mock.md)
49+
* [eBPF网络调用数据](plugins/input/metric-observer.md)
50+
* [主机监控数据](plugins/input/metric-system.md)
51+
* [MySQL Binlog](plugins/input/service-canal.md)
52+
* [GO Profile](plugins/input/service-goprofile.md)
53+
* [GPU数据](plugins/input/service-gpu.md)
54+
* [HTTP数据](plugins/input/service-http-server.md)
55+
* [ServiceInput示例插件](plugins/input/service-input-example.md)
56+
* [Journal数据](plugins/input/service-journal.md)
57+
* [Kafka](plugins/input/service-kafka.md)
58+
* [Mock数据-Service](plugins/input/service-mock.md)
59+
* [SqlServer 查询数据](plugins/input/service-mssql.md)
60+
* [OTLP数据](plugins/input/service-otlp.md)
61+
* [PostgreSQL 查询数据](plugins/input/service-pgsql.md)
62+
* [Syslog数据](plugins/input/service-syslog.md)
63+
* [处理](plugins/processor/README.md)
64+
* [添加字段](plugins/processor/processor-add-fields.md)
65+
* [添加云资产信息](plugins/processor/processor-cloudmeta.md)
66+
* [原始数据](plugins/processor/processor-default.md)
67+
* [数据脱敏](plugins/processor/processor-desensitize.md)
68+
* [丢弃字段](plugins/processor/processor-drop.md)
69+
* [字段加密](plugins/processor/processor-encrypy.md)
70+
* [条件字段处理](plugins/processor/processor-fields-with-condition.md)
71+
* [日志过滤](plugins/processor/processor-filter-regex.md)
72+
* [Go时间格式解析](plugins/processor/processor-gotime.md)
73+
* [Grok](plugins/processor/processor-grok.md)
74+
* [Json](plugins/processor/processor-json.md)
75+
* [日志转SLS Metric](plugins/processor/processor-log-to-sls-metric.md)
76+
* [正则](plugins/processor/processor-regex.md)
77+
* [重命名字段](plugins/processor/processor-rename.md)
78+
* [分隔符](plugins/processor/processor-delimiter.md)
79+
* [键值对](plugins/processor/processor-split-key-value.md)
80+
* [多行切分](plugins/processor/processor-split-log-regex.md)
81+
* [字符串替换](plugins/processor/processor-string-replace.md)
82+
* [聚合](plugins/aggregator/README.md)
83+
* [基础](plugins/aggregator/aggregator-base.md)
84+
* [上下文](plugins/aggregator/aggregator-context.md)
85+
* [按Key分组](plugins/aggregator/aggregator-content-value-group.md)
86+
* [按GroupMetadata分组](plugins/aggregator/aggregator-metadata-group.md)
87+
* [输出](plugins/flusher/README.md)
88+
* [Kafka(Deprecated)](plugins/flusher/flusher-kafka.md)
89+
* [kafkaV2](plugins/flusher/flusher-kafka_v2.md)
90+
* [ClickHouse](plugins/flusher/flusher-clickhouse.md)
91+
* [ElasticSearch](plugins/flusher/flusher-elasticsearch.md)
92+
* [SLS](plugins/flusher/flusher-sls.md)
93+
* [标准输出/文件](plugins/flusher/flusher-stdout.md)
94+
* [OTLP日志](plugins/flusher/flusher-otlp.md)
95+
* [Pulsar](plugins/flusher/flusher-pulsar.md)
96+
* [HTTP](plugins/flusher/flusher-http.md)
97+
* [Loki](plugins/flusher/loki.md)
98+
* [加速](plugins/accelerator/README.md)
99+
* [分隔符加速](plugins/accelerator/delimiter-accelerate.md)
100+
* [Json加速](plugins/accelerator/json-accelerate.md)
101+
* [正则加速](plugins/accelerator/regex-accelerate.md)
102102

103103
## 工作原理 <a href="#principle" id="principle"></a>
104104

docs/cn/about/brief-history.md

+5-5
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,9 @@
22

33
秉承着阿里人简单的特点,iLogtail的命名也非常简单,我们最开始期望的就是能够有一个统一去Tail日志的工具,所以就叫做Logtail,添加上“i”的原因主要当时使用了inotify的技术,能够让日志采集的延迟控制在毫秒级,因此最后叫做iLogtail。从2013年开始研发,iLogtail整个发展历程概括起来大致可以分为四个阶段,分别是飞天5K阶段、阿里集团阶段、云原生阶段和开源共建阶段。
44

5-
![](<../.gitbook/assets/ilogtail-history.png>)
5+
![ilogtail发展历史](<../.gitbook/assets/ilogtail-history.png>)
66

7-
### 飞天5K阶段 <a href="#4ever-bi-127" id="4ever-bi-127"></a>
7+
## 飞天5K阶段 <a href="#4ever-bi-127" id="4ever-bi-127"></a>
88

99
作为中国云计算领域的里程碑,2013年8月15日,阿里巴巴集团正式运营服务器规模达到5000(5K)的“飞天”集群,成为中国第一个独立研发拥有大规模通用计算平台的公司,也是世界上第一个对外提供5K云计算服务能力的公司。飞天5K项目从2009年开始,从最开始的30台逐渐发展到5000,不断解决系统核心的问题,比如说规模、稳定性、运维、容灾等等。而iLogtail在这一阶段诞生,最开始就是要解决5000台机器的监控、问题分析、定位的工作(如今的词语叫做“可观测性”)。从30到5000的跃升中,对于可观测问题有着诸多的挑战,包括单机瓶颈、问题复杂度、排查便捷性、管理复杂度等。
1010

@@ -17,7 +17,7 @@
1717
* 运维:加入集团yum源,运行状态监控,异常自动上报
1818
* 规模:3W+部署规模,上千采集配置项,日10TB数据
1919

20-
### 阿里集团阶段 <a href="#4ever-bi-265" id="4ever-bi-265"></a>
20+
## 阿里集团阶段 <a href="#4ever-bi-265" id="4ever-bi-265"></a>
2121

2222
iLogtail在阿里云飞天5K项目中的应用解决了日志、监控统一收集的问题,而当时阿里巴巴集团、蚂蚁等还缺少一套统一、可靠的日志采集系统,因此我们开始推动iLogtail作为集团、蚂蚁的日志采集基础设施。从5K这种相对独立的项目到全集团应用,不是简单复制的问题,而我们要面对的是更多的部署量、更高的要求以及更多的部门:
2323

@@ -34,7 +34,7 @@ iLogtail在阿里云飞天5K项目中的应用解决了日志、监控统一收
3434
* 运维:基于集团StarAgent自动安装与守护,异常主动通知,提供多种问题自查工具
3535
* 规模:百万+部署规模,千级别内部租户,10万+采集配置,日采集PB级数据
3636

37-
### 云原生阶段 <a href="#4ever-bi-329" id="4ever-bi-329"></a>
37+
## 云原生阶段 <a href="#4ever-bi-329" id="4ever-bi-329"></a>
3838

3939
随着阿里所有IT基础设施全面云化,以及iLogtail所属产品[SLS](https://www.aliyun.com/product/sls)(日志服务)正式在阿里云上商业化,iLogtail开始全面拥抱云原生。从阿里内部商业化并对外部各行各业的公司提供服务,对于iLogtail的挑战的重心已经不是性能和可靠性,而是如何适应云原生(容器化、K8s,适应云上环境)、如何兼容开源协议、如何去处理碎片化需求。这一阶段是iLogtail发展最快的时期,经历了非常多重要的变革:
4040

@@ -43,7 +43,7 @@ iLogtail在阿里云飞天5K项目中的应用解决了日志、监控统一收
4343
* 插件化扩展:iLogtail增加插件系统,可自由扩展Input、Processor、Aggregator、Flusher插件用以实现各类自定义的功能
4444
* 规模:千万部署规模,数万内外部客户,百万+采集配置项,日采集数十PB数据
4545

46-
### 开源共建阶段
46+
## 开源共建阶段
4747

4848
闭源自建的软件永远无法紧跟时代潮流,尤其在当今云原生的时代,我们坚信开源才是iLogtail最优的发展策略,也是释放其最大价值的方法。iLogtail作为可观测领域最基础的软件,我们将之开源,也希望能够和开源社区一起共建,持续优化,争取成为世界一流的可观测数据采集器。对于未来iLogail的发展,我们期待:
4949

docs/cn/about/license.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
[iLogtail](https://github.com/alibaba/ilogtail),包括内核、插件和工具均遵循[Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0)分发:
44

5-
```
5+
```plain
66
Apache License
77
Version 2.0, January 2004
88
http://www.apache.org/licenses/

0 commit comments

Comments
 (0)