Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support host monitor #1890

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Conversation

Abingcbc
Copy link
Collaborator

  1. 打通进程元信息采集链路

TODO:

  1. 支持更多字段
  2. 可观测指标

@Abingcbc Abingcbc force-pushed the host_monitor branch 6 times, most recently from 6150e52 to bfdd9c2 Compare November 19, 2024 06:27
core/common/timer/HostMonitorTimerEvent.cpp Outdated Show resolved Hide resolved
core/models/PipelineEventGroup.h Outdated Show resolved Hide resolved
core/runner/sink/http/HttpSink.cpp Outdated Show resolved Hide resolved
core/host_monitor/collector/MockCollector.cpp Outdated Show resolved Hide resolved
core/host_monitor/collector/CollectorManager.cpp Outdated Show resolved Hide resolved
core/plugin/processor/inner/ProcessorHostMetaNative.cpp Outdated Show resolved Hide resolved
core/constants/EntityConstants.cpp Outdated Show resolved Hide resolved
core/host_monitor/SystemInformationTools.cpp Show resolved Hide resolved
core/plugin/processor/inner/ProcessorHostMetaNative.cpp Outdated Show resolved Hide resolved
core/host_monitor/collector/ProcessCollector.cpp Outdated Show resolved Hide resolved
core/common/FileSystemUtil.h Outdated Show resolved Hide resolved
core/common/StringTools.cpp Outdated Show resolved Hide resolved
core/common/timer/HostMonitorTimerEvent.cpp Outdated Show resolved Hide resolved
core/plugin/input/InputHostMeta.h Outdated Show resolved Hide resolved
core/plugin/input/InputHostMeta.h Outdated Show resolved Hide resolved
core/plugin/input/InputHostMeta.cpp Outdated Show resolved Hide resolved
core/plugin/input/InputHostMeta.cpp Outdated Show resolved Hide resolved
core/host_monitor/HostMonitorInputRunner.cpp Outdated Show resolved Hide resolved
core/host_monitor/HostMonitorInputRunner.cpp Outdated Show resolved Hide resolved
core/host_monitor/HostMonitorInputRunner.cpp Outdated Show resolved Hide resolved
core/common/timer/Timer.h Show resolved Hide resolved
core/plugin/processor/inner/ProcessorHostMetaNative.cpp Outdated Show resolved Hide resolved
core/runner/sink/http/HttpSink.cpp Outdated Show resolved Hide resolved
core/plugin/processor/inner/ProcessorHostMetaNative.cpp Outdated Show resolved Hide resolved
core/host_monitor/collector/ProcessCollector.h Outdated Show resolved Hide resolved
core/host_monitor/collector/ProcessCollector.cpp Outdated Show resolved Hide resolved
core/host_monitor/collector/ProcessCollector.cpp Outdated Show resolved Hide resolved
core/plugin/processor/inner/ProcessorHostMetaNative.cpp Outdated Show resolved Hide resolved
core/host_monitor/collector/ProcessCollector.cpp Outdated Show resolved Hide resolved

int readCount = 0;
WalkAllProcess(PROCESS_DIR, [&](const std::string& dirName) {
if (++readCount > mProcessSilentCount) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

个数控制是不是也在控制缓存里的?不应该只是控制去直接操作系统交互的吗

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

缓存是为了计算增量的CPU使用率,前一次和后一次的top n进程可能不一样,所以需要保存所有的

core/common/MachineInfoUtil.cpp Outdated Show resolved Hide resolved
core/common/MachineInfoUtil.cpp Outdated Show resolved Hide resolved
core/common/timer/Timer.h Show resolved Hide resolved
core/plugin/input/InputHostMeta.cpp Show resolved Hide resolved
// See the License for the specific language governing permissions and
// limitations under the License.

#include "InputHostMeta.h"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

文档

core/plugin/input/InputHostMeta.cpp Show resolved Hide resolved
core/host_monitor/HostMonitorInputRunner.cpp Show resolved Hide resolved
ThreadPool mThreadPool;

mutable std::shared_mutex mRegisteredCollectorMapMutex;
std::unordered_map<std::string, bool> mRegisteredCollectorMap;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

std::unordered_map<std::string, bool> mRegisteredCollectorMap;
std::unordered_map<std::string, std::shared_ptr<BaseCollector>> mCollectorInstanceMap;

这两个变量有点不合理。
包括
1、HostMonitorInputRunner::GetCollector的使用上。应该是自闭环合法性
2、锁的关系等
3、RegisterCollector 操作的变量是 mCollectorInstanceMap,但是另一个没操作的变量却叫reg

可能的优化
1、保持现有逻辑。一组功能不用过多发散变量,以上改成一个map。value用结构体表示。整体实现结构优化下。
2、UpdateCollector时再注册ProcessEntityCollector实例,这样能省一个变量。

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants