feat: added support for reading locally generated file logs via vector inst…

[aayushdhiman01]

Type of Change

• Bugfix
• New feature
• Enhancement
• Refactoring
• Dependency updates
• Documentation
• CI/CD

Description

Added the code for reading locally generated file logs via vector instance, but need to verify that local logs are coming, can u guide how to check that?

Additional Changes

• This PR modifies the API contract
• This PR modifies the database schema
• This PR modifies application configuration/environment variables

Motivation and Context

How did you test it?

Checklist

• I formatted the code cargo +nightly fmt --all
• I addressed lints thrown by cargo clippy
• I reviewed the submitted code
• I added unit tests for my changes where possible

[lsampras]

Hey @aayushdhiman01 ,
Can you take a look at this for reference.

So the logs would be written in a shared local folder and we mount that folder in vector to read them,

You can enable file logging via the config file(development/docker_compose.toml),

The vector/loki/grafana cotainers can be run via docker compose --profile monitoring up -d

[aayushdhiman01]

@lsampras can you check the changes made, mounted the logs folder in vector , in docker-compose file and enabled file logging in development.toml and docker_compose.toml file.

[lsampras]

We need to add extra labels to distinguish between file & console logs,
can add a label like source = console/file

Also can you share screenshots of the file logs in loki

[aayushdhiman01]

@lsampras i am not able to connect to loki server in grafana,monitoring containers are running. Made the commented changes.