Skip to content

Commit

Permalink
Logster v2
Browse files Browse the repository at this point in the history
  • Loading branch information
navinpeiris committed Jul 26, 2023
1 parent d4d8629 commit 8420cfe
Show file tree
Hide file tree
Showing 20 changed files with 1,777 additions and 348 deletions.
14 changes: 14 additions & 0 deletions .cspell.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"version": "0.2",
"language": "en",
"ignorePaths": [
"/.git/",
"/cover/",
"/deps/",
"/mix.lock"
],
"words": [
"logfmt",
"loggable"
]
}
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,22 @@
# 2.0.0-rc.1

- Introduce `Logster.attach_phoenix_logger/1`, which attaches a phoenix telemetry event listener for logging. This is the recommended way to use Logster with Phoenix.
- Introduce `Logster.info/3`, `Logster.debug/3`, `Logster.warning/3`, `Logster.error/3` etc. These functions mimic those provided by the elixir logger, which outputs messages in your chosen log format.
- Logs socket events.
- Move all configuration options to be set through `config.exs`.
- Rename `Logster.Plugs.Logger` to `Logster.Plug`.
- Rename `Logster.Plugs.ChangeLogLevel` to `Logster.ChangeLogLevel`.
- Rename `Logster.StringFormatter` to `Logster.Formatters.String`.
- Rename `Logster.JSONFormatter` to `Logster.Formatters.JSON`.
- Specify formatter using atoms instead of module:
- `formatter: :json`
- `formatter: :string`
- Request state output in logs changed:
- `state=set` => `state=sent`
- `state=set_chunked` => `state=chunked`
- Config option `allowed_headers` changed to `headers`.
- Better `mix docs` output.

# 1.1.0

- By default logs responses with 5xx status code with error level and 4xx responses with warning level.
Expand Down
221 changes: 157 additions & 64 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,105 +5,228 @@
[![Hex downloads](https://img.shields.io/hexpm/dt/logster.svg "Hex downloads")](https://hex.pm/packages/logster)
[![License](http://img.shields.io/:license-mit-blue.svg)](http://doge.mit-license.org)

An easy to parse, one line logger for Elixir Plug.Conn and Phoenix, inspired by [lograge](https://github.com/roidrage/lograge).
> **Note**\
> This is the documentation for v2 of Logster. If you're looking for v1, see the [v1 branch](https://github.com/navinpeiris/logster/tree/v1.x).
With the default `Plug.Logger`, the log output for a request looks like:
<!-- MDOC -->

An easy-to-parse, single-line logger for Elixir Phoenix and Plug.Conn, inspired by [lograge](https://github.com/roidrage/lograge). Supports logfmt, JSON and custom output formatting.

By default, Phoenix log output for a request looks similar to the following:

```
[info] GET /articles/some-article
[debug] Processing with HelloPhoenix.ArticleController.show/2
Parameters: %{"id" => "some-article"}
Pipelines: [:browser]
[info] Sent 200 in 21ms
```

With Logster, we've got logging that's much easier to parse and search through, such as:
This can be handy for development, but cumbersome for production. The log output is spread across multiple lines, making it difficult to parse and search.

Logster aims to solve this problem by logging the request in a single line:

```
[info] method=GET path=/articles/some-article format=html controller=HelloPhoenix.ArticleController action=show params={"id":"some-article"} status=200 duration=0.402 state=set
[info] state=sent method=GET path=/articles/some-article format=html controller=HelloPhoenix.ArticleController action=show params={"id":"some-article"} status=200 duration=0.402
```

This becomes handy specially when integrating with log management services such as [Logentries](https://logentries.com/) or [Papertrail](https://papertrailapp.com/).
This is especially handy when integrating with log management services such as [Logentries](https://logentries.com/) or [Papertrail](https://papertrailapp.com/).

Alternatively, Logster can also output JSON formatted logs (see configuration section below):

```
[info] {"state":"sent","method":"GET","path":"/articles/some-article","format":"html","controller":"HelloPhoenix.ArticleController","action":"show","params":{"id":"some-article"},"status":200,"duration":0.402}
```

## Installation

First, add Logster to your `mix.exs` dependencies:
Add `:logster` to the list of dependencies in `mix.exs`:

```elixir
def deps do
[{:logster, "~> 1.1"}]
[{:logster, "~> 2.0.0-rc.1"}]
end
```

Then, update your dependencies:
## Upgrading

See [Upgrade Guide](UPGRADE_GUIDE.md) for more information.

## Usage

### Using with Phoenix

Attach the Logster Phoenix logger in the `start` function in your project's `application.ex` file:

```elixir
# lib/my_app/application.ex
def start(_type, _args) do
children = [
# ...
]

#
# Add the line below:
#
:ok = Logster.attach_phoenix_logger()

opts = [strategy: :one_for_one, name: MyApp.Supervisor]
Supervisor.start_link(children, opts)
end
```
$ mix deps.get

Next, disable the default Phoenix logger by adding the following line to your `config.exs` file:

```elixir
config :phoenix, :logger, false
```

## Usage
### Using with Plug

To use with a Phoenix application, replace `Plug.Logger` in the projects `endpoint.ex` file with `Logster.Plugs.Logger`:
Add `Logster.Plug` to your plug pipeline, or in the relevant module:

```elixir
# plug Plug.Logger
plug Logster.Plugs.Logger
plug Logster.Plug
```

To use it in as a plug, just add `plug Logster.Plugs.Logger` into the relevant module.
### Using standalone logger

### Filtering parameters
Logster provides `debug`, `info`, `warning`, `error` etc convenience functions that mimic those provided by the elixir logger, which outputs messages in your chosen log format.

For example:

By default, Logster filters parameters named `password`, and replaces the content with `[FILTERED]`.
```elixir
Logger.info(service: :payments, event: :received, amount: 1000, customer: 123)
```

You can update the list of parameters that are filtered by adding the following to your configuration file:
will output the following to the logs:

```
[info] service=payments event=received amount=1000 customer=123
```

You can also provide a function to be called lazily, which will only be called if the log level is enabled:

```elixir
Logger.debug(fn ->
# some potentially expensive operation
# won't be called if the log level is not enabled
customer = get_customer_id()

[service: :payments, event: :received, amount: 1000, customer: customer]
end)
```

## Configuration

The following configuration options can be set through your `config.exs` file

### Formatter

#### JSON formatter

```elixir
config :logster, :filter_parameters, ["password", "secret", "token"]
config :logster,
formatter: :json
```

### HTTP headers support
_Caution:_ There is no guarantee that what reaches your console will be valid JSON. The Elixir `Logger` module has its own formatting which may be appended to your message. See the [Logger documentation](http://elixir-lang.org/docs/stable/logger/Logger.html) for more information.

#### Custom formatter

Provide a function that takes one argument, the parameters as input, and returns formatted output

```elixir
config :logster,
formatter: &MyCustomFormatter.format/1
```

### Filtering parameters

By default, Logster filters parameters named `password`.

To change the filtered parameters:

```elixir
config :logster, filter_parameters: ["password", "secret", "token"]
```

By default, Logster won't parse and log HTTP headers.
### Logging HTTP request headers

But you can update the list of headers that should be parsed and logged. The logged headers will be added under `headers`. Both plain text and JSON formatters are supported.
By default, Logster won't log any request headers. To log specific headers, you can use the `:headers` option:

```elixir
config :logster, :allowed_headers, ["my-header-one", "my-header-two"]
config :logster, headers: ["my-header-one", "my-header-two"]
```

### Changing the log level for a specific controller/action

To change the Logster log level for a specific controller and/or action, you use the `Logster.Plugs.ChangeLogLevel` plug.
#### Through Logster.ChangeLogLevel plug

To change the Logster log level for a specific controller and/or action, you use the `Logster.ChangeLogLevel` plug.

For example, to change the logging of all requests in a controller to `debug`, add the following to that controller:

```elixir
plug Logster.Plugs.ChangeLogLevel, to: :debug
plug Logster.ChangeLogLevel, to: :debug
```

And to change it only for `index` and `show` actions:

```elixir
plug Logster.Plugs.ChangeLogLevel, [to: :debug] when action in [:index, :show]
plug Logster.ChangeLogLevel, [to: :debug] when action in [:index, :show]
```

This is specially useful for cases such as when you want to lower the log level for a healthcheck endpoint that gets hit every few seconds.

### Changing the formatter
#### Through endpoint configuration

You can set the `Plug.Telemetry` `:log` option to a tuple, `{Mod, Fun, Args}`. `The Plug.Conn.t()` for the request will be prepended to the provided list of arguments.

When invoked, your function must return a `Logger.level()` or `false` to disable logging for the request.

```elixir
# lib/my_app_web/endpoint.ex
plug Plug.Telemetry,
event_prefix: [:phoenix, :endpoint],
log: {__MODULE__, :log_level, []}

# Disables logging for routes like /status/*
def log_level(%{status: status}) when status >= 500, do: :error
def log_level(%{status: status}) when status >= 400, do: :warning
def log_level(%{path_info: ["status" | _]}), do: false
def log_level(_), do: :info
```

Logster allows you to use a different formatter to get your log lines looking just how you want. It comes with two built-in formatters: `Logster.StringFormatter` and `Logster.JSONFormatter`
### Renaming default fields

To use `Logster.JSONFormatter`, supply the `formatter` option when you use the `Logster.Plugs.Logger` plug:
You can rename the default keys passing a map like `%{key: :new_key}`:

```elixir
plug Logster.Plugs.Logger, formatter: Logster.JSONFormatter
config :logster,
renames: [duration: :response_time, params: :parameters]
```

That means your log messages will be formatted thusly:
Example output:

```
{"status":200,"state":"set","path":"hello","params":{},"method":"GET","format":"json","duration":20.647,"controller":"App.HelloController","action":"show"
[info] method=GET path=/articles/some-article format=html controller=HelloPhoenix.ArticleController action=show parameters={"id":"some-article"} status=200 response_time=0.402 state=set
```

_Caution:_ There is no guarantee that what reaches your console will be valid JSON. The Elixir `Logger` module has its own formatting which may be appended to your message. See the [Logger documentation](http://elixir-lang.org/docs/stable/logger/Logger.html) for more information.
### Excluding fields

You can exclude fields with `:excludes`:

```elixir
config :logster,
excludes: [:params, :status, :state]
```

Example output:

```
[info] method=GET path=/articles/some-article format=html controller=HelloPhoenix.ArticleController action=show duration=0.402
```

### Metadata

Expand All @@ -119,7 +242,7 @@ Logger.metadata(%{user_id: "123", foo: "bar"})
config :logger, :console, metadata: [:user_id, :foo]
```

The easiest way to do this app wide is to introduce a new plug which you can include in your phoenix router pipeline.
The easiest way to do this app wide is to introduce a new plug which you can include in your Phoenix router pipeline.

For example:

Expand Down Expand Up @@ -154,37 +277,7 @@ pipeline :browser do
end
```

### Renaming default fields

You can rename the default keys passing a map like `%{key: :new_key}`:

```elixir
plug Logster.Plugs.Logger, renames: %{duration: :response_time, params: :parameters}
```

It will log the following:

```
[info] method=GET path=/articles/some-article format=html controller=HelloPhoenix.ArticleController action=show parameters={"id":"some-article"} status=200 response_time=0.402 state=set
```

### Excluding fields

You can exclude fields with `:excludes`:

```elixir
plug Logster.Plugs.Logger, excludes: [:params, :status, :state]
```

It will log the following:

```
[info] method=GET path=/articles/some-article format=html controller=HelloPhoenix.ArticleController action=show duration=0.402
```

#### Writing your own formatter

To write your own formatter, all that is required is a module which defines a `format/1` function, which accepts a keyword list and returns a string.
<!-- MDOC -->

## Development

Expand Down
Loading

0 comments on commit 8420cfe

Please sign in to comment.