Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make chunk key configurable from plugins #314

Closed
wants to merge 1 commit into from

Conversation

sabottenda
Copy link
Contributor

A key name of chunk can be specified in 4th argument of BufferedOutput#emit in BufferedOutput class, but no one use this argument and no hook point for setting the value. It is good to be able to configure it from plugins like format.

If this feature is available, plugins can determine how to handle chunks depending on the chunk key. For instance, destination server is determined by the tag. I know the example can be achieved by using fluent-plugin-forest. Though I think it is easy to provide a plugin with built-in features like fluent-plugin-hash-forward.

Actually I want to set the key name from log data, but it is inefficient to call chunk_key for each message just like format. For that purpose, I think Mixin approach is better and allow format to also return a key name of chunk. How do you think about it?

@repeatedly
Copy link
Member

Quick reply from my view point for the patch:

  • We can't merge this patch because it breaks several plugins, e.g. fluent-plugin-mongo and tag_mapped like approach plugins,
  • chunk_key is achieved by current key argument. I can't see the merit of this approach excluding human readable.

@sabottenda
Copy link
Contributor Author

I read fluent-plugin-mongo implementation, so it's enough for the purpose definitely.
I'll close this PR, thanks.

@sabottenda sabottenda closed this May 10, 2014
@sabottenda sabottenda deleted the chunk_key branch May 10, 2014 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Feature request or improve operations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants