Skip to content

[gelf] Replace '/' with '_' in keys when converting from msgpack.#1166

Merged
edsiper merged 1 commit intofluent:masterfrom
lbogdan:fix/gelf-k8s
Jul 23, 2019
Merged

[gelf] Replace '/' with '_' in keys when converting from msgpack.#1166
edsiper merged 1 commit intofluent:masterfrom
lbogdan:fix/gelf-k8s

Conversation

@lbogdan
Copy link
Contributor

@lbogdan lbogdan commented Mar 4, 2019

Allows logs form Kubernetes pods having labels or annotations containing '/' in the name, which is not a valid GELF field name.

Also changes the log level to error and adds the invalid char position to "invalid key char" errors.

Fixes #1070 , #1291 .

@nigels-com
Copy link
Contributor

Looks like it does as described.

Seems like there are potential leaks in the early-outs (out of memory situation).
Perhaps a goto cleanup block that centralises the cleanup of temporary copies?

@lbogdan
Copy link
Contributor Author

lbogdan commented Mar 5, 2019

Hey @nigels-com , thanks for your review and suggestion! I was aware of the memory leaks, but I didn't have any obvious fix for it, other than just freeing the copies in every branch, which would get quite verbose (on the other hand, I agree that's not really a reason to ignore memory leaks 🙂 ). I'll look for a goto pattern in the codebase and update the PR.

P.S.: I have used this patch in production for a week now, without any issues (the leak didn't trigger, as there were no further encoding errors).

@nigels-com
Copy link
Contributor

nigels-com commented Mar 5, 2019

In C++ this tends to be less of an issue, of course.

The sort of pattern I suggest is something like:

  void * a = NULL;
  void * b = NULL;
  void * c = NULL;
  void * result = NULL;

  a = malloc(len+1);
  ...
  {
    if (!b) goto cleanup;  /* Uh oh! Exit gracefully! */
    ...
  }
  /* Normal path of execution gets here */
  result = ...  /* Not a, b or c! */

  /* Do the necessary cleanup, whether or not there was an error */

cleanup:
  free(a);
  free(b);
  free(c);
  return result;

It's one of the few remaining legitimate uses of goto but a good one, in my opinion.
Assuming free(NULL) is harmless.

@lbogdan
Copy link
Contributor Author

lbogdan commented Mar 5, 2019

Done. I also safe-guarded the free()s with

if (copy) {
  flb_free(copy);
}

, just in case.

@nigels-com
Copy link
Contributor

For the sake of portability, I'd recommend the NULL check. I do see NULL checks elsewhere in the codebase:

    if (ctx->merge_log_key) {
        flb_free(ctx->merge_log_key);
    }

@lbogdan
Copy link
Contributor Author

lbogdan commented Mar 6, 2019

@nigels-com I'm not sure I get what you mean, didn't I do exactly that?

@knackaron
Copy link

I applied this against our 1.1.0 image to fix the the same issue, works as intended.

@lbogdan Are you able to implement the modifications that @edsiper requested so this can get merged for the next release?

@lbogdan
Copy link
Contributor Author

lbogdan commented May 10, 2019

Hey @knackaron , sorry for the radio silence, I've been busy with work. I'll try to work on this next week, but feel free to take over, if you want.

@lbogdan lbogdan force-pushed the fix/gelf-k8s branch 3 times, most recently from ab2d7a2 to 201beec Compare May 11, 2019 15:54
Allows logs from Kubernetes pods to have labels or annotations containing '/' by replacing it with '_', as '/' is not a valid GELF field name character.

Signed-off-by: Bogdan Luca <luca.bogdan@gmail.com>
@lbogdan
Copy link
Contributor Author

lbogdan commented May 11, 2019

Squashed the commits, updated the commit subject, should be ready for merge.

@knackaron
Copy link

@lbogdan No worries. Thanks for kicking out an updated patch over the weekend.

@PortaltechGithub
Copy link

Is there anything blocking the pull request? The requested change by @edsiper seems to be incorporated in the new commit.

Copy link

@l-lotz l-lotz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look good to me

@l-lotz
Copy link

l-lotz commented Jul 1, 2019

@edsiper can you confirm that your requested change is now included?

@lbogdan
Copy link
Contributor Author

lbogdan commented Jul 1, 2019

There's also #1365 , which is replacing more than just / (although that's the only k8s "offending" character), so I guess only one of the two should be merged.

@isaacegglestone
Copy link

@lbogdan @edsiper This is definitely needed for kubernetes any chance we can get the slash tr merged since it appears to be done and only @edspiper needs to approve the change i think so its mergeable?

@edsiper edsiper merged commit 4b5ad3b into fluent:master Jul 23, 2019
@edsiper
Copy link
Member

edsiper commented Jul 23, 2019

thanks everyone!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error encoding to GELF message applied Kubernetes fileter

7 participants