PARQUET-1761: Lower Logging Level in ParquetOutputFormat

[belugabehr]

Make sure you have checked all steps below.

Jira

• My PR addresses the following PARQUET-1761 issues and references them in the PR title. For example, "PARQUET-1234: My Parquet PR"
  • https://issues.apache.org/jira/browse/PARQUET-1761
  • In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.

Tests

• My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

• My commits all reference Jira issues in their subject lines. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters (not including Jira issue reference)
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

Documentation

• In case of new functionality, my PR adds documentation that describes how to use it.
  • All the public functions and the classes in the PR contain Javadoc that explain what it does

[dossett]

+1 (non-binding) this seems sensible

[gszadovszky]

It is always a tough question to decide the correct log level. If we use DEBUG it may make the investigation of a problem really hard if the issue is not practically reproducible. Meanwhile, I am not sure if these lines can be crucial in such cases. In case of file size issues the blockSize and maxPaddingSize can be interesting.
What do you think?

[belugabehr]

@gszadovszky For me, since the properties are passed in, it is on the caller to log them in the client code before passing to Parquet. The DEBUG logging allows the user to validate.

Anyway, I have proposed simplifying the logging into a single statement (to reduce SPAM in the logs) and to debug only the large properties block that is unwieldy, and introduces new-lines which can make parsing the logs tricky (breaks the normal pattern of one log message per line).

[gszadovszky]

I've written that some data might worth logging at info level because block size might be used automatically based on the file system instead of the configuration. I've checked the code and unfortunately, at this point this is the value from the config. So, I am fine with both ways of having all these values logged at DEBUG level or the current solution. (Sorry for the confusion.)

[belugabehr]

@gszadovszky Let us start with the current change and we can re-visit later if this is still too verbose.