Lower log level in Kafka product tests

[nevillelyh]

Description

Lower log level in Kafka containers and test code used in Kafka product tests

• Related to Lower Kafka log level in product tests #14149, which lowers the log level in Trino containers only
• This change also lowers the level in Kafka containers and test code

Non-technical explanation

Release notes

(x) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
( ) Release notes are required, with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

[lukasz-walkiewicz]

👍

[nevillelyh]

Turns out the environment depends on a started (kafka.server.KafkaServer) log message.

trino/testing/trino-product-tests-launcher/src/main/java/io/trino/tests/product/launcher/env/common/Kafka.java

Line 97 in 756be87

.waitingForAll(forSelectedPorts(9092), forLogMessage(".*started \\(kafka.server.KafkaServer\\).*", 1))

[nevillelyh]

Two possible flakiness issues. Otherwise this is good to go. @lukasz-walkiewicz do we need another approval?
#3049
#6799

[kokosing]

Starting download for product tests and server tarball
Error: Unable to find an artifact with the name: product tests and server tarball

The ci / pt (default, suite-2, ) failed because of the above. I am retrying.

[nevillelyh]

Also discovered that TestingKafka containers have the same problem. Added a commit to address that.

[kokosing]

ci / pt (default, suite-2, ) fails again. So it does not look as flake.

[nevillelyh]

Dead lock should be fixed now.

Also noticed noisy logging from io.confluent.kafka which I missed from #14149. Added a commit to address that.
https://github.com/trinodb/trino/actions/runs/3329251804/jobs/5507115101.

INFO	pool-3-thread-2	io.confluent.kafka.serializers.KafkaAvroSerializerConfig	KafkaAvroSerializerConfig values: 
...

[nevillelyh]

Hit flaky test in Pinot #14239

[kokosing]

Thanks!