RANGER-5310:Include Apache Tez as the process framework for ranger-hive docker #660
Conversation
… docker Signed-off-by: Ramesh Mani <[email protected]>
There was a problem hiding this comment.
Pull Request Overview
This PR integrates Apache Tez as the processing framework for the ranger-hive Docker setup to enable faster data processing through DAG execution and resolve issues with INSERT commands in beeline.
- Adds Tez binary distribution and configuration files for Hive integration
- Updates Hadoop YARN configuration to support Tez execution
- Creates comprehensive Tez configuration across all Hive database variants
Reviewed Changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tez-site.xml | New Tez configuration template with memory and execution settings |
| ranger-hive-setup.sh | Adds Tez setup, YARN configuration, and HDFS directory creation |
| ranger-hadoop-setup.sh | Enhances YARN configuration and installs Tez JARs for NodeManager |
| hive-site-*.xml | Adds Tez execution engine configuration to all database variants |
| hive-site-metastore-mysql.xml | New metastore-specific configuration with Tez support |
| create-users.sh | New script for creating test users (alice, abram) |
| download-archives.sh | Adds Tez binary download support |
| docker-compose files | Updates build arguments and environment variables for Tez |
| Dockerfiles | Integrates Tez installation and user creation across containers |
| .env | Updates Hadoop version compatibility and adds Tez version |
Comments suppressed due to low confidence (1)
dev-support/ranger-docker/.env:1
- The KAFKA_VERSION line appears to be missing after the HIVE_HADOOP_VERSION change. This could break Kafka-related builds that depend on this environment variable.
BUILD_HOST_SRC=true
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
Thank you @rameeshm for the patch, I believe this is tested with Ubuntu base image, please see if this can be tested with UBI base image as well, this change needs to made in |
… docker - changes to use ranger base image for user creation, fix issue with usage of ranger base image in other containers
@kumaab current patch with the review comments tested with RANGER_BASE_VERSION=[20250712-1-ubi-8] |
… docker - addressed review comment on issue related to base immage
… docker - address review comment on the switch user statement
| ln -s /opt/hadoop-${HADOOP_VERSION} /opt/hadoop && \ | ||
| rm -f /home/ranger/dist/hadoop-${HADOOP_VERSION}.tar.gz && \ | ||
| tar xvfz /home/ranger/dist/apache-tez-${TEZ_VERSION}-bin.tar.gz --directory=/opt/ && \ | ||
| ln -s /opt/apache-tez-${TEZ_VERSION}-bin /opt/tez && \ |
There was a problem hiding this comment.
Let's remove the apache-tez-<version>-bin.tar.gz after un-taring ? as it's done for other tarballs.
| <name>hive.zookeeper.client.port</name> | ||
| <value>2181</value> | ||
| </property> | ||
|
|
There was a problem hiding this comment.
This file may be removed, as support for SQL Server is going to be deprecated soon.
3 participants
What changes were proposed in this pull request?
Include Apache Tez as the process framework for ranger-hive docker
How was this patch tested?
Testing in Docker running HiveServer 2 beeline and execute "INSERT" statement for DAG.
