diff --git a/hadoop-hdds/common/src/main/resources/ozone-default.xml b/hadoop-hdds/common/src/main/resources/ozone-default.xml index dbc02cb4ed1..872e256342a 100644 --- a/hadoop-hdds/common/src/main/resources/ozone-default.xml +++ b/hadoop-hdds/common/src/main/resources/ozone-default.xml @@ -554,6 +554,9 @@ The address of the Ozone OM service. This allows clients to discover the address of the OM. + + When HA mode is enabled, append the service ID and node ID to each OM property. For example: + ozone.om.address.service1.om1 @@ -583,6 +586,9 @@ If the port is 0, then the server will start on a free port. However, it is best to specify a well-known port, so it is easy to connect and see the OM management UI. + + When HA mode is enabled, append the service ID and node ID to each OM property. For example: + ozone.om.http-address.service1.om1 @@ -593,6 +599,9 @@ The actual address the OM web server will bind to. If this optional the address is set, it overrides only the hostname portion of ozone.om.http-address. + + When HA mode is enabled, append the service ID and node ID to each OM property. For example: + ozone.om.http-bind-host.service1.om1 @@ -611,6 +620,9 @@ The address and the base port where the OM web UI will listen on using HTTPS. If the port is 0 then the server will start on a free port. + + When HA mode is enabled, append the service ID and node ID to each OM property. For example: + ozone.om.https-address.service1.om1 @@ -621,6 +633,9 @@ The actual address the OM web server will bind to using HTTPS. If this optional address is set, it overrides only the hostname portion of ozone.om.https-address. + + When HA mode is enabled, append the service ID and node ID to each OM property. For example: + ozone.om.https-bind-host.service1.om1 diff --git a/hadoop-hdds/docs/content/concept/NetworkPorts.md b/hadoop-hdds/docs/content/concept/NetworkPorts.md new file mode 100644 index 00000000000..521c2568a1f --- /dev/null +++ b/hadoop-hdds/docs/content/concept/NetworkPorts.md @@ -0,0 +1,102 @@ +--- +title: "Network Ports" +date: "2025-04-03" +weight: 9 +menu: + main: + parent: Architecture +summary: Understanding and correctly configuring the network ports used by Apache Ozone is essential for the successful deployment, operation, and maintenance of Apache Ozone clusters. + +--- + + +This document provides a comprehensive overview of the network ports utilized by Apache Ozone. Due to its distributed nature and the requirement for high performance in handling data-intensive tasks, understanding and correctly configuring these network ports is essential for the successful deployment, operation, and maintenance of Apache Ozone clusters. + +# **Ozone Manager (OM)** + +| Default Port Number | Configuration Key | Endpoint Protocol | Purpose | +|:--------------------|:----------------------------------------------| :---- | :---- | +| 9862 | `ozone.om.address..` | Hadoop RPC | Primary RPC endpoint for Ozone clients | +| 9874 | `ozone.om.http-address..` | HTTP | Web UI for monitoring OM status and metadata | +| 9875 | `ozone.om.https-address..` | HTTPS | Secure Web UI for monitoring OM status and metadata | +| 9872 | `ozone.om.ratis.port..` | HTTP/2 | RPC endpoint for OM HA instances to form a RAFT consensus ring | +| 8981 | `ozone.om.grpc.port` | HTTP/2 | gRPC endpoint for Ozone Manager clients | + +Note: except for `ozone.om.grpc.port`, all the above OM properties are suffixed with `service_id.node_id`. +For example: `ozone.om.address.cluster1.om1`. + +# **Storage Container Manager (SCM)** + +| Default Port Number | Configuration Key | Endpoint Protocol | Purpose | +| :---- | :---- | :---- | :---- | +| 9860 | `ozone.scm.client.port..` | Hadoop RPC | Communication with Ozone clients for namespace and container management. `ozone.scm.client.address` overrides it, if defined (default is empty) | +| 9863 | `ozone.scm.block.client.port..` | Hadoop RPC | Communication with Datanodes for block-level operations | +| 9861 | `ozone.scm.datanode.port..` | Hadoop RPC | Port used by Datanodes to communicate with the SCM. `ozone.scm.datanode.address` overrides it, if defined (default is empty) | +| 9961 | `ozone.scm.security.service.port..` | Hadoop RPC | SCM security server port | +| 9876 | `ozone.scm.http-address..` | HTTP | Web UI for monitoring SCM status | +| 9877 | `ozone.scm.https-address..` | HTTPS | Secure Web UI for monitoring SCM status | +| 9894 | `ozone.scm.ratis.port..` | HTTP/2 | SCM Ratis HA | +| 9895 | `ozone.scm.grpc.port..` | HTTP/2 | SCM GRPC server port | + +Note: all the above SCM properties are suffixed with `service_id.node_id`. +For example: `ozone.scm.client.port.scmservice.scm1`. + +# **Recon** + +| Default Port Number | Configuration Key | Endpoint Protocol | Purpose | +| :---- | :---- | :---- | :---- | +| 9891 | `ozone.recon.address` | Hadoop RPC | RPC address for Recon to collect metadata from other Ozone services | +| 9888 | `ozone.recon.http-address` | HTTP | Web-based management and monitoring console for the entire Ozone cluster | +| 9889 | `ozone.recon.https-address` | HTTPS | Web-based management and monitoring console for the entire Ozone cluster | + +# **S3 Gateway (S3G)** + +| Default Port Number | Configuration Key | Endpoint Protocol | Purpose | +| :---- | :---- | :---- | :---- | +| 9878 | `ozone.s3g.http-address` | HTTP | S3-compatible RESTful API endpoint | +| 9879 | `ozone.s3g.https-address` | HTTPS | Secure S3-compatible RESTful API endpoint | +| 19878 | `ozone.s3g.webadmin.http-address` | HTTP | Ozone S3Gateway serves web content | +| 19879 | `ozone.s3g.webadmin.https-address` | HTTPS | Ozone S3Gateway serves web content | + +# **HttpFS Server** + +| Default Port Number | Configuration Key | Endpoint Protocol | Purpose | +| :---- | :---- | :---- | :---- | +| 14000 | `httpfs.http.port` | HTTP or HTTPS | The HTTP port for HttpFS REST API. TLS is enabled if `httpfs.ssl.enabled` is true | + +# **Datanode** + +| Default Port Number | Configuration Key | Endpoint Protocol | Purpose | +| :---- | :---- | :---- | :---- | +| 9859 | `hdds.container.ipc.port` | HTTP/2 | Inter-process communication related to container operations | +| 9855 | `hdds.container.ratis.datastream.port` | TCP | Ratis data streaming for container replication (if enabled) | +| 9858 | `hdds.container.ratis.ipc.port` | HTTP/2 | Communication with embedded Ratis server for replication coordination | +| 9857 | `hdds.container.ratis.admin.port` | HTTP/2 | Administrative requests to the Ratis server | +| 9856 | `hdds.container.ratis.server.port` | HTTP/2 | Communication between Ratis peers in a replication pipeline | +| 9882 | `hdds.datanode.http-address` | HTTP | Web UI for monitoring Datanode status and resource utilization | +| 9883 | `hdds.datanode.https-address` | HTTPS | Secure Web UI for monitoring Datanode status and resource utilization | +| 19864 | `hdds.datanode.client.port` | Hadoop RPC | The port number of the Ozone Datanode client service | + +Note: + +* The default port values can be overridden within the `ozone-site.xml` configuration file, with the exception of Httpfs ports, which are configurable via the `httpfs-site.xml` configuration file. +* Hadoop RPC, a binary protocol operating over TCP, may be authenticated and encrypted using the Java SASL mechanism. +* Authentication of the Web UI and HttpFS is achievable through Kerberos/SPNEGO, with encryption facilitated by HTTPS. +* S3 client connections to the S3 Gateway undergo authentication utilizing S3 secrets and encryption via TLS. +* A series of Ozone service ports are established by gRPC and Ratis, the latter employing gRPC. gRPC, a protocol based on HTTP/2, is capable of being encrypted with TLS. +* Ratis streaming ports, initiated by Netty, can be secured through TLS encryption.