diff --git a/internal/gossip/libserf/serf.go b/internal/gossip/libserf/serf.go index f5d6a90e748c..4e4f7de55609 100644 --- a/internal/gossip/libserf/serf.go +++ b/internal/gossip/libserf/serf.go @@ -29,7 +29,7 @@ func DefaultConfig() *serf.Config { // This gives leaves some time to propagate through the cluster before // we shut down. The value was chosen to be reasonably short, but to // allow a leave to get to over 99.99% of the cluster with 100k nodes - // (using https://www.serf.io/docs/internals/simulator.html). + // (using https://github.com/hashicorp/serf/blob/master/docs/internals/simulator.html.erb). base.LeavePropagateDelay = 3 * time.Second return base diff --git a/website/content/docs/architecture/coordinates.mdx b/website/content/docs/architecture/coordinates.mdx index ad8f7722e2ae..7bc37cc9c029 100644 --- a/website/content/docs/architecture/coordinates.mdx +++ b/website/content/docs/architecture/coordinates.mdx @@ -14,10 +14,10 @@ very simple calculation. This allows for many useful applications, such as findi the service node nearest a requesting node, or failing over to services in the next closest datacenter. -All of this is provided through the use of the [Serf library](https://www.serf.io/). +All of this is provided through the use of the [Serf library](https://github.com/hashicorp/serf/). Serf's network tomography is based on ["Vivaldi: A Decentralized Network Coordinate System"](http://www.cs.ucsb.edu/~ravenben/classes/276/papers/vivaldi-sigcomm04.pdf), with some enhancements based on other research. There are more details about -[Serf's network coordinates here](https://www.serf.io/docs/internals/coordinates.html). +[Serf's network coordinates here](https://github.com/hashicorp/serf/blob/master/docs/internals/coordinates.html.markdown). ## Network Coordinates in Consul diff --git a/website/content/docs/architecture/gossip.mdx b/website/content/docs/architecture/gossip.mdx index 09f85b19bd5c..12a4ef8de7ac 100644 --- a/website/content/docs/architecture/gossip.mdx +++ b/website/content/docs/architecture/gossip.mdx @@ -9,15 +9,15 @@ description: >- Consul uses a [gossip protocol](https://en.wikipedia.org/wiki/Gossip_protocol) to manage membership and broadcast messages to the cluster. The protocol, membership management, and message broadcasting is provided -through the [Serf library](https://www.serf.io/). The gossip protocol +through the [Serf library](https://github.com/hashicorp/serf/). The gossip protocol used by Serf is based on a modified version of the [SWIM (Scalable Weakly-consistent Infection-style Process Group Membership)](https://www.cs.cornell.edu/projects/Quicksilver/public_pdfs/SWIM.pdf) protocol. -Refer to the [Serf documentation](https://www.serf.io/docs/internals/gossip.html) for additional information about the gossip protocol. +Refer to the [Serf documentation](https://github.com/hashicorp/serf/blob/master/docs/internals/gossip.html.markdown) for additional information about the gossip protocol. ## Gossip in Consul Consul uses a LAN gossip pool and a WAN gossip pool to perform different functions. The pools -are able to perform their functions by leveraging an embedded [Serf](https://www.serf.io/) +are able to perform their functions by leveraging an embedded [Serf](https://github.com/hashicorp/serf/) library. The library is abstracted and masked by Consul to simplify the user experience, but developers may find it useful to understand how the library is leveraged. @@ -52,5 +52,5 @@ For more details about Lifeguard, please see the [Making Gossip More Robust with Lifeguard](https://www.hashicorp.com/blog/making-gossip-more-robust-with-lifeguard/) blog post, which provides a high level overview of the HashiCorp Research paper [Lifeguard : SWIM-ing with Situational Awareness](https://arxiv.org/abs/1707.00788). The -[Serf gossip protocol guide](https://www.serf.io/docs/internals/gossip.html#lifeguard) +[Serf gossip protocol guide](https://github.com/hashicorp/serf/blob/master/docs/internals/gossip.html.markdown#lifeguard-enhancements) also provides some lower-level details about the gossip protocol and Lifeguard. diff --git a/website/content/docs/install/glossary.mdx b/website/content/docs/install/glossary.mdx index f11ae21e7be0..c8bee74d6d77 100644 --- a/website/content/docs/install/glossary.mdx +++ b/website/content/docs/install/glossary.mdx @@ -51,7 +51,7 @@ and our implementation is described [here](/consul/docs/architecture/consensus). ## Gossip -Consul is built on top of [Serf](https://www.serf.io/) which provides a full +Consul is built on top of [Serf](https://github.com/hashicorp/serf/) which provides a full [gossip protocol](https://en.wikipedia.org/wiki/Gossip_protocol) that is used for multiple purposes. Serf provides membership, failure detection, and event broadcast. Our use of these is described more in the [gossip documentation](/consul/docs/architecture/gossip). It is enough to know @@ -78,171 +78,171 @@ This section collects brief definitions of some of the terms used in the discuss ## Access Control List (ACL) -An Access Control List (ACL) is a list of user permissions for a file, folder, or -other object. It defines what users and groups can access the object and what +An Access Control List (ACL) is a list of user permissions for a file, folder, or +other object. It defines what users and groups can access the object and what operations they can perform. Consul uses Access Control Lists (ACLs) to secure the UI, API, CLI, service communications, and agent communications. Visit [Consul ACL Documentation and Guides](/consul/docs/security/acl) -## API Gateway -An Application Programming Interface (API) is a common software interface that -allows two applications to communicate. Most modern applications are built using -APIs. An API Gateway is a single point of entry into these modern applications +## API Gateway +An Application Programming Interface (API) is a common software interface that +allows two applications to communicate. Most modern applications are built using +APIs. An API Gateway is a single point of entry into these modern applications built using APIs. -## Application Security -Application Security is the process of making applications secure by detecting -and fixing any threats or information leaks. This can be done during or after -the app development lifecycle; although, it is easier for app teams and security -teams to incorporate security into an app even before the development process +## Application Security +Application Security is the process of making applications secure by detecting +and fixing any threats or information leaks. This can be done during or after +the app development lifecycle; although, it is easier for app teams and security +teams to incorporate security into an app even before the development process begins. -## Application Services -Application Services are a group of services, such as application performance -monitoring, load balancing, service discovery, service proxy, security, -autoscaling, etc. needed to deploy, run, and improve applications. +## Application Services +Application Services are a group of services, such as application performance +monitoring, load balancing, service discovery, service proxy, security, +autoscaling, etc. needed to deploy, run, and improve applications. -## Authentication and Authorization (AuthN and AuthZ) +## Authentication and Authorization (AuthN and AuthZ) Authentication (AuthN) deals with establishing user identity while Authorization (AuthZ) allows or denies access to the user based on user identity. -## Auto Scaling Groups -An Auto Scaling Group is an AWS specific term that represents a collection of -Amazon EC2 instances that are treated as a logical grouping for the purposes of -automatic scaling and management. -Learn more about Auto Scaling Groups +## Auto Scaling Groups +An Auto Scaling Group is an AWS specific term that represents a collection of +Amazon EC2 instances that are treated as a logical grouping for the purposes of +automatic scaling and management. +Learn more about Auto Scaling Groups [here](https://docs.aws.amazon.com/autoscaling/ec2/userguide/AutoScalingGroup.html). -## Autoscaling -Autoscaling is the process of automatically scaling computational resources based -on network traffic requirements. Autoscaling can be done either horizontally or -vertically. Horizontal scaling is done by adding more machines into the pool of -resources whereas vertical scaling means increasing the capacity of an existing +## Autoscaling +Autoscaling is the process of automatically scaling computational resources based +on network traffic requirements. Autoscaling can be done either horizontally or +vertically. Horizontal scaling is done by adding more machines into the pool of +resources whereas vertical scaling means increasing the capacity of an existing machine. -## Blue-Green Deployments -Blue-Green Deployment is a deployment method designed to reduce downtime by -running two identical production environments labeled Blue and Green. Blue is -the active while Green is the idle environment. +## Blue-Green Deployments +Blue-Green Deployment is a deployment method designed to reduce downtime by +running two identical production environments labeled Blue and Green. Blue is +the active while Green is the idle environment. -## Canary Deployments -Canary deployment is the pattern used for rolling out releases to a subset of -users or servers. The goal is deploy the updates to a subset of users, test it, -and then roll out the changes to everyone. +## Canary Deployments +Canary deployment is the pattern used for rolling out releases to a subset of +users or servers. The goal is deploy the updates to a subset of users, test it, +and then roll out the changes to everyone. -## Client-side Load Balancing -Client-side load balancing is a load balancing approach that relies on clients' -decision to call the right servers. As the name indicates, this approach is part -of the client application. Servers can still have their own load balancer +## Client-side Load Balancing +Client-side load balancing is a load balancing approach that relies on clients' +decision to call the right servers. As the name indicates, this approach is part +of the client application. Servers can still have their own load balancer alongside the client-side load balancer. -## Cloud Native Computing Foundation -The [Cloud Native Computing Foundation (CNCF)](https://github.com/cncf/foundation) -is a Linux Foundation project that was founded in 2015 to help advance +## Cloud Native Computing Foundation +The [Cloud Native Computing Foundation (CNCF)](https://github.com/cncf/foundation) +is a Linux Foundation project that was founded in 2015 to help advance container technology and align the tech industry around its evolution. -HashiCorp joined Cloud Native Computing Foundation to further HashiCorp -product integrations with CNCF projects and to work more closely with the -broader cloud-native community of cloud engineers. Read more +HashiCorp joined Cloud Native Computing Foundation to further HashiCorp +product integrations with CNCF projects and to work more closely with the +broader cloud-native community of cloud engineers. Read more [here](https://www.hashicorp.com/blog/hashicorp-joins-the-cncf/). -## Custom Resource Definition (CRD) -Custom resources are the extensions of the Kubernetes API. A Custom Resource -Definition (CRD) file allows users to define their own custom resources and +## Custom Resource Definition (CRD) +Custom resources are the extensions of the Kubernetes API. A Custom Resource +Definition (CRD) file allows users to define their own custom resources and allows the API server to handle the lifecycle. -## Egress Traffic -Egress traffic is network traffic that begins inside a network and proceeds +## Egress Traffic +Egress traffic is network traffic that begins inside a network and proceeds through its routers to a destination outside the network. -## Elastic Provisioning -Elastic Provisioning is the ability to provision computing resources +## Elastic Provisioning +Elastic Provisioning is the ability to provision computing resources dynamically to meet user demand. -## Envoy Proxy -[Envoy Proxy](https://www.envoyproxy.io/) is a modern, high performance, -small footprint edge and service proxy. Originally written and deployed at +## Envoy Proxy +[Envoy Proxy](https://www.envoyproxy.io/) is a modern, high performance, +small footprint edge and service proxy. Originally written and deployed at [Lyft](https://eng.lyft.com/announcing-envoy-c-l7-proxy-and-communication-bus-92520b6c8191), - Envoy Proxy is now an official project at [Cloud Native Computing Foundation - (CNCF)](https://www.cncf.io/cncf-envoy-project-journey/) + Envoy Proxy is now an official project at [Cloud Native Computing Foundation + (CNCF)](https://www.cncf.io/cncf-envoy-project-journey/) -## Forward Proxy -A forward proxy is used to forward outgoing requests from inside the network -to the Internet, usually through a firewall. The objective is to provide a level +## Forward Proxy +A forward proxy is used to forward outgoing requests from inside the network +to the Internet, usually through a firewall. The objective is to provide a level of security and to reduce network traffic. -## Hybrid Cloud Architecture -A hybrid cloud architecture is an IT architectural approach that mixes -on-premises, private cloud, and public cloud services. A hybrid cloud -environment incorporates workload portability, orchestration, and management +## Hybrid Cloud Architecture +A hybrid cloud architecture is an IT architectural approach that mixes +on-premises, private cloud, and public cloud services. A hybrid cloud +environment incorporates workload portability, orchestration, and management across the environments. -A private cloud, traditionally on-premises, is referred to an infrastructure +A private cloud, traditionally on-premises, is referred to an infrastructure environment managed by the user themselves. -A public cloud, traditionally off-premises, is referred to an infrastructure +A public cloud, traditionally off-premises, is referred to an infrastructure service provided by a third party. -## Identity-based authorization -Identity-based authorization is a security approach to restrict or allow access +## Identity-based authorization +Identity-based authorization is a security approach to restrict or allow access based on the authenticated identity of an individual. -## Infrastructure as a Service -Infrastructure as a Service, often referred to as IaaS, is a cloud computing -approach where the computing resources are delivered online via APIs. These +## Infrastructure as a Service +Infrastructure as a Service, often referred to as IaaS, is a cloud computing +approach where the computing resources are delivered online via APIs. These APIs communicate with underlying infrastructure like physical computing resources, - location, data partitioning, scaling, security, backup, etc. + location, data partitioning, scaling, security, backup, etc. -IaaS is one of the four types of cloud services along with SaaS +IaaS is one of the four types of cloud services along with SaaS (Software as a Service), PaaS (Platform as a Service), and Serverless. -## Infrastructure as Code -Infrastructure as Code (IaC) is the process of developers and operations teams' -ability of provisioning and managing computing resources automatically through +## Infrastructure as Code +Infrastructure as Code (IaC) is the process of developers and operations teams' +ability of provisioning and managing computing resources automatically through software, instead of using configuration tools. -## Ingress Controller -In Kubernetes, "ingress" is an object that allows access Kubernetes services -from outside the Kubernetes cluster. An ingress controller is responsible for -ingress, generally with a load balancer or an edge router that can help with +## Ingress Controller +In Kubernetes, "ingress" is an object that allows access Kubernetes services +from outside the Kubernetes cluster. An ingress controller is responsible for +ingress, generally with a load balancer or an edge router that can help with traffic management. -## Ingress Gateway -An Ingress Gateway is an edge of the mesh load balancer that provides secure and -reliable access from external networks to Kubernetes clusters. +## Ingress Gateway +An Ingress Gateway is an edge of the mesh load balancer that provides secure and +reliable access from external networks to Kubernetes clusters. -## Ingress Traffic -Ingress Traffic is the network traffic that originates outside the network and +## Ingress Traffic +Ingress Traffic is the network traffic that originates outside the network and has a destination inside the network. -## Key-Value Store -A Key-Value Store (or a KV Store) also referred to as a Key-Value Database is -a data model where each key is associated with one and only one value in +## Key-Value Store +A Key-Value Store (or a KV Store) also referred to as a Key-Value Database is +a data model where each key is associated with one and only one value in a collection. -## L4 - L7 Services -L4-L7 Services are a set of functions such as load balancing, web application -firewalls, service discovery, and monitoring for network layers within the +## L4 - L7 Services +L4-L7 Services are a set of functions such as load balancing, web application +firewalls, service discovery, and monitoring for network layers within the Open Systems Interconnection (OSI) model. -## Layer 7 Observability -Layer 7 Observability is a feature of Consul Service Mesh that enables a -unified workflow for metric collection, distributed tracking, and logging. -It also allows centralized configuration and management for a distributed -data plane. +## Layer 7 Observability +Layer 7 Observability is a feature of Consul Service Mesh that enables a +unified workflow for metric collection, distributed tracking, and logging. +It also allows centralized configuration and management for a distributed +data plane. -## Load Balancer -A load balancer is a network appliance that acts as a [reverse proxy](#reverse-proxy) +## Load Balancer +A load balancer is a network appliance that acts as a [reverse proxy](#reverse-proxy) and distributes network and application traffic across the servers. -## Load Balancing -Load Balancing is the process of distributing network and application traffic -across multiple servers. +## Load Balancing +Load Balancing is the process of distributing network and application traffic +across multiple servers. -## Load Balancing Algorithms -Load balancers follow an algorithm to determine how to route the traffic across +## Load Balancing Algorithms +Load balancers follow an algorithm to determine how to route the traffic across the server farm. Some of the commonly used algorithms are: 1. Round Robin 2. Least Connections @@ -251,127 +251,127 @@ the server farm. Some of the commonly used algorithms are: 5. Least Response Time Method 6. Least Bandwidth Method -## Multi-cloud -A multi-cloud environment generally uses two or more cloud computing services -from different vendors in a single architecture. This refers to the distribution -of compute resources, storage, and networking aspects across cloud environments. -A multi-cloud environment could be either all private cloud or all public cloud -or a combination of both. +## Multi-cloud +A multi-cloud environment generally uses two or more cloud computing services +from different vendors in a single architecture. This refers to the distribution +of compute resources, storage, and networking aspects across cloud environments. +A multi-cloud environment could be either all private cloud or all public cloud +or a combination of both. -## Multi-cloud Networking -Multi-cloud Networking provides network configuration and management across +## Multi-cloud Networking +Multi-cloud Networking provides network configuration and management across multiple cloud providers via APIs. -## Mutual Transport Layer Security (mTLS) -Mutual Transport Layer Security, also known as mTLS, is an authentication -mechanism that ensures network traffic security in both directions between -a client and server. +## Mutual Transport Layer Security (mTLS) +Mutual Transport Layer Security, also known as mTLS, is an authentication +mechanism that ensures network traffic security in both directions between +a client and server. -## Network Middleware Automation -The process of publishing service changes to network middleware such as -load balancers and firewalls and automating network tasks is called Network +## Network Middleware Automation +The process of publishing service changes to network middleware such as +load balancers and firewalls and automating network tasks is called Network Middleware Automation. -## Network security -Network security is the process of protecting data and network. It consists -of a set of policies and practices that are designed to prevent and monitor -unauthorized access, misuse, modification, or denial of a computer network +## Network security +Network security is the process of protecting data and network. It consists +of a set of policies and practices that are designed to prevent and monitor +unauthorized access, misuse, modification, or denial of a computer network and network-accessible resources. -## Network traffic management -Network Traffic Management is the process of ensuring optimal network operation -by using a set of network monitoring tools. Network traffic management also -focuses on traffic management techniques such as bandwidth monitoring, deep +## Network traffic management +Network Traffic Management is the process of ensuring optimal network operation +by using a set of network monitoring tools. Network traffic management also +focuses on traffic management techniques such as bandwidth monitoring, deep packet inspection, and application based routing. -## Network Visualization -Network Visualization is the process of visually displaying networks and -connected entities in a "boxes and lines" kind of a diagram. +## Network Visualization +Network Visualization is the process of visually displaying networks and +connected entities in a "boxes and lines" kind of a diagram. -In the context of microservices architecture, visualization can provide a clear -picture of how services are connected to each other, the service-to-service +In the context of microservices architecture, visualization can provide a clear +picture of how services are connected to each other, the service-to-service communication, and resource utilization of each service. -## Observability -Observability is the process of logging, monitoring, and alerting on the +## Observability +Observability is the process of logging, monitoring, and alerting on the events of a deployment or an instance. -## Elastic Scaling -Elastic Scaling is the ability to automatically add or remove compute or +## Elastic Scaling +Elastic Scaling is the ability to automatically add or remove compute or networking resources based on the changes in application traffic patterns. -## Platform as a Service -Platform-as-a-Service (PaaS) is a category of cloud computing that allows -users to develop, run, and manage applications without the complexity of -building and maintaining the infrastructure typically associated with -developing and launching the application. - -## Reverse Proxy -A reverse proxy handles requests coming from outside, to the internal -network. Reverse Proxy provides a level of security that prevents the -external clients from having direct access to data on the corporate servers. -The reverse proxy is usually placed between the web server and the external -traffic. - -## Role-based Access Controls -The act of restricting or provisioning access +## Platform as a Service +Platform-as-a-Service (PaaS) is a category of cloud computing that allows +users to develop, run, and manage applications without the complexity of +building and maintaining the infrastructure typically associated with +developing and launching the application. + +## Reverse Proxy +A reverse proxy handles requests coming from outside, to the internal +network. Reverse Proxy provides a level of security that prevents the +external clients from having direct access to data on the corporate servers. +The reverse proxy is usually placed between the web server and the external +traffic. + +## Role-based Access Controls +The act of restricting or provisioning access to a user based on their specific role in the organization. -## Server side load balancing -A Server-side Load Balancer sits between the client and the server farm, -accepts incoming traffic, and distributes the traffic across multiple backend +## Server side load balancing +A Server-side Load Balancer sits between the client and the server farm, +accepts incoming traffic, and distributes the traffic across multiple backend servers using various load balancing methods. -## Service configuration -A service configuration includes the name, description, and the specific -function of a service. In a microservices application architecture setting, +## Service configuration +A service configuration includes the name, description, and the specific +function of a service. In a microservices application architecture setting, a service configuration file includes a service definition. -## Service Catalog -A service catalog is an organized and curated collection of services that +## Service Catalog +A service catalog is an organized and curated collection of services that are available for developers to bind to their applications. -## Service Discovery -Service Discovery is the process of detecting services and devices on a -network. In a microservices context, service discovery is how applications +## Service Discovery +Service Discovery is the process of detecting services and devices on a +network. In a microservices context, service discovery is how applications and microservices locate each other on a network. -## Service Mesh -Service Mesh is the infrastructure layer that facilitates service-to-service -communication between microservices, often using a sidecar proxy. This -network of microservices make up microservice applications and the +## Service Mesh +Service Mesh is the infrastructure layer that facilitates service-to-service +communication between microservices, often using a sidecar proxy. This +network of microservices make up microservice applications and the interactions between them. -## Service Networking -Service networking brings several entities together to deliver a particular -service. Service Networking acts as the brain of an organization's +## Service Networking +Service networking brings several entities together to deliver a particular +service. Service Networking acts as the brain of an organization's networking and monitoring operations. -## Service Proxy -A service proxy is the client-side proxy for a microservice application. -It allows applications to send and receive messages over a proxy server. +## Service Proxy +A service proxy is the client-side proxy for a microservice application. +It allows applications to send and receive messages over a proxy server. -## Service Registration -Service registration is the process of letting clients (of the service) -and routers know about the available instances of the service. +## Service Registration +Service registration is the process of letting clients (of the service) +and routers know about the available instances of the service. Service instances are registered with a service registry on startup and deregistered at shutdown. -## Service Registry -Service Registry is a database of service instances and information on +## Service Registry +Service Registry is a database of service instances and information on how to send requests to these service instances. -## Microservice Segmentation -Microservice segmentation, sometimes visual, of microservices is the -segmentation in a microservices application architecture that enables +## Microservice Segmentation +Microservice segmentation, sometimes visual, of microservices is the +segmentation in a microservices application architecture that enables administrators to view their functions and interactions. -## Service-to-service communication -Service-to-service communication, sometimes referred to as -inter-service communication, is the ability of a microservice -application instance to communicate with another to collaborate and +## Service-to-service communication +Service-to-service communication, sometimes referred to as +inter-service communication, is the ability of a microservice +application instance to communicate with another to collaborate and handle client requests. -## Software as a Service -Software as a Service is a licensing and delivery approach to software -delivery where the software is hosted by a provider and licensed -to users on a subscription basis. +## Software as a Service +Software as a Service is a licensing and delivery approach to software +delivery where the software is hosted by a provider and licensed +to users on a subscription basis. diff --git a/website/content/docs/troubleshoot/faq.mdx b/website/content/docs/troubleshoot/faq.mdx index c57ad30232ab..5e733f1cd1af 100644 --- a/website/content/docs/troubleshoot/faq.mdx +++ b/website/content/docs/troubleshoot/faq.mdx @@ -69,7 +69,7 @@ and [`disable_update_check`](/consul/docs/agent/config/config-files#disable_upda ### Q: Does Consul rely on UDP Broadcast or Multicast? -Consul uses the [Serf](https://www.serf.io) gossip protocol which relies on +Consul uses the [Serf](https://github.com/hashicorp/serf/) gossip protocol which relies on TCP and UDP unicast. Broadcast and Multicast are rarely available in a multi-tenant or cloud network environment. For that reason, Consul and Serf were both designed to avoid any dependence on those capabilities.