[FSSDK-9973] doc: update readme with webhook & notification synchroni…

…zation doc (#413)

* update webhook doc

* update doc

* add line break

* Update README.md

Co-authored-by: Jae Kim <[email protected]>

* Update README.md

Co-authored-by: Jae Kim <[email protected]>

* Update README.md

Co-authored-by: Jae Kim <[email protected]>

---------

Co-authored-by: Jae Kim <[email protected]>

README.md

@@ -186,6 +186,29 @@ To accept webhook requests Agent must be configured by mapping an Optimizely Pro
 with the associated secret used for validating the inbound request. An example webhook configuration can
 be found in the the provided [config.yaml](./config.yaml).
 
+When running Agent in High Availability (HA) mode, it's important to ensure that all nodes are updated promptly when a webhook event (datafile updated) is received. By default, only one Agent node or instance will receive the webhook notification. A pub-sub system can be used to ensure this.
+
+Redis, a powerful in-memory data structure store, can be used as a relay to propagate the datafile webhook event to all other nodes in the HA setup. This ensures that all nodes are notified about the event and can update their datafiles accordingly.
+
+To set up Redis as a relay, you need to enable the datafile synchronization in your Optimizely Agent configuration. The PubSub feature of Redis is used to publish the webhook notifications to all subscribed Agent nodes.
+
+Here's an example of how you can enable the datafile synchronization with Redis:
+
+```yaml
+## synchronization should be enabled when features for multiple nodes like notification streaming are deployed
+synchronization:
+    pubsub:
+        redis:
+            host: "localhost:6379"
+            password: ""
+            database: 0
+    ## if datafile synchronization is enabled, then for each webhook API call
+    ## the datafile will be sent to all available replicas to achieve better eventual consistency
+    datafile:
+        enable: true
+        default: "redis"
+```
+
 ## Admin API
 
 The Admin API provides system information about the running process. This can be used to check the availability
@@ -432,6 +455,24 @@ Optimizely Agent supports authorization workflows based on OAuth and JWT standar
 
 Just as you can use Notification Listeners to subscribe to events of interest with Optimizely SDKs, you can use the Notifications endpoint to subscribe to events in Agent. For more information, see the [Notifications Guide](https://docs.developers.optimizely.com/experimentation/v4.0.0-full-stack/docs/agent-notifications).
 
+When the Agent is operating in High Availability (HA) mode, you need to enable notification synchronization to get notifications from all nodes in an HA setup. A PubSub system (Redis) is now used to ensure consistent retrieval of notification events across all nodes in an HA setup.
+Here's an example of how you can enable the notification synchronization with Redis:
+
+```yaml
+## synchronization should be enabled when features for multiple nodes like notification streaming are deployed
+synchronization:
+    pubsub:
+        redis:
+            host: "localhost:6379"
+            password: ""
+            database: 0
+    ## if notification synchronization is enabled, then the active notification event-stream API
+    ## will get the notifications from available replicas
+    notification:
+        enable: true
+        default: "redis"
+```
+
 ## Agent Development
 
 ### Package Structure