[Core] Discussion to support Valkey instead of or alongside Redis

[dtaivpp]

Description

Discussion thread for whether to support Valkey

With the change of Redis' licensing many users may not be able to deploy Redis due to company policy. I wanted to start a discussion thread to see if Ray should move to support Valkey alongside Redis or instead of Redis.

I am not overly familiar with Ray's core but with a quick look it seems the code as ~200 references to Redis. I think out the gate supporting Valkey and Redis both may be fairly straightforward however as Valkey matures it may deprecate API's supported by Redis and the other way around.

Context for those not familiar: Redis has moved from a OSI approved open source license to a dual license model with RSALv2 and SSPL. These licenses prohibit companies from running Redis as a service without paying appropriate license fees. While many companies may still run it there are many more who will be unwilling to take the risk (from my experience with the Elasticsearch -> OpenSearch fork).

I think supporting Valkey is important to ensure that users are able to freely adopt Ray but wanted to get others thoughts.

Edit:
I forgot to mention this work probably does not to be immediate as Valkey has only just recently been formed but it is likely going to be the primary distribution going forward supported by AWS, Google Cloud, and Oracle due to the license change. See Linux Foundation thread: https://www.linuxfoundation.org/press/linux-foundation-launches-open-source-valkey-community

[anyscalesam]

thanks for raising this @dtaivpp > valkey's a bit new but we understand the friction point on the new redis license. will make a call on when we add this support (i think it'll most likely be a when and not if) for our next 6w sprint planning.

[dtaivpp]

Sounds good to me. Just my two cents, if it's possible I think it would be best to support both if you all can swing it. For the self-hosters I feel like Valkey is going to be the go to option. At least for the next few years though I feel like most people will keep redis. I imagine they will change in their core features at some point which would make it handy to have an adapter that can support either or.