Refactor agent rpc retry

[6543]

address #6470 (comment)

[codecov]

Codecov Report

❌ Patch coverage is 63.58025% with 59 lines in your changes missing coverage. Please review.
✅ Project coverage is 41.31%. Comparing base (52ed3f1) to head (234cea2).
⚠️ Report is 34 commits behind head on main.

Files with missing lines	Patch %	Lines
agent/rpc/client_grpc.go	63.58%	53 Missing and 6 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6515      +/-   ##
==========================================
+ Coverage   41.29%   41.31%   +0.01%     
==========================================
  Files         431      431              
  Lines       28792    28615     -177     
==========================================
- Hits        11890    11821      -69     
+ Misses      15836    15724     -112     
- Partials     1066     1070       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[6543]

I want to have the release first, to not get regressions!