Skip to content

feat(backend): add trace to outgoing payment lifecycle #2884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 27, 2024
Merged

feat(backend): add trace to outgoing payment lifecycle #2884

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2c8fc82
feat(backend): add trace to outgoing payment lifecycle
njlie Aug 22, 2024
d49321c
feat(backend): encapsulate query lookup in trace
mkurapov Aug 27, 2024
dfe6b1d
chore(localenv): remove traces from default localenv
mkurapov Aug 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions localenv/cloud-nine-wallet/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ services:
WALLET_ADDRESS_URL: ${CLOUD_NINE_WALLET_ADDRESS_URL:-https://cloud-nine-wallet-backend/.well-known/pay}
ILP_CONNECTOR_URL: ${CLOUD_NINE_CONNECTOR_URL:-http://cloud-nine-wallet-backend:3002}
ENABLE_TELEMETRY: true
ENABLE_TELEMETRY_TRACES: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think since we have this in the telemetry docker compose we can keep it turned off here IMO

KEY_ID: 7097F83B-CB84-469E-96C6-2141C72E22C0
depends_on:
- shared-database
Expand Down
1 change: 1 addition & 0 deletions localenv/happy-life-bank/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ services:
REDIS_URL: redis://shared-redis:6379/2
WALLET_ADDRESS_URL: ${HAPPY_LIFE_BANK_WALLET_ADDRESS_URL:-https://happy-life-bank-backend/.well-known/pay}
ENABLE_TELEMETRY: true
ENABLE_TELEMETRY_TRACES: true
KEY_ID: 53f2d913-e98a-40b9-b270-372d0547f23d
depends_on:
- cloud-nine-backend
Expand Down
27 changes: 18 additions & 9 deletions packages/backend/src/open_payments/payment/outgoing/worker.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { OutgoingPayment, OutgoingPaymentState } from './model'
import { LifecycleError, PaymentError } from './errors'
import * as lifecycle from './lifecycle'
import { PaymentMethodHandlerError } from '../../../payment-method/handler/errors'
import { trace, Span } from '@opentelemetry/api'

// First retry waits 10 seconds, second retry waits 20 (more) seconds, etc.
export const RETRY_BACKOFF_SECONDS = 10
Expand Down Expand Up @@ -63,16 +64,24 @@ async function handlePaymentLifecycle(
deps: ServiceDependencies,
payment: OutgoingPayment
): Promise<void> {
if (payment.state !== OutgoingPaymentState.Sending) {
deps.logger.warn('unexpected payment in lifecycle')
return
}
const tracer = trace.getTracer('outgoing_payment_worker')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, let us also include the code where we actually fetch the outgoing payment from the DB (it's happening in getPendingPayment)

await tracer.startActiveSpan(
'outgoingPaymentLifecycle',
async (span: Span) => {
if (payment.state !== OutgoingPaymentState.Sending) {
deps.logger.warn('unexpected payment in lifecycle')
span.end()
return
}

try {
await lifecycle.handleSending(deps, payment)
} catch (error) {
await onLifecycleError(deps, payment, error as Error | PaymentError)
}
try {
await lifecycle.handleSending(deps, payment)
} catch (error) {
await onLifecycleError(deps, payment, error as Error | PaymentError)
}
span.end()
}
)
}

async function onLifecycleError(
Expand Down