Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Register
  • Sign in
  • A Ahad Mobile
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Terraform modules
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • holatech
  • Ahad Mobile
  • Merge requests
  • !90

fix(fireblocks): lock the balance row before every crypto ledger write

  • Review changes

  • Download
  • Patches
  • Plain diff
Merged Khanh Pham requested to merge fix/crypto-balance-cache-lock into develop Oct 02, 2026
  • Overview 0
  • Commits 3
  • Pipelines 1
  • Changes 4

Problem

Every crypto balance writer in FireblocksLedgerService (deposit, withdrawal, refund, transfer, conversion in/out, manual adjustment) computed balance_after from an unlocked read of user_crypto_balances and then upserted the absolute value. Two concurrent writers on the same row lost an update.

Example: a user has 100 USDC and converts 30, while a 50 USDC deposit webhook arrives at the same moment. The deposit reads the stale 100 and writes 150, but the correct balance is 120. The ledger entries are still complete, so the cache silently drifts from the ledger until reconciliation catches it.

Fireblocks webhooks are only serialised per transaction. Two deposits, or a deposit and a withdrawal for the same user, can therefore run in parallel without any user action.

Fix

  • calculateBalancesFromCache reads through lockUserBalanceForUpdate (SELECT ... FOR UPDATE), and manager is now required. This single change covers every writer.
  • recordTransfer locks the sender and receiver rows up front, in postLendingJournal's order: control rows first, then user rows by id. This avoids an A→B / B→A deadlock and matches the order used by the conversion provider, ramp reversal and lending.
  • approveManualAdjustment now locks the HOUSE row before the user row, following the same global order.

Tests

  • New fireblocks-ledger.concurrency.postgres.spec.ts, run on real Postgres over two connections:
    • a deposit blocked behind an open locking transaction (checked with pg_blocking_pids)
    • two concurrent deposits
    • opposite-direction transfers with no 40P01
  • Mutation checks:
    • Removing the lock turns the spec red (expected 190, got 140).
    • Removing the up-front transfer locks turns it red with deadlock detected.
    • Swapping the manual-adjustment order back turns its new unit test red.
  • Results:
    • tsc is clean, and yarn jest src/modules/fireblocks passes (180 tests).
    • The fireblocks-ledger DB specs pass.
    • On MR !88 (merged) plus this branch, 34/34 conversion and ledger suites pass, including 16 simultaneous mixed-direction converts with no deadlock.
  • Already failing on develop (not caused by this branch):
    • The ramp-order and lending postgres specs fail at seed with CK_user_crypto_balances_account_code, identically on the base commit.
    • admin-lending.risk.spec.ts fails with a Nest DI error and is unrelated to this diff.

Opus review gate: PASS after 2 rounds.

Follow-ups (not in this MR)

  • The internal user-to-user transfer checks the sender's balance outside the transaction and without a lock (fireblocks.service.ts ~2857). Two concurrent transfers can therefore push the sender negative.
  • syncUserBalance is dead code and still does an unlocked read with an absolute write.
  • Lock-order inversion on USD accounts between Convert and card deposit/transfer. It is rare, can cause a 500 on the deposit side, and never misposts money.

🤖 Generated with Claude Code

Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: fix/crypto-balance-cache-lock