refactor: Pathfinding 2.0 using Dijkstra's Algorithm - #7392
refactor: Pathfinding 2.0 using Dijkstra's Algorithm#7392shortthefomo wants to merge 25 commits into
Conversation
shortthefomo
changed the title
dbf762f to
a2bdd34
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #7392 +/- ##
=========================================
- Coverage 82.4% 82.2% -0.2%
=========================================
Files 1011 1014 +3
Lines 76544 76734 +190
Branches 7322 7381 +59
=========================================
+ Hits 63061 63096 +35
- Misses 13483 13638 +155
... and 12 files with indirect coverage changes 🚀 New features to boost your workflow:
|
…shortest by 3x so rankPaths has spares when paths fail rippleCalc
… only the current PayGraph snapshot
…de-cleaner warnings
…ation suppression for Apple libc++ compat
|
This PR has conflicts, please resolve them in order for the PR to be reviewed. |
|
If only the most recent commit is unsigned, you can run:
If multiple commits are unsigned, you can run:
If you're new to commit signing, there are different ways to set it up: Sign commits with
|
|
All conflicts have been resolved. Assigned reviewers can now start or resume their review. |
12d5190 to
e3099f4
Compare
|
This PR has conflicts, please resolve them in order for the PR to be reviewed. |
Resolved merge conflicts by keeping pathfinding-2 (HEAD) changes where conflicts occurred, while incorporating develop features that don't overlap: - .cspell.config.yaml: Added 'decryptor' and 'summands' words from develop - OrderBookDB.h: Kept pathfinding methods (getAllTakerPaysAssets, processTxn, getBookListeners, makeBookListeners) and added affectedBooks function from develop - ConfigSections.h: Kept HEAD version with SECTION_PATH_WORKERS - Config.cpp, PathRequest.h, PathRequestManager.h: Kept HEAD versions - Pathfinder.cpp, Pathfinder.h: Deleted as they were removed in HEAD This preserves pathfinding features while integrating compatible develop changes.
…tes compose multiplicatively via log-space edge weights fix, Top-of-book weights would ignore liquidity depth and exchange rates compose multiplicatively via log-space edge weights
Resolve conflict by keeping Pathfinder.h deleted: pathfinding-2 uses GraphPathfinder as the replacement; develop only had style updates to the old Pathfinder header.
|
All conflicts have been resolved. Assigned reviewers can now start or resume their review. |
![depthfirst-app[bot]](https://avatars.githubusercontent.com/in/1021002?s=60&v=4){kind=small}
…Exception skip so we dont touch consensys
|
This PR has conflicts, please resolve them in order for the PR to be reviewed. |
|
#7962 is the better approach |
1 participant
High Level Overview of Change
XLS: XRPLF/XRPL-Standards#561
Replaces the existing BFS-based
Pathfinderwith a new graph-based pathfinding engine (GraphPathfinder+PayGraph) that uses Dijkstra/Yen's K-Shortest algorithm on a persistent, incrementally-updated asset-exchange graph.The new engine is dramatically faster: the asset graph on mainnet has ~500 vertices and ~1,000 edges — small enough that Yen's K-Shortest runs in microseconds rather than the milliseconds-to-seconds of the previous BFS over the full account+asset space.
Related: pathfinding 2.0 using Dijkstra's Algorithm.
Context of Change
Previous implementation:
Pathfinderperforms BFS over a combined account+asset graph with O(A^D) fanout (A ≈ 20 trust-line neighbours, D ≈ 7 hops) before any pruning, then callsrippleCalculateon up to 1,000 candidates. This is CPU-intensive and latency-sensitive, especially for multi-hop cross-currency payments.New implementation:
PayGraph— a persistent, process-lifetime asset-exchange graph.{currency, issuer}, MPT IDs, XRP)OrderBookDBsignals "ready" (so the first request after startup never pays for the initial walk) and updated incrementally on each ledger close viaapplyLedgerDelta()— typically patching fewer than 100 edges in a few microseconds by re-querying top-of-book for changed books only.std::atomic<shared_ptr<Snapshot>>): pathfinder threads take a snapshot and hold zero locks during search; the ledger-close thread publishes a new snapshot atomically.GraphPathfinder— drop-in replacement forPathfinder.PayGraphsnapshot in O((V+E) log V).kMaxK * 3 = 18candidates per asset pair) so the ranking stage has spares to fall back on when a candidate failsrippleCalculate(thin order books, AMM overflow, etc).STPath(book nodes only; the XRPL payment engine handles trust-line rippling implicitly).rippleCalculate, returning at mostkMaxK = 6viable paths. Early-exits ranking oncekMaxKsucceed and 3 consecutive failures follow, so wasted work is bounded.PathRequest/PathRequestManagerprogressive-refinement WebSocket pipeline unchanged.PayGraphDelta— extracts the set of changed order books from transaction metadata (ltOFFER,ltAMMchanges) to driveapplyLedgerDelta().Configuration changes:
[path_search]is now a boolean flag (0 = disabled [default], 1 = enabled). The old aggressiveness scale (path_search_fast,path_search_max,path_search_old) is removed.OrderBookDB— addedgetAllTakerPaysAssets()to support initial graph population.API Impact
path_find/ripple_path_findresponse shape is unchanged)libxrplchangeOrderBookDB.h(part oflibxrpl) gains a new pure virtual methodgetAllTakerPaysAssets(). Dependents that subclassOrderBookDBwill need to implement it.Before / After
Before:
path_findtriggers BFS over account+asset graph: O(A^D) node expansions, up to ~1,000rippleCalculatecalls per update tick.path_search(2),path_search_fast(2),path_search_max(3),path_search_old(2).After:
path_findruns Yen's K-Shortest on the tiny asset graph (μs), materialises ≤18 candidates, callsrippleCalculateuntil 6 succeed (bounded early-exit on consecutive failures).path_search = 1to enable). Off by default.Test Plan
Path_test.cpp,PathMPT_test.cpp) updated to compile against the new engine interface.path_findWebSocket subscription returns alternatives with paths within the first update tick after ledger close, and consistently returns the requested number of paths (6) for liquid pairs like XRP→RLUSD.ripple_path_find(synchronous RPC) confirmed working end-to-end.