Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel weighted shortest path #4699

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Parallel weighted shortest path #4699

wants to merge 2 commits into from

Conversation

andyfengHKU
Copy link
Contributor

@andyfengHKU andyfengHKU commented Jan 12, 2025

Description

This PR implements the initial version of bellman-ford parallel weighted shortest path. The current version only tracks total weight and dst nodes. The grammar is [* WSHORTEST (cost)]. We only support edge property as weight but not general expression. All numerical types except int128 & int128-backed decimal are supported.

This PR contains a refactor that removes DestinationOutput(Writer) and let SingleShortestPath & AllShortestPath creating their own ones. This creates a bit of code duplication but the previous inheritance is killing me and I don't think it's actually maintainable for the long term. I'll add more PRs to get rid of this Output & OutputWriter design.

There will be following PRs that supports

  • tracking path
  • all weighted shortest path.

Fixes # (issue)

Contributor agreement

@andyfengHKU andyfengHKU changed the title Draft parallel weighted shortest path Parallel weighted shortest path Jan 14, 2025
@andyfengHKU andyfengHKU marked this pull request as ready for review January 14, 2025 13:44
Copy link

Benchmark Result

Master commit hash: f23ed68f79eaf187d7b554a15a5e18280c5e7f3f
Branch commit hash: 861ab119bbb1fe011c122848c0e537faabb8aadb

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 640.35 655.21 -14.86 (-2.27%)
aggregation q28 11823.51 11572.14 251.37 (2.17%)
filter q14 118.87 135.43 -16.56 (-12.23%)
filter q15 119.64 133.20 -13.56 (-10.18%)
filter q16 291.94 312.37 -20.44 (-6.54%)
filter q17 437.56 458.02 -20.47 (-4.47%)
filter q18 1961.09 1967.33 -6.25 (-0.32%)
filter zonemap-node 81.13 97.40 -16.27 (-16.70%)
filter zonemap-node-lhs-cast 81.41 97.26 -15.85 (-16.29%)
filter zonemap-node-null 77.03 93.74 -16.72 (-17.83%)
filter zonemap-rel 5890.83 5831.50 59.33 (1.02%)
fixed_size_expr_evaluator q07 573.04 579.82 -6.79 (-1.17%)
fixed_size_expr_evaluator q08 804.23 812.66 -8.43 (-1.04%)
fixed_size_expr_evaluator q09 801.84 808.65 -6.80 (-0.84%)
fixed_size_expr_evaluator q10 237.75 245.36 -7.60 (-3.10%)
fixed_size_expr_evaluator q11 230.52 238.47 -7.95 (-3.34%)
fixed_size_expr_evaluator q12 225.47 235.94 -10.47 (-4.44%)
fixed_size_expr_evaluator q13 1452.75 1459.84 -7.09 (-0.49%)
fixed_size_seq_scan q23 112.24 122.12 -9.89 (-8.10%)
join q29 599.08 592.70 6.38 (1.08%)
join q30 9692.73 10267.07 -574.34 (-5.59%)
join q31 4.26 7.03 -2.77 (-39.35%)
join SelectiveTwoHopJoin 51.63 54.65 -3.02 (-5.52%)
ldbc_snb_ic q35 2714.01 2695.09 18.92 (0.70%)
ldbc_snb_ic q36 449.31 439.63 9.69 (2.20%)
ldbc_snb_is q32 7.14 7.60 -0.46 (-6.02%)
ldbc_snb_is q33 16.35 15.94 0.41 (2.59%)
ldbc_snb_is q34 1.35 1.29 0.07 (5.05%)
multi-rel multi-rel-large-scan 1347.81 1332.79 15.02 (1.13%)
multi-rel multi-rel-lookup 22.21 31.93 -9.72 (-30.44%)
multi-rel multi-rel-small-scan 63.81 84.18 -20.36 (-24.19%)
order_by q25 120.46 141.75 -21.30 (-15.02%)
order_by q26 446.86 459.68 -12.82 (-2.79%)
order_by q27 1483.83 1465.23 18.60 (1.27%)
recursive_join recursive-join-bidirection 307.86 302.16 5.70 (1.89%)
recursive_join recursive-join-dense 7405.71 7367.05 38.66 (0.52%)
recursive_join recursive-join-path 24056.92 24040.54 16.38 (0.07%)
recursive_join recursive-join-sparse 1065.00 1059.02 5.98 (0.56%)
recursive_join recursive-join-trail 7334.77 7351.04 -16.27 (-0.22%)
scan_after_filter q01 165.22 177.67 -12.45 (-7.01%)
scan_after_filter q02 150.94 165.91 -14.97 (-9.02%)
shortest_path_ldbc100 q37 89.67 84.14 5.53 (6.57%)
shortest_path_ldbc100 q38 374.07 376.26 -2.19 (-0.58%)
shortest_path_ldbc100 q39 64.45 63.48 0.97 (1.53%)
shortest_path_ldbc100 q40 453.61 434.82 18.79 (4.32%)
var_size_expr_evaluator q03 2116.30 2049.98 66.32 (3.23%)
var_size_expr_evaluator q04 2256.80 2246.99 9.81 (0.44%)
var_size_expr_evaluator q05 2626.17 2622.11 4.06 (0.15%)
var_size_expr_evaluator q06 1349.47 1334.33 15.14 (1.13%)
var_size_seq_scan q19 1472.54 1441.76 30.78 (2.14%)
var_size_seq_scan q20 2738.29 2771.87 -33.58 (-1.21%)
var_size_seq_scan q21 2375.45 2307.66 67.79 (2.94%)
var_size_seq_scan q22 129.39 128.50 0.89 (0.69%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant