Record · corpus.blog/posts/01a0877c-4f7d-709c-90ad-c4fe62045879

Optimization adventures: making a parallel Rust workload 10x faster with (or without) Rayon

gendignoux.com · published 18 November 2024

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at gendignoux.com Wayback capture

0blogs citing

First cited: —
Most recent: —
Rank this month: not ranked
Links from this post: 101
Words captured: 5,979

Cited by

No blog we hold has cited this post yet.

Links from this post

Link

Host

STV-rs: Single Transferable Vote implementation in Rust Read at the source ↗

gendignoux.com

https://docs.rs/rayon/

docs.rs

https://gendignoux.com/blog/images/rust-rayon-optimized/plot-parallel-rand2x10.6H1A44kAkKxR.svg

gendignoux.com

Optimization adventures: making a parallel Rust workload even faster with data-oriented design (and other tricks) Read at the source ↗

gendignoux.com

https://gendignoux.com/blog/images/rust-data-oriented-design/plot-stv.473bcd6-parallel-rand2x10.OD8zhsdTu2Gu.svg

gendignoux.com

https://github.com/gendx/stv-rs/tree/473bcd652372e341e934b630dd71c02ec65fd5fa

github.com

Lessons learned from stracing a password manager in Docker | Blog | Guillaume Endignoux Read at the source ↗

gendignoux.com

Tutorial: Profiling Rust applications in Docker with perf | Blog | Guillaume Endignoux Read at the source ↗

gendignoux.com

https://man7.org/linux/man-pages/man1/strace.1.html

man7.org

https://man7.org/linux/man-pages/man2/futex.2.html

man7.org

https://man7.org/linux/man-pages/man2/sched_yield.2.html

man7.org

https://man7.org/linux/man-pages/man2/mmap.2.html

man7.org

https://man7.org/linux/man-pages/man2/write.2.html

man7.org

https://www.brendangregg.com/blog/2014-05-11/strace-wow-much-syscall.html

brendangregg.com

https://www.brendangregg.com/perf.html

brendangregg.com

https://github.com/moby/moby/issues/39457

github.com

https://www.man7.org/linux/man-pages/man1/perf.1.html

man7.org

https://www.brendangregg.com/flamegraphs.html

brendangregg.com

https://perfwiki.github.io/main/tutorial/

perfwiki.github.io

https://doc.rust-lang.org/cargo/reference/profiles.html

doc.rust-lang.org

https://man7.org/linux/man-pages/man1/perf-annotate.1.html

man7.org

https://profiler.firefox.com

profiler.firefox.com

https://man7.org/linux/man-pages/man1/perf-script.1.html

man7.org

https://gendignoux.com/blog/images/rust-rayon-optimized/firefox-profiler-tree-view.9RCOOeaIZuqM.png

gendignoux.com

https://docs.rs/rayon/1.10.0/rayon/iter/plumbing/fn.bridge_producer_consumer.html

docs.rs

https://docs.rs/rayon-core/1.12.1/rayon_core/fn.join_context.html

docs.rs

https://github.com/rayon-rs/rayon/issues/591

github.com

https://www.reddit.com/r/rust/comments/bto10h/update_a_scaling_comparison_between_rustrayon_and/

reddit.com

https://gendignoux.com/blog/images/rust-rayon-optimized/firefox-profiler-flame-graph.KTuEz_ZuAmzu.png

gendignoux.com

https://docs.rs/stv-rs/0.4.0/stv_rs/vote_count/struct.VoteCount.html

docs.rs

https://gendignoux.com/blog/images/rust-rayon-optimized/firefox-profiler-collapse-recursion.aZXhPOsYxBFY.png

gendignoux.com

https://gendignoux.com/blog/images/rust-rayon-optimized/firefox-profiler-invert-call-stack.3ae6qbf5XwvP.png

gendignoux.com

https://docs.rs/num-bigint/0.4.6/num_bigint/struct.BigUint.html

docs.rs

https://man7.org/linux/man-pages/man1/perf-stat.1.html

man7.org

https://en.wikipedia.org/wiki/Superscalar_processor

en.wikipedia.org

https://en.wikipedia.org/wiki/Branch_(computer_science

en.wikipedia.org

https://en.wikipedia.org/wiki/Speculative_execution

en.wikipedia.org

https://perf.rust-lang.org/index.html

perf.rust-lang.org

https://man7.org/linux/man-pages/man1/perf-list.1.html

man7.org

https://en.wikipedia.org/wiki/CPU_cache

en.wikipedia.org

https://man7.org/linux/man-pages/man1/lscpu.1.html

man7.org

https://www.youtube.com/watch?v=WDIkqP4JbkE

youtube.com

https://en.wikipedia.org/wiki/Simultaneous_multithreading

en.wikipedia.org

https://doc.rust-lang.org/stable/std/thread/fn.available_parallelism.html

doc.rust-lang.org

https://lib.rs/crates/rayon

lib.rs

https://blog.rust-lang.org/2023/11/09/parallel-rustc.html

blog.rust-lang.org

https://github.com/nikomatsakis

github.com

Rayon: data parallelism in Rust Read at the source ↗

smallcultfollowing.com

Parallel Iterators Part 1: Foundations Read at the source ↗

smallcultfollowing.com

Parallel Iterators Part 2: Producers Read at the source ↗

smallcultfollowing.com

Parallel iterators, part 3: Consumers Read at the source ↗

smallcultfollowing.com

https://doc.rust-lang.org/std/iter/trait.Iterator.html

doc.rust-lang.org

https://docs.rs/rayon-core/

docs.rs

https://github.com/rayon-rs/rayon/blob/v1.10.0/rayon-core/src/sleep/README.md

github.com

https://docs.rs/crossbeam/0.8.4/crossbeam/deque/index.html

docs.rs

https://gendignoux.com/blog/images/rust-rayon-optimized/diagram-xbeam-deque.sX_dAFfyPfkl.svg

gendignoux.com

https://docs.rs/rayon/1.10.0/rayon/iter/trait.ParallelIterator.html

docs.rs

https://docs.rs/rayon/1.10.0/rayon/iter/plumbing/index.html

docs.rs

https://github.com/rayon-rs/rayon/blob/v1.10.0/src/iter/plumbing/README.md

github.com

https://docs.rs/rayon/1.10.0/rayon/iter/plumbing/trait.Producer.html

docs.rs

https://docs.rs/rayon/1.10.0/rayon/iter/plumbing/trait.Consumer.html

docs.rs

https://docs.rs/rayon/1.10.0/rayon/iter/plumbing/trait.Folder.html

docs.rs

https://docs.rs/rayon/1.10.0/rayon/iter/plumbing/trait.Reducer.html

docs.rs

https://docs.rs/rayon/1.10.0/rayon/iter/plumbing/trait.ProducerCallback.html

docs.rs

https://docs.rs/rayon/1.10.0/rayon/iter/plumbing/fn.bridge.html

docs.rs

https://docs.rs/rayon/1.10.0/rayon/iter/trait.IntoParallelRefIterator.html

docs.rs

https://gendignoux.com/blog/images/rust-rayon-optimized/diagram-iterator-pipeline.FthmFKlvoetE.svg

gendignoux.com

https://gendignoux.com/blog/images/rust-rayon-optimized/diagram-rayon-bridge.SbD08MV8eBps.svg

gendignoux.com

https://gendignoux.com/blog/images/rust-rayon-optimized/diagram-rayon-split-reduce.PQaqdFEWLsKG.svg

gendignoux.com

https://docs.rs/rayon/1.10.0/rayon/iter/trait.IndexedParallelIterator.html

docs.rs

https://docs.rs/rayon/1.10.0/src/rayon/iter/plumbing/mod.rs.html

docs.rs

https://gendignoux.com/blog/images/rust-rayon-optimized/diagram-work-stealing-tree.IuFfzMvksmuz.svg

gendignoux.com

https://github.com/gendx/stv-rs/blob/v0.5.0/testdata/ballots/skewed.blt

github.com

https://doc.rust-lang.org/std/sync/struct.Mutex.html

doc.rust-lang.org

https://doc.rust-lang.org/std/sync/struct.Condvar.html

doc.rust-lang.org

https://gendignoux.com/blog/images/rust-rayon-optimized/diagram-partition._i4DgPpWvSPa.svg

gendignoux.com

https://doc.rust-lang.org/stable/std/thread/fn.scope.html

doc.rust-lang.org

https://doc.rust-lang.org/std/sync/struct.Arc.html

doc.rust-lang.org

https://doc.rust-lang.org/std/sync/struct.RwLock.html

doc.rust-lang.org

https://github.com/gendx/stv-rs/commit/93f36a67d04df8e958fb2523cdff9662cbca6472

github.com

https://gendignoux.com/blog/images/rust-rayon-optimized/diagram-cpu-migration.SsgVp5XzlqyS.svg

gendignoux.com

https://docs.rs/nix/0.29.0/nix/sched/fn.sched_setaffinity.html

docs.rs

https://github.com/rayon-rs/rayon/issues/319

github.com

https://github.com/gendx/stv-rs/commit/20069a43ba2a1f35b4cac56a8105679c6e1bb39e

github.com

https://gendignoux.com/blog/images/rust-rayon-optimized/diagram-unbalanced.7FakR9__G8e_.svg

gendignoux.com

https://github.com/gendx/stv-rs/tree/v0.5.0/testdata/shuffle_ballots

github.com

https://en.wikipedia.org/wiki/Work_stealing

en.wikipedia.org

https://github.com/gendx/stv-rs/blob/v0.5.0/src/parallelism/range.rs

github.com

https://gendignoux.com/blog/images/rust-rayon-optimized/diagram-work-stealing.MWPI2yOsEnAr.svg

gendignoux.com

https://doc.rust-lang.org/std/ops/struct.Range.html

doc.rust-lang.org

https://gendignoux.com/blog/images/rust-rayon-optimized/diagram-work-stealing-half.fp82nhHRvodr.svg

gendignoux.com

https://en.wikipedia.org/wiki/Cache_prefetching

en.wikipedia.org

https://github.com/gendx/stv-rs/commit/f324c337ba958a099bd8473653741dd5dd6c8559

github.com

https://github.com/gendx/paralight

github.com

https://www.reddit.com/r/rust/comments/1gu8t2j/optimization_adventures_making_a_parallel_rust/

reddit.com

https://github.com/gendx/stv-rs/commit/6f5fdf7a3a008ee8c2eb2c428ec52b7913c36ad1

github.com

https://gendignoux.com/blog/images/rust-rayon-optimized/plot-stv-max-len-rand2x10.LtzulBTNnmjH.svg

gendignoux.com

https://github.com/gendx/stv-rs/blob/v0.5.0/testdata/ballots/random/rand_2x10.blt

github.com

https://gendignoux.com/blog/images/rust-rayon-optimized/plot-stv-max-len-skewed.7U9gvg3iNsvX.svg

gendignoux.com

https://infosec.exchange/@gendx/113504908007896810

infosec.exchange

https://lobste.rs/s/kfw82m/optimization_adventures_making

lobste.rs