Skip to content
This repository was archived by the owner on Apr 22, 2020. It is now read-only.

Parallelise batching of writes for similarity algorithms #814

Open
wants to merge 11 commits into
base: 3.4
Choose a base branch
from
Open

Parallelise batching of writes for similarity algorithms #814

wants to merge 11 commits into from

Conversation

mneedham
Copy link
Contributor

No description provided.

@mneedham
Copy link
Contributor Author

Testing on the recommendations dataset, there are so many partitions that the sequential approach is still faster.

selection_249
@mneedham mneedham force-pushed the 3.4-parallel-similarity-batching branch from 62cae0a to 17d4502 Compare February 12, 2019 08:24
@mneedham mneedham changed the title batching wip Feb 12, 2019
@mneedham mneedham force-pushed the 3.4-parallel-similarity-batching branch from 75cc75e to aed8535 Compare February 12, 2019 10:22
@jexp
Copy link
Member

jexp commented Mar 5, 2019

Hmm do we want to add this as an option to be able to experiment more with it?
@mneedham
Copy link
Contributor Author

We can merge this in but I need to fix the conflicts first
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
2 participants
@mneedham @jexp
close