Skip to content

Parallelize interpolation build #264

Open

Description

@orangejulius

This issue is to record the fact that the process of building the interpolation database is single threaded and can be quite slow.

For a full planet build you can expect it to take 6+ days. We recommend avoiding setting up the interpolation database when initially experimenting with Pelias to save yourself some time.

We welcome efforts to parallelize it, but beware: it's not as easy as it sounds. Feel free to shoot an email to team@pelias.io if you are interested in helping out.

Over time we may add more details here about the architecture of the interpolation service and why it's single threaded :)

Activity

  1. pinned this issue on Sep 15, 2020
  2. joacub commented on Nov 7, 2020

    @joacub

    Why it is single thread, i can see that could be mutlithread processed bounch of files at the same time will reduce a lot the times

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions