Continually adding path to sys.path

Question

I have inherited quite a bit of python code and all over it is the following snippet which adds the file path of the parent directory to the system path.

from os.path import join, dirname
sys.path.insert(0, join(dirname(sys.argv[0]), "..\\"))
from utilities import find, execute

My understanding of this is that it adds a path to the search path. Which during the running of a program adds numerous pathsto the search path and presumably make it slower. As each file adds it's own parent directory.

I prefer the syntax

from scm_tools.general.utilities import find, execute

because this is easier to understand and far less code. This might have implications if I am moving the code around but it's all in a single package.

Am I right in assuming that inside a package that the latter syntax is the more pythonic way of doing things ?

or does it not really matter as under the hood python is doing some magic ?

your two examples are not equivelent ... with python 2.7 it introduced relative imports ... I think you would do something like from ..utilities import find,execute ... but yeah generally speaking it is bad form to edit the path from inside your program (every rule has exceptions however) — Joran Beasley
– Joran Beasley, Commented Jan 21, 2016 at 17:47
This is an awful pattern. If the code you mention is all in the same package (scm_tools?), then your preferred syntax is much better. — MattDMo
– MattDMo, Commented Jan 21, 2016 at 17:47
or just use your actual system ENVIRNOMENT to set the path or pythonpath to the desired values (or even your IDE typically) — Joran Beasley
– Joran Beasley, Commented Jan 21, 2016 at 17:48
Another problem with that pattern is that modules inside scm_tools could mask other top level modules. Suppose somebody creates scm_tools/xml.py - a perfectly reasonable thing to do. Then you spend a day wondering why your deployments are crashing on import xml.etree.ElementTree. — tdelaney
– tdelaney, Commented Jan 21, 2016 at 17:59

Pyrce · Accepted Answer · 2016-01-21 18:11:39Z

Use relative imports when you can:

from ..utilities import find, execute

This requires that you stay within the module space, which means each directory you traverse requires an __init__.py file.

There are cases where this breaks down, for example if your tests directory isn't inside the module structure. In these cases you need to edit the path, but you shouldn't edit the path blindly like the above example.

Either add to the PYTHONPATH environment variable before you code starts so you can always reference the root of the directory or only add paths that aren't already in the sys.path and try avoiding adding anything but module roots.

The PYTHONPATH change is a bit risky for code you wish to distribute. It's easy to have a change in PYTHONPATH you can't control or for you to not define that addition in a way that transfers to distributed code. It also adds an annoying module requirement that other's have to deal with -- so reserve this for adding whole swaths of modules that you want to include, like custom site-package directories. It's almost always better to use virtualenv for such situations.

If you do need to change a sys.path inside code you should try to at least avoid clobbering it all over the place or you'll have a headache trying to fix it when it goes awry. To avoid this try to only add root module paths so you can always import in a root.submodule.desiredmodule pattern. Additionally check if a path is already present before you insert it into sys.path to avoid very long sys.paths. In my test directories I oftentimes have an importable file that fixes the sys.path to the root of the directory structures I am testing:

# Add parent import capabilities
parentdir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
if parentdir not in sys.path:
    sys.path.insert(0, parentdir)

Collectives™ on Stack Overflow

Continually adding path to sys.path

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related