-
Notifications
You must be signed in to change notification settings - Fork 892
fix: better parsing for pep dependencies in pyodide #7670
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR improves dependency parsing for PEP 440 version specifiers in Pyodide environments. The changes enhance the strip_version function to properly handle extras (e.g., package[extra]), environment markers (e.g., ;python_version>='3.8'), URL dependencies, and various version specifiers (==, >=, ~=, etc.). Additionally, the code now deduplicates module names and filters out nested imports like matplotlib.pyplot to just matplotlib.
Key Changes
- Enhanced
strip_versionfunction with comprehensive PEP 440 version specifier support including extras, environment markers, and URL dependencies - Changed module name handling from list to set to deduplicate imports and filter nested module imports (e.g.,
matplotlib.pyplot→matplotlib) - Added comprehensive test coverage for various PEP 440 patterns including edge cases
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| marimo/_pyodide/pyodide_session.py | Enhanced strip_version function with better PEP 440 parsing, added support for extras and environment markers, and improved module name deduplication by converting to set and filtering nested imports |
| tests/_pyodide/test_pyodide_session.py | Added test cases for complex dependencies with extras, comprehensive PEP 440 version specifier tests, and updated existing tests to handle nested imports like matplotlib.pyplot |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
marimo/_pyodide/pyodide_session.py
Outdated
|
|
||
| # Pyodide find_imports is aggressive and grabs nested imports | ||
| # so we filter them out | ||
| module_names = set(mod.split(".")[0] for mod in module_names) |
Copilot
AI
Jan 5, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code on line 225 converts module_names to a set again using a set comprehension, but module_names is already a set from line 219. This redundant conversion is unnecessary since sets are mutable and can be directly reassigned. Consider: module_names = {mod.split(".")[0] for mod in module_names} or simply reassigning the variable.
| module_names = set(mod.split(".")[0] for mod in module_names) | |
| module_names = {mod.split(".")[0] for mod in module_names} |
marimo/_pyodide/pyodide_session.py
Outdated
| # Handle URL dependencies (package @ https://...) - leave as-is | ||
| if "@" in dep and ("http://" in dep or "https://" in dep): | ||
| return dep |
Copilot
AI
Jan 5, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The URL dependency detection on line 194 only checks for "http://" and "https://" schemes. However, PEP 508 allows other URL schemes for direct references including "git+https://", "git+ssh://", "file://", "ftp://", etc. For example, a dependency like package @ git+https://github.com/user/repo.git would not be properly handled as a URL dependency. Consider expanding the check to handle other valid URL schemes or use a more robust URL detection method such as checking for common URL schemes or using regex to detect URLs after the "@" symbol.
| # Handle URL dependencies (package @ https://...) - leave as-is | |
| if "@" in dep and ("http://" in dep or "https://" in dep): | |
| return dep | |
| # Handle URL dependencies (package @ <url>) - leave as-is. | |
| # Detect a URL scheme after the "@" using a generic scheme pattern, | |
| # e.g. git+https, git+ssh, file, ftp, http, https, etc. | |
| if "@" in dep: | |
| _, rhs = dep.split("@", 1) | |
| rhs = rhs.strip() | |
| if re.match(r"^[a-zA-Z][a-zA-Z0-9+.\-]*://", rhs): | |
| return dep |
marimo/_pyodide/pyodide_session.py
Outdated
| module_names: set[str] = set(pyodide.code.find_imports(code)) # type: ignore | ||
| if not isinstance(module_names, set): | ||
| return [] | ||
|
|
Copilot
AI
Jan 5, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The type check on line 220 is ineffective because the code explicitly wraps the result in set() on line 219. Since set(...) always returns a set, the isinstance check will always be True. If the goal is to validate the return value from pyodide.code.find_imports, the check should be performed before the set conversion. Consider checking the type before wrapping: result = pyodide.code.find_imports(code) followed by if not isinstance(result, list): return [], then module_names = set(result).
| module_names: set[str] = set(pyodide.code.find_imports(code)) # type: ignore | |
| if not isinstance(module_names, set): | |
| return [] | |
| imports = pyodide.code.find_imports(code) # type: ignore | |
| if not isinstance(imports, list): | |
| return [] | |
| module_names: set[str] = set(imports) |
|
|
||
| # Handle URL dependencies (package @ <url>) - leave as-is | ||
| # PEP 508 allows various URL schemes: http, https, git+https, git+ssh, file, ftp, etc. | ||
| if "@" in dep: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should come after ; check
package@https://; python_version>='3.8'
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is actually fine. I think micropip should be the one handling it.
dmadisetti
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Woops, small bug I think
Better parsing for script header in pyodide following
PEP 440