Fix docstring removing regex to support empty parentheses

[noisycat3]

While loading a Qwen3VL-derived model I got the following error on from_pretrained call:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/unsloth_zoo/compiler.py", line 667, in create_new_function
    new_module, old_path = import_module(compile_folder, name)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/unsloth_zoo/compiler.py", line 663, in import_module
    raise e
  File "/usr/local/lib/python3.12/dist-packages/unsloth_zoo/compiler.py", line 658, in import_module
    new_module = importlib.import_module(name)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 991, in exec_module
  File "<frozen importlib._bootstrap_external>", line 1129, in get_code
  File "<frozen importlib._bootstrap_external>", line 1059, in source_to_code
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/trainer/unsloth_compiled_cache/unsloth_compiled_module_qwen3_vl.py", line 612
    ()
    ^
SyntaxError: invalid syntax
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/unsloth_zoo/compiler.py", line 694, in create_new_function
    spec.loader.exec_module(new_module)
  File "<frozen importlib._bootstrap_external>", line 991, in exec_module
  File "<frozen importlib._bootstrap_external>", line 1129, in get_code
  File "<frozen importlib._bootstrap_external>", line 1059, in source_to_code
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/tmp/unsloth_compiled_cache/unsloth_compiled_module_qwen3_vl.py", line 612
    ()
    ^
SyntaxError: invalid syntax
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/unsloth_zoo/compiler.py", line 2971, in unsloth_compile_transformers
    combined_module = create_new_function(
                      ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/unsloth_zoo/compiler.py", line 696, in create_new_function
    raise RuntimeError(f"Direct module loading failed for {name}: {e}")
RuntimeError: Direct module loading failed for unsloth_compiled_module_qwen3_vl: invalid syntax (unsloth_compiled_module_qwen3_vl.py, line 612)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/trainer/app.py", line 54, in <module>
    raise SystemExit(main())
                     ^^^^^^
  File "/trainer/app.py", line 50, in main
    return run_training(config)
           ^^^^^^^^^^^^^^^^^^^^
  File "/trainer/trainer/pipeline.py", line 78, in run_training
    model, tokenizer = initialize_model(config.model_loader, config.peft)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/trainer/trainer/modeling.py", line 38, in initialize_model
    model, tokenizer = FastVisionModel.from_pretrained(**loader_kwargs)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/unsloth/models/loader.py", line 1009, in from_pretrained
    model_types, supports_sdpa = unsloth_compile_transformers(
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/unsloth/models/_utils.py", line 1858, in unsloth_compile_transformers
    _unsloth_compile_transformers(
  File "/usr/local/lib/python3.12/dist-packages/unsloth_zoo/compiler.py", line 2983, in unsloth_compile_transformers
    raise RuntimeError(exception)
RuntimeError: Direct module loading failed for unsloth_compiled_module_qwen3_vl: invalid syntax (unsloth_compiled_module_qwen3_vl.py, line 612)

I've traced the issue to a syntax error in the generated unsloth code - notice the faulty ():

@torch.compiler.disable(recursive = False)
()
def Qwen3VLModel_forward(
    self,
    input_ids: torch.LongTensor = None,
    attention_mask: Optional[torch.Tensor] = None,
    position_ids: Optional[torch.LongTensor] = None,
    past_key_values: Optional[Cache] = None,
    inputs_embeds: Optional[torch.FloatTensor] = None,
    pixel_values: Optional[torch.Tensor] = None,
    pixel_values_videos: Optional[torch.FloatTensor] = None,
    image_grid_thw: Optional[torch.LongTensor] = None,
    video_grid_thw: Optional[torch.LongTensor] = None,
    cache_position: Optional[torch.LongTensor] = None,
    **kwargs: Unpack[TransformersKwargs],
) -> Union[tuple, Qwen3VLModelOutputWithPast]:

It's caused by the regex not matching decorator with empty argument parentheses.

Ref: https://github.com/huggingface/transformers/blob/8f129d256dfbd414b47f7f4a495a7834d95ffbf8/src/transformers/models/qwen3_vl/modeling_qwen3_vl.py#L1132

    @auto_docstring
    @check_model_inputs() # <-- this wasn't matched
    def forward(
        self,
        input_ids: torch.LongTensor = None,
        attention_mask: Optional[torch.Tensor] = None,

Seems to be caused by a faulty regex in create_standalone_class. My fix is to allow for zero or more characters between ( and ) in the replacement regex instead of expecting at least one non-) character.

Not sure if this was the intended behavior, but it seems to fix my use case.

[gemini-code-assist]

Summary of Changes

Hello @noisycat3, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical SyntaxError encountered during the loading of Qwen3VL-derived models. The root cause was an overly strict regular expression within unsloth_zoo/compiler.py that failed to correctly handle decorators with empty argument lists, such as @check_model_inputs(). The fix modifies this regex to properly account for such cases, preventing the generation of invalid Python syntax and enhancing the robustness of the model compilation process.

Highlights

• Bug Fix: Resolves a SyntaxError that occurred when loading Qwen3VL-derived models due to incorrect processing of decorators with empty parentheses.
• Regex Update: The regular expressions used to remove @auto_docstring and @check_model_inputs decorators have been updated to correctly match decorators with zero or more characters within their parentheses.
• Improved Compatibility: Ensures that the unsloth_zoo/compiler.py can process code containing decorators like @check_model_inputs() without introducing syntax errors into the generated code.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request addresses a SyntaxError that occurs when loading certain models. The error is caused by a regular expression that fails to remove decorators with empty parentheses, such as @check_model_inputs(). The fix correctly modifies the regex to handle empty parentheses by changing the quantifier from {1,} (one or more) to {0,} (zero or more). The change is correct and resolves the issue. I've added a suggestion to further simplify the regex for better readability.

[gemini-code-assist]

While the change is functionally correct, the regular expressions can be simplified for better readability by using common regex quantifiers. [\s]{0,} can be replaced with \s*, and [^\)]{0,} can be replaced with [^\)]*.

Suggested change

	source = re.sub(r"@auto_docstring[\s]{0,}(\([^\)]{0,}\))?", "", source)
	source = re.sub(r"@check_model_inputs[\s]{0,}(\([^\)]{0,}\))?", "", source)
	source = re.sub(r"@auto_docstring\s*(\([^\)]*\))?", "", source)
	source = re.sub(r"@check_model_inputs\s*(\([^\)]*\))?", "", source)

[danielhanchen]

Oh thank you!