Bug report

Bug description:

I have a python program (let's call it my_prog.py) that starts multiple threads of periodic tasks in one process. Some periodic tasks involvs writing to a file and calls "setfacl" command after the file is written.

I recently observed one of the threads hangs at subprocess.Popen() when trying to calling setfacl command. I managed to print out the stacktrace of all the threads by using py-spy dump. Here are the detailed infos:

1. There are 2 my_prog.py processes exists, they are parent and child processes, while normally there should only be 1.
2. I dumped out the 2 processes' stacktrace, it looks like (The python version is 3.9.9):

parent process:

Thread 44167 (idle): "Thread-get-lldp-ip"

    _execute_child (subprocess.py:1777)

    __init__ (subprocess.py:951)

   ... some irrelevant stacktraces...

    run (xxx.py: xxx)

    _bootstrap_inner (threading.py:980)

    _bootstrap (threading.py:937)

child process:

Thread 89047 (idle): "Thread-get-lldp-ip"

    _execute_child (subprocess.py:1754)

    __init__ (subprocess.py:951)

   ... some irrelevant stacktraces...

    run (xxx.py: xxx)

    _bootstrap_inner (threading.py:980)

    _bootstrap (threading.py:937)

so the parent process is waiting to read from errpipe_read as

3. looking into _posixsubprocess.c，the fork_exec() function does some prepation work and then forks a child process and calls execv() to invoke the actual command called. So I assume the child process is forked in the above procedure but somehow stucks before execv() as the child's process name is still the parent's process name rather than the actual command called.

4. some other infos I dont know if it's related to the problem:
   a) The stuck subprocess.Popen() call is inside a thread lock, and other threads got blocked acquiring this lock since subprocess.Popen() is stuck. I've read some posts on lock inheritance or what, but I don't know if it's related.
   b) When the subprocess is forked, there's another thread calling subprocess and is waiting on communicate(), as seen in child process's stacktrace:

Thread 0x14F2A5F37640 (idle): "ComponentManager"

    select (selectors.py:416)

    _communicate (subprocess.py:1979)

    communicate (subprocess.py:1134)

   ... some irrelevant stacktraces...

    _bootstrap_inner (threading.py:980)

    _bootstrap (threading.py:937)

while in parent process, this "ComponentManager" thread has continued running.

This issue only appeared once, and I didn't manage to reproduce after that.

Based on the above infos, could someone give some clues on what may be the cause? Thanks.

CPython versions tested on:

3.9

Operating systems tested on:

Linux