xds: give up pool lock before closing xdsclient channel

[dfawley]

This prevents deadlocks with nested xds channels.

I believe this must be cherry-picked into the 1.74 release branch, too.

RELEASE NOTES: none

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.27%. Comparing base (3d0cb79) to head (de39873).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8445      +/-   ##
==========================================
- Coverage   82.48%   82.27%   -0.21%     
==========================================
  Files         414      414              
  Lines       40418    40422       +4     
==========================================
- Hits        33338    33257      -81     
- Misses       5732     5800      +68     
- Partials     1348     1365      +17     

Files with missing lines	Coverage Δ
xds/internal/xdsclient/pool.go	75.55% <100.00%> (+0.74%)	⬆️

... and 21 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[purnesh42H]

Thanks for finding and fixing it back. Perhaps we should file an issue for writing the test?

[purnesh42H]

nit: Even these 2 cleanups of authorities and xds servers were being done after releasing the lock before because they were within client's close method. May be we should move them down as well after closing the generic client.

[dfawley]

I'm worried about actual races here, since the bootstrapConfig appears to be a pointer, and could be modified later.

[arjan-bal]

Thanks for finding and fixing it back. Perhaps we should file an issue for writing the test?

In my opinion, we should have added a test when we first fixed the issue. Now that it has occurred again, I think we should add the test as soon as possible. I can try to write it before we attempt to import the fix back into google3.