When using Discord, other people on my call can hear other PC sounds (such as YouTube) despite me being on a headset. Things I have checked:
- Audio is not playing over speakers.
- Stereo Mix is disabled.
- Audio drivers are up-to-date.
My headset had a built-in microphone that I wasn't using (I was using a higher-quality attachable microphone in this case). Despite Discord and Windows not showing it was in use, Discord, for whatever reason, was still using that microphone. Disabling that mike in Windows fixed it.
In Windows, I disabled my headset's built-in microphone in sound settings. (Control Panel -> Sound -> Recording tab)

To ensure no sound is picked up by any device, you can also simply globally mute yourself in the main menu bar:
which you can also toggle with Ctrl + Shift + M.