I don’t know why this is happening, but it could be related to file ownership and/or permissions.
So try to force the delete with the -f parameter like this:
rm -f "${FILE_TO_DELETE}"
If that doesn’t work, try changing the rm to ls -la to see if you can glean any insight on what might be happening:
ls -la "${FILE_TO_DELETE}"
Or even use the Z parameter to check for SELinux context with:
ls -laZ "${FILE_TO_DELETE}"