Fix controller model invisible sometimes when toggling between hands and controllers

Fix controller model invisible sometimes when toggling between hands and controllers #1567