Hmmm. Refresh the page and try to remove again (even if the 'Removing...' text is still there).
You could also press the 'Reset' button in blue ribbon bar at top. Though, this will reset all widgets and you'll have to re-do the settings, so be aware.
I will try to reproduce this by adding a widget to a pin that also has nothing connected to it. Perhaps we never tested removing a widget from a pin where nothing can be removed. You should be able to remove a widget regardless. We'll investigate this.