To add or deactivate a user you need to have Admin rights. If you have admin rights, you will have the "plus" button visible on the Collaborators Page.
To deactivate a user, you need to go to the Collaborators Page and add the Active column, by editing Visible columns. To deactivate, you can uncheck the box of the corresponding user. Refer Admin Features for detailed information and gifs