You have the option of deleting or unsubscribing a contact from your list. What's the difference?
Deleting a contact will remove them from your list and remove all data and list activity from when the user was part of that list. If this contact decides to subscribe again in the future, the subscriber's previous history will not be available in the system. This will not affect past campaign reports.
Unsubscribing a contact will remove them from your active subscriber list, but keeps all of their data and list activity from when the user was part of the active list. If the user decides to subscribe again, all of their contact history will still be available.