There are a number of ways that you can delete references.
References in Trash will not appear in any other group, and they cannot be searched or inserted into Word documents using Cite While You Write.
To delete references permanently, go to the References menu and select Empty trash. This cannot be undone.
This video (3 mins 16 secs) covers: