In DW and Security Assist the method is to have an extra recordset that looks for any records where the email matches the one entered by the user. If any matches are found, meaning the recordset is not empty, then the user already exists and that email address should not be allowed.
That is the concept in general but there is a problem with letting the user know if a password has been used. This is a security risk as you can verify exact password matches in your users table. It should not matter if two or more users have the same password so long as they don't have the same email or login it will not affect anything.
So the check should only be done with the email. Your recordset should select from your user table and filter on the posted email address. If there are any rows in the rs then the insert should not occur and you should display an error to the user. In your check for the rows of the recordset you would have a header call that will redirect the user to this same page with a URL parameter to indicate that the email address was a duplicate.
Please post back with any further questions that you have about this so we can help you get it worked out.