The behavior that is used to redirect to another page if the email already exists is the Dreamweaver Check New user Name behavior.
You cannot use both the Check new User Name and Validation toolkit to check the email address at the same time.
The validation toolkit behavior is being triggered first, and causing the first probelm you report, you will need to choose between one or the other, you can do both.
The issue with validation toolkit, is that the code for the recordset is at line 87 - 95, this is after the validation toolkit code. When validation toolkit is validating the recordset, it has not been created yet.
The validation code is at line 9 - 33, move that code black to line 99 so that it is after the record set is created.