The issue was with the key value being set to the session variable value in the server validation.
That setting is only used on update pages to allow the user to set the same value of their existing record, since it is getting overwritten anyway.
In your case the second registration was being allowed with the same email because the validation thought that was the record being overwritten, so it allowed it to pass.
The error was an invalid duplicate key error when the insert tried to process.
I fixed the problem by updating the server validations unique database value server behaviors and set the Key value to 0 (zero) since that will never exist. That is the right way to use it on insert pages.