By default the redirect should go to the thank you page as you saw in the code.
To test this please add this line of code to the top of the page of the thank you page:
<?php die('thankyoupage'); ?>
I'm guessing that on the thank you page you have a access page rule and it is redirecting you to the login page.
Please let me know if you are getting the 'thankyoupage' output after registering.