It looks like you are using an older version of Validation Toolkit that doesn't include the new validation option. It would be available if you upgraded to Data Bridge.
In your version you have to do it manually. You would add a recordset filtered by the unique value. Then you can use server validation on the number of rows that are returned and make sure it is less than 1, which means it doesn't already exist.
You should be able to find examples of this on our forums, or get as far as you can and post back when you have questions.