Are you saying that it is not renaming the new file if it is a duplicate? Have you double checked in the upload folder? The file system will balk at creating a new file with the same name as the file that already exist, the new file should be given a new name if the file already exists.
Keep in mind though that MyFile.jpg is not a duplicate of myfile.jpg because of the case difference.
To restrict the space in the file name, you need to use the Restrict Content validation type. set it to restrict the space and use the includes type.