Using server side validation is the best way to accomplish this.
Choose File Extension validation, and click the lightning bolt next to server variable. From the Form collection, chose the fileField Extension variable.
If you are still not able to get it working, please send a copy if your page so I can take a look at the code.