It sounds like server validation isn't getting applied properly. Everything else is caught with the spry validation, but the captcha and security question are only handled with server validation.
Did you upload the action page? Maybe the ServerValidation server behavior wasn't applied to the action page, maybe it wasn't uploaded, maybe the trigger is wrong, maybe the validations are wrong somehow. It is not a problem I am familiar with.
The form builder isn't great as a form editor. Once you build a form I would suggest adding and removing security features using the standard DW page editing experience from there. You can "edit", but it will wipe out any customization you do when you re-apply, so it often isn't the best option.