when creating the coupon, there is a checkbox at the bottom that you can check to select which products the coupon applies to, leave that un checked and it will apply to all products.
if your check it, you can search for products to apply the coupon to, leave the criteria blank and it will return all products, you can use the Apply All button to add all products to the coupon.