Groups are a way for users to opt-in to mailings that you send. It is not intended for the Site administrator to manage Group Members, you add the group signup for to your page to allow users to decide to become part of the mailing group.
Audiences are they way the site admin sets up who the email goes to.
You can set the Audience to include the group, and also set other parameters to target specific users by First Name, last Name, email address or any other column of the users table.