ecart 6 remove product from cart on product page

I have tried to modify my cart to achieve this effect but with no luck. I have a product page that has an "Add" button for each product listed in columns and rows. When clicked it processes the Add To Cart (ATC) functions and returns to the same page to allow a user to select more items with a single click. When added I have the Add button turn green with a check mark in it to indicate that the product is already in the cart.

Now I want a user to be able to click that green button to trigger a remove from cart and return to the same page for a seamless effect. I added a new form around the green button and copied the Add to Cart button to try to get it to modify the cart but this is where I am not sure which code to modify and which values to pass in the form to get the product removed from the cart.

Any thoughts or help appreciated! I hope I explained it well enough!