Use advanced options for the size. With advanced options, you can set unique price desciption or images for each option.
on the product update page,in the "Choose the variation of this product to edit..." section, select the XL or XXL size, and check the box to only edit that sizes options and change the price.