I think you're right to take a step back.
Having pricing dependent on size and/or colour is more complicated than by item, but if you plan well, it isn't too hard.
As an idea, take a look at a couple of sites, with price options dependent on colour and size, these may help cement your ideas...
The first one i did about a year ago, the second is an early version of something i am currently working on
(Go to one of the gallery pages, each of the thumbnails has pricing options dependent on size)
(work in progress)
Choose a product category from the products menu, then click on a thumbnail to go to product details, where you can choose size and colour
(-- of the two the second one is easier to implement)