yes, that is the way dynamic drop downs is supposed to work.
I think we need to back up 10 steps here and determine if dynamic drop downs is needed for what you are trying to accomplish.
Question: Do the available colors change depending on the selected size?
if the answer to this is no, than you do not need to be using dynamic dropdowns.
dynamic dropdowns is used to drill down available selections in a child list based on the selection made in a parent list.
think in terms of a car site
you would have a selection for make:
if the selection in the parent list was Ford, you would not want the child list to show Sentra or Outback, but only show Explorer or Escape, that is what dynamic dropdowns is used for.
I have a suspicion that 90% of your problem is that you dont need to use dynamic dropdowns.