There is an interesting question and answer in the Google AdWords Help forums about how tied are ad groups to the domain names in those ad groups.

Tommy asked Google, “Is there any validity to the idea that since there is a ‘history’ of everything in our accounts that once a domain is added to an ad group, that domain is partially/forever attached to that ad group?”

Google’s Charvi said no. Charvi explained in more detail:

I think the perception was that this was the case but Google says no, so it is good to clear it up.

Forum discussion at Google AdWords Help.