I think without a house to move in, the Clonks would look like stupid zombies standing around your base. There are so many ways of using the "Clonk shelter" (under water; for some kind of ambush, when the enemy lurks around your place; to hide the Clonks from your enemys eye's at all, when you put them all just in one special building, the enemy is always going to blow it first), that you can't just take it away for practical reasons. I think new players would soon ask, why you can't move in buildings, and when you then show them CR, their heads would explode instantly! 
And grabbing a buildind to move stuff in and out looks just plain unrealistic.    

And where do you think the Inuits going to have their "rendezvous" then? ;)