Thankfully this is no longer a problem, and there is now a way to restrict group invites easily.
To do so, go to the parameters menu of your app.
Tap on the Account option, followed by Privacy.
The parameters
Scroll down to Groups and tap it.
Youll get a choice between three controls.
Nobody, Your Contacts and Everyone.
Nobody means that, well, nobody can add you to any groups.
