I do like this idea. The leader of the group could appoint "moderators" of their group to allow for more invites in the leader's absence. What Shurtle said is true, however. This would have to be something that the source code would have to be altered to provide. Which, unfortunately, we are at a loss and do not have it.
Wonderful idea, nonetheless!