Does "leaving" amount to "unsubscribing"? Such that, leaving a group simply means it won't:

1. show up in your sidebar/stream content
2. won't send you email for additions in that group
3. won't be in your list of groups to publish into

I.e. if I wanted back in, I could find that email with the invite link and re-join (or perhaps there's UI that let's me re-join past groups).