Receiving chat messages after leaving a group

There have been cases where someone hwo is not a member of a group anymore is still receiving Karrot group messages via e-mail

issue on github: Users sometimes stay conversation participants after leaving group: https://github.com/karrot-dev/karrot-frontend/issues/2571