I think maybe you guys just need to take a break. Everything can be fun for awhile, and after a bit it seems not-so-wonderful. I don't really think you have anything to worry about, unless you're loosing sleep over it. Or if there are any other signs that you're having serious anxiety over things. If not, I think maybe you just need to take a break, tell your wife you don't want to stop swinging, but just a recess from all the fun
Then talk about your limits and all.. you know, couples renew their vows to show their devotion to each other after some time has passed from their marriage... and I think open relationships need to renew their conduct code with each other from time to time.