When I first started working to get into shape I walked in water to get resistance, but no impact. Since you swim, you'd probably like that better - but there is the whole thing of being in a gym full of people.

Maybe hiking would work? Or doing yoga at home?

I know people also swear by using resistance band training. I've only done that as part of physical therapy, but it did work well for that.