Doug,
I'm having a hard visualizing this, because my ports have the oval ring on the outside. The inner part is all part of the main porthole, which could have been originally installed with a bedding compound or sealant and later have been re-done by a previous owner. Hopefully 5200 was not used!
Mike's right that heat could help make the sealant softer. Putty knife will help too. You may also be able to give it a tap with a rubber mallet from the opposite side to help break it free.
Regarding the new sealant, I have had excellent results on installing portlights with a bedding compound called Dolphinite. It can be removed at a later time with no fuss. I used 5200 once on a previous boat, and the portlight was impossible to remove and permanently attached as far as I was concerned! Lesson learned.
Matthew