Xbox: How do I remove mercenaries?

I goofed. I created a mercenary and then backed out and then made another with an identical picture and name as the first. Now its a real guessing game everytime I party up. Will I spend the next half hour picking a mercenary, loading,picking a mercenary, loading…

I see the ‘X’ on the portraits, but Im at a loss how to click on it on an Xbox.