How to manage entity visibility/representation over multiple screens

[Bytekeeper]

Suppose you have entities that are visible on one screen, but not on another. For example: A group of characters are visible in combat, but not visible on the travel map.
One way is to add a "graphical" entity to represent them in the combat view and remove that entity if the travel screen is open. But that means a lot of syncing while adding characters, removing characters etc.

Another way is to remove the graphical components (Sprite in my case). But removing Sprite and all required components will make Bevy crash (while syncing the render world, because I also will have removed SyncToRenderWorld, which you're not supposed to do). I could only remove Sprite, or leave keep SyncToRenderWorld around - a bit ugly, but the best thing I can imagine to fix this.

Is there another way?