Shared hosting is not for open world level design
After trying to implement some more complicated interactions like sharing inventory and vehicles, I have learned that prefab loading can only be done on master-client. This breaks dynamic scene loading for all other players.