Vast majority of modern network games use "lobby" conception: when limited amount of players connect to a single lobby server and play the game in isolation from everyone else who is not in thу same lobby.
Were the any attempts to create a game where players would have had one shared world to play altogether? If no, what are the major problems which made that idea non-viable?
Yes there are games, and yes there are problems.
Listen, light has a set speed. Even if the connection were at light speed all the way from client to server (which probably it isn't), it takes time for information to go from one place to the other.
But I can't explain that better than Grace Hopper: Admiral Grace Hopper Explains the Nanosecond.
That time the information needs time to travel adds latency (which is a component of "lag"). Another source of latency is that the server needs to process the information of every player. The server has only so many CPU cores… And yes, I'm counting a data center where you have rows of stacks of computers. That gets expensive.
Furthermore. Each player has to receive information about what all the other players are doing in that same area. And that can be a lot when areas that get crowded. So the server (and the client) need a large transference rate (which you might mistakenly refer as bandwidth), and large transference rate is also expensive (low transference rate is the other component of "lag").
Games with a single coherent world available world wide exist. Except they often don't have real-time physics between players. The less real-time interaction they can get away with, the better. That way if there is some lag between them, it isn't a problem. To this category belong a few select MMOs, where people walk through each other. Although they still need to deal with crowded areas, and they cope with it in multiple ways (including limiting the number of players a player can see).
Albion Online praises itself on having "one massive, sprawling world". My personal experience with it is huge lag on crowded towns.
Because of that, games which are available world wide would prefer to have multiple servers for different geographical regions.
Ok, I hear you, surely we can have a single coherent world per region, right? That is much more manageable, and much more common. Although, even for these regions real time physics between everybody is not practical. If we can limit the number of players that can interact with each other at a time, it is much more manageable. So we don't see as many huge open worlds as networks of maps. Or, again, no real-time physics between players.
You may also be interested in: Time Dilation (Eve Online). Yes, they have multiple servers.
There are other considerations for game design. For example, competitive games should strive for good match making. Because the game is interesting when the skill level of the players is similar. If you have highly skilled players vs beginners, the beginners get wiped and discouraged from continuing playing and the highly skilled players get bored, so the player base would dwindle.
Also consider that if the maps of the game are not huge, and it makes sense to have a maximum number of players per map, then lobbies makes sense.
By the way MMORPGs also suffer of something similar to the match making issue. You don't want high level players in the areas intended for lower level players. A variant of this happens for other cooperative multiplayer games: the high level players do everything (e.g. kill all the monsters) and leave nothing for the low level players. So, instead, you want people to move to other areas as they level up. A consequence of that is that the players with the highest levels will run out of content (it takes time for developers to add new areas for them). In competitive games, the players are the content. Sometimes players want to start over just to have something to do while they wait for more content… They would welcome another server where they could do that without messing with their higher level characters (although allowing players to have multiple characters and switch between them would also work, developers also need to consider if that breaks the game design or economy).
And please consider that not having this hube world for everybody also lower the requirements for the players (they don't need to download/load huge maps, and they don't need to have the transference rate for the information of as many players), which means more people can actually enjoy the game.
And do not forget that having lobbies also allows players to have private sessions with only the people they want. Yes, people enjoy that. No troll and no griefers for a while can be good. Also games can be useful to strengthen bonds with people.
