George: question about system generation time

[alterecco]

OK, so, this is more of a question to learn your reasoning, rather than a suggestion. :)

A couple of years back, when I first started getting into modding Transcendence, one of the first things that I wanted to work on was economy improvements (grand dreams!). I remember pestering Betel on IRC about these things :D
Essential to building an economy is that you know your surroundings. In Transcendence, the only surrounding that you ever really know is the system you are currently in, so that put some limitations on that idea. What I really wanted was to implement inter-system travel and trading. I have pondered on this question a lot of times, talking to others who share the same "dreams", and in the end it has all fallen down on one single problem: we don't know what is ahead.

So. My question is: Why are all systems not generated at the start of the game?

Is it a performance issue? Is it a holdover from rogue/nethack? Is it to allow level scumming (i doubt it)? Or is it for practical implementation issues?

What I would really like to have access to, in script, is the elements of others systems. What stations does system X contain? What is their inventory? What gates does it have (well, where are they placed!). I realize that i would only have access to a snapshot, but that would be perfectly fine. I am not asking why the entire Transcendence universe is not "live". Basically, we are talking about positioning and type data. I would like to be able to ask questions like:

Is there an overwhelming enemy presence in the next system, then perhaps enemies should slowly be spreading to it's adjacent systems (that is, enemies should be spawning out of stargates when the player is in that system).

Is the next system incredibly rich on ore, then miners will be flocking to it, at a greater level at least.

Send trade convoys to and from systems with particular resources. Ice farms are a precious commodity, so freighters might travel from near and far to visit them.

--------------------------------------------------------------------------

I realize that I might never actually build a mod that provides those features, so, it is not so much that I want you to implement it, it's more that I am curious as to your decision to generate systems first on entry. When I know you reason, then perhaps I can say to those that ask about inter-system travel, "No, because so and so and so..." and leave it be... and maybe I can finally stop thinking about this myself XD.

[george moromisato]

There are really two issues here:

1. Systems are created the first time a player enters them.
2. Systems other than the one the player is in are flushed from memory and left on disk.

The reason for #1 is historical, and mostly for performance. Today, I don't think it would be too bad to generate all systems at game start (though this will get worse once Part II comes out).

I think it would be reasonable to change this behavior.

#2 is a different problem. It isn't scalable to hold all the systems in memory, so accessing the data in other systems would be (at best) time-consuming: load the system from disk.

One possible work-around for #2 would be to add code in OnGlobalSystemCreated to squirrel away the info you needed from the system into some place that doesn't get flushed, such as data on the topology node.

[Atarlost]

You don't need full data. If systems were all generated at game start* and oncreate events could populate a global list then the required data about other systems could be available without ever directly addressing issue 2.


* and people would actually read the scroll text if it were a loading screen

[alterecco]

Thanks for the answer George.

I am glad that you would consider changing the system generation time. I agree with Atarlost that it would hardly post a problem if the load screen took a small while longer to finish. However, if it is not possible to access at least some of the information of systems the player has not accessed yet, then there is little point to it.

I would consider the most interesting information about systems to be station presence and location. With that information realistic trade routes (with travel times) could be calculated. If we could also access information about the items in the stations inventory, then trading could be enhanced even more (cross-system supply/demand trading, traders union with tidbits of information on good places to buy/sell).

If you would be willing to add an OnCreate event to all systems in vanilla, I would be happy to help draft a script that gathered and structured this information. Of course, this would not make sense to do unless all systems are generated at the start of the game.

[PKodon]

George:

Two comments, with regard to this.

I realize that alterecco is asking about generating the entire system map at startup.

Here is one possibility I was considering: You start in system A, system B is created at start. System B and System A are in memory, but events in System B are only virtual, not all of System B is "live".

For later example, with regard to Charon, if one is in the system before Charon, one would not wish events that require player interaction, such as the attack on the Korolov station, to happen till the player got there. However, any other randomly generated stations that would generate trade between there and the system you're in, would be known, and appropriate trade ships could travel to/from the gate to Charon (and an appropriate pirate presence could "visit" the system before Charon).

If Charon is then system F, and the player is in system E, systems A through C would be stored on disk, but system D would still have a presence in memory, enough so that it's randomly generated stations would also influence the type and number of trade vessels (and others) present in system E.

It may not help alterecco's situation completely, but would again be a sort of "sliding window" solution to local system economy.

[Teraton]

Here's another, somewhat similar suggestion:

Instead of generating the entire universe at startup, just generate a single random seed, that is used by an algorithm (or several algorithms) to generate everything else: system topology, station presence, ships, etc.

If you were to use the same seed in another new game, you'd end up with an identical universe.

The saved games, rather than storing every minute detail of the systems you've been in, only store the seed value, the state of your ship, and the changes you've made in those systems from their default state.

Then you can have some code that simulates what's going on in nearby systems, based on the seed value (and any changes made by the player). Alterecco, that simulation area would be your playground. Trade economics, inter-system warfare, you name it. By abstracting it into a simulation (instead of running it at 1-to-1 realism), you can drastically reduce the processing and memory overhead, but still end up with a more dynamic universe.

Okay, so far, this is close to the way that Minecraft generates its worlds (at least as I understand it, and thinking in terms of "systems" instead of "chunks"). But we can take it one step further in Transcendence - one tiny trick, that will add tons of realism to the game:

As soon as you gate to a new system, while it is being generated, it checks to see if any other ships gated in ahead of you (say, within the last 30 seconds), as well as any that were about to take the same gate. The new system adds those ships in as well, simulating where the early ships should have been by now, and popping in the late ones in at the time that they would have popped in.

We're probably talking an average of 1 ship, maybe 5 at the most, that would be pulled in from the old system to the new one. But it would add just enough continuity, that when combined with the simulations running for nearby systems, the entire universe would seem alive and dynamic.

Does that sound like it would work?

[Atarlost]

Storing changes is not practical. You'd hardly save any space because most stuff in a system is ships that will change continuously, and load times would go up quite a bit.

[Teraton]

Storing changes is not practical. You'd hardly save any space because most stuff in a system is ships that will change continuously, and load times would go up quite a bit.

Individual ships wouldn't be tracked outside of your system, unless they were crossing over with you when you gate.

I'd disagree with you on the practicality of storing changes, but honestly, saving space isn't even the main point. The point is that once you start the universe, once that seed has been formed, then the initial state of the rest of the systems can be predicted. They haven't been "generated" yet, they aren't clogging up space in your save game file... but a mod can still use that information, and work with it, just by looking at the seed value. That way, when you enter a new system, you'll get to see the results of multi-system dynamics, without having to keep track of the minutest details.

[Atarlost]

Storing changes is not practical. You'd hardly save any space because most stuff in a system is ships that will change continuously, and load times would go up quite a bit.

Individual ships wouldn't be tracked outside of your system, unless they were crossing over with you when you gate.

I'd disagree with you on the practicality of storing changes, but honestly, saving space isn't even the main point. The point is that once you start the universe, once that seed has been formed, then the initial state of the rest of the systems can be predicted. They haven't been "generated" yet, they aren't clogging up space in your save game file... but a mod can still use that information, and work with it, just by looking at the seed value. That way, when you enter a new system, you'll get to see the results of multi-system dynamics, without having to keep track of the minutest details.

Ship status, inventory, and location are stored for systems you have departed.

You're also going to have a lot of lag any time you access data that requires generating systems from seeds.

[Teraton]

Let's try an analogy to a different game.

Have you ever played Mount & Blade? Before entering a battle, you get a screen that looks something like this.

If you choose the option to order your troops to attack while you stay back, then it "simulates" how the battle would have gone, based on an estimate of each side's strength. And compared to actually entering the battle itself, that simulation process is lightning fast.

It doesn't bother to "think" about every individual soldier as it moves across the field, or calculate each slash and wound... instead, it reduces everything down to a few simple numbers, flips a few coins, and tells you how the battle ended.

You're also going to have a lot of lag any time you access data that requires generating systems from seeds.

But you're not... generating... those systems. At least, not the way you're thinking. (I'm trying to give you the benefit of the doubt here.) You're not loading sprites, you're not thrashing the hard drive (which can be a major performance bottleneck), and you're not moving individual ships around in those systems. You're simplifying everything down to a few important numbers, running them through a formula of your choice, and flipping some coins.

[Cirevam]

Teraton's idea is excellent. I'll offer an example of how I think it would work. Let's say you gate to a new system. The system has already been defined at the start of the game through the seed value and whatever algorithm(s). When you gate into the system, the game says "okay, this system has X number of enemy stations, Y number of friendly stations, of which there are Z shops, Q fuel depots, etc. The stations are located at these positions. Based on these numbers plus a few others, so many stations have been destroyed, so many are being contested, and these surviving stations have so much money left." It wouldn't take more than a second to calculate this, and since it uses information that was generated from the start of the game, it will be the same every time as long as the seed remains the same.

[shanejfilomena]

the game selects the systems on start up while loading Eridani: the math for system selection is done in the time you get in the ship:
from there the game projects states of the systems as you gate.
the number of station in each system is set by the design of that type of system: the game has tables of items to select stations, asteroids, etc...
If you want a FAST loading system: build an empty system.

The problem is that your over thinking the future of the present. computerz only go but so fast.

as to speeding up the game generations: you can if you strip out everything you require for a active event......
but then you got DOS.

[Atarlost]

You're also going to have a lot of lag any time you access data that requires generating systems from seeds.

But you're not... generating... those systems. At least, not the way you're thinking. (I'm trying to give you the benefit of the doubt here.) You're not loading sprites, you're not thrashing the hard drive (which can be a major performance bottleneck), and you're not moving individual ships around in those systems. You're simplifying everything down to a few important numbers, running them through a formula of your choice, and flipping some coins.

You forgot the big problem: script. You have to run all the <oncreate> scripts and the system entry scripts for the playership, all of which can place or remove stations or influence their inventories. Any of which may rely on global variables that they can break the game by accessing when you're not really going through a stargate, or actually lock the game up by trying to access if they're prevented from altering them in "fake" generation mode.

[alterecco]

While I think the idea has merit, I do agree with Atarlost. Every call to a function that would work off the seed would incur a penalty. Plus, even more so, I don't think Georges system generation is built to work off of a random seed. I have a feeling a very large part of the system generation code would have to be changed, thereby leading to a whole new slew of bugs. If this idea were for a new game, I would say "go ahead". As it stands, I think we would be better served with one of the other models. At least they would be easier to implement, although leading to a memory penalty. But, a memory penalty would be better for T than a computational penalty, imo.

[shanejfilomena]

I thinks I got to learn your language you said it far better then I tried

but you understand these things in deeper terms and I am thankful you are there to protect us from ourselves:)

by the way: this "seed" thing.....i was running experiments with ( censored by commonwealth security forces): i hate it. you have no control over your ship , nothing thinks ; i lost 50K worth of new ships ......I do not think I want to see a "seed" of anything unless it's a teraton building a defender.....that stuff is just not my cup of battle action.

And I am happy to sit behind my transcendence ships knowing that I MIGHT beat that Ventari ( and sometimes can ) instead of some "Seed" deciding I totally lost before I fired a shot.