[George] In what order should I parse the XML files?

Freeform discussion about anything related to modding Transcendence.
User avatar
pixelfck
Militia Captain
Militia Captain
Posts: 571
Joined: Tue Aug 11, 2009 8:47 pm
Location: Travelling around in Europe

Tue May 26, 2015 9:02 pm

Hi,

I'm writing a script for parsing the xml files so that we can auto-update the wiki and have some questions about the xml structure of transcendence.

In what order should I process the Modules/Libraries/Adventures/etc?

I start with reading <TranscendenceUniverse>. The universe contains several other building blocks:
  • <CoreLibrary
    • <Module
  • <TranscendenceExtension
    • <Library
    • <Module
The libraries may themselves contain <Module> and other <Library> references, etc.


Is there any particular order in parsing those files?
Should I stick with the order as specified? or can I, for example, first parse all modules and then continue with the libraries?

Are the DocType entities as used in a <Module> guaranteed to be defined in the building block which includes said <Module>?
In other words, is it guaranteed that all entities used in a module are specified in its parent, or can DocType entities from, for example a <Library>, be used as well?
Additionally, can DocType entities as found in other building blocks come from previously specified libraries, universes or adventures? or worse, could they be defined in a later file? (I need to know all entities that could be used in a file before parsing it).

----------
Also, why are libraries sometimes referenced by file name and at other times by unid?

Why are some coreLibraries referenced twice? first by file name and, at some later point, by unid?

Cheers,
Pixelfck
Image
Download the Black Market Expansion from Xelerus.de today!
My other mods at xelerus.de

george moromisato
Developer
Developer
Posts: 2979
Joined: Thu Jul 24, 2003 9:53 pm
Contact:

Wed May 27, 2015 2:22 am

The process for loading the universe is very tricky, unfortunately. If you could possibly rely on TransData, or even the TSE C++ library, you might be better off. Still, I understand that there are some things that you can only do if you parse it yourself. Sometimes you just gotta re-invent the wheel. I'll try to help where I can:

DEFINITIONS
A "type" is a single definitions of some object--a ship, an item type, a dock screen, whatever. All types have an UNID. For example, the <ItemType> definition for a laser cannon is the type with UNID = 0x0000400D. A type can be overridden if you define an object with the same UNID.

An "extension" is a collection of types. And extension also has an UNID (but it is not technically a type because you can't override it). There are three kinds of extensions: adventures, expansions, and libraries. [Forget for a moment, the common usage of extension as something that extends an adventure. At the engine level, everything is an extension.]

An "adventure" is a special kind of extension that is required to have certain types (such as an <AdventureDesc>) and is required to defined a topology. But other than that, it is just a plain old extension.

Similarly, a "library" is also a special kind of extension, which is included in a special way (described below).

Unless I specify otherwise, when I say "extension," I mean any one of the three types: adventures, expansions, or libraries.

A "module" is an XML file that is included syntactically into some other file. It DOES NOT have an UNID, and it is equivalent to an include file in C. If you could copy the contents of a module XML file and paste into the place where it is referenced, it should behave identically (minus prologue, etc.). Modules, of course, can include other modules recursively.

An extension has a root XML file, which starts with one of the following tags: TranscendenceAdventure, TranscendenceExtension, TranscendenceLibrary. This root XML file may reference modules (which are included syntactically) and it may reference libraries.

TRANSCENDENCE.XML
The Transcendence.xml file is a special file that is used to generate Transcendence.tdb. Unlike every other TDB file, Transcendence.tdb contains MULTIPLE extensions. Transcendence.xml does not follow the same syntax as extensions because it is parsed by TransCompiler to generate Transcendence.tdb.

This is why it uses a filename to refer to the libraries and adventure. Once they're loaded into the TDB file, they are referenced by UNID.

ENTITIES
1. The biggest divergence from true XML is in the handling of entity declarations and libraries. In particular, a library reference (<Library ...>) pulls in all entities defined in that library. It is (currently) illegal to use an entity defined in a library BEFORE the library reference. But it is legal afterwards.

2. Entities declared in the root XML of an extension are visible to all modules included by that extension. And if you use a library, the entities defined by that library are visible to any module you subsequently import.

3. Entity declarations are only exported from the root XML of a library. That is, if a library has multiple <Modules>, and those modules define entities, those entities stay PRIVATE to the module; they are not exported. Only the entities defined in the library's root XML are exported.

4. Obviously, loading order matters if you're trying to resolve an entity. If you're loading a module and it has an entity, you can't resolve that entity until you've loaded all the libraries used by the root extension of the module.

PROCESSING ORDER
When loading the universe, the engine follows a recursive procedure:

Code: Select all

LoadExtension (E)
{
   for each library L used by E
      LoadExtension(L)
}

LoadUniverse ()
{
   LoadExtension(the-adventure-chosen-by-the-user)
   for each expansion E chosen by the user
      LoadExtension(E)
}
This order is important for entity definitions but also for overrides. There are very few cases of core extensions overriding types, but it does happen. Expansions, of course, very often override types.

EditorRUS
Militia Lieutenant
Militia Lieutenant
Posts: 148
Joined: Tue Oct 30, 2012 6:30 pm

Wed May 27, 2015 8:40 pm

I can definitely say you won't be able to distinguish between real and virtual objects. As virtual I mean those objects used only for storing data.

User avatar
KaoS
Miner
Miner
Posts: 34
Joined: Sun Aug 19, 2012 3:25 pm
Location: 404
Contact:

Wed May 27, 2015 11:32 pm

My brain just exploded.

User avatar
catfighter
Militia Commander
Militia Commander
Posts: 466
Joined: Fri Nov 08, 2013 5:17 am
Location: Laughing manically amidst the wreckage of the Iocrym fleet.

Thu May 28, 2015 5:42 am

KaoS wrote:My brain just exploded.
Welcome to coding, hehe, where brain-splosions are as common as grains of sand on a beach. :mrgreen:

George, under what category would an XML file containing nothing but variables fall? I was thinking about making a single separate file for holding entites and/or variables used in multiple mods so that they don't have to be redownloaded for each individual new mod (we're talking in the hundreds of entities/variables, a real pain to have to scroll through in every file), but I was unsure if this would have to be classified as an extension, a library, or if it could be either one.
Behold my avatar, one of the few ships to be drawn out pixel by pixel in the dreaded... Microsoft Paint!

Day 31: "I have successfully completed my time reversal experiment! Muahahaha!!!"
Day 30: "I might have run into a little problem here."

User avatar
pixelfck
Militia Captain
Militia Captain
Posts: 571
Joined: Tue Aug 11, 2009 8:47 pm
Location: Travelling around in Europe

Thu May 28, 2015 9:13 am

Ok, thanks George.

From your code example I read that it is basically a depth first recursive read, that, whenever a type is encountered twice, will overwrite the previous one with the new definition.
george moromisato wrote:It is (currently) illegal to use an entity defined in a library BEFORE the library reference. But it is legal afterwards.
I'm happy to hear this, it would be a real pain (= require a hacky solution) otherwise.
george moromisato wrote:3. Entity declarations are only exported from the root XML of a library. That is, if a library has multiple <Modules>, and those modules define entities, those entities stay PRIVATE to the module; they are not exported. Only the entities defined in the library's root XML are exported.
By this, do you mean that it is legal to declare entities in a <module>? or do you mean stuff like <Image UNID="">?

Cheers,
Pixelfck
Image
Download the Black Market Expansion from Xelerus.de today!
My other mods at xelerus.de

george moromisato
Developer
Developer
Posts: 2979
Joined: Thu Jul 24, 2003 9:53 pm
Contact:

Thu May 28, 2015 2:20 pm

pixelfck wrote:From your code example I read that it is basically a depth first recursive read, that, whenever a type is encountered twice, will overwrite the previous one with the new definition.
I think, if an extension tries to redefine an ENTITY, it gets an error (though it might be OK if it defines it to the same value). If you define a TYPE (e.g., <ShipClass>) then, yes, it will override the previous definition.
pixelfck wrote:By this, do you mean that it is legal to declare entities in a <module>? or do you mean stuff like <Image UNID="">?
I believe it is legal to define an entity inside a module (though I never do it). Those entity definitions would only be valid within that module (not in the rest of the extension).

User avatar
KaoS
Miner
Miner
Posts: 34
Joined: Sun Aug 19, 2012 3:25 pm
Location: 404
Contact:

Thu May 28, 2015 10:37 pm

catfighter wrote:
KaoS wrote:My brain just exploded.
Welcome to coding, hehe, where brain-splosions are as common as grains of sand on a beach. :mrgreen:

Yeah, logic is trippy stuff. I've never really given that much thought to this. I usually just start reading and following references until I find what I want and then I plug in a new set of variables for it. Seeing George spell it out in words like that just made me think of all the stuff he's got floating around in that brain of his.

User avatar
pixelfck
Militia Captain
Militia Captain
Posts: 571
Joined: Tue Aug 11, 2009 8:47 pm
Location: Travelling around in Europe

Fri May 29, 2015 10:16 pm

Some additional questions.

george moromisato wrote:There are three kinds of extensions: adventures, expansions, and libraries.
What about the <CoreLibrary> ? is this just another library or should I be aware of something special?
george moromisato wrote: A "module" is an XML file that is included syntactically into some other file. It DOES NOT have an UNID, and it is equivalent to an include file in C. If you could copy the contents of a module XML file and paste into the place where it is referenced, it should behave identically (minus prologue, etc.). Modules, of course, can include other modules recursively.
Does this mean that a module will not include a library/extension/adventure ? (since a module acts as an inline replacement, I would expect them to be able to also include libraries/extension/adventures, but just checking).


Cheers,
Pixelfck
Image
Download the Black Market Expansion from Xelerus.de today!
My other mods at xelerus.de

george moromisato
Developer
Developer
Posts: 2979
Joined: Thu Jul 24, 2003 9:53 pm
Contact:

Fri May 29, 2015 10:42 pm

pixelfck wrote:What about the <CoreLibrary> ? is this just another library or should I be aware of something special?
This is just a normal library, but it's being included into Transcendence.tdb (the core file). It has a different name only because some core libraries are also included in CSC America. But for Transcendence purposes, there is no difference.
pixelfck wrote:Does this mean that a module will not include a library/extension/adventure ? (since a module acts as an inline replacement, I would expect them to be able to also include libraries/extension/adventures, but just checking).
Your expectations are correct. You can include a library inside a module (though I don't do it in any official extension).

User avatar
pixelfck
Militia Captain
Militia Captain
Posts: 571
Joined: Tue Aug 11, 2009 8:47 pm
Location: Travelling around in Europe

Sat May 30, 2015 3:36 pm

Well, my script now reads itself through all the files as specified by <Module>, <CoreLibrary>, <Library>, <TranscendenceAdventure> and <TranscendenceExtension>.

I'm somewhat surprised at the amount of double reading/traversing the inclusion tree; CoreTypesLibrary.xml is, for example, visited 22 times?

Is this correct?:

Code: Select all

Starting from Universe: Transcendence.xml
> Stepping into CoreLibrary: CoreTypesLibrary.xml
- > Stepping into Module: CoreCode.xml
- > Stepping into Module: CoreEffects.xml
- > Stepping into Module: CoreExplosions.xml
- > Stepping into Module: CorePlaceholders.xml
> Stepping into CoreLibrary: RPGLibrary.xml
- > Stepping into Library: CoreTypesLibrary.xml
- - > Stepping into Module: CoreCode.xml
- - > Stepping into Module: CoreEffects.xml
- - > Stepping into Module: CoreExplosions.xml
- - > Stepping into Module: CorePlaceholders.xml
- > Stepping into Module: HUD.xml
- > Stepping into Module: RPGAnalyzers.xml
- > Stepping into Module: RPGAutons.xml
- > Stepping into Module: RPGAuxDevices.xml
- > Stepping into Module: RPGBehaviors.xml
- > Stepping into Module: RPGCharacterClasses.xml
- > Stepping into Module: RPGCharacters.xml
- > Stepping into Module: RPGCode.xml
- > Stepping into Module: RPGCompatibility.xml
- > Stepping into Module: RPGArmorDockScreens.xml
- > Stepping into Module: RPGDeviceDockScreens.xml
- > Stepping into Module: RPGDialog.xml
- > Stepping into Module: RPGDockScreens.xml
- > Stepping into Module: RPGMissions.xml
- > Stepping into Module: RPGPlayer.xml
- > Stepping into Module: RPGShipScreens.xml
- > Stepping into Module: RPGTypes.xml
- > Stepping into Module: RPGWingmen.xml
> Stepping into CoreLibrary: GalaxyLibrary.xml
- > Stepping into Library: CoreTypesLibrary.xml
- - > Stepping into Module: CoreCode.xml
- - > Stepping into Module: CoreEffects.xml
- - > Stepping into Module: CoreExplosions.xml
- - > Stepping into Module: CorePlaceholders.xml
- > Stepping into Library: RPGLibrary.xml
- - > Stepping into Library: CoreTypesLibrary.xml
- - - > Stepping into Module: CoreCode.xml
- - - > Stepping into Module: CoreEffects.xml
- - - > Stepping into Module: CoreExplosions.xml
- - - > Stepping into Module: CorePlaceholders.xml
- - > Stepping into Module: HUD.xml
- - > Stepping into Module: RPGAnalyzers.xml
- - > Stepping into Module: RPGAutons.xml
- - > Stepping into Module: RPGAuxDevices.xml
- - > Stepping into Module: RPGBehaviors.xml
- - > Stepping into Module: RPGCharacterClasses.xml
- - > Stepping into Module: RPGCharacters.xml
- - > Stepping into Module: RPGCode.xml
- - > Stepping into Module: RPGCompatibility.xml
- - > Stepping into Module: RPGArmorDockScreens.xml
- - > Stepping into Module: RPGDeviceDockScreens.xml
- - > Stepping into Module: RPGDialog.xml
- - > Stepping into Module: RPGDockScreens.xml
- - > Stepping into Module: RPGMissions.xml
- - > Stepping into Module: RPGPlayer.xml
- - > Stepping into Module: RPGShipScreens.xml
- - > Stepping into Module: RPGTypes.xml
- - > Stepping into Module: RPGWingmen.xml
- > Stepping into Module: Environments.xml
- > Stepping into Module: GalaxyCompatibility.xml
- > Stepping into Module: GalaxyEffects.xml
- > Stepping into Module: Groups.xml
- > Stepping into Module: Ice.xml
- > Stepping into Module: Metallic.xml
- > Stepping into Module: Mining.xml
- > Stepping into Module: Ore.xml
- > Stepping into Module: Ovoids.xml
- > Stepping into Module: Neurohack.xml
- > Stepping into Module: Primordial.xml
- > Stepping into Module: Rocky.xml
- > Stepping into Module: Stargates.xml
- > Stepping into Module: Stars.xml
- > Stepping into Module: StarSystems.xml
- > Stepping into Module: Volcanic.xml
- > Stepping into Module: Worlds.xml
> Stepping into CoreLibrary: HumanSpaceVol01.xml
- > Stepping into Library: CoreTypesLibrary.xml
- - > Stepping into Module: CoreCode.xml
- - > Stepping into Module: CoreEffects.xml
- - > Stepping into Module: CoreExplosions.xml
- - > Stepping into Module: CorePlaceholders.xml
- > Stepping into Library: RPGLibrary.xml
- - > Stepping into Library: CoreTypesLibrary.xml
- - - > Stepping into Module: CoreCode.xml
- - - > Stepping into Module: CoreEffects.xml
- - - > Stepping into Module: CoreExplosions.xml
- - - > Stepping into Module: CorePlaceholders.xml
- - > Stepping into Module: HUD.xml
- - > Stepping into Module: RPGAnalyzers.xml
- - > Stepping into Module: RPGAutons.xml
- - > Stepping into Module: RPGAuxDevices.xml
- - > Stepping into Module: RPGBehaviors.xml
- - > Stepping into Module: RPGCharacterClasses.xml
- - > Stepping into Module: RPGCharacters.xml
- - > Stepping into Module: RPGCode.xml
- - > Stepping into Module: RPGCompatibility.xml
- - > Stepping into Module: RPGArmorDockScreens.xml
- - > Stepping into Module: RPGDeviceDockScreens.xml
- - > Stepping into Module: RPGDialog.xml
- - > Stepping into Module: RPGDockScreens.xml
- - > Stepping into Module: RPGMissions.xml
- - > Stepping into Module: RPGPlayer.xml
- - > Stepping into Module: RPGShipScreens.xml
- - > Stepping into Module: RPGTypes.xml
- - > Stepping into Module: RPGWingmen.xml
- > Stepping into Library: GalaxyLibrary.xml
- - > Stepping into Library: CoreTypesLibrary.xml
- - - > Stepping into Module: CoreCode.xml
- - - > Stepping into Module: CoreEffects.xml
- - - > Stepping into Module: CoreExplosions.xml
- - - > Stepping into Module: CorePlaceholders.xml
- - > Stepping into Library: RPGLibrary.xml
- - - > Stepping into Library: CoreTypesLibrary.xml
- - - - > Stepping into Module: CoreCode.xml
- - - - > Stepping into Module: CoreEffects.xml
- - - - > Stepping into Module: CoreExplosions.xml
- - - - > Stepping into Module: CorePlaceholders.xml
- - - > Stepping into Module: HUD.xml
- - - > Stepping into Module: RPGAnalyzers.xml
- - - > Stepping into Module: RPGAutons.xml
- - - > Stepping into Module: RPGAuxDevices.xml
- - - > Stepping into Module: RPGBehaviors.xml
- - - > Stepping into Module: RPGCharacterClasses.xml
- - - > Stepping into Module: RPGCharacters.xml
- - - > Stepping into Module: RPGCode.xml
- - - > Stepping into Module: RPGCompatibility.xml
- - - > Stepping into Module: RPGArmorDockScreens.xml
- - - > Stepping into Module: RPGDeviceDockScreens.xml
- - - > Stepping into Module: RPGDialog.xml
- - - > Stepping into Module: RPGDockScreens.xml
- - - > Stepping into Module: RPGMissions.xml
- - - > Stepping into Module: RPGPlayer.xml
- - - > Stepping into Module: RPGShipScreens.xml
- - - > Stepping into Module: RPGTypes.xml
- - - > Stepping into Module: RPGWingmen.xml
- - > Stepping into Module: Environments.xml
- - > Stepping into Module: GalaxyCompatibility.xml
- - > Stepping into Module: GalaxyEffects.xml
- - > Stepping into Module: Groups.xml
- - > Stepping into Module: Ice.xml
- - > Stepping into Module: Metallic.xml
- - > Stepping into Module: Mining.xml
- - > Stepping into Module: Ore.xml
- - > Stepping into Module: Ovoids.xml
- - > Stepping into Module: Neurohack.xml
- - > Stepping into Module: Primordial.xml
- - > Stepping into Module: Rocky.xml
- - > Stepping into Module: Stargates.xml
- - > Stepping into Module: Stars.xml
- - > Stepping into Module: StarSystems.xml
- - > Stepping into Module: Volcanic.xml
- - > Stepping into Module: Worlds.xml
- > Stepping into Module: Domina.xml
- > Stepping into Module: HSCompatibility.xml
- > Stepping into Module: HumanSpaceMap.xml
- > Stepping into Module: Antarctica.xml
- > Stepping into Module: BattleArena.xml
- > Stepping into Module: BattleArenaGladiators.xml
- > Stepping into Module: CharonFortress.xml
- > Stepping into Module: Eridani.xml
- > Stepping into Module: Heretic.xml
- > Stepping into Module: PointJuno.xml
- > Stepping into Module: StKatharines.xml
- > Stepping into Module: AresOrthodoxy.xml
- > Stepping into Module: Benedict.xml
- > Stepping into Module: BlackMarket.xml
- > Stepping into Module: CentauriWarlords.xml
- > Stepping into Module: CharonPirates.xml
- > Stepping into Module: Commonwealth.xml
- > Stepping into Module: CommonwealthAgricultural.xml
- > Stepping into Module: CommonwealthFleet.xml
- > Stepping into Module: CommonwealthMetropolis.xml
- > Stepping into Module: CommonwealthMilitia.xml
- > Stepping into Module: CommonwealthMining.xml
- > Stepping into Module: CorporateHierarchy.xml
- > Stepping into Module: DwargRaiders.xml
- > Stepping into Module: EncountersVol01.xml
- > Stepping into Module: Iocrym.xml
- > Stepping into Module: KateMorgental.xml
- > Stepping into Module: KorolovShipping.xml
- > Stepping into Module: Luminous.xml
- > Stepping into Module: OutlawMiners.xml
- > Stepping into Module: Rasiermesser.xml
- > Stepping into Module: Ringers.xml
- > Stepping into Module: RogueFleet.xml
- > Stepping into Module: SistersOfDomina.xml
- > Stepping into Module: SungSlavers.xml
- > Stepping into Module: Teratons.xml
- > Stepping into Module: Tinkers.xml
- > Stepping into Module: UrakWarlords.xml
- > Stepping into Module: MiscItems.xml
- > Stepping into Module: StdArmor.xml
- > Stepping into Module: StdAutons.xml
- > Stepping into Module: StdDevices.xml
- > Stepping into Module: StdEffects.xml
- > Stepping into Module: StdFuel.xml
- > Stepping into Module: StdPlayerShips.xml
- > Stepping into Module: StdShields.xml
- > Stepping into Module: StdSovereigns.xml
- > Stepping into Module: StdSystems.xml
- > Stepping into Module: StdUnknownItems.xml
- > Stepping into Module: StdWeapons.xml
- > Stepping into Module: UsefulItems.xml
- > Stepping into Module: Code.xml
- > Stepping into Module: StdDockScreens.xml
- > Stepping into Module: StdTypes.xml
> Stepping into CoreLibrary: CompatibilityLibrary.xml
- > Stepping into Library: CoreTypesLibrary.xml
- - > Stepping into Module: CoreCode.xml
- - > Stepping into Module: CoreEffects.xml
- - > Stepping into Module: CoreExplosions.xml
- - > Stepping into Module: CorePlaceholders.xml
- > Stepping into Library: RPGLibrary.xml
- - > Stepping into Library: CoreTypesLibrary.xml
- - - > Stepping into Module: CoreCode.xml
- - - > Stepping into Module: CoreEffects.xml
- - - > Stepping into Module: CoreExplosions.xml
- - - > Stepping into Module: CorePlaceholders.xml
- - > Stepping into Module: HUD.xml
- - > Stepping into Module: RPGAnalyzers.xml
- - > Stepping into Module: RPGAutons.xml
- - > Stepping into Module: RPGAuxDevices.xml
- - > Stepping into Module: RPGBehaviors.xml
- - > Stepping into Module: RPGCharacterClasses.xml
- - > Stepping into Module: RPGCharacters.xml
- - > Stepping into Module: RPGCode.xml
- - > Stepping into Module: RPGCompatibility.xml
- - > Stepping into Module: RPGArmorDockScreens.xml
- - > Stepping into Module: RPGDeviceDockScreens.xml
- - > Stepping into Module: RPGDialog.xml
- - > Stepping into Module: RPGDockScreens.xml
- - > Stepping into Module: RPGMissions.xml
- - > Stepping into Module: RPGPlayer.xml
- - > Stepping into Module: RPGShipScreens.xml
- - > Stepping into Module: RPGTypes.xml
- - > Stepping into Module: RPGWingmen.xml
- > Stepping into Library: HumanSpaceVol01.xml
- - > Stepping into Library: CoreTypesLibrary.xml
- - - > Stepping into Module: CoreCode.xml
- - - > Stepping into Module: CoreEffects.xml
- - - > Stepping into Module: CoreExplosions.xml
- - - > Stepping into Module: CorePlaceholders.xml
- - > Stepping into Library: RPGLibrary.xml
- - - > Stepping into Library: CoreTypesLibrary.xml
- - - - > Stepping into Module: CoreCode.xml
- - - - > Stepping into Module: CoreEffects.xml
- - - - > Stepping into Module: CoreExplosions.xml
- - - - > Stepping into Module: CorePlaceholders.xml
- - - > Stepping into Module: HUD.xml
- - - > Stepping into Module: RPGAnalyzers.xml
- - - > Stepping into Module: RPGAutons.xml
- - - > Stepping into Module: RPGAuxDevices.xml
- - - > Stepping into Module: RPGBehaviors.xml
- - - > Stepping into Module: RPGCharacterClasses.xml
- - - > Stepping into Module: RPGCharacters.xml
- - - > Stepping into Module: RPGCode.xml
- - - > Stepping into Module: RPGCompatibility.xml
- - - > Stepping into Module: RPGArmorDockScreens.xml
- - - > Stepping into Module: RPGDeviceDockScreens.xml
- - - > Stepping into Module: RPGDialog.xml
- - - > Stepping into Module: RPGDockScreens.xml
- - - > Stepping into Module: RPGMissions.xml
- - - > Stepping into Module: RPGPlayer.xml
- - - > Stepping into Module: RPGShipScreens.xml
- - - > Stepping into Module: RPGTypes.xml
- - - > Stepping into Module: RPGWingmen.xml
- - > Stepping into Library: GalaxyLibrary.xml
- - - > Stepping into Library: CoreTypesLibrary.xml
- - - - > Stepping into Module: CoreCode.xml
- - - - > Stepping into Module: CoreEffects.xml
- - - - > Stepping into Module: CoreExplosions.xml
- - - - > Stepping into Module: CorePlaceholders.xml
- - - > Stepping into Library: RPGLibrary.xml
- - - - > Stepping into Library: CoreTypesLibrary.xml
- - - - - > Stepping into Module: CoreCode.xml
- - - - - > Stepping into Module: CoreEffects.xml
- - - - - > Stepping into Module: CoreExplosions.xml
- - - - - > Stepping into Module: CorePlaceholders.xml
- - - - > Stepping into Module: HUD.xml
- - - - > Stepping into Module: RPGAnalyzers.xml
- - - - > Stepping into Module: RPGAutons.xml
- - - - > Stepping into Module: RPGAuxDevices.xml
- - - - > Stepping into Module: RPGBehaviors.xml
- - - - > Stepping into Module: RPGCharacterClasses.xml
- - - - > Stepping into Module: RPGCharacters.xml
- - - - > Stepping into Module: RPGCode.xml
- - - - > Stepping into Module: RPGCompatibility.xml
- - - - > Stepping into Module: RPGArmorDockScreens.xml
- - - - > Stepping into Module: RPGDeviceDockScreens.xml
- - - - > Stepping into Module: RPGDialog.xml
- - - - > Stepping into Module: RPGDockScreens.xml
- - - - > Stepping into Module: RPGMissions.xml
- - - - > Stepping into Module: RPGPlayer.xml
- - - - > Stepping into Module: RPGShipScreens.xml
- - - - > Stepping into Module: RPGTypes.xml
- - - - > Stepping into Module: RPGWingmen.xml
- - - > Stepping into Module: Environments.xml
- - - > Stepping into Module: GalaxyCompatibility.xml
- - - > Stepping into Module: GalaxyEffects.xml
- - - > Stepping into Module: Groups.xml
- - - > Stepping into Module: Ice.xml
- - - > Stepping into Module: Metallic.xml
- - - > Stepping into Module: Mining.xml
- - - > Stepping into Module: Ore.xml
- - - > Stepping into Module: Ovoids.xml
- - - > Stepping into Module: Neurohack.xml
- - - > Stepping into Module: Primordial.xml
- - - > Stepping into Module: Rocky.xml
- - - > Stepping into Module: Stargates.xml
- - - > Stepping into Module: Stars.xml
- - - > Stepping into Module: StarSystems.xml
- - - > Stepping into Module: Volcanic.xml
- - - > Stepping into Module: Worlds.xml
- - > Stepping into Module: Domina.xml
- - > Stepping into Module: HSCompatibility.xml
- - > Stepping into Module: HumanSpaceMap.xml
- - > Stepping into Module: Antarctica.xml
- - > Stepping into Module: BattleArena.xml
- - > Stepping into Module: BattleArenaGladiators.xml
- - > Stepping into Module: CharonFortress.xml
- - > Stepping into Module: Eridani.xml
- - > Stepping into Module: Heretic.xml
- - > Stepping into Module: PointJuno.xml
- - > Stepping into Module: StKatharines.xml
- - > Stepping into Module: AresOrthodoxy.xml
- - > Stepping into Module: Benedict.xml
- - > Stepping into Module: BlackMarket.xml
- - > Stepping into Module: CentauriWarlords.xml
- - > Stepping into Module: CharonPirates.xml
- - > Stepping into Module: Commonwealth.xml
- - > Stepping into Module: CommonwealthAgricultural.xml
- - > Stepping into Module: CommonwealthFleet.xml
- - > Stepping into Module: CommonwealthMetropolis.xml
- - > Stepping into Module: CommonwealthMilitia.xml
- - > Stepping into Module: CommonwealthMining.xml
- - > Stepping into Module: CorporateHierarchy.xml
- - > Stepping into Module: DwargRaiders.xml
- - > Stepping into Module: EncountersVol01.xml
- - > Stepping into Module: Iocrym.xml
- - > Stepping into Module: KateMorgental.xml
- - > Stepping into Module: KorolovShipping.xml
- - > Stepping into Module: Luminous.xml
- - > Stepping into Module: OutlawMiners.xml
- - > Stepping into Module: Rasiermesser.xml
- - > Stepping into Module: Ringers.xml
- - > Stepping into Module: RogueFleet.xml
- - > Stepping into Module: SistersOfDomina.xml
- - > Stepping into Module: SungSlavers.xml
- - > Stepping into Module: Teratons.xml
- - > Stepping into Module: Tinkers.xml
- - > Stepping into Module: UrakWarlords.xml
- - > Stepping into Module: MiscItems.xml
- - > Stepping into Module: StdArmor.xml
- - > Stepping into Module: StdAutons.xml
- - > Stepping into Module: StdDevices.xml
- - > Stepping into Module: StdEffects.xml
- - > Stepping into Module: StdFuel.xml
- - > Stepping into Module: StdPlayerShips.xml
- - > Stepping into Module: StdShields.xml
- - > Stepping into Module: StdSovereigns.xml
- - > Stepping into Module: StdSystems.xml
- - > Stepping into Module: StdUnknownItems.xml
- - > Stepping into Module: StdWeapons.xml
- - > Stepping into Module: UsefulItems.xml
- - > Stepping into Module: Code.xml
- - > Stepping into Module: StdDockScreens.xml
- - > Stepping into Module: StdTypes.xml
- > Stepping into Module: Compatibility10.xml
- > Stepping into Module: PlayerShips.xml
> Stepping into TranscendenceAdventure: StarsOfThePilgrim.xml
- > Stepping into Library: CoreTypesLibrary.xml
- - > Stepping into Module: CoreCode.xml
- - > Stepping into Module: CoreEffects.xml
- - > Stepping into Module: CoreExplosions.xml
- - > Stepping into Module: CorePlaceholders.xml
- > Stepping into Library: RPGLibrary.xml
- - > Stepping into Library: CoreTypesLibrary.xml
- - - > Stepping into Module: CoreCode.xml
- - - > Stepping into Module: CoreEffects.xml
- - - > Stepping into Module: CoreExplosions.xml
- - - > Stepping into Module: CorePlaceholders.xml
- - > Stepping into Module: HUD.xml
- - > Stepping into Module: RPGAnalyzers.xml
- - > Stepping into Module: RPGAutons.xml
- - > Stepping into Module: RPGAuxDevices.xml
- - > Stepping into Module: RPGBehaviors.xml
- - > Stepping into Module: RPGCharacterClasses.xml
- - > Stepping into Module: RPGCharacters.xml
- - > Stepping into Module: RPGCode.xml
- - > Stepping into Module: RPGCompatibility.xml
- - > Stepping into Module: RPGArmorDockScreens.xml
- - > Stepping into Module: RPGDeviceDockScreens.xml
- - > Stepping into Module: RPGDialog.xml
- - > Stepping into Module: RPGDockScreens.xml
- - > Stepping into Module: RPGMissions.xml
- - > Stepping into Module: RPGPlayer.xml
- - > Stepping into Module: RPGShipScreens.xml
- - > Stepping into Module: RPGTypes.xml
- - > Stepping into Module: RPGWingmen.xml
- > Stepping into Library: GalaxyLibrary.xml
- - > Stepping into Library: CoreTypesLibrary.xml
- - - > Stepping into Module: CoreCode.xml
- - - > Stepping into Module: CoreEffects.xml
- - - > Stepping into Module: CoreExplosions.xml
- - - > Stepping into Module: CorePlaceholders.xml
- - > Stepping into Library: RPGLibrary.xml
- - - > Stepping into Library: CoreTypesLibrary.xml
- - - - > Stepping into Module: CoreCode.xml
- - - - > Stepping into Module: CoreEffects.xml
- - - - > Stepping into Module: CoreExplosions.xml
- - - - > Stepping into Module: CorePlaceholders.xml
- - - > Stepping into Module: HUD.xml
- - - > Stepping into Module: RPGAnalyzers.xml
- - - > Stepping into Module: RPGAutons.xml
- - - > Stepping into Module: RPGAuxDevices.xml
- - - > Stepping into Module: RPGBehaviors.xml
- - - > Stepping into Module: RPGCharacterClasses.xml
- - - > Stepping into Module: RPGCharacters.xml
- - - > Stepping into Module: RPGCode.xml
- - - > Stepping into Module: RPGCompatibility.xml
- - - > Stepping into Module: RPGArmorDockScreens.xml
- - - > Stepping into Module: RPGDeviceDockScreens.xml
- - - > Stepping into Module: RPGDialog.xml
- - - > Stepping into Module: RPGDockScreens.xml
- - - > Stepping into Module: RPGMissions.xml
- - - > Stepping into Module: RPGPlayer.xml
- - - > Stepping into Module: RPGShipScreens.xml
- - - > Stepping into Module: RPGTypes.xml
- - - > Stepping into Module: RPGWingmen.xml
- - > Stepping into Module: Environments.xml
- - > Stepping into Module: GalaxyCompatibility.xml
- - > Stepping into Module: GalaxyEffects.xml
- - > Stepping into Module: Groups.xml
- - > Stepping into Module: Ice.xml
- - > Stepping into Module: Metallic.xml
- - > Stepping into Module: Mining.xml
- - > Stepping into Module: Ore.xml
- - > Stepping into Module: Ovoids.xml
- - > Stepping into Module: Neurohack.xml
- - > Stepping into Module: Primordial.xml
- - > Stepping into Module: Rocky.xml
- - > Stepping into Module: Stargates.xml
- - > Stepping into Module: Stars.xml
- - > Stepping into Module: StarSystems.xml
- - > Stepping into Module: Volcanic.xml
- - > Stepping into Module: Worlds.xml
- > Stepping into Library: HumanSpaceVol01.xml
- - > Stepping into Library: CoreTypesLibrary.xml
- - - > Stepping into Module: CoreCode.xml
- - - > Stepping into Module: CoreEffects.xml
- - - > Stepping into Module: CoreExplosions.xml
- - - > Stepping into Module: CorePlaceholders.xml
- - > Stepping into Library: RPGLibrary.xml
- - - > Stepping into Library: CoreTypesLibrary.xml
- - - - > Stepping into Module: CoreCode.xml
- - - - > Stepping into Module: CoreEffects.xml
- - - - > Stepping into Module: CoreExplosions.xml
- - - - > Stepping into Module: CorePlaceholders.xml
- - - > Stepping into Module: HUD.xml
- - - > Stepping into Module: RPGAnalyzers.xml
- - - > Stepping into Module: RPGAutons.xml
- - - > Stepping into Module: RPGAuxDevices.xml
- - - > Stepping into Module: RPGBehaviors.xml
- - - > Stepping into Module: RPGCharacterClasses.xml
- - - > Stepping into Module: RPGCharacters.xml
- - - > Stepping into Module: RPGCode.xml
- - - > Stepping into Module: RPGCompatibility.xml
- - - > Stepping into Module: RPGArmorDockScreens.xml
- - - > Stepping into Module: RPGDeviceDockScreens.xml
- - - > Stepping into Module: RPGDialog.xml
- - - > Stepping into Module: RPGDockScreens.xml
- - - > Stepping into Module: RPGMissions.xml
- - - > Stepping into Module: RPGPlayer.xml
- - - > Stepping into Module: RPGShipScreens.xml
- - - > Stepping into Module: RPGTypes.xml
- - - > Stepping into Module: RPGWingmen.xml
- - > Stepping into Library: GalaxyLibrary.xml
- - - > Stepping into Library: CoreTypesLibrary.xml
- - - - > Stepping into Module: CoreCode.xml
- - - - > Stepping into Module: CoreEffects.xml
- - - - > Stepping into Module: CoreExplosions.xml
- - - - > Stepping into Module: CorePlaceholders.xml
- - - > Stepping into Library: RPGLibrary.xml
- - - - > Stepping into Library: CoreTypesLibrary.xml
- - - - - > Stepping into Module: CoreCode.xml
- - - - - > Stepping into Module: CoreEffects.xml
- - - - - > Stepping into Module: CoreExplosions.xml
- - - - - > Stepping into Module: CorePlaceholders.xml
- - - - > Stepping into Module: HUD.xml
- - - - > Stepping into Module: RPGAnalyzers.xml
- - - - > Stepping into Module: RPGAutons.xml
- - - - > Stepping into Module: RPGAuxDevices.xml
- - - - > Stepping into Module: RPGBehaviors.xml
- - - - > Stepping into Module: RPGCharacterClasses.xml
- - - - > Stepping into Module: RPGCharacters.xml
- - - - > Stepping into Module: RPGCode.xml
- - - - > Stepping into Module: RPGCompatibility.xml
- - - - > Stepping into Module: RPGArmorDockScreens.xml
- - - - > Stepping into Module: RPGDeviceDockScreens.xml
- - - - > Stepping into Module: RPGDialog.xml
- - - - > Stepping into Module: RPGDockScreens.xml
- - - - > Stepping into Module: RPGMissions.xml
- - - - > Stepping into Module: RPGPlayer.xml
- - - - > Stepping into Module: RPGShipScreens.xml
- - - - > Stepping into Module: RPGTypes.xml
- - - - > Stepping into Module: RPGWingmen.xml
- - - > Stepping into Module: Environments.xml
- - - > Stepping into Module: GalaxyCompatibility.xml
- - - > Stepping into Module: GalaxyEffects.xml
- - - > Stepping into Module: Groups.xml
- - - > Stepping into Module: Ice.xml
- - - > Stepping into Module: Metallic.xml
- - - > Stepping into Module: Mining.xml
- - - > Stepping into Module: Ore.xml
- - - > Stepping into Module: Ovoids.xml
- - - > Stepping into Module: Neurohack.xml
- - - > Stepping into Module: Primordial.xml
- - - > Stepping into Module: Rocky.xml
- - - > Stepping into Module: Stargates.xml
- - - > Stepping into Module: Stars.xml
- - - > Stepping into Module: StarSystems.xml
- - - > Stepping into Module: Volcanic.xml
- - - > Stepping into Module: Worlds.xml
- - > Stepping into Module: Domina.xml
- - > Stepping into Module: HSCompatibility.xml
- - > Stepping into Module: HumanSpaceMap.xml
- - > Stepping into Module: Antarctica.xml
- - > Stepping into Module: BattleArena.xml
- - > Stepping into Module: BattleArenaGladiators.xml
- - > Stepping into Module: CharonFortress.xml
- - > Stepping into Module: Eridani.xml
- - > Stepping into Module: Heretic.xml
- - > Stepping into Module: PointJuno.xml
- - > Stepping into Module: StKatharines.xml
- - > Stepping into Module: AresOrthodoxy.xml
- - > Stepping into Module: Benedict.xml
- - > Stepping into Module: BlackMarket.xml
- - > Stepping into Module: CentauriWarlords.xml
- - > Stepping into Module: CharonPirates.xml
- - > Stepping into Module: Commonwealth.xml
- - > Stepping into Module: CommonwealthAgricultural.xml
- - > Stepping into Module: CommonwealthFleet.xml
- - > Stepping into Module: CommonwealthMetropolis.xml
- - > Stepping into Module: CommonwealthMilitia.xml
- - > Stepping into Module: CommonwealthMining.xml
- - > Stepping into Module: CorporateHierarchy.xml
- - > Stepping into Module: DwargRaiders.xml
- - > Stepping into Module: EncountersVol01.xml
- - > Stepping into Module: Iocrym.xml
- - > Stepping into Module: KateMorgental.xml
- - > Stepping into Module: KorolovShipping.xml
- - > Stepping into Module: Luminous.xml
- - > Stepping into Module: OutlawMiners.xml
- - > Stepping into Module: Rasiermesser.xml
- - > Stepping into Module: Ringers.xml
- - > Stepping into Module: RogueFleet.xml
- - > Stepping into Module: SistersOfDomina.xml
- - > Stepping into Module: SungSlavers.xml
- - > Stepping into Module: Teratons.xml
- - > Stepping into Module: Tinkers.xml
- - > Stepping into Module: UrakWarlords.xml
- - > Stepping into Module: MiscItems.xml
- - > Stepping into Module: StdArmor.xml
- - > Stepping into Module: StdAutons.xml
- - > Stepping into Module: StdDevices.xml
- - > Stepping into Module: StdEffects.xml
- - > Stepping into Module: StdFuel.xml
- - > Stepping into Module: StdPlayerShips.xml
- - > Stepping into Module: StdShields.xml
- - > Stepping into Module: StdSovereigns.xml
- - > Stepping into Module: StdSystems.xml
- - > Stepping into Module: StdUnknownItems.xml
- - > Stepping into Module: StdWeapons.xml
- - > Stepping into Module: UsefulItems.xml
- - > Stepping into Module: Code.xml
- - > Stepping into Module: StdDockScreens.xml
- - > Stepping into Module: StdTypes.xml
- > Stepping into Module: Benedict00.xml
- > Stepping into Module: Benedict01.xml
- > Stepping into Module: Benedict02.xml
- > Stepping into Module: Benedict03.xml
- > Stepping into Module: BenedictStoryArc.xml
- > Stepping into Module: Elysium.xml
- > Stepping into Module: Huari.xml
- > Stepping into Module: PlayerShips.xml
(Maybe I should include some sort of infinite recursion protection.)


Problems encountered

1.
According to the XML standard, apparently '--' is not allowed within a comment block ('<!-- ... -->').
So, for example (Eridani.xml):

Code: Select all

<!--
	<Station type="&stTeratonFactory;"/>
	<Station type="&stCommonwealthFortress;"/>
	<Station type="&stSungSlaveCamp;"/>
	<Station type="&stCentauriOccupation;"/>
	<Station type="&stCSCAmericaEncounter;"/>
	<LevelTable>
		<Station levelFrequency="c----" type="&stWondrousDevices;"/>
		<Station levelFrequency="-----" type="&stInsuranceCompany;"/>
	</LevelTable>
	<Station type="&stCSCSaharaEncounter;"/>
	<Station type="&stInsuranceCompany;"/>
	<Station type="&stKorolovShipping;"/>
	<Station type="&stAgriculturalStation;" />
	<Station type="&stCorporateTradingPost;" />
	<Station type="&stCSCArcticaEncounter;"/>
	<Station type="&stRingerShipyard;"/>
	<Station type="&stGaianProcessorEncounter;" />
	<Station type="&stFerianColony;" />
	-->
Gives me an error:

Code: Select all

'<Station levelFrequency="c----" typ'
                             ^---- Parser error : Comment not terminated 
so I'll need to come up with a method to work around it (probably I'm going to pre-process the files and remove all comment blocks).

2.
Non-well formed XML:

Code: Select all

'<00182001_Donation'
  ^---- StartTag: invalid element name
This one is harder. Simple solution would be to prepend a _ to each element that starts with a number. It would probably be an ok-enough solution for our purpose (automatically updating the wiki) but at the same time it would qualify as an pretty ugly hackish solution, which I would rather avoid (because of forward compatibility with possible later features e.a.).

This may turn out harder than anticipated :-(


(If I skip the files:
  • Eridani.xml
  • StKatharines.xml
  • CharonPirates.xml
  • RogueFleet.xml
  • MiscItems.xml
  • StdDockScreens.xml
all is fine though)[/size]


Edit:
Updated with some clarifications.

Cheers,
Pixelfck
Image
Download the Black Market Expansion from Xelerus.de today!
My other mods at xelerus.de

User avatar
pixelfck
Militia Captain
Militia Captain
Posts: 571
Joined: Tue Aug 11, 2009 8:47 pm
Location: Travelling around in Europe

Wed Jun 03, 2015 10:45 am

(Updated previous post)
~Pixelfck
Image
Download the Black Market Expansion from Xelerus.de today!
My other mods at xelerus.de

george moromisato
Developer
Developer
Posts: 2979
Joined: Thu Jul 24, 2003 9:53 pm
Contact:

Wed Jun 03, 2015 2:19 pm

pixelfck wrote:This may turn out harder than anticipated :-(
~Pixelfck
Sorry about that! It's my fault for not using a standard XML parser.

User avatar
pixelfck
Militia Captain
Militia Captain
Posts: 571
Joined: Tue Aug 11, 2009 8:47 pm
Location: Travelling around in Europe

Wed Jun 03, 2015 2:58 pm

george moromisato wrote:
pixelfck wrote:This may turn out harder than anticipated :-(
~Pixelfck
Sorry about that! It's my fault for not using a standard XML parser.
Don't worry about it. It should all work out Ok, and the fun is in writing the script anyway :-)

question:
pixelfck wrote: I'm somewhat surprised at the amount of double reading/traversing the inclusion tree; CoreTypesLibrary.xml is, for example, visited 22 times?
Is this correct?:
~Pixelfck
Image
Download the Black Market Expansion from Xelerus.de today!
My other mods at xelerus.de

george moromisato
Developer
Developer
Posts: 2979
Joined: Thu Jul 24, 2003 9:53 pm
Contact:

Wed Jun 03, 2015 7:53 pm

pixelfck wrote:I'm somewhat surprised at the amount of double reading/traversing the inclusion tree; CoreTypesLibrary.xml is, for example, visited 22 times?
Is this correct?:
A <Library> statement is a reference to a library, not an include directive (like a module).

The core engine keeps a list of all extensions (libraries, adventures, whatever) by UNID. When it sees a <Library> reference it just looks up the extension, makes sure it's loaded, and then pulls the list of entities. We never have to parse a library twice.

For your purposes, I'm not sure it matters, but that's the way it works in the engine.

Post Reply