ordoor/doc/formats/index.md

# Formats

Chaos Gate has over 700MB of data, organized (seemingly) logically into files in
directories. Being able to parse, display, and use that data is paramount to any
remake.

## Filesystem layout

* `Anim/`
    * `WarHammer.ani` # Doesn't seem to be a RIFF file. 398M so very important.
      * There's a pcx image header at `dd ... bs=1 skip=213` but it seems to be a false alert
      * Hits for "AmigaOS bitmap font"... probably a false positive
      * Lots of 8-byte reads when loading stuff in the mission editor
      * Some ~4K reads, havent found one corresponding to a known format yet
    * foo
* [`Assign/`](obj.md#assign)
    * `*.asn` # Unknown, seems to be related to .obj files 
* `Cursor/`
    * `*.ani` # RIFF data
    * `*.cur` # Presumably standard windows-format non-animated cursors
* `Data/`
    * `*.dat` # plaintext files defining properties of objects. No single format
        * **PARSED**
            * `Accounting.dat`  # key = value => internal/data/accounting.go
            * `AniObjDef.dat`   # animated object definitions
            * `GenericData.dat` # Generic Game Settings
        * **TODO**
            * `ChaNames.dat`    # list of character names
            * `Coordinates.dat` # Weapon Firing Coordinates
            * `Defs.dat`        # defines properties for objects and tiles, each of which seems to have an id
            * `GDestroy.dat`    # table of what destroys what?
            * `HasAction.dat`   # "Are there animation for each of the character" - list of booleans
            * `MiniMap.dat`     # lots of seemingly random numbers. IDs?
            * `MissionBriefing.dat` # Contains all Campaign Mission Briefing Text. Sections: "CAMPAIGN MISSION X ... END CAMPAIGN MISSION X"
            * `PWeight.dat`     # Personality weights for the individual character types
            * `Random_AI.dat`   # contains the percentage of the different AI types for each of the different Chaos Squad Types
            * `RandomPlanets.dat` #  Campaign Primary and Secondary Objectives
            * `Sounds.dat`      # Sound Effect Data
            * `SpellDef.dat`    # SPELL DEFINITIONS
            * `StdWeap.dat`     # SQUAD STANDARD WEAPONS
            * `Ultnames.dat`    # List of names for ultramarines
            * `VehicDef.dat`    # VEHICLE DEFINITIONS
            * `WeapDef.dat`     # Weapon definitions
        * **PROBABLY NOT NEEDED**
            * `BugHunt.dat`     # Contains SMF text for Bug hunt random missions
            * `Credits.dat`     # list of credits
            * `GenArm.dat`      # "Random campaign armory levels - space marine"
            * `HeroArm.dat`     # "Random campaign armory levels - Veteran"
            * `MHeroArm.dat`    # "Random campaign armory levels - Veteran"
            * `PlanetDesc.dat`  # well-documented params. random mission generator params per planet.
            * `RandChar.dat`    # Random character matrix. Information on what goes into a character matrix.
            * `RandSMFtext.dat` # SuperMacro Objectives for Campaign Random Mission Generator.
            * `SpArm.dat`       # "RANDOM CAMPAIGN ARMORY LEVELS - Veteran"
            * `VetArm.dat`      # "RANDOM CAMPAIGN ARMORY LEVELS - Veteran"
            * `*.chk` # checksums? Mentions all the .dat files
    * `*.cyc` # ColorCycle DataFile.
    * `*.dta` # localized strings and things
    * `Encyclopedia.dta` # encyclopedia entries
    * `KeyMap.dta`       # unknown
    * `Keymap.dta` # unknown
    * `USEng.dta`  # Localized strings
* `EquipmentMenuData` # gzip-compressed, presumably to do with (initial?) squad configuration
* `Filters/`
    * `wh40k.flt` # Audio filter(s?)
* `Fonts/`
    * `cboxfont` # unknown
    * `*.fnt` # ASCII text. Map keycodes to sprites?
    * `*.spr` # sprites, presumably
* `Idx/`
    * `WarHammer.idx` # unknown, 1.8M
* [`Maps/`](maps.md)
    * `*.MAP`
    * `*.TXT`
* `Menu/`
    * `*.mnu` # plain text, linked to .obj file, contains definitions
    * `*.obj` # unknown, not 3D .obj format
* `Misc/`
    * `occlusio.lis` # plain text, presumably occlusion mappings?
* [`MultiMaps/`](maps.md#multimaps)
    * `*.MAP`
    * `*.TXT`
* [`Obj/`](obj.md)
    * `*.obj` # must be visual data of some sort, one per placeable map object?
* [✓] `Pic/`
    * `*.pcx` # Standard .pcx format
* `RandomMaps/`
    * `*.chk` # multiplayer. worry about these another day
    * `*.dat`
* `Save_G/`
    * `*.sav` # savedata, gzip-compressed, custom format
    * `*.txt` # Seems to be a copy of one of Maps/*.txt
* [✓] [`Sets/`](sets.md)
    * `Data.chk` # checksums? Mentions all the .set files
    * `*.set` # plain text, related to maps. Editor has a concept of map sets, which these must be
* [✓] `SMK/`
    * `*.smk` # Videos: RAD Game Tools Smacker Multimedia version 2
* `Sounds/`
    * `wh40k.ds` # 0xffffffff then a list of .wav file names. Some sort of index?
* [✓] `Wav/`
    * `*.wav` # Normal WAV files

Phew.

## General notes

* For binary formats, integers seem to be stored in little-endian
* *Almost everything* seems to be in a data file somewhere. Helpful!
* `make loader` creates a `load` binary that will try to load various bits
  of data from `investigation/` and `orig/`. I use it to investigate file
  formats and the parsers I'm writing. Ensure you're in a `GOPATH`!

## Cross-links / associations

(Just some noticed so far)

* [`Assign/*.asn`](assign.md):
    * [`Obj/*.obj`](obj.md).
* [`Maps/*.map`](maps.md):
    * [`Maps/*.txt`](maps.md#associated-txt-file)
    * [`Sets/*.set`](sets.md)
* [`Sounds/wh40k.ds`](sounds.md)
    * `Wav/*.wav`