From 87c0aae54b5ccab2b12a294e2a346a8d247e469d Mon Sep 17 00:00:00 2001 From: Nick Thomas Date: Wed, 15 Apr 2020 22:18:53 +0100 Subject: [PATCH] More file format musing --- doc/formats/ani.md | 86 +++++++++++---- doc/formats/index.md | 12 +-- doc/formats/obj.md | 250 ++++++++++++++++++++++--------------------- 3 files changed, 201 insertions(+), 147 deletions(-) diff --git a/doc/formats/ani.md b/doc/formats/ani.md index 3100efb..fdde390 100644 --- a/doc/formats/ani.md +++ b/doc/formats/ani.md @@ -1,18 +1,71 @@ # `Anim/WarHammer.ani` -This turns out to simply be an [`obj`](obj.md#WarHammer.ani) file. However, some -other files are implicated in its interpretation: +This turns out to simply be an [`obj`](obj.md) file. -- `Data/AniObDef.dat` +The first 1,064 sprites are all of the same Ultramarine, carryng a bolter. There +are eight "facing" orientations: + +* North +* Northeast +* East +* Southeast +* South +* Southwest +* West +* Northwest + +For each orientation, an action is pictured in a variable number of frames. The +final frame for each action appears to be "stationary". + +* Walk (13 frames) +* Run (9 frames) +* Crouch down (8 frames) +* Stand up (8 frames) +* Take aim (standing) (6 frames) +* Fire (standing) (6 frames) +* Relax aim (standing) (6 frames) +* Throw grenade (standing) (18 frames) +* Take aim (crouched) (5 frames) +* Fire (crouched) (5 frames) +* Relax aim (crouched) (5 frames) +* Throw grenade (crouched) (17 frames) +* Draw melee weapon (standing) (10 frames) +* Strike down with melee weapon (standing) (8 frames) +* Stab with melee weapon (standing) (9 frames) + +Added together and multiplied by 87, that's 1064. + +The next sprite is a walking-north action for an ultramarine with a flamer. The +total number of frames for this character is 1120 - 56 additional frames, or 7 +per orientation. Could be an extra action, or an extra frame per action. + +Also notable is that while the bolter showed muzzle flash in the animation, the +flamer only showed a tiny hint of fire. I think the animation for spewing flame +is held elsewhere. + +I strongly suspect the actions and the number of frames in each action are +configurable. So, what other files are implicated in its interpretation? Here's +a few possibilities: + +* `Data/AniObDef.dat` +* `Data/Coordinates.dat` +* `Data/HasAction.dat` +* `Data/VehicDef.dat` +* `Data/WeapDef.dat` - `Idx/WarHammer.idx` -Including comments, the former is 4098 lines, giving approx. 45 lines for each +## `Data/AniObDef.dat` + +Including comments, this is 4098 lines, giving approx. 45 lines for each of the ~188 characters in the `ani`. That doesn't seem many, and there's no obvious correspondence between the commented-on names (`SMOKE01`?) and the viewed frames... but then, I've not viewed all the frames. -Still, I think a focus in `WarHammer.idx` (1,880,078 bytes, binary, so around -10KiB per character, in theory) is more reasonable. + +## `Idx/WarHammer.idx` + +`WarHammer.idx` (1,880,078 bytes, binary, so around 10KiB per character, in +theory) is more reasonable. Here's a list of operations on the file when `WH40K_TD.EXE` is instructed to place an Ultramarine squad: @@ -310,22 +363,15 @@ read(, "\xbe\xe5\xb6\x09\x0c\x0f\x00\x00", 8) = 8 -Notable is that we still load 5 type 2 records, even though there's just a -single librarian, and 8 compass points. Why 5? +Notable is that we read from `idx` **before** we read from `ani` - so it does +seem like the former should tell us where to pull from the latter. -Also notable is that we read from `idx` **before** we read from `ani`. +We still load 5 type 2 records, even though there's just a single librarian, and +8 compass points. Why 5? After looking in `idx`, we load 10 sprites from `ani`, +which is at least a multiple. Do any of the records we load from `idx` specify +sprite directory offsets that match? -Other data may be loaded at program start, of course. What other files seem -relevant? - -* `Data/AniObDef.dat` -* `Data/HasAction.dat` -* `Data/VehicDef.dat` -* `Data/WeapDef.dat` - -These all reference animations in one way or another. - -## `HasAction.dat` +## `Data/HasAction.dat` This file seems relevant as it says whether or not particular animations exist for the different types of character, which maps directly to what is stored in diff --git a/doc/formats/index.md b/doc/formats/index.md index 53fd865..0681491 100644 --- a/doc/formats/index.md +++ b/doc/formats/index.md @@ -9,7 +9,7 @@ remake. * [✓] [`Anim/`](obj.md#WarHammer.ani) * [`WarHammer.ani`](obj.md#WarHammer.ani) * [`Assign/`](obj.md#assign) - * `*.asn` # Unknown, seems to be related to .obj files + * `*.asn` # Specify properties for frames in .obj files * `Cursor/` * `*.ani` # RIFF data, standard ANI format \o/ * [`Cursors.cur`](obj.md) # `obj` file containing pointers and drag elements @@ -23,6 +23,7 @@ remake. * **TODO** * `ChaNames.dat` # list of character names * `Coordinates.dat` # Weapon Firing Coordinates + * `Credits.dat` # list of credits * `Defs.dat` # defines properties for objects and tiles, each of which seems to have an id * `GDestroy.dat` # table of what destroys what? @@ -39,7 +40,6 @@ remake. * `WeapDef.dat` # Weapon definitions * **PROBABLY NOT NEEDED** * `BugHunt.dat` # Contains SMF text for Bug hunt random missions - * `Credits.dat` # list of credits * `GenArm.dat` # "Random campaign armory levels - space marine" * `HeroArm.dat` # "Random campaign armory levels - Veteran" * `MHeroArm.dat` # "Random campaign armory levels - Veteran" @@ -49,11 +49,9 @@ remake. * `SpArm.dat` # "RANDOM CAMPAIGN ARMORY LEVELS - Veteran" * `VetArm.dat` # "RANDOM CAMPAIGN ARMORY LEVELS - Veteran" * `*.chk` # checksums? Mentions all the .dat files - * `*.cyc` # ColorCycle DataFile. - * `*.dta` # localized strings and things + * `Cycle.cyc` # ColorCycle DataFile. * `Encyclopedia.dta` # encyclopedia entries - * `KeyMap.dta` # unknown - * `Keymap.dta` # unknown + * `keymap.dta` # unknown * `USEng.dta` # Localized strings * `EquipmentMenuData` # gzip-compressed, presumably to do with (initial?) squad configuration * `Filters/` @@ -105,7 +103,7 @@ Phew. * *Almost everything* seems to be in a data file somewhere. Helpful! * `make loader` creates a `load` binary that will try to load various bits of data from `investigation/` and `orig/`. I use it to investigate file - formats and the parsers I'm writing. Ensure you're in a `GOPATH`! + formats and the parsers I'm writing. ## Cross-links / associations diff --git a/doc/formats/obj.md b/doc/formats/obj.md index 8c3bb21..5fc8b45 100644 --- a/doc/formats/obj.md +++ b/doc/formats/obj.md @@ -6,80 +6,6 @@ be placed on it - a `SURFACE`, `LEFT`, `RIGHT`, and `CENTER`. Maps reference objects in a space-efficient way via sets, which seem to be a kind of object palette. -## Assign - -The `Assign/` directory contains a matching `.asn` file for each `Obj/*.obj`. -It's a plain-text format which seems to assign properties to frames, and has -references to a `.flc` file which does not exist in the tree. - -Theory: .obj files were originally generated from `.flc` files. This is an -AutoDesk format for visual data, so this suggests the .obj files contain pixels -\o/ - -`blank.asn` references 6 frames (0-5): - -``` -# single pixel tile -# transpix.obj/.asn -#/--> transpix.flc -# - -0-5:DEF 1; - - -0-5:TYPE 13; - - -END OF FILE -``` - -`jungtil.asn` references 18 frames (0-17): - -``` -# jungle floor -# jungtil.obj/.asn -# /--> d:\warflics\missions\jungtil.flc -# - -0:DEF 2; -1-11:DEF 454; - - -#damaged frames!!!! -12:DEF 2; -13-16:DEF 454; -17:DEF 454; - - -0:TYPE 2; -1-11:TYPE 0; -12:TYPE 2; -13-16:TYPE 0; -17:TYPE 0; - - -0:DESTROY 12; -1-3:DESTROY 13; -4-6:DESTROY 14; -7-9:DESTROY 15; -10-11:DESTROY 16; -17:DESTROY 15; - -1-11:Dmg1Lnk 17; - -END OF FILE -``` - -So it seems this visual data can have quite complicated attributes. At a minimum -we see: - -* `TYPE` -* `DEF` -* `DESTROY` -* `Dmg1Lnk` - -The `type` field **may** tell us what format each sprite is in. - ## OBJ container structure `.obj` files represent visual data. They contain a number of sprites, which are @@ -512,60 +438,144 @@ break *0x41DD10 This lets me focus very narrowly on what happens when loading sprites, and might give clues. +## Assign + +The `Assign/` directory contains a matching `.asn` file for each `Obj/*.obj`. +It's a plain-text format which seems to assign properties to frames, and has +references to a `.flc` file which does not exist in the tree. + +Theory: .obj files were originally generated from `.flc` files. This is an +AutoDesk format for visual data, so this suggests the .obj files contain pixels +\o/ + +`blank.asn` references 6 frames (0-5): + +``` +# single pixel tile +# transpix.obj/.asn +#/--> transpix.flc +# + +0-5:DEF 1; + + +0-5:TYPE 13; + + +END OF FILE +``` + +`jungtil.asn` references 18 frames (0-17): + +``` +# jungle floor +# jungtil.obj/.asn +# /--> d:\warflics\missions\jungtil.flc +# + +0:DEF 2; +1-11:DEF 454; + + +#damaged frames!!!! +12:DEF 2; +13-16:DEF 454; +17:DEF 454; + + +0:TYPE 2; +1-11:TYPE 0; +12:TYPE 2; +13-16:TYPE 0; +17:TYPE 0; + + +0:DESTROY 12; +1-3:DESTROY 13; +4-6:DESTROY 14; +7-9:DESTROY 15; +10-11:DESTROY 16; +17:DESTROY 15; + +1-11:Dmg1Lnk 17; + +END OF FILE +``` + +So it seems this visual data can have quite complicated attributes. We see: + +* `DEF` +* `DESTROY` +* `Dmg1Lnk` +* `MacroLnk` +* `TYPE` + +Each command takes the form: + +``` +: ; +``` + +The sprite-spec can either be a single number, or a range of n-m. + +### `DESTROY` + +### `DEF` + +I wonder if this `DEF`ines sprites an object, represented by an integer. In +`Data/Defs.dat`, we see sections that `EDIT `. Perhaps the two point to +the same thing? + +`Assign/cabbage.asn` has `DEF 17` and `DEF 20`, covering two ranges of sprites. +`Data/Defs.dat` has `EDIT 17 # TREE LEAVES` and `EDIT 20 # NON-CENTERED CENTER +BUSHES`. + +`Assign/TZEENTCH.asn` has `DEF 24` and `DEF 25`. `Data/Defs.dat` has `EDIT 24 # +SWITCHES & HOSES AS CENTER OBJS...RIGHT` and `EDIT 25 # SWITCHES & HOSES AS +CENTER OBJS...LEFT`. + +`Assign/altar.asn` has `DEF 232`, and `Data/Defs.data` has `EDIT 232 # MRS +BUILDING PIECES - POOL TABLE`. + +These all seem really close. + +Then we have `EDIT 19 #big rocks` and these instances of `DEF 19;`: + +``` +Assign/grayroks.asn:5:0-7:DEF 19; +Assign/grayroks.asn:9:8-15:DEF 19; +Assign/sn_roks.asn:6:0-7:DEF 19; +Assign/sn_roks.asn:10:8-15:DEF 19; +``` + +The sprites for `grayroks` depicts some grey rocks, while `sn_roks` depicts +the same, but with snow on them. So this seems correct to me. + +Does each object ID represent a unique thing? Or is it an index for a set of +characteristics that may be shared across multiple things? A set could quite +easily specify both `grayroks` and `sn_roks` for a map... + +### `Dmg1Lnk` + +This command is fairly easy. It takes a sprite-spec and says "if damage has been +sustained and you would normally display these sprites, display this sprite +instead". This is particularly obvious for the `cabbage` pair. + +### `MacroLnk` + +### `TYPE` + + ## WarHammer.ani This 400MiB file appears to be a standard object file, it's just very large. The directory contains 188,286 sprites! -The first 1,064 sprites are all of the same Ultramarine, carryng a bolter. There -are eight "facing" orientations: - -* North -* Northeast -* East -* Southeast -* South -* Southwest -* West -* Northwest - -For each orientation, an action is pictured in a variable number of frames. The -final frame appears to be "stationary". - -* Walk (13 frames) -* Run (9 frames) -* Crouch down (8 frames) -* Stand up (8 frames) -* Take aim (standing) (6 frames) -* Fire (standing) (6 frames) -* Relax aim (standing) (6 frames) -* Throw grenade (standing) (18 frames) -* Take aim (crouched) (5 frames) -* Fire (crouched) (5 frames) -* Relax aim (crouched) (5 frames) -* Throw grenade (crouched) (17 frames) -* Draw melee weapon (standing) (10 frames) -* Strike down with melee weapon (standing) (8 frames) -* Stab with melee weapon (standing) (9 frames) - -Added together and multiplied by 87, that's 1064. - -The next sprite is a walking-north action for an ultramarine with a flamer. The -total number of frames for this character is 1120 - 56 additional frames, or 7 -per orientation. Could be an extra action, or an extra frame per action. - -Also notable is that while the bolter showed muzzle flash in the animation, the -flamer only showed a tiny hint of fire. I think the animation for spewing flame -is held elsewhere. - -I strongly suspect the actions and the number of frames in each action are -configurable. But where? - -The field at 0x10 in the sprite header is set in `WarHammer.ani`, but not in +The field at 0x10 in each sprite header is set in `WarHammer.ani`, but not in the other object files encountered so far. However, it seems to be set statically to the bytes `[212 113 59 1]` for all of them. Assuming ~1000 sprites per character, `WarHammer.ani` contains 188 characters. Two other files have been implicated in animation - `Data/AniObDefs.dat` and -`Idx/WarHammer.idx`. More on those in `ani.md`. +`Idx/WarHammer.idx`. More on those in [ani.md](ani.md)