# Map format information ## Overview The `Map/` directory contains linked pairs of files: a `.TXT` and a `.MAP`. The former appears to be structured text, with tilde-separated sections. The latter is a custom format of some kind, GZip-compressed, with a `WHMAP` constant and character names embedded WH40K_TD.exe can read in and write out maps for us, for investigation \o/. I update [structure](#structure) with details as I work them out, and put the working in appropriately-named subsections *below* that section. ## Directory listing ``` total 1.5M -rw-r--r-- 1 65K Nov 4 1998 Chapter01.MAP -rw-r--r-- 1 429 Nov 7 1998 Chapter01.TXT -rw-r--r-- 1 81K Nov 4 1998 Chapter02.MAP -rw-r--r-- 1 587 Nov 7 1998 Chapter02.TXT -rw-r--r-- 1 88K Nov 4 1998 Chapter03.MAP -rw-r--r-- 1 597 Nov 7 1998 Chapter03.TXT -rw-r--r-- 1 100K Jan 26 1999 Chapter04.MAP -rw-r--r-- 1 538 Jan 26 1999 Chapter04.TXT -rw-r--r-- 1 42K Jan 26 1999 Chapter05.MAP -rw-r--r-- 1 1.3K Nov 7 1998 Chapter05.TXT -rw-r--r-- 1 56K Nov 4 1998 Chapter06.MAP -rw-r--r-- 1 572 Nov 7 1998 Chapter06.TXT -rw-r--r-- 1 63K Jan 26 1999 Chapter07.MAP -rw-r--r-- 1 750 Jan 26 1999 Chapter07.TXT -rw-r--r-- 1 66K Nov 20 1998 Chapter08.MAP -rw-r--r-- 1 638 Nov 7 1998 Chapter08.TXT -rw-r--r-- 1 98K Jan 26 1999 Chapter09.MAP -rw-r--r-- 1 656 Jan 26 1999 Chapter09.TXT -rw-r--r-- 1 100K Nov 5 1998 Chapter10.MAP -rw-r--r-- 1 622 Nov 7 1998 Chapter10.TXT -rw-r--r-- 1 110K Nov 4 1998 Chapter11.MAP -rw-r--r-- 1 396 Nov 7 1998 Chapter11.TXT -rw-r--r-- 1 118K Jan 26 1999 Chapter12.MAP -rw-r--r-- 1 593 Nov 7 1998 Chapter12.TXT -rw-r--r-- 1 133K Nov 4 1998 Chapter13.MAP -rw-r--r-- 1 1.1K Nov 7 1998 Chapter13.TXT -rw-r--r-- 1 137K Jan 26 1999 Chapter14.MAP -rw-r--r-- 1 641 Nov 7 1998 Chapter14.TXT -rw-r--r-- 1 140K Nov 5 1998 Chapter15.MAP -rw-r--r-- 1 338 Nov 7 1998 Chapter15.TXT ``` ## Multimaps There are also maps in the `MultiMaps/` directory. I guess they're reworks of the `Maps/` entries for multiplayer with differing numbers of players - `Chapter1_3P.MAP` would be the chapter1 map, just modified for 3-player multiplayer. Ignore for now, but here's the directory listing: ``` total 1.4M -rw-r--r-- 1 93K Nov 5 1998 Chapter10_2P.MAP -rw-r--r-- 1 21 Nov 4 1998 Chapter10_2P.TXT -rw-r--r-- 1 103K Nov 5 1998 Chapter11_4P.MAP -rw-r--r-- 1 36 Nov 4 1998 Chapter11_4P.TXT -rw-r--r-- 1 111K Jan 26 1999 Chapter12_2P.MAP -rw-r--r-- 1 198 Nov 4 1998 Chapter12_2P.TXT -rw-r--r-- 1 125K Jan 26 1999 Chapter13_4P.MAP -rw-r--r-- 1 394 Nov 4 1998 Chapter13_4P.TXT -rw-r--r-- 1 64K Nov 5 1998 Chapter1_3P.MAP -rw-r--r-- 1 168 Nov 4 1998 Chapter1_3P.txt -rw-r--r-- 1 127K Nov 4 1998 Chapter14_4P.MAP -rw-r--r-- 1 21 Nov 4 1998 Chapter14_4P.TXT -rw-r--r-- 1 127K Nov 7 1998 Chapter15_2P.MAP -rw-r--r-- 1 21 Nov 4 1998 Chapter15_2P.TXT -rw-r--r-- 1 75K Nov 4 1998 Chapter2_4P.MAP -rw-r--r-- 1 109 Nov 4 1998 Chapter2_4P.txt -rw-r--r-- 1 83K Nov 5 1998 Chapter3_2P.MAP -rw-r--r-- 1 119 Nov 4 1998 Chapter3_2P.txt -rw-r--r-- 1 92K Jan 26 1999 Chapter4_2P.MAP -rw-r--r-- 1 21 Nov 4 1998 Chapter4_2P.txt -rw-r--r-- 1 35K Nov 4 1998 Chapter5_3P.MAP -rw-r--r-- 1 416 Nov 4 1998 Chapter5_3P.txt -rw-r--r-- 1 48K Nov 5 1998 Chapter6_2P.MAP -rw-r--r-- 1 357 Nov 4 1998 Chapter6_2P.txt -rw-r--r-- 1 56K Nov 7 1998 Chapter7_2P.MAP -rw-r--r-- 1 233 Nov 4 1998 Chapter7_2P.txt -rw-r--r-- 1 59K Nov 4 1998 Chapter8_2P.MAP -rw-r--r-- 1 240 Nov 4 1998 Chapter8_2P.txt -rw-r--r-- 1 91K Nov 4 1998 Chapter9_4P.MAP -rw-r--r-- 1 367 Nov 4 1998 Chapter9_4P.TXT ``` ## Structure The .MAP files are compressed with gzip. All offsets are on the uncompressed size. (Incomplete) parser can be found [here](../../internal/maps/) The start of Chapter01.MAP. It's header then per-map-coordinate data. Must remember that maps have 7 height levels (7), so the volume we're describing is actually WxLxH in all cases ``` 00000000 01 00 00 00 .... # IsCampaignMap 00000004 1A 00 00 00 .... # MinWidth 00000008 14 00 00 00 .... # MinLength 0000000C 68 00 00 00 h... # MaxWidth 00000010 50 00 00 00 P... # MaxLength 00000014 01 00 00 00 .... # Unknown1 00000018 00 00 00 00 .... # Unknown2 0000001C 00 00 00 00 .... # Unknown3 00000020 00 00 00 00 .... # Unknown4 00000024 08 00 57 48 ..WH # Magic value 00000028 4D 41 50 00 MAP. # ... 0000002C 00 00 00 00 .... # Unknown 00000030 91 09 00 00 .... # Unknown - varies between maps of the same size 00000034 6D 61 70 30 map0 # Name of object set to load from `Sets/` 00000038 31 00 00 00 1... # ... 0000003C 00 00 00 00 .... # Unknown from here.... 00000040 00 00 00 00 .... 00000044 00 00 00 00 .... 00000048 00 00 00 00 .... 0000004C 00 00 00 00 .... 00000050 00 00 00 00 .... 00000054 00 00 00 00 .... 00000058 00 00 00 00 .... 0000005C 00 00 00 00 .... 00000060 00 00 00 00 .... 00000064 00 00 00 00 .... 00000068 00 00 00 00 .... 0000006C 00 00 00 00 .... 00000070 00 00 00 00 .... 00000074 00 00 00 00 .... 00000078 00 00 00 00 .... 0000007C 00 00 00 00 .... 00000080 00 00 00 00 .... 00000084 00 00 00 00 .... 00000088 00 00 00 00 .... 0000008C 00 00 00 00 .... 00000090 00 00 00 00 .... 00000094 00 00 00 00 .... 00000098 00 00 00 00 .... 0000009C 00 00 00 00 .... 000000A0 00 00 00 00 .... 000000A4 00 00 00 00 .... 000000A8 00 00 00 00 .... 000000AC 00 00 00 00 .... 000000B0 00 00 00 00 .... 000000B4 38 00 30 00 8.0. 000000B8 00 00 3F 00 ..?. 000000BC 32 00 00 00 2... 000000C0 00 00 00 00 .... 000000C4 00 00 00 00 .... 000000C8 00 00 00 00 .... 000000CC 00 00 00 00 .... 000000D0 00 00 3F 00 ..?. 000000D4 32 00 01 00 2... 000000D8 00 00 0F 00 .... 000000DC 05 00 00 00 .... 000000E0 01 00 00 00 .... 000000E4 A2 FF 80 00 .... 000000E8 17 00 00 00 .... 000000EC FF FF FF FF .... 000000F0 FF FF FF FF .... 000000F4 FF FF FF FF .... 000000F8 FF FF FF FF .... 000000FC FF FF FF FF .... 00000100 FB 00 00 00 .... # It's plausible that per-coord data begins here ``` Comparing some values across all maps bundled with the game: * `Maps/` * `Chapter10.MAP`: IsCampaignMap=1 W=26:104 L=20:80 SetName=map10 * `Chapter12.MAP`: IsCampaignMap=1 W=26:104 L=20:80 SetName=map12 * `Chapter15.MAP`: IsCampaignMap=1 W=0:130 L=0:100 SetName=map15 * `Chapter04.MAP`: IsCampaignMap=1 W=26:104 L=20:80 SetName=map04 * `Chapter08.MAP`: IsCampaignMap=1 W=26:104 L=20:80 SetName=map08 * `Chapter09.MAP`: IsCampaignMap=1 W=13:117 L=10:90 SetName=map09 * `Chapter03.MAP`: IsCampaignMap=1 W=26:104 L=20:80 SetName=map03 * `Chapter11.MAP`: IsCampaignMap=1 W=26:104 L=20:80 SetName=map11 * `Chapter13.MAP`: IsCampaignMap=1 W=26:104 L=20:80 SetName=map13 * `Chapter02.MAP`: IsCampaignMap=1 W=13:117 L=10:90 SetName=map02 * `Chapter06.MAP`: IsCampaignMap=1 W=26:104 L=20:80 SetName=map06 * `Chapter07.MAP`: IsCampaignMap=1 W=26:104 L=20:80 SetName=map07 * `Chapter14.MAP`: IsCampaignMap=1 W=13:117 L=10:90 SetName=map14 * `Chapter01.MAP`: IsCampaignMap=1 W=26:104 L=20:80 SetName=map01 * `Chapter05.MAP`: IsCampaignMap=1 W=26:104 L=20:80 SetName=map05 * `MultiMaps/` * `Chapter13_4P.MAP`: IsCampaignMap=0 W=26:104 L=20:80 SetName=map13 * `Chapter1_3P.MAP`: IsCampaignMap=0 W=26:104 L=20:80 SetName=map01 * `Chapter5_3P.MAP`: IsCampaignMap=0 W=26:104 L=20:80 SetName=map05 * `Chapter8_2P.MAP`: IsCampaignMap=0 W=26:104 L=20:80 SetName=map08 * `Chapter11_4P.MAP`: IsCampaignMap=0 W=26:104 L=20:80 SetName=map11 * `Chapter9_4P.MAP`: IsCampaignMap=0 W=13:117 L=10:90 SetName=map09 * `Chapter12_2P.MAP`: IsCampaignMap=1 W=26:104 L=20:80 SetName=map12 * `Chapter15_2P.MAP`: IsCampaignMap=0 W=0:130 L=0:100 SetName=map15 * `Chapter2_4P.MAP`: IsCampaignMap=0 W=13:117 L=10:90 SetName=map02 * `Chapter4_2P.MAP`: IsCampaignMap=1 W=26:104 L=20:80 SetName=map04 * `Chapter10_2P.MAP`: IsCampaignMap=0 W=26:104 L=20:80 SetName=map10 * `Chapter14_4P.MAP`: IsCampaignMap=0 W=13:117 L=10:90 SetName=map14 * `Chapter3_2P.MAP`: IsCampaignMap=0 W=26:104 L=20:80 SetName=map03 * `Chapter6_2P.MAP`: IsCampaignMap=0 W=26:104 L=20:80 SetName=map06 * `Chapter7_2P.MAP`: IsCampaignMap=0 W=26:104 L=20:80 SetName=map07 ## IsCampaignMap This is just a theory, but every map that forms part of the single-player campaign has a value of 0x01 for 0x0000 - 0x0003. Most of the MP maps and the maps I generate for myself have the value of 0x0 instead. Perhaps `Chapter4_2P.MAP` and `Chapter12_2P.MAP` have it set by accident? Or maybe it's another thing entirely. TODO: try to trace what WH40K.exe does with this flag, if anything. ## Width and Length Maximum allowed size by WH40K_TD.exe is 130x100. Minimum is 20x20. To investigate this, I created maps of different sizes **only**, and observed what differences this made in the putative header: | Offset | 20x20 | 20x21 | 21x20 | 21x21 | 100x100 | 130x100 | | ------ | ----- | ----- | ----- | ----- | ------- | ------- | | 0x04 | 0x37 | 0x37 | 0x36 | 0x36 | 0x0F | 0x00 | | 0x08 | 0x28 | 0x27 | 0x28 | 0x27 | 0x00 | 0x00 | | 0x0C | 0x4B | 0x4B | 0x4B | 0x4B | 0x73 | 0x82 | | 0x10 | 0x3C | 0x3C | 0x3C | 0x3C | 0x64 | 0x64 | WH40K_TD.exe gives us coordinates for each point on each of these maps: | Size | North | East | South | West | 0x04 | 0x08 | 0x0c | 0x10 | | ------- | ----- | ------ | ------ | ----- | ---- | ---- | ---- | ---- | | 20x20 | 55,40 | 74,40 | 74,59 | 55,59 | 55 | 40 | 75 | 60 | | 20x21 | 55,39 | 74,39 | 74,59 | 55,59 | 55 | 39 | 75 | 60 | | 21x20 | 54,40 | 74,40 | 74,59 | 54,59 | 54 | 40 | 75 | 60 | | 100x100 | 15, 0 | 114, 0 | 114,99 | 15,99 | 15 | 0 | 115 | 100 | | 130x100 | 0, 0 | 129, 0 | 129,99 | 0,99 | 0 | 0 | 130 | 100 | So our valid coordinates range in width from `(*0x04, (*0x0c)-1)`, and for length, from `(*0x08, (*0x10)-1)` We're specifying a viewport on a statically-defined area with a width of 130, and a length of 100. The viewport is centered on the middle of that area and anything outside it is clipped away. ## Per-cell data All .MAP files are the same size once uncompressed, regardles of width and length parameters, suggesting that they each have fixed-sized records for every possible cell. Skipping the header we know about, we have data like this: ``` 00000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 000000A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 000000B0 00 00 00 00 1F 00 2C 00 00 00 00 00 00 00 00 00 ......,......... 000000C0 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 ................ 000000D0 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 ................ 000000E0 01 00 FC FF F3 FF 1C 00 10 00 00 00 FF FF FF FF ................ 000000F0 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF ................ 00000100 FB 00 00 00 24 FF FF FF 51 02 00 00 00 06 00 FF ....$...Q....... 00000110 3F 00 00 00 82 01 00 00 00 00 00 FF 00 00 00 00 ?............... 00000120 39 00 00 00 87 01 00 00 00 00 00 FF 00 00 00 00 9............... 00000130 38 00 00 00 89 01 00 00 00 00 00 FF 00 00 00 00 8............... 00000140 38 00 00 00 85 01 00 00 00 00 00 FF 00 00 00 00 8............... 00000150 38 00 00 00 88 01 00 00 00 00 00 FF 00 00 00 00 8............... 00000160 38 00 00 00 89 01 00 00 00 00 00 FF 00 00 00 00 8............... 00000170 38 00 00 00 84 01 00 00 00 00 00 FF 00 00 00 00 8............... 00000180 38 00 00 00 89 01 00 00 00 00 00 FF 00 00 00 00 8............... 00000190 38 00 00 00 85 01 00 00 00 00 00 FF 00 00 00 00 8............... 000001A0 38 00 00 00 8A 01 00 00 00 00 00 FF 00 00 00 00 8............... 000001B0 38 00 00 00 82 01 00 00 00 00 00 FF 00 00 00 00 8............... 000001C0 38 00 00 00 8A 01 00 00 00 00 00 FF 00 00 00 00 8............... 000001D0 38 00 00 00 83 01 00 00 00 00 00 FF 00 00 00 00 8............... 000001E0 38 00 00 00 83 01 00 00 00 00 00 FF 00 00 00 00 8............... 000001F0 38 00 00 00 88 01 00 00 00 00 00 FF 00 00 00 00 8............... ``` It would be very neat if the per-cell data started at 0x100 and took 16 bytes per coordinate, but to get objects in my map to line up properly with cells in WH40K_TD.exe, I've had to start parsing these rows at `0x0110` instead. Still tentative! Total number of possible coordinates is 100x130x7 = 91,000 = 1,456,000 bytes. Uncompressed file size is always 1,458,915 bytes, leaving 2,915 bytes over for header and any trailing data. How are the rows structured? `[z][y][x]`? Looking at the data around 0x163890, we see: ``` 00163800 38 00 00 00 00 01 00 00 00 00 00 FF 00 00 00 00 8............... 00163810 38 00 00 00 00 01 00 00 00 00 00 FF 00 00 00 00 8............... 00163820 38 00 00 00 00 01 00 00 00 00 00 FF 00 00 00 00 8............... 00163830 38 00 00 00 00 01 00 00 00 00 00 FF 00 00 00 00 8............... 00163840 38 00 00 00 00 01 00 00 00 00 00 FF 00 00 00 00 8............... 00163850 38 00 00 00 00 01 00 00 00 00 00 FF 00 00 00 00 8............... 00163860 38 00 00 00 00 01 00 00 00 00 00 FF 00 00 00 00 8............... 00163870 38 00 00 00 00 01 00 00 00 00 00 FF 00 00 00 00 8............... 00163880 38 00 00 00 00 01 00 00 00 00 00 FF 00 00 00 00 8............... 00163890 38 00 00 68 00 00 00 50 00 00 00 1A 00 00 00 14 8..h...P........ 001638A0 00 00 00 3A 00 00 00 00 38 25 00 04 00 00 00 00 ...:....8%...... 001638B0 00 00 00 1A 00 00 00 00 00 00 00 00 00 00 00 00 ................ 001638C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 001638D0 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00 ................ 001638E0 00 00 00 32 00 00 00 00 00 00 00 00 00 00 00 00 ...2............ 001638F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00163900 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00163910 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00163920 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00163930 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00163940 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00163950 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00163960 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00163970 00 00 00 00 00 00 00 00 00 00 00 00 00 17 4D 61 ..............Ma 00163980 6E 69 61 00 00 00 00 00 00 00 00 00 00 00 00 00 nia............. ``` So this looks like a good working theory. Wherever they start, these rows must refer to the object sets somehow. We also need to work out how they're arranged. Disassembly of WH40K_TD.EXE suggests a file `Engine::CellInfo` at 0x4226F0. It dumps the following information: * Mission exit: true / false * Smoke: true / false * Sprite: true / false * Vehicle: true / false * Net start: true / false * Spell: true / false * Trigger: true / false * Breadcrumb: true / false * Door: true / false * Lock: true / false * Reactor: true / false * Objective: true / false * Canister number %d * Cell visible %d * Tile visible %d * N0 - N7: true / false (seems to be a bitfield) * Object 0 SURFACE: true / false * Object area %d * Curr sprite %d * Object 1 LEFT: true / false * Object area %d * Curr sprite %d * Object 2 RIGHT: true / false * Object area %d * Curr sprite %d * Object 3 CENTRE: true / false * Object area %d * Curr sprite %d Presumably at least some of these attributes come from the records identified above. Will need to experiment to establish correlations. Items of interest: * A cell can have 4 objects in it - a surface, a left, a right, and a centre. * Presumably a surface is what we show on the ground, and it will be set for "ground level" data. Press "U" to get to the screen! ![Debug screen](img/cell-information-debug.png) I've added a `view-minimap` command to explore the data graphically. Each of the 16 bytes in a cell row must have a function; comparing a known map to how it looks in WH40K_TD.exe can help me to unravel that function. Here's an (out-of-date) side-by-side comparison of Chapter3.MAP, investigating CellIndex=3 and Z index = 0. ![Map comparison](img/chapter_01_cell_index_3.png) Incrementing the Z index shows the building and terrain progressively changing as expected, so this is a good hint that the data is actually arranged in `[z][y][x][16]` blocks and that I either have the right offset, or I'm out by a well-aligned amount. Investigation has so far suggested the following: * `Cell[0]` seems related to doors and canisters. Observed: * Nothing special: 0x38 * ???: 0x39 * Imperial crate: 0x28 * Door: 0xB8 * `Cell[1]` seems related to special placeables (but not triggers). Bitfield. Observed: * 0x01: Reactor * 0x20: Door or door lock? * 0x40: Animated object * `Cell[2]` hasn't been seen with a value > 0 yet * `Cell[3]` Object 0 (Surface) Area (Sets/*.set lookup) * `Cell[4]` Object 0 (Surface) Sprite + active flag * Bottom bits encode the sprite (frame number in the .obj file) * 0x80 is set too. A flag? * `Cell[5]` Object 1 (Left) Area (Sets/*.set lookup) * `Cell[6]` Object 1 (Surface) Sprite + active flag * `Cell[7]` Object 2 (Right) Area (Sets/*.set lookup) * `Cell[6]` Object 2 (Right) Sprite + active flag * `Cell[9]` Object 3 (Center) Area (Sets/*.set lookup) * `Cell[10]` Object 3 (Center) Sprite + active flag * `Cell[11]` all 255? Vehicle? * `Cell[12]` all 0? * `Cell[13]` all 0? * `Cell[14]` all 0? * `Cell[15]` shows squad positions, MP start positions, etc, as 0x04. Bitfield? Mapping the altar in Chapter01 to the map01 set suggests it's a palette entry lookup, 0-indexed. `U` debug in WH40K_TD.exe says the cell's `Object 3-Center` has `Area 67` and `Sprite 1`. In `Sets/map01.set`, entry 67, zero-indexed and ignoring non-palette data, is an altar. The data of that cell is: ``` maps.Cell{ 0x18, 0x0, 0x0, 0x2, 0x99, 0x1, 0x0, 0x0, 0x0, 0x43, 0x81, 0xff, 0x0, 0x0, 0x0, 0x0 } ``` So `CellIdx == 9` points to the center object's Area, looked up in the set file! ![Pinning down cell index 9](img/chapter01_cell_index_9.png) It seems the area numbers are absolute indexes into the set, rather than having a new set of indices for each type. We have to remove 0x80 from the sprite byte to get a valid reference. This seems to act as an "active" flag - without it, Chapter01.MAP has a "ghost" of the template close to the 0,0 boundary. Theory: the devs originally started at 0,0 but then they decided to center smaller maps rather than being there, so the layout got moved! With this information, we can render a given Z index for a map quite easily, using the new `view-map` binary. It draws the four objects for every cell, and gives results like this: ![Rendering all map objects](img/chapter01_rendered_2018-09-08.png) ## Trailer Assuming the theory above is correct, we have trailer data starting at `0x163890`. Browsing through the `Chapter01.MAP` data, I see: * Names: * `0x163970`: Mania * Dagon * Nihasa * Samnu * Gigamen * Valefor * Apollyon * Vassago * Diabolus * Asmodee * Gamigin * Beherit * Marbas * Lucifer * Ahpuch * Gorgon * Loki * Lucifer * Mammon * Baphomet * Magot * Belial * Mamon * ... So we must have enemy characters placed on the map, although I don't remember Chapter01 having *this many*... At 0017C9DF we see the start of scenario text: ``` 0017C9A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0017C9B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0017C9C0 00 00 00 02 00 00 00 00 00 00 00 32 00 00 00 01 ...........2.... 0017C9D0 03 FF FF 00 00 00 00 00 00 00 00 00 00 00 00 59 ...............Y 0017C9E0 6F 75 20 68 61 76 65 20 73 70 6F 74 74 65 64 20 ou have spotted 0017C9F0 61 6D 6D 75 6E 69 74 69 6F 6E 20 63 72 61 74 65 ammunition crate 0017CA00 73 20 77 69 74 68 20 49 6D 70 65 72 69 61 6C 20 s with Imperial 0017CA10 6D 61 72 6B 69 6E 67 73 2E 00 00 00 00 00 00 00 markings........ 0017CA20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ ``` Several records like this. Around 001841A0: mission objectives! ``` 00184190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 001841A0 00 00 00 00 00 00 4D 69 73 73 69 6F 6E 20 4F 62 ......Mission Ob 001841B0 6A 65 63 74 69 76 65 3A 0A 0A 45 6E 74 65 72 20 jective:..Enter 001841C0 74 68 65 20 74 65 6D 70 6C 65 20 6F 66 20 74 68 the temple of th 001841D0 65 20 43 68 61 6F 73 20 43 75 6C 74 69 73 74 20 e Chaos Cultist 001841E0 61 6E 64 20 72 65 74 72 69 65 76 65 20 74 68 65 and retrieve the 001841F0 20 72 65 6C 69 63 2E 20 0A 0A 52 65 63 6F 6D 6D relic. ..Recomm 00184200 65 6E 64 65 64 20 45 71 75 69 70 6D 65 6E 74 3A ended Equipment: 00184210 20 0A 0A 4B 72 61 6B 20 47 72 65 6E 61 64 65 73 ..Krak Grenades 00184220 20 0A 0A 41 6E 74 69 2D 50 6C 61 6E 74 20 47 72 ..Anti-Plant Gr 00184230 65 6E 61 64 65 73 0A 00 00 00 00 00 00 00 00 00 enades.......... 00184240 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ ``` Relative offsets from the start of the trailer, we have: | Offset | Text | | -------- | ---- | | `0xEE` | Mania | | `0x78A` | Dagon | | `0xE26` | Nihasa | | `0x14C2` | Samnu | | `0x1b5e` | Bael | | `0x2896` | Gigamen | | `0x2f32` | Valefor | | `0x35ce` | Baalberith | | `0x3c6a` | Fenriz | | `0x4306` | #Character | | `0x49a2` | Apollyon | So there are 1692 bytes between each name (the names probably don't come at the start of each block, but it's still a useful stride). Presumably `#Character` is a space for one of the player characters, while the others specify an NPC placed on the map. There's 56 of these records between the first and last name we see - `Ahpuch`. Then there are a number of other strings that seem related to triggers / events, including lots that say `NO FILE`. The first two are 96 bytes apart; from then on they seem to be placed variably apart from each other; I've seen 96, 256, and 352 byte offsets. At 0x20916 the mission objective is readable. At 0x2092a the mission description is readable. Generating another map with just 5 characters on it, things look different: * Trailer size is 13543 bytes * There are only 5 names * There are none of the trigger/event strings * Mission title is found at 0x2b93 * Mission briefing is found at 0x2c92 Since the trailer is a variable size, there must be a header that tells us how many of each type of record to read. Peeking at the differences in `vbindiff`: ``` Chapter01.MAP.Trailer 0000 0000: 38 00 00 68 00 00 00 50 00 00 00 1A 00 00 00 14 8..h...P ........ 0000 0010: 00 00 00 3A 00 00 00 00 38 25 00 04 00 00 00 00 ...:.... 8%...... 0000 0020: 00 00 00 1A 00 00 00 00 00 00 00 00 00 00 00 00 ........ ........ 0000 0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ........ ........ 0000 0040: 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00 ........ ........ 0000 0050: 00 00 00 32 00 00 00 00 00 00 00 00 00 00 00 00 ...2.... ........ TINYSQUAD.MAP.Trailer 0000 0000: 38 00 00 4B 00 00 00 3C 00 00 00 37 00 00 00 28 8..K...< ...7...( 0000 0010: 00 00 00 05 00 00 00 00 2B 3A 00 04 00 00 00 05 ........ +:...... 0000 0020: 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 ........ ........ 0000 0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ........ ........ 0000 0040: 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00 ........ ........ 0000 0050: 00 00 00 32 00 00 00 00 00 00 00 00 00 00 00 00 ...2.... ........ ``` The size of the trailer for Chapter01 is 139,483 bytes, assuming it starts at `0x163890`. However, things may be a lot more sensible if we drop 3 bytes off the start of that to get the fields into little-endian alignment. Have I made a maths error above somewhere? Is it some sort of alignment thing? Do those 3 bytes actually have meaning? Ignoring them for now, here's a first guess at a header: | Offset | Size | Meaning | | ------ | ---- | ------- | | 0 | 4 | Map maximum X + 1 | | 4 | 4 | Map maximum Y + 1 | | 8 | 4 | Map minimum X | | 12 | 4 | Map minimum Y | | 16 | 4 | Number of character records | | 20 | 4 | Padding? - invariant `00 00 00 00` | | 24 | 2 | ??? - varies. Seems related to character/squad position? | | 26 | 2 | ??? - invariant `00 04` | | 28 | 4 | ??? - varies (0 vs 5) | | 32 | 1 | Number of thingies | | 33 | 3 | ???. With a Lord of Change on the map, only one byte works as thingies count | | 36 | 20 | Padding? | 56 bytes of data is interesting because the value of that first, ignored byte is 0x38 - perhaps it's a skip value + 2 bytes of padding? It's just weird. Keep ignoring it for now. 0x4b contains the next non-null byte; is the gap between the the number of thingies, and it, padding? Minus a bit? 0x50 is another non-null byte. Then it's all zeroes until one byte before the first name at 0xee. It's hard to say where the alignment should be at this point. We need to compare more characters with each other. Other notes... Characters are organised into Squads somehow. Individual cells seem to have a flag to say "We have a character in us", but not the number for the character themselves, so the coordinates must be in the per-character records also. There are several candidates for this. Placing a single character at (64,49) causes those bytes to show up at four offsets - 0x18 (!), 0x1F4, 0x1F8, and 0x6C8. Generating a map with no characters at all, the trailer is 2,447 bytes, and the mission title starts at 0x3B (59). So we can say we have 20 bytes of padding as a first approximation? Here's where we're at with the per-character data, going from the padding values suggested above: | Offset | Size | Meaning | | ------ | ---- | ------- | | 0 | 179 | ??? | | 179 | 80 | Character name | | 259 | 10 | Character attributes | | 269 | 495 | ??? | | 764 | 1(?) | Squad number? | | 765 | 927 | ??? | There's still a lot of bytes to dig through, but this allows me to load the character names from Chapter01 correctly, with the exception of record 57 which just contains `\x02` and is then null-terminated all the way through. Looking at the bytes for one character record, I can easily correlate certain bytes to various attributes; that's just done in code for the moment. Given two characters of the same time, just in different locations, differing values are seen at: * `0x103 - 0x10c` (hodgepodge) * `0x178 - 0x1be` (hodgepodge) * `0x2fc` (0, 1) - squad number? In Chapter01, picking a random character (Gorgon) and looking at his squadmates, they are all in the same squad, and no other characters are in that squad, so it looks pretty diagnostic to me. There's nothing in the UI to indicate the squad, though. Now let's look for position. In my 2-character map, they're at 65,50 and 70,55. Within a character, I see those numbers repeated twice - around `0x1b{9,a}` and `0x1b{d,e}`. Characters don't seem to take up multiple x,y squares, but they *can* take up multiple Z levels. So maybe this is a bounding box of some kind? Adding a (tall) Lord of Change to the map gave me `02 36 45 00 02 37 45`, which doesn't quite match what my eyes are telling me for Z,Y,X. In addition, the data immediately after this offset changed into a large number of coordinate-like sets of values - far too many for it to actually be a bounding box. However, the first one remains good as a position specifier. Down in `0x679` (Chaos Sorcerer) or `0x68D` (Lord of Change), the map coords for the *other* character appears, which is downright odd. For now, just use the first-indexed value. How about their types? We need to be able to work out which animations to show, so they must be uniquely identified somehow. Ultramarine captain is animation group 12, while chaos lord is group... 14? Not certain. I don't see either of those numbers in a promising spot, anyway. I do see differences at around `0x170`: ``` Ultramarine Captain: 00 00 00 00 00 00 00 00 50 03 00 00 00 00 00 00 Chaos Lord: 00 00 00 00 11 01 00 00 47 03 00 04 00 00 02 00 ``` Maybe they're somewhat relevant, it's hard to say. Thingies next: these aren't decoded at all yet, and the sizes seem to be variable. | Offset | Size | Meaning | | ------ | ---- | ------- | | | | | Finally, the "trailer trailer", for want of a better term, seems to be organised as: | Offset | Size | Meaning | | ----- | ---- | ------- | | 0 | 255 | Title | | 255 | 2048 | Briefing | | 2304 | 85 | ??? - each byte is 1 or 0. Spaced so it may be partly uint32 | This duplicates the information found in the `.TXT` files. No idea what the end data is yet. ## Soldiers At War All the above applies to Chaos Gate maps. Maps for Soldiers At War seem to have a lot of similarities, but also some differences. For a start, the maps are a variable size! Starting with the header, given a tiny 26x20 generated map, the first 256 bytes look like this: ``` 00000000: 1500414d 425f4d41 50005041 52495300 ..AMB_MAP.PARIS. 00000010: 00000000 00000000 00000000 00000000 ................ 00000020: 00000000 00000000 00000000 00000000 ................ 00000030: 00000000 00000000 00000000 00000000 ................ 00000040: 00000000 00000000 00000000 00000000 ................ 00000050: 00000000 00000000 00000000 00000000 ................ 00000060: 00000000 00000000 00000000 00000000 ................ 00000070: 00000000 00000000 00000000 00000000 ................ 00000080: 00000000 00000000 00001e00 45000100 ............E... 00000090: 1f004600 10010000 52000000 00001b00 ..F.....R....... 000000a0: 38000100 00000500 0a000001 00f0f9ff 8............... 000000b0: ffb60500 00000100 ff370a00 64006400 .........7..d.d. 000000c0: 08008501 00000000 00ff0000 1f008082 ................ 000000d0: 01000000 0000ff00 001f0080 84010000 ................ 000000e0: 000000ff 00001f00 00810100 00000000 ................ 000000f0: ff00001f 00808301 00000000 00ff0000 ................ ``` Almost everything we knew is out of the window, but a few things look familiar. First, the header seems simplified down to just two recognisable-at-first-glance fields: Magic bytes (now `\x15\x00AMV_MAP\x00`) and the set name, coming immediately after. Like Chaos Gate, all map files are the same size once uncompressed, but they are smaller - at 1,214,559 bytes, they are 76% the size. This is quite significant. We now have 13.3 bytes per voxel, rather than the 17.5 bytes per voxel that was available to Chaos Gate. This means that the number of bytes *per cell* must be reduced, in addition to the header (and trailer?) values. Looking at data from 0x110, it seems to group naturally into 13-byte records: ``` $ xxd -s 0x110 -c 13 -l 65 -g 1 TINYMAP.MAP 00000110: 80 01 00 00 00 00 00 ff 00 00 1f 00 00 ............. 0000011d: 85 01 00 00 00 00 00 ff 00 00 1f 00 00 ............. 0000012a: 82 01 00 00 00 00 00 ff 00 00 1f 00 80 ............. 00000137: 82 01 00 00 00 00 00 ff 00 00 1f 00 00 ............. 00000144: 82 01 00 00 00 00 00 ff 00 00 1f 00 80 ............. ``` It's a strange number. Chaos Gate cells group nicely on 16 bytes: ``` $ xxd -s 0x110 -c 16 -l 64 -g 1 Chapter01.MAP 00000110: 3f 00 00 00 83 01 00 00 00 00 00 ff 00 00 00 00 ?............... 00000120: 38 00 00 00 85 01 00 00 00 00 00 ff 00 00 00 00 8............... 00000130: 38 00 00 00 84 01 00 00 00 00 00 ff 00 00 00 00 8............... 00000140: 38 00 00 00 8a 01 00 00 00 00 00 ff 00 00 00 00 8............... 00000150: 38 00 00 00 83 01 00 00 00 00 00 ff 00 00 00 00 8............... ``` That grouping is very enticing, though. I feel strongly that it's the right number. Now we need to ask about start offset. Where is byte 0 of the per-cell data, and do the 13 bytes it has line up neatly to the functions of some of the 16 bytes seen in Chaos Gate? I generated a `BIGGESTMAP` (130x100) to investigate. It's just grass, nothing but grass, and 0xC0 is the first offset where it starts to look nicely grouped: ``` xxd -s 0xc0 -c 13 -l 260 -g 13 BIGGESTMAP.MAP 000000c0: 08 80 81 01 00 00 00 00 00 ff 00 00 1f ............. 000000cd: 00 80 81 01 00 00 00 00 00 ff 00 00 1f ............. 000000da: 00 00 81 01 00 00 00 00 00 ff 00 00 1f ............. 000000e7: 00 00 85 01 00 00 00 00 00 ff 00 00 1f ............. # ... ``` This can be interpreted more or less the same way as the Chaos Gate maps now, and the `soldiers-at-war` branch contains a hacked-up implementation that kind of works \o/. Does the same trailer apply? Seemingly not. Looking at `PARIS.MAP`, there's no similarity at first glance. However, I did manage to track down 4 32-bit ints inside the trailer, starting at `0x121ad1`, which specify dimensions of the map, at least. Perhaps the position has moved, but some of the data is the same? It's 3320 bytes into the trailer.