Trim map based on trailer data

doc/formats/maps.md

@@ -392,8 +392,11 @@ well-aligned amount.
 Investigation has so far suggested the following:
 
 * `Cell[0]` seems related to doors and canisters. Observed:
+    * Nothing special: 0x38
+    * ???: 0x39
     * Imperial crate: 0x28
     * Door: 0xB8
+
 * `Cell[1]` seems related to special placeables (but not triggers). Bitfield. Observed:
     *  0x01: Reactor
     *  0x20: Door or door lock?
@@ -408,12 +411,12 @@ Investigation has so far suggested the following:
 * `Cell[7]` Object 2 (Right) Area (Sets/*.set lookup)
 * `Cell[6]` Object 2 (Right) Sprite + active flag
 * `Cell[9]` Object 3 (Center) Area (Sets/*.set lookup)
-* `Cell[10]` Object 3 (Right) Sprite + active flag
-* `Cell[11]` all 255?
+* `Cell[10]` Object 3 (Center) Sprite + active flag
+* `Cell[11]` all 255? Vehicle?
 * `Cell[12]` all 0?
 * `Cell[13]` all 0?
 * `Cell[14]` all 0?
-* `Cell[15]` shows squad positions, MP start positions, etc, as 0x04
+* `Cell[15]` shows squad positions, MP start positions, etc, as 0x04. Bitfield?
 
 Mapping the altar in Chapter01 to the map01 set suggests it's a palette entry
 lookup, 0-indexed. `U` debug in WH40K_TD.exe says the cell's `Object 3-Center`
@@ -515,10 +518,120 @@ Around 001841A0: mission objectives!
 00184240   00 00 00 00  00 00 00 00  00 00 00 00  00 00 00 00  ................
 ```
 
-Since all the files are exactly the same length uncompressed, I'm going to
-assume these are all a fixed number of fixed-size records when looking into it.
+Relative offsets from the start of the trailer, we have:
 
+| Offset   | Text |
+| -------- | ---- |
+| `0xEE`   | Mania |
+| `0x78A`  | Dagon |
+| `0xE26`  | Nihasa |
+| `0x14C2` | Samnu |
+| `0x1b5e` | Bael |
+| `0x2896` | Gigamen |
+| `0x2f32` | Valefor |
+| `0x35ce` | Baalberith |
+| `0x3c6a` | Fenriz |
+| `0x4306` | #Character |
+| `0x49a2` | Apollyon |
 
+So there are 1692 bytes between each name (the names probably don't come at the
+start of each block, but it's still a useful stride). Presumably `#Character` is
+a space for one of the player characters, while the others specify an NPC placed
+on the map.
+
+There's 56 of these records between the first and last name we see - `Ahpuch`.
+
+Then there are a number of other strings that seem related to triggers / events,
+including lots that say `NO FILE`. The first two are 96 bytes apart; from then
+on they seem to be placed variably apart from each other; I've seen 96, 256, and
+352 byte offsets.
+
+At 0x20916 the mission objective is readable.
+
+At 0x2092a the mission description is readable.
+
+Generating another map with just 5 characters on it, things look different:
+
+* Trailer size is 13543 bytes
+* There are only 5 names
+* There are none of the trigger/event strings
+* Mission title is found at 0x2b93
+* Mission briefing is found at 0x2c92
+
+Since the trailer is a variable size, there must be a header that tells us how
+many of each type of record to read. Peeking at the differences in `vbindiff`:
+
+```
+Chapter01.MAP.Trailer                                                           
+0000 0000: 38 00 00 68 00 00 00 50  00 00 00 1A 00 00 00 14  8..h...P ........  
+0000 0010: 00 00 00 3A 00 00 00 00  38 25 00 04 00 00 00 00  ...:.... 8%......  
+0000 0020: 00 00 00 1A 00 00 00 00  00 00 00 00 00 00 00 00  ........ ........  
+0000 0030: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ........ ........  
+0000 0040: 00 00 00 00 00 00 00 00  00 00 00 02 00 00 00 00  ........ ........  
+0000 0050: 00 00 00 32 00 00 00 00  00 00 00 00 00 00 00 00  ...2.... ........  
+
+TINYSQUAD.MAP.Trailer                                                           
+0000 0000: 38 00 00 4B 00 00 00 3C  00 00 00 37 00 00 00 28  8..K...< ...7...(  
+0000 0010: 00 00 00 05 00 00 00 00  2B 3A 00 04 00 00 00 05  ........ +:......  
+0000 0020: 00 00 00 01 00 00 00 00  00 00 00 00 00 00 00 00  ........ ........  
+0000 0030: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ........ ........  
+0000 0040: 00 00 00 00 00 00 00 00  00 00 00 02 00 00 00 00  ........ ........  
+0000 0050: 00 00 00 32 00 00 00 00  00 00 00 00 00 00 00 00  ...2.... ........  
+```
+
+The size of the trailer for Chapter01 is 139,483 bytes, assuming it starts at
+`0x163890`. However, things may be a lot more sensible if we drop 3 bytes off
+the start of that to get the fields into little-endian alignment. Have I made a
+maths error above somewhere? Is it some sort of alignment thing? Do those 3
+bytes actually have meaning?
+
+Ignoring them for now, here's a first guess at a header:
+
+| Offset | Size | Meaning |
+| ------ | ---- | ------- |
+| 0      | 4    | Map maximum X + 1 |
+| 4      | 4    | Map maximum Y + 1 |
+| 8      | 4    | Map minimum X |
+| 12     | 4    | Map minimum Y |
+| 16     | 4    | Number of character records |
+| 20     | 4    | Padding? - invariant `00 00 00 00` |
+| 24     | 2    | ??? - varies. Seems related to character/squad position? |
+| 26     | 2    | ??? - invariant `00 04` |
+| 28     | 4    | ??? - varies (0 vs 5) |
+| 32     | 4    | Number of thingies (26 vs 1) |
+| 36     | 20   | Padding? |
+
+56 bytes of data is interesting because the value of that first, ignored byte is
+0x38 - perhaps it's a skip value + 2 bytes of padding? It's just weird. Keep
+ignoring it for now.
+
+0x4b contains the next non-null byte; is the gap between the the number of
+thingies, and it, padding? Minus a bit? 0x50 is another non-null byte. Then
+it's all zeroes until one byte before the first name at 0xee.
+
+It's hard to say where the alignment should be at this point. We need to compare
+more characters with each other. Other notes...
+
+Characters are organised into Squads somehow.
+
+Individual cells seem to have a flag to say "We have a character in us", but not
+the number for the character themselves, so the coordinates must be in the
+per-character records also. There are several candidates for this.
+
+Placing a single character at (64,49) causes those bytes to show up at four
+offsets - 0x18 (!), 0x1F4, 0x1F8, and 0x6C8.
+
+Generating a map with no characters at all, the trailer is 2,447 bytes, and the
+mission title starts at 0x3B (59). So we can say we have 20 bytes of padding as
+a first approximation?
+
+The "trailer trailer", for want of a better term, seems to be organised as:
+
+| Offset | Size | Meaning |
+| -----  | ---- | ------- |
+| 0      | 255  | Title   |
+| 255    | 2048 | Briefing |
+| 2304   | 85   | ??? - each byte is 1 or 0. Spaced so it may be partly uint32 |
 
 
 ## Soldiers At War
@@ -600,3 +713,15 @@ xxd -s 0xc0 -c 13 -l 260 -g 13 BIGGESTMAP.MAP
 000000e7: 00 00 85 01 00 00 00 00 00 ff 00 00 1f  .............
 # ...
 ```
+
+This can be interpreted more or less the same way as the Chaos Gate maps now,
+and the `soldiers-at-war` branch contains a hacked-up implementation that kind
+of works \o/.
+
+Does the same trailer apply? Seemingly not. Looking at `PARIS.MAP`, there's no
+similarity at first glance.
+
+However, I did manage to track down 4 32-bit ints inside the trailer, starting
+at `0x121ad1`, which specify dimensions of the map, at least. Perhaps the
+position has moved, but some of the data is the same? It's 3320 bytes into the
+trailer.

internal/maps/maps.go

@@ -38,18 +38,40 @@ type Header struct {
 	IsCampaignMap byte
 }
 
-func (h Header) Width() int {
-	return MaxWidth
+type TrailerHeader struct {
+	Discard1 [3]byte // No idea what this lot is
+	MaxWidth  uint32
+	MaxLength uint32
+	MinWidth  uint32
+	MinLength uint32
+
+	NumCharacters uint32
+
+	Unknown1 uint32
+	Unknown2 uint16
+	Unknown3 uint16
+	Unknown4 uint32
+	NumThingies uint32
+	Padding1 [20]byte
 }
 
-func (h Header) Length() int {
-	return MaxLength
+type TrailerTrailer struct {
+	Title    [255]byte
+	Briefing [2048]byte
+	Unknown1 [85]uint8 // Maybe? each contains either 0 or 1? Hard to say
 }
 
-func (h Header) Height() int {
-	return MaxHeight
+type Character struct {
+	Unknown1 uint32
 }
 
+type Characters []Character
+
+// TODO. These are triggers/reactors/etc.
+type Thingy struct {}
+
+type Thingies []Thingy
+
 func (h Header) MapSetName() string {
 	idx := bytes.IndexByte(h.SetName[:], 0)
 	if idx < 0 {
@@ -186,16 +208,23 @@ func (h Header) Check() []error {
 type GameMap struct {
 	Header
 	Cells
-	// TODO: parse this into sections
+	TrailerHeader
+	Characters
+	Thingies
+	TrailerTrailer
+
+	// TODO: parse this into sections. This is the text that comes from the
+	// .TXT file. Maybe we don't need it at all since it should be duplicated in
+	// the trailer.
 	Text string
 }
 
 func (m *GameMap) Rect() image.Rectangle {
 	return image.Rect(
-		int(0),
-		int(0),
-		int(m.Width()-1),
-		int(m.Length()-1),
+		int(m.TrailerHeader.MinWidth),
+		int(m.TrailerHeader.MinLength),
+		int(m.TrailerHeader.MaxWidth-1),
+		int(m.TrailerHeader.MaxLength-1),
 	)
 }
 
@@ -302,5 +331,42 @@ func loadMapFile(filename string) (*GameMap, error) {
 		return nil, fmt.Errorf("Error parsing cells for %s: %v", filename, err)
 	}
 
+	// no gzip.SeekReader, so discard unread trailer bytes for now
+	if _, err := io.CopyN(ioutil.Discard, zr, int64(3320-2)); err != nil { // observed
+		return nil, err
+	}
+
+	if err := binary.Read(zr, binary.LittleEndian, &out.TrailerHeader); err != nil {
+		return nil, fmt.Errorf("Error parsing trailer header for %s: %v", filename, err)
+	}
+
+	log.Printf("Trailer Header: %#+v", out.TrailerHeader)
+	/*
+	// TODO: until we know how large each character record should be, we can't read this lot
+
+	out.Characters = make(Characters, int(out.TrailerHeader.NumCharacters))
+	if err := binary.Read(zr, binary.LittleEndian, &out.Characters); err != nil {
+		return nil, fmt.Errorf("Error parsing characters for %s: %v", filename, err)
+	}
+
+	out.Thingies = make(Thingies, int(out.TrailerHeader.NumThingies))
+	if err := binary.Read(zr, binary.LittleEndian, &out.Thingies); err != nil {
+		return nil, fmt.Errorf("Error parsing thingies for %s: %v", filename, err)
+	}
+
+	if err := binary.Read(zr, binary.LittleEndian, &out.TrailerTrailer); err != nil {
+		return nil, fmt.Errorf("Error parsing trailer trailer for %s: %v", filename, err)
+	}
+	log.Printf("Trailer Trailer: %s", out.TrailerTrailer.String())
+*/
 	return &out, nil
 }
+
+func (t *TrailerTrailer) String() string {
+	return fmt.Sprintf(
+		"title=%q briefing=%q rest=%#+v",
+		strings.TrimRight(string(t.Title[:]), "\x00"),
+		strings.TrimRight(string(t.Briefing[:]), "\x00"),
+		t.Unknown1,
+	)
+}