Nick Thomas
7a8e9dbd97
This makes menus display more correctly, and also fixes trees and other objects on the main map, although it messes up bounds clipping (sigh).
19 KiB
Obj format information
The names of Obj files are highly suggestive of them being objects that can be
placed on maps. Each map cell seems to have space for four objects to
be placed on it - a SURFACE, LEFT, RIGHT, and CENTER. Maps reference
objects in a space-efficient way via sets, which seem to be a kind of object
palette.
Assign
The Assign/ directory contains a matching .asn file for each Obj/*.obj.
It's a plain-text format which seems to assign properties to frames, and has
references to a <name>.flc file which does not exist in the tree.
Theory: .obj files were originally generated from .flc files. This is an
AutoDesk format for visual data, so this suggests the .obj files contain pixels
\o/
blank.asn references 6 frames (0-5):
# single pixel tile
# transpix.obj/.asn
#/--> transpix.flc
#
0-5:DEF 1;
0-5:TYPE 13;
END OF FILE
jungtil.asn references 18 frames (0-17):
# jungle floor
# jungtil.obj/.asn
# /--> d:\warflics\missions\jungtil.flc
#
0:DEF 2;
1-11:DEF 454;
#damaged frames!!!!
12:DEF 2;
13-16:DEF 454;
17:DEF 454;
0:TYPE 2;
1-11:TYPE 0;
12:TYPE 2;
13-16:TYPE 0;
17:TYPE 0;
0:DESTROY 12;
1-3:DESTROY 13;
4-6:DESTROY 14;
7-9:DESTROY 15;
10-11:DESTROY 16;
17:DESTROY 15;
1-11:Dmg1Lnk 17;
END OF FILE
So it seems this visual data can have quite complicated attributes. At a minimum we see:
TYPEDEFDESTROYDmg1Lnk
The type field may tell us what format each sprite is in.
OBJ container structure
.obj files represent visual data. They contain a number of sprites, which are
assigned attributes in .asn files and referenced from .map files.
The container format is worked out, but the per-sprite data is still WIP, so I'm documenting the former here while still investigating the latter.
The file begins with a header, with all values 32-bit little-endians:
| Offset | Meaning |
|---|---|
| 0x0000 | Number of sprites |
| 0x0004 | Offset of sprite directory |
| 0x0008 | Size of sprite directory |
| 0x000c | Offset of the sprite data |
| 0x0010 | Size of the sprite data |
The sprite directory contains an 8-byte record per sprite. That record contains two 32-bit little-endians:
| Offset | Meaning |
|---|---|
| 0x0000 | Relative offset of sprite in data block |
| 0x0004 | Size of sprite in data block |
For sanity checks, we can ensure that:
- The header, sprite directory and data blocks do not overlap or extend past the end of the file
- None of the individual sprites overlap each other or extend past the end of the data block
Sprite structure
First, the blank.obj file. blank.asn helpfully tells us that it's a single-
pixel tile and assigns it a type of 13. There are six sprites, all of which are
identical. Data dump:
-------------------------------------------------------------------------------
0 1 2 3 4 5 6 7 01234567 0 1 2 3 4 5 6 7
-------------------------------------------------------------------------------
0x0000 10 01 61 01 01 00 01 00 | a | 16 1 97 1 1 0 1 0
0x0008 00 00 00 00 03 00 00 00 | | 0 0 0 0 3 0 0 0
0x0010 00 00 00 00 00 00 00 00 | | 0 0 0 0 0 0 0 0
0x0018 01 fd 00 | | 1 253 0
Total size is 27 bytes, so if there is only one pixel, it's tempting to suggest that the first 24 bytes are header, and the final 3 bytes represent that one pixel.
I think this is rendered as a 1px dot with the colour #ff00ff in WH40K_TD.exe:
In the tile, the dot is in the very centre.
The colour itself doesn't show up in the data directly, so it's not a simple RGB array of pixels. WH40K_TD.exe is a 256-colour application, so the pixel data may account for 1 of the 3 bytes.
----------------------------------------------
0 1 2 3 4 5 6 7 01234567
----------------------------------------------
0x0000 fa 00 25 01 2e 00 48 00 | H | 250 0 37 1 46 0 72 0
0x0008 00 00 00 00 f4 0c 00 00 | | 0 0 0 0 244 12 0 0
0x0010 00 00 00 00 00 00 00 00 | | 0 0 0 0 0 0 0 0
0x0018 80 0f 90 1a 4b 1a 1b 4b | K K | 128 15 144 26 75 26 27 75
0x0020 1a 4c 1a 4b 19 19 49 19 | L K I | 26 76 26 75 25 25 73 25
0x0028 4a 18 4a 80 0f 00 80 0b | J J | 74 24 74 128 15 0 128 11
0x0030 98 1a 4c 1a 4c 1a 9e 1a | L L | 152 26 76 26 76 26 158 26
0x0038 4c 1b 4b 1a 1a 49 19 48 | L K I H | 76 27 75 26 26 73 25 72
0x0040 18 18 48 19 4b 19 4b 19 | H K K | 24 24 72 25 75 25 75 25
0x0048 9d 80 0b 00 80 08 9e 1a | | 157 128 11 0 128 8 158 26
0x0050 4a 1a 1b 4d 9e 1a 4c 1b | J M L | 74 26 27 77 158 26 76 27
0x0058 1b 4c 1b 4c 1a 1a 49 18 | L L I | 27 76 27 76 26 26 73 24
0x0060 48 17 49 18 19 4a 19 4b | H I J K | 72 23 73 24 25 74 25 75
0x0068 19 9d 4a 1a 49 80 08 00 | J I | 25 157 74 26 73 128 8 0
0x0070 80 07 a0 19 4b 1a 1b 4c | K L | 128 7 160 25 75 26 27 76
0x0078 1b 1b 4d 9e 1b 4c 1b 4d | M L M | 27 27 77 158 27 76 27 77
Total size is 3340. The first 24 bytes look quite different to the remainder of this file, lending weight to the 24-byte header theory.
Line-by-line comparisons of first 16 bytes (all TYPE 13). 0x10..0x17 are all 0x00 in all examples so far.
a: blank.obj sprite 0, 1x1 tile (27 bytes)
b: pillar.obj sprite 1 (3340 bytes)
c: altar.obj sprite 0 (7465 bytes)
d: altar.obj sprite 1 (7368 bytes)
a 0x0000 10 01 61 01 01 00 01 00 | a | 16 1 97 1 1 0 1 0
b 0x0000 fa 00 25 01 2e 00 48 00 | H | 250 0 37 1 46 0 72 0
c 0x0000 d6 00 16 01 74 00 67 00 | t g | 214 0 22 1 116 0 103 0
d 0x0000 d6 00 19 01 74 00 64 00 | t d | 214 0 25 1 116 0 100 0
a 0x0008 00 00 00 00 03 00 00 00 | | 0 0 0 0 3 0 0 0
b 0x0008 00 00 00 00 f4 0c 00 00 | | 0 0 0 0 244 12 0 0
c 0x0008 00 00 00 00 11 1d 00 00 | | 0 0 0 0 17 29 0 0
d 0x0008 00 00 00 00 b0 1c 00 00 | | 0 0 0 0 176 28 0 0
Assuming a 24-byte header, 0x0c matches "remaining pixeldata" size in all cases.
0x04 makes sense as x,y dimension in blank.obj where we know it's 1x1 pixel.
The pillar has more Y than X; the altar more X than Y, suggesting 0x04-0x05 are X and 0x06-0x07 are Y. Measuring the rendered pixels by WH40K_TD.exe gives a close correspondence in all cases.
WH40K_TD.exe crashes trying to load a set referencing pillar in it unless it's
in the CENTER position. Interesting.
| Offset | Purpose |
|---|---|
| 0x0000 | x,y offset (16 bits each) |
| 0x0004 | x,y size (16 bits each) |
| 0x0008 | ? (blank in all cases so far) |
| 0x000c | Size of remaining pixeldata |
| 0x0010 | Padding? |
| 0x0014 | Padding? |
The volume represented by a cell is a little odd. We see three faces of a fake 3D volume of size 64x64x32(ish). This is presented in an isomorphic fashion, so the height is 32px at the leftmost and rightmost extents and 96 in the centre. Overall width is 128px, and the minimum rectangle covering the whole space would be 128x96, or 12288 bytes at 8bpp.
It seems pixels can be larger than a cell - TZEENTCH.OBJ is almost 2 cells high, for instance.
Loading a constructed palette.obj containing 1x63 pixels, with 0x0000 for the
first 32 bits, caused WH40K_TD.exe to render the pixels far to the left and above
the cell the palette objects were being placed into. Menus have a .obj file
associated with them too, and the elements for Main.mnu draw in the correct
places if we treat it as such.
Considering sprites with a 1,1 x,y.
u0 u1 u2 u3 X Y pixeldata
bgtree_m 24 : ee 00 5e 01 01 01 01 1f 00
blank 0 : 10 01 61 01 01 01 01 fd 00
transpix 0 : 11 01 3c 01 01 01 01 ae 00
Sprites seem to end with a \x00 byte in-general, but we only see 0x01 in the first byte of data for these 1x1 tiles.
Sprites with X= 128 Y=63 almost always seem to have a u0..u3 of
d1 00 42 01 with a few exceptions, e.g. treemac{1,2}.obj
Investigating the sprite data a little more, it seems that we tend to have Y null-separated records: 1 record for sprites with a height of 1, 100 for those with a height of 100, etc.
Blank:
0x0000 d1 00 42 01
0x0004 80 00 3f 00
0x0008 00 00 00 00
b7 01 00 00
0x0010 00 00 00 00
00 00 00 00
jungtil sprite 0 (blank, draws nothing to screen
0x0018 80 3e 04 1f 80 3e 00 128 062 004 031 128 062 000
80 3c 08 1f 80 3c 00 128 060 008 031 128 060 000
80 3a 0c 1f 80 3a 00
80 38 10 1f 80 38 00
80 36 14 1f 80 36 00
80 34 18 1f 80 34 00
80 32 1c 1f 80 32 00
80 30 20 1f 80 30 00
0x0050 80 2e 24 1f 80 2e 00
80 2c 28 1f 80 2c 00
80 2a 2c 1f 80 2a 00
80 28 30 1f 80 28 00
80 26 34 1f 80 26 00
80 24 38 1f 80 24 00
80 22 3c 1f 80 22 00
0x0080 80 20 40 1f 80 20 00
0x0088 80 1e 44 1f 80 1e 00
80 1c 48 1f 80 1c 00
80 1a 4c 1f 80 1a 00
80 18 50 1f 80 18 00
80 16 54 1f 80 16 00
80 14 58 1f 80 14 00
80 12 5c 1f 80 12 00
80 10 60 1f 80 10 00
0x00c0 80 0e 64 1f 80 0e 00
80 0c 68 1f 80 0c 00
80 0a 6c 1f 80 0a 00
80 08 70 1f 80 08 00
80 06 74 1f 80 06 00
80 04 78 1f 80 04 00
80 02 7c 1f 80 02 00 128 002 124 031 128 002 000 = row 31
7f 1f 01 1f 00 127 031 001 031 000 = row 32
80 02 7c 1f 80 02 00 128 002 124 031 128 002 000 = roe 33
80 04 78 1f 80 04 00
80 06 74 1f 80 06 00
80 08 70 1f 80 08 00
80 0a 6c 1f 80 0a 00
80 0c 68 1f 80 0c 00
0x0120 80 0e 64 1f 80 0e 00
80 10 60 1f 80 10 00
80 12 5c 1f 80 12 00
80 14 58 1f 80 14 00
80 16 54 1f 80 16 00
80 18 50 1f 80 18 00
80 1a 4c 1f 80 1a 00
80 1c 48 1f 80 1c 00
0x0158 80 1e 44 1f 80 1e 00
80 20 40 1f 80 20 00
80 22 3c 1f 80 22 00
80 24 38 1f 80 24 00
80 26 34 1f 80 26 00
80 28 30 1f 80 28 00
80 2a 2c 1f 80 2a 00
80 2c 28 1f 80 2c 00
0x0190 80 2e 24 1f 80 2e 00
80 30 20 1f 80 30 00
80 32 1c 1f 80 32 00
80 34 18 1f 80 34 00
80 36 14 1f 80 36 00
80 38 10 1f 80 38 00
80 3a 0c 1f 80 3a 00
80 3c 08 1f 80 3c 00
0x01c8 80 3e 04 1f 80 3e 00 = row 63
Sprite 1:
0x0000 d1 00 42 01 80 00 3f 00
0x0008 00 00 00 00 5a 11 00 00
0x0010 00 00 00 00 00 00 00 00
0x0018 80 3e 84 6d 6c 6e 1e 80 3e 00 128 062 132 109 108 110 30 128 062 000
80 3c 88 bf 76 6e 6d 6e 76 76 6e 80 3c 00
0x0030 80 3a 84 bf 76 6e 76 04 6d 84 6e 76 7d 97 80 3a 00
80 38 86 6d 76 6e 76 6e 87 04 6d 86 6e 97 1e 6e 6e 97 80 38 00
80 36 85 bf 76 1d 97 96 04 6d 8b 87 76 6e 97 97 6d 87 6d 1c 97 97 80 36 00
80 34 90 76 6e 97 76 6e 6d 6e 97 6c 6d 6e 76 6e 97 97 1e 03 6d 85 97 97 6e 6c 6c 80 34 00
80 32 89 6d 6e 1e 97 97 1e 6d 6d 6e 03 6d 90 97 6e 76 6e 97 76 6e 6e 6d 97 6c 96 6d 6e 97 1c 80 32 00
80 30 a0 bf 1e 97 76 1e 97 1e bf 6e 6d 6d 8f 6e 8f 1c 76 97 6d 6e 76 6e 96 6e 97 1d 6c 97 6d 6e 97 97 87 80 30 00
80 2e a4 1c af 7e 7d 97 97 7f 6e 97 1e 1e 6e 87 6c 8f 6c 8e 6e 76 1e 6e 97 6d 1c 97 97 76 6e 6d 97 97 6e 87 6e 87 6e 80 2e 00
80 2c a8 bf 1e bf 97 6e 1e 1e 87 6d 6e 96 6e 97 1e 6d 6d 6e 6d 1c 97 7d 97 6e 1e 76 6e 76 97 97 6d 6e 6d 97 96 6e 6e 87 96 97 6e 80 2c 00
80 2a a0 bf 97 97 6e 97 1c 97 1e 6d 6e 97 76 6d 6d 6e 97 6e 6d 76 6e 6e 76 6e 97 7f 1d 6e 97 76 1d 97 6e 03 97 89 6e 87 6e 6e 96 97 96 97 6d 80 2a 00
80 28 b0 bf 97 6e 97 6e 97 1e 6e 87 6e 97 97 1e 76 6e 6d 6e 97 6e 1d 1e 75 1e 1e 7f 1e 6e 97 1e 97 76 97 1d 97 6e 97 87 6e 6e 96 96 8f 6e 97 97 6d 6e 97 80 28 00
80 26 87 bf 6e 97 97 6e 97 6e 03 97 a2 1c 1e 6e 1e 97 1c 97 6d 6e 97 1c 97 1c 7f 97 6e 76 97 1c 97 76 97 6e 97 1c 6e 6e 97 6e 87 6d 96 97 ad 03 97 85 6e 97 76 6e 6e 80 26 00
80 24 81 76 03 97 b4 6e 97 97 6e 97 6e 97 97 1d 97 7f 6e 97 6e 1c 6e 97 6e 1e 76 7f bf 6e 97 1e 76 6e 6e 1e 97 6d 6e 97 1e 97 6e 97 6d 97 be 6e 87 6e 97 1c 97 1d 97 97 6c 97 6d 80 24 00
80 22 b7 bf 1e 6d 6d 6e 97 97 96 76 5f 1c 87 97 97 1e 97 6e 4f 1e 76 97 1e 97 1e 7e 97 1c 1e 1e 6c 6d 97 6d 76 6e 97 1e 1e 7e 1e 97 6d 76 6e ad 87 1c 6d 87 97 1d 87 97 be 97 03 6d 82 6e 97 80 22 00
80 20 81 bf 03 97 83 6e 97 6e 03 97 b6 1c 1c 6d ad 6e 97 1e 6e 97 76 6c 8f 6d 6c 96 97 1e 97 1d 97 1e 76 6e 6d 76 6e 6d 97 1e bf 1d 76 6e 97 6e 76 87 6e 87 97 96 97 6e 97 6d 6e 6e 76 6e 87 6c 6d 6d 6e 80 20 00
80 1e c4 6c 87 76 6e 97 97 6e 1c 97 97 6d 97 97 6e 97 1e 97 1e 1c 6d 6d 87 76 6d be 76 6d 97 1c 97 1e bf 76 7d 97 1d 76 96 6e 6e 97 1e 97 97 6e 97 87 6e 97 97 87 76 87 6e 1c 97 6e ad 1e 6e 97 6c 76 6d 87 6d 1c 76 80 1e 00
80 1c c8 bf 97 76 97 76 ad 6e 6d 97 97 6e 6e 76 6e 97 97 1e 1c 97 97 4f 1e 76 6d 6d be 6d 6d be 7d
The first byte of each record seems width-related. Mostly matches sprite width, at least in the cases I've looked at so far. Not in the 0x01 case, but certainly in the 0x80 case, we then get a byte that seems to specify an offset for the following pixeldata, and then a trailing pair of bytes with the same value.
For these tiles, the central (widest) row has 0x7f instead of 0x80.
Comparative rendering
Here, I'm focusing on the altar.obj file. It has just two sprites and is a
CENTER object. Here's how it looks when rendered by WH40K_TD.exe (there's
a cloud of uncertainty around the skulls at present):
And here's the rendering of the first sprite, done by my code as of 2018-09-07:
Initial impressions:
- The original clearly has some transparent pixels
- Skulls are scrambled, more so to the right
- Starting X-axis along the top half is well out
- The tabletop has become basically invisible
I think some sort of RLE must be in place for repeated pixels in a line.
Digging into the pixeldata, we have 3 distinct stages in the first 16 records (which are null-terminated):
- Top of skull
- Right hand side is marked by a vertical antenna-like thing
- The stepped altar resumes on the right.
Header:
------------------
0 1 2 3
------------------
0x0000 d6 00 16 01
0x0004 74 00 67 00 (x = 116, y = 103)
0x0008 00 00 00 00
0x000c 11 1d 00 00
0x0010 00 00 00 00
0x0014 00 00 00 00
Here's the first row of pixeldata:
80 3d 82 65 1d 80 35 00 (0)
There are just two visible pixels in this row, and they are specified correctly by the middle two bytes.
The 3 on either side are framing data, for certain. It's not clear exactly what they specify though.
Onto the next row:
80 38 87 97 19 17 18 17 18 16 80 35 00 (1)
These pixels end in the same position on the right, but start 5 pixels earlier. So there are 7 pixels total, specified by the middle bytes again.
The RHS bytes are identical. The LHS bytes are 5 lower and higher, respectively.
Hmm. More rows, not analyzed in depth yet:
80 37 82 17 19 03 17 04 16 80 34 00 ( 2)
80 36 84 18 16 15 15 04 16 83 14 14 16 80 33 00 ( 3)
80 35 83 17 17 16 03 17 81 18 03 17 83 a8 16 17 80 32 00 ( 4)
80 35 8e 18 19 aa 18 17 17 18 18 17 a8 a8 17 18 18 80 02 81 ad 80 2e 00 ( 5) - right vertical bar starts
80 34 8f 18 ab 19 18 aa 19 19 ab ab 19 aa aa 17 17 19 80 02 81 ad 80 2e 00 ( 6)
80 34 8c 19 ab 19 19 1a ad ad 1b ad ad 1a ab 03 17 81 19 80 01 81 ad 80 2e 00 ( 7)
80 34 90 ac 19 18 18 ab 1b ae 1c ae 1c ad ab 17 16 aa 1b 80 01 81 ae 80 2e 00 ( 8)
80 33 91 1a 1c ac 18 19 1a 1b 1c ae 1c 1c ad 1a 19 17 19 ae 80 01 81 ae 80 2e 00 ( 9)
80 33 91 1a 1c ad 1a ad 1c ae 1c ae 1c 1c ad 1c ad 1a ab ad 80 01 81 ae 80 2e 00 (10)
80 31 81 8b 80 01 91 19 1c 1a ad ae 1d 1c af 1d 1c af 1c ae ae 1b ab ad 80 01 81 af 80 2e 00 (11) - left vertical bar starts
80 31 81 ab 80 02 86 1c ad 1b 1c 1c af 04 1d 86 1c ae 1b ad 1a ae 80 01 81 af 80 2e 00 (12)
80 31 81 ad 80 01 93 1f 1c 1c 1b 1c 1b ae 1d 1c 1d ae 1c ad ad 1b ae 1c 1f ad 80 2e 00 (13)
Now for something more interesting. At line 14, an antenna starts to sprout on the far left. It is one pixel wide, followed by 19 transparent pixels, followed by another antenna and the skull.
1 2 3 4
80 1d 81 ad 80 13 85 ac 1f 1f af 1d 08 1c 89 1b 1b 1c ae 1c 1a 1f af 1f 80 2d 00 (14) - right vertical bar is no longer the last thing
80 1d 81 ac 80 11 83 1f 1f ac 03 1f 81 af 09 1d 87 1c 1d ae 1d 1f 1f ae 03 1f 80 2b 00 (15)
80 1d 81 ac 80 0f 04 1f 81 ab 03 1f 88 1c 1d 17 15 15 1d 1e 1c 05 1d 03 1f 81 af 05 1f 80 29 00
Looking at y=14, the first couplet must define where we start, and
that all bytes before then are transparent. Looks like 0x80 <count>
The second couplet writes 1 byte in a defined colour. Palette indices 0xad and
0xac match the pixels rendered by WH40K_TD.exe.
The third couplet defines the transparent section between the far left antenna
and the left antenna or table. Like the first couplet, it is 0x80 <count>.
At 4, the couplet idea starts to break down, but this is definitely starting to look RLE-ish.
Let's go back to line 2 with the knowledge above. We have 9 literal bytes to display. Decoded, they should equal:
17 19 17 17 17 16 16 16 16
(these are the indexes for the colours each pixel is, as rendered by
WH40K_TD.EXE).
The encoded bytes:
80 37 82 17 19 03 17 04 16 80 34 00 ( 2)
We can get to the output by applying these rules to each byte of encoded data:
< 0x80: repeat the next byte that many times= 0x80: skip<count>bytes> 0x80: take literal - 0x80 bytes
Doing so, the altar displays, but more importantly, so does all of
Chapter01.MAP!
The annoying "static" is gone, and the only rendering issue visible is that the objects from different Z levels don't stack correctly. Hooray for progress!
I haven't tried any of the other objects in different chapters yet. Perhaps some of them will fail to render?
Debugger
WH40K_TD.exe loops around "ReadInMissionFLCs", incl. address 0x0041dd10, where it loads in all the .asc and .obj files in a set.
break *0x41DD10
This lets me focus very narrowly on what happens when loading sprites, and might give clues.