User:BenLubar/save research

Partly based on User:Rick/Save research and User:Andux/Format research.

Primitives

intn	an n bit little endian two's complement signed integer
uintn	an n bit little endian unsigned integer
string	uint16 (length) followed by length bytes of CP437-encoded text.
list of type	uint32 (count) followed by count elements of type. can be nested.
set of type	same as list, but always of a numeric type, has no duplicate elements, and is always in ascending order.
n-array of type	n elements of type. n is always a constant.
optional type	uint8 (flag); if flag is 0, nothing follows; if flag is 1, an element of type type follows.
mapping from type1 to type2	uint32 (count) followed by count instances of a type1 followed by a type2, alternating.

Data structures

name

string	first name
string	nickname
7-array of int32	word index (index into WORD string table or -1)
7-array of uint16	word form
uint32	language (index into TRANSLATION string table)
int16	unknown

version header

uint32	version (this page only concerns version 1451)
uint32	compression. when 0, the remainder of the file is verbatim. when 1, the remainder of the file is in compressed blocks.

compressed block

list of uint8	a block of data, compressed with zlib

Files

world.dat

world header

version header	version header
uint16	always zero?[Verify]
28-array of int32	unknown
optional name	the world name, like "The Realm of Daggers"
uint8	unknown
int16	unknown
3-array of int32	unknown
string	the translated world name, like "Thadar Rabin"
4-array of list of list of string	generated raws: inorganic, item, creature, interaction
20-array of list of string	string tables: inorganic, plant, body, bodygloss, creature, item, building, entity, word, symbol, translation, color, shape, color_pattern, reaction, material_template, tissue_template, body_detail_plan, creature_variation, and interaction Note: at this point, it is a good idea to append the generated raw names (index 2 of each entry, surrounded by : on the left and ] on the right) to the corresponding string tables.
mapping from uint32 to uint32	unknown
uint32	always zero?[Verify]
14-array of list of uint32	unknown

books

Still working on this; here's what I have so far:

the byte sequence d0 8a d0 8a d0 8a 00 00
a 32-bit bitfield - 0x00000804 has 6 bytes at the end that 0x00000004 doesn't. (A)
there are 48 bits, all of them zeroes.
there's a 32 bit integer that monotonically increases. (B)
the following 32 bit integers: 0, 1, 1.
B is repeated.
the following 32 bit integers: -1, -1, 1.
24 bits, all of them zeroes.
the 16 bit integer 0x2742 (ASCII B', which looks like "Book"(?))
the following 32 bit integers: 0, 0, -1.
16 bit integer. (C)
if C is 0x00, this book is bound with an INORGANIC material.
if C is 0x03, this book is bound with green glass and D is -1.
if C is 0x04, this book is bound with clear glass and D is -1.
if C is 0x05, this book is bound with crystal glass and D is -1.
if C is 0x15 or 0x16, this book is bound with CREATURE bone.
if C is 0x17, this book is bound with CREATURE cartilage.
if C is 0x19, this book is bound with CREATURE teeth.
if C is 0x23, this book is bound with CREATURE spleen.
if C is 0x24 or 0x25 or 0x26, this book is bound with CREATURE leather.
if C is 0x27, this book is bound with CREATURE shell.
if C is 0x28, this book is bound with CREATURE feather.
if C is 0x29, this book is bound with CREATURE hoof.
if C is 0x2a, this book is bound with CREATURE ivory.
if C is 0x1a4, this book is bound with PLANT wood.
if C is 0x1a5, this book is bound with PLANT fiber.

32 bit signed integer. (D)
D is the index into the string table mentioned by C that further describes the material binding the book.
For example, if the INORGANIC string table has "QUARTZITE" in position 178 and C=0, D=178, this book is bound with quartzite.

the following 16 bit integer: -1.

16 bit integer. (G)
if G is 0, this is a book and the string H is a title.
if G is 5, this is a slab and the string H is a description.
A will have the 0x800 bit set iff G is 5.

the following 16 bit integers: 0, 0.
32 bit integer. (E)
the following 32 bit integer: -1.
32 bit integer (bitfield). (F)


if F & 2,
        74 bytes (TODO)
if F & 1,
        40 bytes (TODO)

a string. (H)

if A & 0x800
        six bytes