User:Jifodus/Dwarf Fortress Utility Framework

I'm currently writing a framework for Dwarf Fortress utilities. The general idea is to use C++ interfaces in a cross-compiler fashion that is very easy to use.

Please note, since this is still currently under development, I don't have the time to make a pretty WikiPage. I also will not post the interface & file format document for the same reason. Once there is a release, I will have a solid data format that will probably never change, as well as unchanging interfaces.

Features Overview

The Dwarf Fortress Utility Framework is designed with the following goals in mind:

• Utilities compiled for the very first release of the framework still work 10 years later
  • With all the updates and patches applied to the library
  • With the latest release of Dwarf Fortress
• The end user of a utility can simply download and run the utility and have it work
• The framework provides a flexible and easy to use way to define types and memory maps, and write utilities to use the types and memory maps

Current Features

• Written in C++, using interfaces
• Memory data stored in Lua format
• Sample utility for implementation reference: reimplementation of StartProfile
• API header & library
• Complete source code
• Identify Dwarf Fortress version via PE executable timestamp
• Cross-compiler/C compatible using Windows COM style interfaces
• Auto-loading of Lua data files
• Library self-contained in DLL

Current Limitations

• Buggy!
  • Memory leaks exist
• Absolutely no documentation whatsoever
• Sparse comments
• Memory data is all there, but the library does not yet use all of it
• The library is designed for C++, the C interface has not been tested (and probably will not work)
  • Also, this was only tested using MSVC++ .net 2005, it will probably work on other version of MSVC++.
• Does not cleanly handle errors
• Some interface functions not implemented
  • Map/Get through pointer functions not yet implemented
  • iDFUF::installDataFile is not yet implemented
• Compiled as debug mode (bloated file sizes)
• Windows XP or later required (from the use of GetModuleHandleEx)
• The library is not extensively tested

Second Release

The next release planned features will include:

• Auto-retrieve memory data off the internet
• Auto-update framework code itself
• Have an installer that will install the DLL and data file to a shared location, so multiple utilities can use the same library

Future

And further down the road (through auto-update):

• Implement specific subsets of the std library; reduce utility size more, no large dependencies for the utilities
  • std::string
  • std::vector
  • std::map
  • Console IO
  • File IO
• Easy to use GUI framework; for making tools with a nice GUI
• Cross-process memory allocation

Data Format

Requirements

The framework requires definitions of the following types:

• raw: a raw array of bytes; internally it allows access to an array of type.size * type.fixed_array bytes.
• pointer: a pointer to a location in Dwarf Fortress's memory; can represent a 32-bit pointer
• dword: an integer type that is 4 bytes
• word: an integer type that is 2 bytes
• byte: an integer type that is 1 byte
• float: a 32-bit floating point type
• double: a 64-bit floating point type
• string: a type that represents a std::string
  • required members:
    • dword length: defines how many characters string contains
    • dword capacity: defines the maximum number of characters the string buffer can contain
    • pointer buffer_ptr: a pointer to a memory location containing the string data
  • optional members:
    • raw buffer: a fixed-size array of characters containing the string when length < the fixed size of the buffer
• vector: a type representing a std::vector
  • required members:
    • pointer begin: a pointer to the begining of the memory block
    • pointer end: a pointer to just beyond of the last valid element in the vector
    • pointer last: a pointer to just beyond the end of the memory block

Data Files

The data files must supply the following information:

• Types: each element of the type list represents one type; there is a special type called Main, Main represents the global memory map
• Signatures: each signature is designed to uniquely identify each version of Dwarf Fortress

Lua Data Files

Classes:

• Type: 'type =' The value can either be a set containing a type override or the string of a type name.
  • Type: 'type =' A string representing the Type Name of the type to override.
  • Subtypes: An array of type overrides and/or Type Names, that are subtypes of this type.
  • Size: 'size =' An integer overriding the size of the type.
  • Fixed Array Size: 'fixed_size =' An integer overriding the fixed array size of the type.
• Types: 'Types'
  • Version: String representing version
    • TypeName: String representing name of the type, special type is Main.
      • Size: 'size =' an integer representing
      • Members: 'members' a table of of name value pairs; The name being the Member Name; The value is a set containing the Type and Offset.
        • Member Name: String representing the name
          • Type: 'type =' The value can either be a set containing a Type override or the name of the of a type.
          • Offset: 'offset =' The member offset (from the base address).
          • Pointer: 'pointer =' The pointer in memory for which the member can be found. Used for the Main type.
• Signatures 'Signatures'
  • Version: String representing the version (same string as types).
    • PE Timestamp: 'pe_timestamp' The PE header timestamp value.
    • .text Adler32: 'adler32' The Adler32 CRC of the ".text" section of the executable.
    • Text Segments: 'text_segments' An array of segments of the ".text" section of the executable.
      • Address: [1] = The offset into the ".text" section that the following raw data can be found.
      • Raw Data: [2] = The data the ".text" segment is supposed to contain.