Skyrim VR Address Tools

Tools for converting a Skyrim SSE skse mod to Skyrim VR.

Description

This repo consists of two main components:

1. Python files for analyzing c++ code.
2. CSV files that include various data.

Python

vr_address_tools.py

This is a python tool that uses the various csv files to analyze c++ code. It is intended to analyze code built using commonlibsse for readiness to compile against commonlibvr or commonlibsseng. This currently requires a commonlibvr that can read csv files.

Setting up

1. Pull git repo.

git clone https://github.com/alandtse/vr_address_tools
cd vr_address_tools

2. Install poetry
3. Install python dependencies

poetry install

analyze

Analyze code to determine if uses of rel::id have been defined in database.csv. This allows the mod to be compiled with rel::id's without further changes. Rel::ids using offsets may require further code changes if the VR function has changed.

Output will be a tab separated with warnings and potential SSE or VR addresses to check:

> ./vr_address_tools.py ../CommonLibVR analyze
Finished scanning 1,820 files. rel_ids: 8351 offsets: 4013 results: 90
Database matched: 3869 ida_suggested: 4234 unverified: 3 mismatch: 16 missing: 4466
include/RE/B/BSFaceGenAnimationData.h:26        REL::ID(25977)  SSE: 0x1403c38e0                        WARNING: VR Address undefined.
include/RE/B/BSFaceGenAnimationData.h:33        REL::ID(25980)  SSE: 0x1403c3f00                        WARNING: VR Address undefined.
include/RE/B/BSMusicManager.h:26        REL::ID(514738) SSE: 0x142ec5ce0                        WARNING: VR Address undefined.
include/RE/B/BSPointerHandle.h:213      REL::ID(15967)  SSE: 0x1401ee670                        WARNING: VR Address undefined.
include/RE/B/BSPointerHandle.h:220      REL::ID(12204), 1234    SSE: 0x1401329d0        REL::Offset(0x0143180)  0x140143180     WARNING: Offset detected; offset may need to be manually updated for VR
include/RE/B/BSPointerHandleManager.h:30        REL::ID(514478) SSE: 0x141ec47c0                        WARNING: VR Address undefined.

Warning: rel::id with offsets may require change if the underlying function has been changed in VR.

REL::Relocation<std::uintptr_t> target{ REL::ID(41659), 0x526 };

In this example, even if 41659 exists in database.csv, the offset to 0x526 may not be the same in VR and will need to be manually updated.

generate

Generate a database.csv or release csv. Database.csv can be edited manually or generated. Release csvs should be generated using the tool.

Generate Release csv:

This will take the database.csv and convert it to a release csv.

./vr_address_tools.py . generate -rv 1.1.25
Finished scanning 0 files. rel_ids: 0 offsets: 0 results: 0
Filtered 749049 to 3884 using min_confidence 2
Wrote 3884 rows into version-1.4.15.0.csv with release version 1.1.25

Generate Database.csv

This is intended to scan an existing project that defines both rel::id and rel::offset files with the same namespace. For example, exit-9b's commonsse vr branch was used to generate the initial database.csv file.

./vr_address_tools.py . generate -d
Finished scanning 0 files. rel_ids: 0 offsets: 0 results: 0
Filtered 749049 to 3884 using min_confidence 2
Wrote 3888 rows into database.csv with release version 0.0.0

merge.py

Quick script to try to merge the offsets files and some comments files. The primary purpose is to generate se_ae_offsets.csv.

CSV Files

database.csv

A csv for generating release csv files for loading in CommonLibVR to replace addresslib. This intended to be a database to identify addresslib ids that represent SkyrimSSE addresses and convert to appropriate VR address. This can be manually edited and is intended to be a community resource. The database.csv can be converted to a release csv using vr_address_tools.py generate.

id	sse	vr	status	name
10878	0x1400f7210	0x1401077c0	3	RE::Offset::BGSDefaultObjectManager::GetSingleton

• id - Addresslib id
• sse - SSE Address with base (e.g., 0x1400f7210)
• vr - VR Address with base (e.g., 0x1401077c0)
• status - The level of confidence in the VR address.
  • 0 - Unknown
  • 1 - Suggested by automatic tools
  • 2 - Manually entered and assumed manually verified
  • 3 - Manually entered with suggested automatic tools verification
  • 4 - Bit for bit identical (offsets and patches should work identically)
• name (optional) - A friendly name to describe the id

Release CSVs

A non-standard csv installed by end users in the data/skse/plugins/ directory. This follows the addresslib naming of version-{skyrim version}.csv. The first row of data is the csv header, second row is meta data, and third row and beyond is the actual data:

id	offset
total entries	version
10878	01077c0

• id - Addresslib id
• offset - VR Address as offset (e.g., 01077c0)
• total entries - The number of entries to reserve space for. WARNING: CTDs may occur if the total entries is less than the actual number of entries since it is allocating space for a memory map.
• version - The release version which is a semantic version.

Analysis CSVs

These are CSVs intended to aid in analysis.

offsets-1.5.97.0.csv

A dump of addresslib for SkyrimSSE 1.5.97.0. This should be considered canonical for the id -> sse mapping.

id	sse
2	10d0

• id - Addresslib id
• sse - SSE offset (e.g., 10d0)

addrlib.csv

A mapping file generated by bakou using ida. Partially automated.

vr	sse	id
0x1400010d0	0x1400010d0	2

• vr - VR Address with base (e.g., 0x1401077c0)
• sse - SSE Address with base (e.g., 0x1400f7210)
• id - Addresslib id

sse_vr.csv

A mapping file generated by meh321 using IDADiffCalculator, the script used to calculate SSE offsets. Partially automated.

sse	vr
0x141992C10	0x141A33D38

• sse - SSE Address with base (e.g., 0x1400f7210)
• vr - VR Address with base (e.g., 0x1401077c0)

offsets-1.6.318.0.csv

A dump of addresslib for SkyrimSSE 1.6.318.0 (AE). This should be considered canonical for the aeid -> ae_addr mapping.

aeid	ae_addr
1	140001022

• aeid - Addresslib id for Anniversary Edition
• ae_addr - AE offset (e.g., 140001022)

1.5.97_comments.csv

Some mappings by Ultra of 1.5.97.0 used in IDA/ghidra. This has some naming info (but should be considered less current then AddressLibraryDatabase)

sse_addr	comments
140001258	anonymous_namespace__MonitorAPO::Func9_140001258

• sse_addr - SSE offset (e.g., 140001022)
• comments - Potential name (e.g., anonymous_namespace__MonitorAPO::Func9_140001258)

se-ae-attempted-match.csv

Algorithmic mapping of SSE to AE addresses. This is a csv version of what is available under misc files at address library.

sse_addr	ae_addr
140001022	140001022

• sse_addr - SSE offset (e.g., 140001022)
• ae_addr - AE offset (e.g., 140001022)

se-ae_offsets.csv

Output of [merge.py]. This tries to map various comments with the offsets* file. This is an algorithmic only mapping and meant as input for vr_address_tools.

sseid	sse_addr	ae_addr	aeid	comments
5	140001258	140001258	11.0	MonitorAPO::Func9_*

• sseid - SSE ID
• sse_addr - SSE offset (e.g., 140001022)
• ae_addr - AE offset (e.g., 140001022)
• aeid - AE ID
• comments - Name

se-ae.csv

Output of [vr_address_tools.py]. This takes the automated items from se_ae_offsets and combines it with se-ae.csv and whatever is scanned by vr_address_tools.

sseid	aeid	confidence	name
5	11	1	MonitorAPO::Func9_*

• sseid - SSE ID
• aeid - AE ID
• confidence - The level of confidence in the mapping.
  • 0 - Unknown
  • 1 - Suggested by automatic tools
  • 2 - Manually entered and assumed manually verified
  • 3 - Manually entered with suggested automatic tools verification
  • 4 - Bit for bit identical (offsets and patches should work identically)
• name - Name

Porting a Skyrim VR mod

Setup CommonLibVR or CommonLibSSE-NG

1. Download CommonLibVR with csv support.
2. Set environment variable for CommonLibVRPath to CommonLibVR location.
3. Set environment variable for SkyrimVRPath to SkyrimVR path
4. Build CommonLibVR. cmake -B buildVR -S . -DBUILD_SKYRIMVR=ON to confirm it builds.

Modify mod

1. Use vr_address_tools to analyze source tree (the tool currently identifies common rel::id formulations. Others may need to be manually found).
2. For any missing rel::ids WARNING: VR Address undefined., modify database.csv with proper address. Consider upstreaming once verified.
3. For any rel::ids with offsets WARNING: Offset detected; offset may need to be manually updated for VR, modify offsets if VR function is different using #ifndef SKYRIMVR as appropriate (see 6).
4. Generate release csv file.
5. Copy release csv to SkyrimVR directory: data/SKSE/Plugins.
6. Use #ifndef SKYRIMVR to identify SSE or VR only sections. For example, the SKSE version check is a common area. Common things (hardest to easiest):
   • x64 Assembly for patchers. This is only needed if functions instructions are not identical. When this happens, the registers used commonly shift in VR vs SSE.
   • Offsets if the function or vtables has changed.
   • Hard coded values such as Skyrim Special Edition like in a path.
   • Version checks
   • Broken functions in VR:
     • ActiveEffectsVisitor
     • Mod Counting
     • Light mod access
7. Modify cmakelists.txt, cmakepresets.json. See example.

Build mod

1. cmake --preset vs2022-windows-vcpkg-vr
2. cmake --build buildvr --config Release or open buildvr/modname.sln. Build release and copy to SkyrimVR.

Address IDs

Address IDs are keyed into SSE. However, occasionally an SSE address will not have an Address ID. In that case, they may still be entered by taking the integer value of the SkyrimSSE address.

id	sse	vr	status	name
5370397616	0x14019c3b0	0x1401ac0e0	3	RE::BSShaderProperty::InvalidateMaterial

Contributions are welcome!

If you want to contribute to this please read the Contribution guidelines

---