Jump to content

[WIPz] TES5Edit


zilav

Recommended Posts

If I wanted to write a script for xEdit, is there a place with documentation to assist in the process?  I actually have a script, but I need to modify it some.  Do I just need to figure it out on my own, or is there someplace I can get some help?

{
Comment
}
unit UserScript;

const
FormIDStart = $00050000;
FormIDLast = $00950000;

function Process(e: IInterface): integer;
var
TDirectory: TDirectory;
FORMID, FORMID2: cardinal;
fileiwb, check: IInterface;
i, j, k: integer;
files: TStringDynArray;
aFolder, q, s, t, m, x, zx, after, temp: string;
begin
aFolder := 'c:\OblivionLM\Data\sound\voice\Desert_AlikR.esp\';

fileiwb := FileByIndex(0);

FORMID := GetLoadOrderFormID(e);
AddMessage (IntToHex64(FORMID, 8));
q := (IntToHex64(FORMID, 8));
q := Copy(q, 3, 8);
q := '00' + q;
AddMessage (q);
for i := FormIDStart to FormIDLast do begin
check := RecordByFormID(fileiwb, i, True);
if not Assigned(check) then begin
AddMessage (IntToHex64(i, 8));
m := (IntToHex64(i, 8));
files := TDirectory.GetFiles(aFolder, '*.*', soAllDirectories);

for j := 0 to Pred(Length(files)) do begin
        s := files[j]; 
        
k := Pos(q, s);
if k > 0 then begin
AddMessage (s); 

x := StringReplace( S, q, m, [ rfReplaceAll, rfIgnoreCase ]);
AddMessage (x);

zx := ExtractFilePath(s);
AddMessage (zx);
after  := StringReplace(x, zx, '', [rfReplaceAll, rfIgnoreCase]);
AddMessage (after); 


     ShellExecute(
    TForm(frmMain).Handle,                  // parent window handle, use 0 for none
    'open',                                 // verb
    'cmd.exe',                              // application  
    '/C ren "' + s + '" "' + after + '"' ,
    ' ',                                     // working directory
    SW_HIDE                           // window mode
  );
  
end; 

end;
SetLoadOrderFormID(e, i);   
exit;

end;

end;

end;

end.

I believe this changes FormIDs within the plugin.  I don't need that.  I just need it to read the FormID associated with LIP files, and change the LIP files to match.

I can be persistent, so should be able to figure this out, but help is always welcome.

Link to comment
Share on other sites

xEdit has discord server.

You need to be more specific on what you want to do. As I see you want to rename .lip files because FormIDs have changed in a plugin, and perform some sort of a weird lookup matching in a cycle.

Lip file name consist of EditorID of QUST + EditorID of DIAL + FormID + Response Number. The only unique part in this name is FormID and the rest can repeat in quite a lot of files, so I don't understand the logic of how you match the old lip file name. This is like solving a single equation with several variables.

Link to comment
Share on other sites

First off, thanks for the help, zilav.

The script I posted is not mine, but I was told (I haven't "figured it out" for myself yet--I'm not familiar with the scripting language) that it checks the plugin for available FormIDs, renumbers those IDs appropriately (as in the "Change FormID" xEdit function), and finally takes those new IDs and renames the lip and mp3 (? I think, right?) accordingly.

I just want the script to get the appropriate FormID from the plugin, copy the rest of the name from the lip/mp3 file, substitute correct FormID in the string, then rename the lip/mp3 file.  All of the FormID changing has been done already.  Now, I just need to rename lip/mp3 files.

Direct scripting assistance is always welcome, but I don't mind going it alone and trying to learn the scripting language.  If that's the route I take, I'm curious if there's information to assist in learning the language?  If not, I'll just study existing scripts and post here with questions.

Thanks again.

EDIT:

Through the Github page, I found this.  As of this writing I haven't seen how much information it contains, but it's a start.  Anything else is more than welcome.

Link to comment
Share on other sites

I'm learning, and enjoying learning.  That xEdit documentation is quite helpful.  But I can't quite seem to figure out what the '$' symbol does?  What in God's name is the purpose of that?

From the supplied change LO script:

NewLoadOrder := StrToInt64('$' + s);

The script calls a GUI, gets user input (which is in the form of a string), the above line takes that string and converts it to an integer.  NP.  But what's the '$' doing?  I Googled for a minute or 5 but couldn't find a reference to that.

Link to comment
Share on other sites

The CKit names files with all those identifiers - but the game does not care what the name of file is ... As long as the record is pointing to correct name.
So, the lip/mp3 files can be renamed anything (MyLip001.lip for example) as long as the voice record points to the new file name not the CKit given name.

Right?

So, in your scenario - renumbering the formids - you don't really need to change the lip file names unless you changed where the records are pointing.

 

$ is symbol for String, it dates back to early programming languages

Edited by Jebbalon
added string thing
Link to comment
Share on other sites

'$' in Pascal is the same as '0x' in C - a prefix indicating a hex number in a string.

Alright, so you are changing FormIDs yourself and want to rename associated assets. First I suggest you to stop treating FormIDs as strings, they are integer 32 bits numbers. If you want to remove/zero the load order byte then

FormID and $00FFFFFF

will do that. To set load order byte (assuming that load order byte is already zero)

FormID or (order shl 24)

So the final code to change load order number in FormID will be

FormID := (FormID and $00FFFFFF) or (order shl 24);

This avoids all those string conversions and $.

Link to comment
Share on other sites

Forum has bugged out and I couldn't type in the same post, so lets continue.

Your logic is fine - you have some FormIDs space to change to, you cycle over it, check that FormID is free, then lookup matching lip file names by FormID (cycle over list of files in a directory and check with Pos for FormID with zero load order byte), rename file(s), change FormID of a record.

1) The code to get the list of lip files should be in Initialize function which is called once when the script starts, you don't want to populate that list over and over again for each processed record.

2) You need to process INFO records only, so if Signature(e) <> 'INFO' then Exit; should be the first line of code in Process function.

3) No need to cycle over FormIDs space for each processed record. Set the initial FormID number to look from using global integer variable in Initialize function, then in Process make a loop to increment it by 1 until FormID number is free. Something like

while Assigned(RecordByFormID(GetFile(e), LookupFormID, True)) do Inc(LookupFormID);

4) There is RenameFile function in Delphi for renaming files, try it instead (not sure if it is exposed for scripting).

Link to comment
Share on other sites

Thanks for the tips, zilav!  I will implement them and update the script.  I've been playing with the scripting functions (remember, the script posted above is not mine, I got it from someone else) and find it easy enough to learn.  I just have no familiarity with Pascal, so going is slow.  Googling, etc.  Prior to this, I've never even looked at a script for xEdit.  So, it'll take a minute to catch on.

If I have questions, I will post back.  Thanks again!

Link to comment
Share on other sites

Okay, zilav, I took a couple hours today and implemented your advice into the script I acquired.  Untested as of this writing, but it looks right to my novice eye.  I would very much appreciate it if you could scan it over and provide any needed feedback.

unit UserScript;

var
	voice_files: TStringDynArray;
	formidstart: Cardinal; 

function Initialize: integer;
var 
	TDirectory: TDirectory;
	voices_root: string;

begin
	formidstart := $00001000;
	voices_root := 'C:\Users\MaLonn\Downloads\Knights\sound\voice\knights.esp\';
	voice_files := TDirectory.GetFiles(voices_root, '*.*', soAllDirectories);
end;

function Process(e: IInterface): integer;
var
	oldid, newid: cardinal;
	j, k: integer;
	freeid, s, x, zx, new_name: string;

begin
	oldid := GetLoadOrderFormID(e);
	newid := (oldid and $00FFFFFF) + ($00 shl 24);
	AddMessage('Current file ID: ' + IntToHex64(newid));

	while Assigned(RecordByFormID(GetFile(e), formidstart, True)) do
		Inc(formidstart);
	for j := 0 to Length(voice_files) - 1 do 
	begin
	    s := voice_files[j]; 
	    k := Pos(IntToHex64(newid), s);
		if k > 0 then 
		begin 
			x := StringReplace(s, newid, IntToHex64(formidstart), [rfReplaceAll, rfIgnoreCase]);
			AddMessage (x);
			zx := ExtractFilePath(s);
			AddMessage (zx);
			new_name  := StringReplace(x, zx, '', [rfReplaceAll, rfIgnoreCase]);
			AddMessage(new_name); 

		    ShellExecute(
		    TForm(frmMain).Handle,                  // parent window handle, use 0 for none
		    'open',                                 // verb
		    'cmd.exe',                              // application  
		    '/C ren "' + s + '" "' + new_name + '"' ,
		    ' ',                                     // working directory
		    SW_HIDE                           // window mode
			);
		end; 
	end;
	SetLoadOrderFormID(e, formidstart);   
	exit;
end;

end.

I can't thank you enough for your patience and help.  Holding a noob's hand can be annoying at times.

Link to comment
Share on other sites

Looks fine (if RenameFile doesn't work, not a fan of starting cmd for each renaming).

The only possible issue is SetLoadOrderFormID - you provide formidstart with the order byte of 00 (which corresponds to the file loaded at index 0 which is almost always Oblivion.esm) but this function expects load order corrected FormID. This is also somewhat relevant to RecordByFormID functon, however it works with file index FormIDs (00XXXXXX are overrides of the first master of a plugin, 01XXXXXX overrides second master, etc. Order byte > highest master index in the header is a new record). So for example if the plugin you are renumbering is loaded at the index 2 and has a single master Oblivion.esm, then RecordByFormID should be used with 01XXXXXX formids and SetLoadOrderFormID with 02XXXXXX.

And you forgot about if Signature(e) <> 'INFO' then Exit;

Link to comment
Share on other sites

I did not even try RenameFile (yet).  The script is untested and I was going to play with that function during the testing process.  It's on my list of things to do, however.

Good catch (!) with the global "formidstart" and those functions.  But, this is a personal script and is designed to be run with two plugins only--master (00) and plugin (01).  Plugin is being groomed to merge into master, so it's okay.  That's the only way it's designed to work and I'll make a comment at the start of the script to remind me of this if time passes between uses (and I forget)

Question about that last line: aren't "DIAL" records recorded and named, thus need to be renamed as well?  I confess, I'm ignorant in regards to this topic.

Link to comment
Share on other sites

Lip files are named after the INFOs only, so if you want to renumber DIALs too then add a check to rename files only for INFOs.

Link to comment
Share on other sites

Gotcha, zilav.  Thanks again for the help.  I'm in a holding pattern for now, but when the time comes to run this script, I may have more questions.

Link to comment
Share on other sites

  • 2 weeks later...

Okay.  I got out of the holding pattern, updated and fixed the script, and it is confirmed working.  Here it is for anyone interested:

{Designed to change dialog formIDs unique to a plugin to free master file formIDs and then rename the associated
 MP3 and Lip files accordingly.
 This script is written for merging from ONE plugin (load order index '01') to a master (load order index '00').
 It does not take into account multiple plugins.}
unit UserScript;

var
	voice_files: TStringDynArray;
	formidstart: Cardinal; 

function Initialize: integer;
var 
	TDirectory: TDirectory;
	voices_root: string;

begin
	formidstart := $00000001;
	//Path to sound files to be renamed
	voices_root := 'C:\Users\MaLonn\Downloads\Sound\Voice\Unofficial Oblivion Patch.esp\';
	//Create array of sound files and path
	voice_files := TDirectory.GetFiles(voices_root, '*.*', soAllDirectories);
end;

function Process(e: IInterface): integer;
var
	oldid, newid: cardinal;
	j: integer;
	s, x, old_name, new_name: string;

begin
	oldid := GetLoadOrderFormID(e);
	//Ensure the load order index (8 bits) of formID are '01', i.e. not the master.
    if (oldid shr 24) and $FF = $01 then
    begin
		//Zero out load order index 8 bits and replace with '00'
		newid := (oldid and $00FFFFFF) + ($00 shl 24);
		//Check for free formID in master to assign
		while Assigned(RecordByFormID(FileByLoadOrder(0), formidstart, True)) do
			Inc(formidstart);
		//Begin loop through sound files array
		for j := 0 to Length(voice_files) - 1 do 
		begin
		    s := voice_files[j];
		    //If the formID is present, proceed
			if Pos(IntToHex64(newid, 8), s) > 0 then 
			begin
				//Replace old formID with new/free formID 
				x := StringReplace(s, IntToHex64(newid, 8), IntToHex64(formidstart, 8), [rfReplaceAll, rfIgnoreCase]);
				//For message printing...
				old_name := StringReplace(s, ExtractFilePath(s), '', [rfReplaceAll, rfIgnoreCase]);
				new_name := StringReplace(x, ExtractFilePath(s), '', [rfReplaceAll, rfIgnoreCase]);
				AddMessage(Format('Replacing "%s" with "%s"', [for_print, new_name]));
				//Delphi function to rename a file 
				RenameFile(s, x)
			end; 
		end;
		//Change referenced records to the new/free formID
		while ReferencedByCount(e) > 0 do
			CompareExchangeFormID(ReferencedByIndex(e, 0), oldid, formidstart);
		SetLoadOrderFormID(e, formidstart);
		Inc(formidstart);
	end;
end;

end.

If anyone wants to use it, make note of the opening comment.  This is written for specific use.  Also, the script posted in this post is the working one.  Earlier posts were WIP.

Another question for you, zilav: in the header (HEDR) section of a plugin there is "NextObjectID".  Is that necessary to be updated in the mod I'm merging into?  xEdit changes it when manually changing formIDs, but I couldn't find a function that seemed to deal with it.

Finally, I'll close by saying thank you, zilav!  I'm continuing to write more scripts to aid in tasks with xEdit.

Link to comment
Share on other sites

Updating HEDR is not necessary.

You can skip overrides and process only the new records in a plugin using if IsMaster(e) then ... instead of checking the load order byte.

By the way

 + ($00 shl 24)

part does nothing :) (you shift zero to the left 24 times producing the same zero and adding it, and as we all know adding zero doesn't change the result).

Also FormIDs should start from $800, below that are reserved.

Link to comment
Share on other sites

  • 3 months later...

Hellos there. I'm trying to create a few xEdit scripts to help modularize Cobl.esm, break it into separate esps for developers to work on (mostly) independently and then merge it all together for releases, and have a couple questions (at least two for now).

Is there a magic trick to getting the file header, or is it really not implemented? I would like to add some information in the description like Module namespace (cobAls, cobGrind, etc.), Dependency/Required Modules. I saw something about injected info in this thread, would that be the preferred method? I can always have a co-file csv and parse it, but I've learned over the years that the closer you place info to the original code the less likely it'll be lost and the more likely it'll actually be updated.

I'm writing a script to gather the dependent modules (not mods, again trying to break apart a single mod), make sure I'm not missing anything. I've got scripts and magic effects so far, so I can use what I've learned to take on pretty much anything, but... Any hints on writing a recursive function that will work with all record types? I can use ElementCount and ElementByIndex to walk down and through the sub-records (so far at least) but then I need a solid way to tell if that element is a record. From the interface, CanContainFormIDs() would work if it returned False correctly, but if I can't trust that can I just use LinksTo() and skip if it returns nil/-1?

Another route I've considered is grabbing all the ReferencedBy entries and building a dependency graph, check it for (module/namespace) cycles. I know of a few general "check on the other quest/container statuses" maintenance quests that are cyclic and will need to be in a last stage module after everything else has come together. Anyway, any suggestions for Delphi graph libraries? I found GraphViz and SimpleGraph but they have not been updated in over 7 years.

Link to comment
Share on other sites

TES4 Header is the first record (element) in a plugin file, so

header := ElementByIndex(f, 0);

where `f` is a file object which you can get either by index from loaded plugins f := FileByIndex(N) or from a record f := GetFile(e), so getting the header from Oblivion.esm which is always loaded first for example

header := ElementByIndex(FileByIndex(0), 0);

Not sure about namespaces. You can place your code into separate .pas files and include them with

using MyFile;

at the top of the script. Storing additional data in separate files is perfectly fine, some demo scripts do that already, for example `Oblivion - Items lookup replacement.pas`.

Recursion while checking for the element type can be seen in the `Skyrim - List used scripts.pas`. You should look into existing scripts more often, they contain a lot of example of how to do things in xEdit scripts and included for that purpose.

No idea why would you need graphs for references, you provided no information on what exactly you want to do. Maybe you won't need graphs, references, recursions, etc, and your task can be implemented way more easier. By the way xEdit has discord server (button in the right upper corner on xEdit windiw) with a separare channel devoted to scripting.

Link to comment
Share on other sites

Thanks for the info on the file header, I'll try it out.

Maybe component is a better word? (It's loaded too for anyone heavy into the .net world.) I'm trying to split up the Cobl Main.esm into many smaller, independent ems that can be developed on their own and then merged back together for release. I thankfully did a lot of the heavy lifting the last time I worked on Cobl (so, like, 10 years ago or so...) by denoting the module that the record belongs to (second part of the name, such as cobAls for the Alchemy Sorters and cobGrind for the grinder) and removing unnecessary "dependencies". I want to make sure I've isolated records so they are practically modules and only reference other records in the same module. I've already found a few records that I need to change, some common modules like cobGen(eneral functional activators), and some dependency cycles (the options menu quest references many modules which in turn reference options buttons/misc items).

Cobl has over 50 different modules and hundreds (thousands?) of records, so if I want to check everything, I'll need a list of references made by each record. As of now, because we already have the modules denoted, I think I can get away with finding all of the references made by all of the records in a module, sort them by editorID, log them, look through them. But there's "grab all references" function like there is a "grab all referenced by" function. Looks like I can handle this with some recursion, thank you for the tip looks like there are no gotchas such as sub-records treated records and becoming infinite loops, so it should be easy, Any shortcuts would be appreciated.

(And I have a start for the actual merge process - just need to use "Merge overrides into Master" as a starter and modify to copy all records into master "Renumber FormID of Matching EDID" is useful too.)

Link to comment
Share on other sites

Only direct "referenced by" records have meaning (what xEdit already shows you), because if you try to go deeper checking references of references, other records can backward reference previous records creating infinite loops, and even recursion itself depending on how deep it goes could potentially grab all records in a plugin in the end depending on it's structure. That's why there is no such function in xEdit as "copy a record and it's references as new into the other plugin", this is like solving a single equotion with several variables. Only a human can decide when to stop and what should be included and what to skip.

Newer versions of xEdit have built-in plugins merging functions, just search this thread for "merge" since I already posted step by step instruction.

Link to comment
Share on other sites

  • 5 months later...

Hey guys, I hope this thread is still active and this is the right place to post for FO4Edit. I've found out what the "Unknown 13" and "Unknown 4" Record flags do. First I want to thank "The ChatBreaker" who, on page 61 of this thread, was the only person I've ever seen reference these stupid records.

Spoiler

xEditRecords.thumb.jpg.d25632a235f370976ce4bfdb8fbe1f71.jpg

So first, The ChatBreaker thought these were references specific to terminals. I looked into that and also found these record flags in loads of other objects; anything that had "SNAM- Marker paramaters" in it. I opened a random terminal (that did not have "Unknown 13 or 4") in Creation kit, saved it without touching anything, and Creation Kit created the Record Flags "Unknown 13" and "Unknown 4" in xEdit. It did this for literally every terminal I checked. So after alot of time I discovered that "Unknown 13" is a Record created or removed by the CK when "Front" is enabled or disabled respectively. "Unknown 4" is the exact same, but for the "Rear" type is enabled or disabled. 

 

Spoiler

EntryMarkers.thumb.jpg.0754d2927982ed9ab92b636425ad72af.jpg

Here you can see "Active Markers" "Interact point 0" which options show a little to the right highlighted in Blue. The "Marker Available Entry types" are what "Unknown 13-4" references. If you uncheck them and save the plugin, CK will remove the respective Record Flags. So yay, we've discovered what the records point to and what creates them, but what the hell does the flag actually do? This took me hours to figure out, and it's so insignificant I wasn't sure I should make this post. But removing the "Unknown 13" Record Flag while keeping the "Front" option enabled; it disabled the two nodes you see highlighted in black, and instead used the two nodes provided by the "Other" checkbox. Re-adding the flag "Unknown 13" uses those two front nodes. 

Now I'm not a modder, gonna say that first. The two farthest blue nodes, highlighted in black, enable the "Walk up to the chair" animation, ya know the animation that plays when you're furthest you can be from the chair and activate it to sit down. Without the "Unknown 13" Flag, even if you're far from the chair when you activate it, it uses the animation closest to the chair. I tried this on every piece of furniture that had these two "Walk up" nodes and the "Unknown 13" record flag being removed, removed the little "Walk up to the chair" Animation on every one of them. Now, as for "Unknown 4" I wasn't actually able to test this. Every piece of furniture that you can walk up to from the back (Highlighted in Red to indicate the "Rear" option) didn't have any blue nodes provided by "Other" The "Other" Option is highlighted in light blue, it provides an alternate "Entry" point blue node, as well as "Exit" yellow node. So best I can tell, this flag tells the game to use the "Front" and "Rear" entry point animations even if the "Other" option provides an alternative. Of course, even with these flags enabled, if you're standing right next to the chair you won't get the little "Walk up to the chair" animation. So I don't actually know how it works. For Example, if you disable "Other" it will remove the nodes in the front and the one exit yellow node in the rear, then if you remove the "Unknown 13 & 4" flags, you'll still get the little walk up animation when facing the chair from the front. Because the Terminals/Ambushs/Power Armor and some other furniture like beds only have options provided by "Other" these flags don't seem to do anything. Terminals especially will have these flags either set, not set, or in some cases only have "Unknown 13" set but the nodes provided by "Front" and "Rear" don't exist, only nodes of entry provided by "Other"

And thats it. Best I can tell through hours and hours of testing, that's all these Record Flags do. I've proven 100% they are tied to the "Front" "Rear" options, but the only noticeable change I've found was the point of entry when the flags are set. Bit anti-climactic really.  I would test this more by changing the animation/NIF/entry points myself, but I'm not a modder, so I don't know how. So I hope this helps you with the "Unknown 13" and "Unknown 4" Record flags.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...