Jump to content

Wrye Bash - All Games


Utumno

Recommended Posts

> the _formVersion and _compressType attributes are only for FO4 and SSE - or TESV too ?

For Skyrim _ formVersion used, yes / _compressType used, no

For Skyrim SE _ formVersion used (In compressed Data), yes / _compressType used, Yes

For Fallout 4 _ formVersion used, yes / _compressType used, No

I have not looked at all the code yet but yes, what you are doing in the Skyrim block of code with def __is_sse(self):  looks like it will work fine. I agree, it won't matter that older Saves from 1.4.132 the version prior to the release of the Fallout 4 CK have version 12.  Because def __is_sse(self): isn't present in class Fallout4SaveHeader.

Skyrim SE Saves:

Please ask the person who made the code to look at the compressed save game header for Skyrim SE to add the Form Version.  The Creation Club will go live soon (possibly within a moth or the next two weeks) and the save game format will change to accommodate ESL files.  The  Form Version is just before the size of the plugins (Still one ubyte) among the compressed data.

Link to comment
Share on other sites

@Utumno Without suppressing the section of code that processes Script Extender CoSaves, (SKSE/F4SE/OBSE), Wrye will produce the following error because that code is still Oblivion specific. It might be best to bypass the CoSave routines except for Oblivion for now.

This is the traceback of the error. What should happen is that the save button sort of greys out and then the screen refreshes. Instead the traceback appears, the Save button is still active as if you should click it again, and things don't refresh. You won't see the changes unless you close and reopen Wrye.

Spoiler

 


Wrye Bash starting
Using Wrye Bash Version 307
OS info: Windows-7-6.1.7601-SP1
Python version: 2.7.12
wxPython version: 2.8.12.1 (msw-unicode)
input encoding: UTF8; output encoding: None; locale: ('en_US', 'cp1252')
filesystem encoding: mbcs 
bash.py  324 main: Searching for game to manage:
bush.py   76 _supportedGames: Detected the following supported games via Windows Registry:
bush.py   78 _supportedGames:  Oblivion: G:\Games\steamapps\common\Oblivion
bush.py   78 _supportedGames:  Skyrim: F:\Games\steamapps\common\Skyrim
bush.py   78 _supportedGames:  Skyrim Special Edition: F:\Games\steamapps\common\Skyrim Special Edition
bush.py   78 _supportedGames:  Fallout4: C:\Games\steamapps\common\Fallout 4
bush.py  136 _detectGames: Detecting games via the -o argument, bash.ini and relative path:
bush.py  142 _detectGames: Set game mode to Fallout4 found in parent directory of Mopy:  C:\Games\steamapps\common\Fallout 4
bush.py  156 __setGame:  Using Fallout4 game: C:\Games\steamapps\common\Fallout 4
testing UAC
mods_metadata.py  228 __init__: Using LOOT API version: 0.10.2
games.py  158 warn_lo: Fixed Load Order: added(Ethreon_Master_Plan.esp), removed(EthreonMasterPlan.esp), reordered (No)
games.py  183 warn_active: Invalid Plugin txt corrected:
Active list contains mods not present in Data/ directory or corrupted: EthreonMasterPlan.esp

Traceback (most recent call last):
  File "bash\balt.py", line 436, in <lambda>
    if onButClick: self.Bind(wx.EVT_BUTTON, lambda __event: onButClick())
  File "bash\basher\__init__.py", line 1999, in DoSave
    saveInfo.write_masters()
  File "bash\bosh\__init__.py", line 1145, in write_masters
    obse.load()
  File "bash\bosh\_saves.py", line 340, in load
    raise FileError(self.name,u'File signature != "OBSE"')
bash.exception.FileError: Save68_A3AA8A36M4C696C6C79_Commonwealth_010652_20170415090643_16_2.f4se: File signature != "OBSE"

 

 

Link to comment
Share on other sites

Updated the FO3/FNV branch, added NSI Installer Info, Added Jip up to v48 and lutana NVSE Support (Thanks to @hlp), and Save Game updates.  It doesn't have the same amount of patchers as Valda's versions (which I still recommend) because of some limitations but it's working again.

EDIT: Nm, CTDA is failing in Fallout NV if I import records from Fallout 3.  I wish I know why, I was very careful of which I deleted and added them to the import list immediately so I wouldn't forget.  If I revert the changes CTDA works fine.

EDIT2: I think I know why. I can't import CTDA because the functions in constants will be different between FO3 and FNV. I don't seem to be able to import any classes that use CTDA because the error keeps saying fallout3.records for CTDA related patching, even when I'm not importing it.

Spoiler

 


Wrye Bash starting
Using Wrye Bash Version 307
OS info: Windows-7-6.1.7601-SP1
Python version: 2.7.12
wxPython version: 2.8.12.1 (msw-unicode)
input encoding: UTF8; output encoding: None; locale: ('en_US', 'cp1252')
filesystem encoding: mbcs 
bash.py  324 main: Searching for game to manage:
bush.py   76 _supportedGames: Detected the following supported games via Windows Registry:
bush.py   78 _supportedGames:  Fallout3: C:\Games\steamapps\common\Fallout 3 goty
bush.py   78 _supportedGames:  Fallout4: C:\Games\steamapps\common\Fallout 4
bush.py   78 _supportedGames:  FalloutNV: C:\Games\steamapps\common\Fallout New Vegas
bush.py   78 _supportedGames:  Oblivion: G:\Games\steamapps\common\Oblivion
bush.py   78 _supportedGames:  Skyrim Special Edition: F:\Games\steamapps\common\Skyrim Special Edition
bush.py   78 _supportedGames:  Skyrim: F:\Games\steamapps\common\Skyrim
bush.py  136 _detectGames: Detecting games via the -o argument, bash.ini and relative path:
bush.py  142 _detectGames: Set game mode to FalloutNV found in parent directory of Mopy:  C:\Games\steamapps\common\Fallout New Vegas
bush.py  156 __setGame:  Using FalloutNV game: C:\Games\steamapps\common\Fallout New Vegas
testing UAC
mods_metadata.py  228 __init__: Using LOOT API version: 0.10.1
Unknown condition function: 586
param1: 00000000
param2: 00000000
Error loading 'ALCH' record and/or subrecord: 00175AAE
  eid = u'CookCooksFiendStew'
  subrecord = 'CTDA'
  subrecord size = 28
  file pos = 12546359
Error in FalloutNV.esm
parsers.py 4068 load:  
Traceback (most recent call last):
  File "bash\parsers.py", line 4062, in load
    self.tops[label].load(ins, unpack and (topClass != MobBase))
  File "bash\record_groups.py", line 70, in load
    ins.tell() + self.size - self.header.__class__.size)
  File "bash\record_groups.py", line 172, in loadData
    record = recClass(header,ins,True)
  File "bash\brec.py", line 1627, in __init__
    self.__class__.melSet.initRecord(self,header,ins,unpack)
  File "bash\brec.py", line 1172, in initRecord
    MreRecord.__init__(record,header,ins,unpack)
  File "bash\brec.py", line 1422, in __init__
    if ins: self.load(ins,unpack)
  File "bash\brec.py", line 1485, in load
    self.loadData(ins,inPos+self.size)
  File "bash\brec.py", line 1636, in loadData
    self.__class__.melSet.loadData(self,ins,endPos)
  File "bash\brec.py", line 1201, in loadData
    loaders[Type].loadData(record,ins,Type,size,readId)
  File "bash\brec.py", line 740, in loadData
    self.loaders[sub_type].loadData(target, ins, sub_type, size, readId)
  File "bash\game\fallout3\records.py", line 105, in loadData
    raise BoltError(u'Unknown condition function: %d\nparam1: %08X\nparam2: %08X' % (ifunc,ins.unpackRef(), ins.unpackRef()))
BoltError: Unknown condition function: 586

 

EDIT3: Yep that's what's doing it, just verified it. At least I can fix it though.

Link to comment
Share on other sites

On 9/2/2017 at 4:49 PM, Sharlikran said:

> the _formVersion and _compressType attributes are only for FO4 and SSE - or TESV too ?

For Skyrim _ formVersion used, yes / _compressType used, no

For Skyrim SE _ formVersion used (In compressed Data), yes / _compressType used, Yes

For Fallout 4 _ formVersion used, yes / _compressType used, No

I have not looked at all the code yet but yes, what you are doing in the Skyrim block of code with def __is_sse(self):  looks like it will work fine. I agree, it won't matter that older Saves from 1.4.132 the version prior to the release of the Fallout 4 CK have version 12.  Because def __is_sse(self): isn't present in class Fallout4SaveHeader.

The comments below are how FO4 handles FormIDs at runtime.  So the next step is to get the Form Version for Skyrim SE also. Because when the Creation Club goes live they will (I don't think Maybe because they have to process the FormIDs), will change the save game format the way it is for Fallout 4. The _ formVersion is just before the size of the plugins (Still one ubyte) among the compressed data.

If you still can't see images.


  wbSaveHeader := wbStruct('Save File Header', [
     wbString('Magic', 13)
    ,wbInteger('Header Size', itU32)
    ,wbHeader
    ,wbByteArray('Hidden: Screenshot Data', ScreenShotDataCounter)
    ,wbInteger('Form Version', itU8)
    ,wbInteger('PluginInfo Size', itU32)
    ,wbArray(wbFilePlugins, wbLenString('PluginName', 2), -4)
    ,wbFileLocationTable
  ]);

Comments from Hlp on handeling FormIDs at Runtime:

  Hide contents

if IsESL (* as in flags bit 9 is set *) then begin
 baseID = formID and 111111111111b; (* yes that is only 12 bits *)
 fileID = formID or 11111110000000000000000000000000b; (* modIndex $FE *)
 eslID = eslLoadOrder shl 12
 formID = fileID + eslID + baseID;
 (* mmmmmmmmeeeeeeeeeeeebbbbbbbbbbbb *)
end else begin
 baseID = formID and 00000000111111111111111111111111b;
 fileID = formID or  11111110000000000000000000000000b; (* modIndex $FE *)
 (* mmmmmmmmbbbbbbbbbbbbbbbbbbbbbbbb *)
end

 

@Sharlikran re: saves thanks, will edit the code - I don't get what are those comments on formIds ? esl load order handling is what remains - the fact that the masters block in saves for esl files is separate maybe means their load order does not matter ?

 

On 9/2/2017 at 6:12 PM, Sharlikran said:

@Utumno Without suppressing the section of code that processes Script Extender CoSaves, (SKSE/F4SE/OBSE), Wrye will produce the following error because that code is still Oblivion specific. By that I mean it has hard coded 'OBSE' in quotes. Hlp (who works on the various script extenders) has told me CoSave structure, and I just have to verify it now.

This is the traceback of the error. What should happen is that the save button sort of greys out and then the screen refreshes. Instead the traceback appears, the Save button is still active as if you should click it again, and things don't refresh. Were you going to implement the changes to bosh/_saves.py, and bosh/__init__.py a different way?

  Reveal hidden contents

 



Wrye Bash starting
Using Wrye Bash Version 307
OS info: Windows-7-6.1.7601-SP1
Python version: 2.7.12
wxPython version: 2.8.12.1 (msw-unicode)
input encoding: UTF8; output encoding: None; locale: ('en_US', 'cp1252')
filesystem encoding: mbcs 
bash.py  324 main: Searching for game to manage:
bush.py   76 _supportedGames: Detected the following supported games via Windows Registry:
bush.py   78 _supportedGames:  Oblivion: G:\Games\steamapps\common\Oblivion
bush.py   78 _supportedGames:  Skyrim: F:\Games\steamapps\common\Skyrim
bush.py   78 _supportedGames:  Skyrim Special Edition: F:\Games\steamapps\common\Skyrim Special Edition
bush.py   78 _supportedGames:  Fallout4: C:\Games\steamapps\common\Fallout 4
bush.py  136 _detectGames: Detecting games via the -o argument, bash.ini and relative path:
bush.py  142 _detectGames: Set game mode to Fallout4 found in parent directory of Mopy:  C:\Games\steamapps\common\Fallout 4
bush.py  156 __setGame:  Using Fallout4 game: C:\Games\steamapps\common\Fallout 4
testing UAC
mods_metadata.py  228 __init__: Using LOOT API version: 0.10.2
games.py  158 warn_lo: Fixed Load Order: added(Ethreon_Master_Plan.esp), removed(EthreonMasterPlan.esp), reordered (No)
games.py  183 warn_active: Invalid Plugin txt corrected:
Active list contains mods not present in Data/ directory or corrupted: EthreonMasterPlan.esp

Traceback (most recent call last):
  File "bash\balt.py", line 436, in <lambda>
    if onButClick: self.Bind(wx.EVT_BUTTON, lambda __event: onButClick())
  File "bash\basher\__init__.py", line 1999, in DoSave
    saveInfo.write_masters()
  File "bash\bosh\__init__.py", line 1145, in write_masters
    obse.load()
  File "bash\bosh\_saves.py", line 340, in load
    raise FileError(self.name,u'File signature != "OBSE"')
bash.exception.FileError: Save68_A3AA8A36M4C696C6C79_Commonwealth_010652_20170415090643_16_2.f4se: File signature != "OBSE"

 


Especially with Fallout 4 if you are into Settlement Building you will need F4SE and a plugin called Place Everywhere because it allows you to precisely place things where you want them. I saw your requests for save games from Supierce so I understand you are testing this as well, but if you aren't using F4SE then keep in mind that the majority of people are for various reasons. Place Everywhere is just one, but some mods require F4SE for other reasons.

Also if you change if self.signature != 'OBSE': in _saves.py to if self.signature != bush.game.se.shortName: while the change is then valid, the code will now continue on from that point and read in the CoSave as it would for oblivion.  For SKSE and F4SE what happened when I was testing is that the CoSave header was changed from F4SE to OBSE. Which breaks the CoSave.  It might be a good idea to keep the doc strings for future reference. However, you might want to use the code in bosh/__init__.py or another alternative you feel comfortable with.

I know and understand all that - that's issue https://github.com/wrye-bash/wrye-bash/issues/236

There's a big fat BETA warning and I prefer it breaks than the users think their saves are edited when in fact they are not. Please ask hlp if he can write code or hlp (sic) me writing code for editing the rest of cosaves (SKSE, F4SE etc). We need proper classes etc and that's a lot of work I can't do now.

On 9/5/2017 at 8:17 AM, Sharlikran said:

Updated the FO3/FNV branch, added NSI Installer Info, Added Jip up to v48 and lutana NVSE Support (Thanks to @hlp), and Save Game updates.  It doesn't have the same amount of patchers as Valda's versions (which I still recommend) because of some limitations but it's working again.

EDIT: Nm, CTDA is failing in Fallout NV if I import records from Fallout 3.  I wish I know why, I was very careful of which I deleted and added them to the import list immediately so I wouldn't forget.  If I revert the changes CTDA works fine.

EDIT2: I think I know why. I can't import CTDA because the functions in constants will be different between FO3 and FNV. I don't seem to be able to import any classes that use CTDA because the error keeps saying fallout3.records for CTDA related patching, even when I'm not importing it.

  Reveal hidden contents

 



Wrye Bash starting
Using Wrye Bash Version 307
OS info: Windows-7-6.1.7601-SP1
Python version: 2.7.12
wxPython version: 2.8.12.1 (msw-unicode)
input encoding: UTF8; output encoding: None; locale: ('en_US', 'cp1252')
filesystem encoding: mbcs 
bash.py  324 main: Searching for game to manage:
bush.py   76 _supportedGames: Detected the following supported games via Windows Registry:
bush.py   78 _supportedGames:  Fallout3: C:\Games\steamapps\common\Fallout 3 goty
bush.py   78 _supportedGames:  Fallout4: C:\Games\steamapps\common\Fallout 4
bush.py   78 _supportedGames:  FalloutNV: C:\Games\steamapps\common\Fallout New Vegas
bush.py   78 _supportedGames:  Oblivion: G:\Games\steamapps\common\Oblivion
bush.py   78 _supportedGames:  Skyrim Special Edition: F:\Games\steamapps\common\Skyrim Special Edition
bush.py   78 _supportedGames:  Skyrim: F:\Games\steamapps\common\Skyrim
bush.py  136 _detectGames: Detecting games via the -o argument, bash.ini and relative path:
bush.py  142 _detectGames: Set game mode to FalloutNV found in parent directory of Mopy:  C:\Games\steamapps\common\Fallout New Vegas
bush.py  156 __setGame:  Using FalloutNV game: C:\Games\steamapps\common\Fallout New Vegas
testing UAC
mods_metadata.py  228 __init__: Using LOOT API version: 0.10.1
Unknown condition function: 586
param1: 00000000
param2: 00000000
Error loading 'ALCH' record and/or subrecord: 00175AAE
  eid = u'CookCooksFiendStew'
  subrecord = 'CTDA'
  subrecord size = 28
  file pos = 12546359
Error in FalloutNV.esm
parsers.py 4068 load:  
Traceback (most recent call last):
  File "bash\parsers.py", line 4062, in load
    self.tops[label].load(ins, unpack and (topClass != MobBase))
  File "bash\record_groups.py", line 70, in load
    ins.tell() + self.size - self.header.__class__.size)
  File "bash\record_groups.py", line 172, in loadData
    record = recClass(header,ins,True)
  File "bash\brec.py", line 1627, in __init__
    self.__class__.melSet.initRecord(self,header,ins,unpack)
  File "bash\brec.py", line 1172, in initRecord
    MreRecord.__init__(record,header,ins,unpack)
  File "bash\brec.py", line 1422, in __init__
    if ins: self.load(ins,unpack)
  File "bash\brec.py", line 1485, in load
    self.loadData(ins,inPos+self.size)
  File "bash\brec.py", line 1636, in loadData
    self.__class__.melSet.loadData(self,ins,endPos)
  File "bash\brec.py", line 1201, in loadData
    loaders[Type].loadData(record,ins,Type,size,readId)
  File "bash\brec.py", line 740, in loadData
    self.loaders[sub_type].loadData(target, ins, sub_type, size, readId)
  File "bash\game\fallout3\records.py", line 105, in loadData
    raise BoltError(u'Unknown condition function: %d\nparam1: %08X\nparam2: %08X' % (ifunc,ins.unpackRef(), ins.unpackRef()))
BoltError: Unknown condition function: 586

 

EDIT3: Yep that's what's doing it, just verified it. At least I can fix it though.

I would rather you not squash commits and add your changes in separate commits on top of the branch otherwise I will have to spend a lot of time to see what changed - nice you gave owneship to valda, so I know who to bug for questions etc.

You have notified me on github on various issues, please give me a 123 rundown of what's needed - I guess this CTDA  is pull request you made ?

 

EDIT: @Supierce and everyone using python wip - I made some last minute changes in install to drop a bunvch of system calls - should be faster but may break something - give a whirl to latest utumno-wip while I merge stuff

Link to comment
Share on other sites

Quote

I don't get what are those comments on formIds ?

I reworded my post.

Quote

There's a big fat BETA warning and I prefer it breaks than the users think their saves

I'm not asking to change the CoSave routines only bypass them except for Oblivion.

Quote

I would rather you not squash commits and add your changes in separate commits on top of the branch otherwise I will have to spend a lot of time to see what changed

Okay I'll redo it, and leave the rebase unchanged.

Quote

I guess this CTDA  is pull request you made?

I'll make one.

Quote

please give me a 123 rundown of what's needed

That's okay.

Quote

There's a big fat BETA warning and I prefer it breaks

I reworded my post.

I want to make changes and it's a shame that as I do they are met with incredulity. Some changes are needed and urgent like the Fallout 4 Save Game changes. Some changes aren't urgent but are useful and I'm finding it hard to have any equanimity toward how you respond to them. So for personal enrichment I will make changes I deem necessary for myself and if you choose to use them that's up to you. Although your compliance is optional of course, all I ask is that you cherry pick my commit as is and make your changes after as shown here.

Keep up the good work. Let me know when your refactoring is finished. I am particularly interested in getting the patchers working. I'm gonna go play Yugioh Duel links as I'm going to miss out on getting Yugi and a Kuriboh, or play some more ESO.

Link to comment
Share on other sites

Quick question: does Wrye Bash allow plugin ghosting for games other than Oblivion? IIRC it was added to work around the 500 plugin limit present in Oblivion, but by the time of Skyrim at least that was no longer present - I'm not sure about Fallout 3 or Fallout New Vegas though.

Link to comment
Share on other sites

4 hours ago, Utumno said:

EDIT: @Supierce and everyone using python wip - I made some last minute changes in install to drop a bunvch of system calls - should be faster but may break something - give a whirl to latest utumno-wip while I merge stuff

All good here so far!  :)

Link to comment
Share on other sites

3 hours ago, WrinklyNinja said:

Quick question: does Wrye Bash allow plugin ghosting for games other than Oblivion? IIRC it was added to work around the 500 plugin limit present in Oblivion, but by the time of Skyrim at least that was no longer present - I'm not sure about Fallout 3 or Fallout New Vegas though.

Not sure about FO3, but FNV limit is roughly 140 so I wouldn't be surprised.  So it definitely needs ghosting ability.  Mentioned earlier to Utumno, but he asked that we wait to remind him when they start on those patchers.

-Dubious-

Link to comment
Share on other sites

3 hours ago, WrinklyNinja said:

Quick question: does Wrye Bash allow plugin ghosting for games other than Oblivion? IIRC it was added to work around the 500 plugin limit present in Oblivion, but by the time of Skyrim at least that was no longer present - I'm not sure about Fallout 3 or Fallout New Vegas though.

Yep ghosting is supported for all games (built into ModInfos). I had recently people asking to be removed for all games apart TESIV but I just removed the pop up asking for enabling autoghost - would cause back compat issues and whatnot. Although it was a bad idea (adds to code complexity all over Bash, I keep fixing it the past few years)

 

@Sharlikran: incredulity ? Nope, just one thing at a time plus bugs - for instance disabling cosaves edits for all games except Oblivion, would simply not edit the cosaves for other games silently breaking those saves.

Plus reviewing those huge f3/fnv branches is a huge pain so I'd rather see changes one by one and issues addressed in order. I do need all help I can get, as in help ;)

  • Like 2
Link to comment
Share on other sites

Quote

would simply not edit the cosaves for other games silently breaking those saves.

With all due respect maybe you should test it with Skyrim or Fallout 4, not SSE.  Because when the traceback occurs, as I explained, then they are not edited... for any other games.

Who wants to test it?

Test 1: When the traceback occurs, the cosave isn't changed.

Steps:

  1. Make sure you have Utumno's wip branch installed.
  2. Go to the data folder for either the original Skyrim or Fallout 4 and change the name of a file. Add or remove an underscore for example, EthreonMasterPlan.esp -> Ethreon_Master_Plan.esp.
  3. Go to your save games folder and backup the save game and its cosave you wish to test with. It must have an accompanying Script Extender file. For example .skse, .f4se.
  4. Open Wrye using Wrye Bash Debug.bat, wait for it to tell you it corrected the plugin list, go to the saves tab.
  5. Highlight the save game to edit, on the right enable "Allow Edit" for save games (unless enabled), highlight the file in the save header that you renamed in your data folder.
  6. Right click the file, choose "change to" and assign it the renamed file.
  7. The save button should become active, click it.

What I explained in my previous post will occur and the traceback will be written into the log. Open the CoSave (Hex editor preferred but even Notepad++ will work) and you will notice it will still start with either SKSE or F4SE. The file has not been edited because neither of the header signatures will be equal to OBSE, which is what the code is looking for. To verify that this is truly what is happening, you have to perform the next test.

Test 2: Change the signature you are looking for and Wrye will attempt to alter the file.

  1. Go to the Mopy folder for the game you are testing with and open Mopy\bash\bosh\_saves.py
  2. change line 339 if self.signature != 'OBSE': to if self.signature != bush.game.se.shortName:
  3. Restore your save game and the cosave form you backup.
  4. Perform all the steps from Test 1

Now open the cosave file (.skse or .f4se) and now Wrye Bash has replaced SKSE or F4SE with OBSE. This time it attempted to change the file because it passed the error check. By doing both steps it shows that unless you are using Wrye with Oblivion the error will occur and the cosave is unaltered. There is no reason not to bypass the routine for now.

If you did this for oblivion you will notice that when you rename the file in the save the filename in the cosave isn't renamed. That's because currently writeMasters records the list of oldMasters and passes that to the cosave routine. Which as I explained some time ago doesn't make sense. If it was never the intention to change the filename in the cosave, then why have a routine at all that reads in the cosave and writes it back out.

In the words of George Carlin, "Amusement? Boredom? Doesn't sound like divine planning to me, sounds like human planning."

gee-wiz-batman-ive-never-heard-of-that-b

Now what will really blow your mind is that the file is a binary equal of the original file except for the signature being changed from SKSE to OBSE.  Check it, use a dif program.  Imagine that! Since the headers are the same from what I can tell looking at some files and reading some xEdit code, the routine would work for other games as is with minor alterations. I also bet you if I passed out the new masterlist it would change the cosave plugin names correctly. Isn't that swell! Golly gosh gee whiz Batman!

Link to comment
Share on other sites

I have a few questions:

1. is this functionality something that existed in WB that suddenly stopped working, or is this an Oblivion-only feature that is missing from Skyrim and later? 

2. Is this directly related to the CC and the new FO4 saves?

2. How much of the user base uses this functionality?  Is this a "several people are temporarily inconvenienced" a "Hannibal ante portas" issue?  In other words, can this wait until Beta 3?

3. Since Utumno has all of his man hour bandwidth consumed at the moment with prepping Beta 2 for release (with the CC stuff adding to the complexity), is there anyone else that could do the change? 

 

Link to comment
Share on other sites

@Beermotor

1. simply put this is the "Edit Save Masters functionality" -> if save has cosave the cosave masters must be edited, but this is not the case - for no game - for oblivion due to an if, for the rest cause there is no code - see #266

2. No

2. Doesn't matter

3. Hopefully - we need someone familiar with ScriptExtender cosave formats and a good deal of save handling refactoring - saves refactoring so we can extend code to handle all games like oblivion is major goal for 308 - hopefully @nycz who's a saves master and a pythonista will lend a hand

@all -> roll up your sleeves (wait for upload!):

307.201709071619

alpha esl, scandir, new saves etc support - basically we need more info on load order handlin g of esls and wait for SSE update - then beta

Link to comment
Share on other sites

5 hours ago, Utumno said:

The correct solution is to edit the cosaves - now who's to write the code ?

I have tested this with 3 of the six games so far. I'm just looking for some other script extender plugins right now to do some more thorough testing.  I know you want the entire code refactored into functions and classes, but simply put, if the headers are exactly the same for every game's cosave, then I don't see the need to refactor that at this time. 

Besides there is no Wiki like Zilav and I linked you for the 24 bytes in the record header. If you ask Hlp he will be happy to tell you the format, but it would be a brief C++ block of a scruct something like the templates I linked you already for 010. So this is something you have to see and actively perform steps to test it and steps to intentionally break it to ensure that everything happens as it should, exactly the way it should. I use both 010 and Beyond Compare to test the files before and after.

If I can get an extra OBSE, SKSE, and F4SE plugin and test with two or more script extender plugins, and the cosaves are binary equals every time, then the routine is fine, and just needs bush.game.se.shortName in two places.

I was working on Fallout 4 records but I might work on this instead, or bounce back and forth between them.

Link to comment
Share on other sites

1 hour ago, Utumno said:

@Beermotor

alpha esl, scandir, new saves etc support - basically we need more info on load order handlin g of esls and wait for SSE update - then beta

Oy that was before my coffee. I didn't even notice my numbering. :P

Quick question regarding scandir: for those of us on python installs, is it necessary to do a 'pip install scandir' or is this built-in now?

Link to comment
Share on other sites

> if the headers are exactly the same for every game's cosave, then I don't see the need to refactor that at this time. 

I agree - let's see

> Quick question regarding scandir: for those of us on python installs, is it necessary to do a 'pip install scandir' or is this built-in now?

Yep, python.exe -m pip install --upgrade scandir

Link to comment
Share on other sites

@UtumnoThat entire block of code is probably overkill. This is all that is needed, period.

Spoiler

 


// CHUNK, struct chunks found in script extender cosave
typedef struct {
    char  chunkType[4];
    uint  chunkVersion;
    uint  chunkLength;
    if (chunkLength != 0)
        char    data[chunkLength];
} CHUNK;

struct FILE {
    struct HEADER {
        char magic[4] <name="Signature">;
        uint formatVersion <name="Format Version">;
        ushort obseVersion <name="OBSE Version">;
        ushort obseMinorVersion <name="OBSE Minor Version">; 
        uint oblivionVersion <name="Oblivion Version">;
    } header <open=true, name="Header">;

    uint numPlugins <name="Number of Plugins">;

    struct chunkBuff {
        uint opcodeBase;
        uint numChunks;
        uint pluginLength;
        CHUNK chunkBuff[numChunks] <open=true, optimize=false, name="chunkBuff">;
    } chunks <open=true, name="Chunk">;

} file <open=true, name="Script Extender Save Game">;

 

That will read to the end of the file. I have tested that with FNV, FO4, TES4, TES5. There is no SKSE for Skyrim Special Edition. Actually I have to redo that but it's close.

Link to comment
Share on other sites

10 minutes ago, Sharlikran said:

@UtumnoThat entire block of code is probably overkill. This is all that is needed, period.

  Hide contents

 



// CHUNK, struct chunks found in script extender cosave
typedef struct {
    char  chunkType[4];
    uint  chunkVersion;
    uint  chunkLength;
    if (chunkLength != 0)
        char    data[chunkLength];
} CHUNK;

struct FILE {
    struct HEADER {
        char magic[4] <name="Signature">;
        uint formatVersion <name="Format Version">;
        ushort obseVersion <name="OBSE Version">;
        ushort obseMinorVersion <name="OBSE Minor Version">; 
        uint oblivionVersion <name="Oblivion Version">;
    } header <open=true, name="Header">;

    uint numPlugins <name="Number of Plugins">;

    struct chunkBuff {
        uint opcodeBase;
        uint numChunks;
        uint pluginLength;
        CHUNK chunkBuff[numChunks] <open=true, optimize=false, name="chunkBuff">;
    } chunks <open=true, name="Chunk">;

} file <open=true, name="Script Extender Save Game">;

 

That will read to the end of the file. I have tested that with FNV, FO4, TES4, TES5. There is no SKSE for Skyrim Special Edition. Actually I have to redo that but it's close.

010 ? What's opcodeBase (for that matter what is chunk) ? What is " That entire block of code" ? (link me to github please)

Also you have in a comment:

# _compressType prevent Skyrim SE saves from editing - currently unused

what does it mean "prevent Skyrim SE saves from editing"?

Link to comment
Share on other sites

opcodebase is the unique identifier for a xSE plugin written for games before Skyrim. It is the opCode of the first function provided by the plugin.

A chunk is an unit of information written to the cosave at once.

 

Link to comment
Share on other sites

@Utumno The 010 Editor, is the name of the program. You can edit anything, and 010 provides Professional text and hex editing with Binary Templates technology. Which I linked to a 30 day trial in this post. The first time Jon posted he was using 010, I didn't know what it was either. So that's a valid question. However, you have to have read all my previous posts. :innocent:

Quote

link me to github please

To open a CoSave and see what I am doing, you have to download the 010 Editor program, and apply the template. I uploaded the template to a fork I have of someone's template for Fallout 4.  However, you can only use that once you have 010 downloaded and installed. That is if you choose to download 010 at all.

@hlp several questions,

1) I am noticing that for Skyrim and above opcodebase is always 0, and for the Oblivion and Fallout New Vegas Save it's 5120 or 0x00001400. What opcode? An OBSE or NVSE code like used in CTDA? From what I am seeing, the opcode might matter to the script extender but it isn't anything Wrye Bash needs to be looking for.

2) In Wrye Bash, in class ObseFile, function def mapMasters: has if opcodeBase == 0x2330: such that if the value is equal to that, then do something, else do something else. So correct me if I'm wrong, but isn't like like saying if you find in Hex 30 23 00 00 or, flipped around 00002330 then perform the action?  Because if I'm correct then the action is only performed when the opcodeBase is 9008, which will never by either of my Oblivion or Fallout New Vegas save games.

3) I see the number of plugins and so far that number is always 1.  If I have 2 NVSE or OBSE plugins (Like NVSE and JIP), will I have one large Chunk for each xSE plugin (dll file), or is the count of plugins just an integer? From what you are saying, if I had two plugins would I end up with a list of File Names from the Data folder in two seperate chuncks, and I would even have opcodebase twice, is that correct?

4) The other observation I have made is that the first chunk is SDOM. Not only is it always first, it is always the list of files in the data folder. Is that Signature reserved for the for the list of files in the Data folder?

EDIT: I installed JIP along with YUP - Base Game + All DLC.esm and Unofficial Patch Plus.esp.  With those installed, I still have 1 for the number of NVSE plugins, I will look for a mod that uses JIP.

EDIT2: Jiminy Christmas, when opcodeBase == 0x2330 that's for Pluggy which is in class SaveFile, function def logStatObse.

 

Link to comment
Share on other sites

6 hours ago, Utumno said:

@all -> roll up your sleeves (wait for upload!):


307.201709071619

alpha esl, scandir, new saves etc support - basically we need more info on load order handlin g of esls and wait for SSE update - then beta

Yaaas! give me to break it! yum. :bunnytroll:

New behaviour spotted : Running a complex BAIN with wizard, on completion of the Wizard we now get a question from windows asking if a folder not previously installed should be created .. Is this now by design ?

000.png

The new Scandir is a vast improvement

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...