Jump to content

Wrye Bash - All Games


Utumno

Recommended Posts

Just made a little change to how the files are presented. Made the Installer the main event, put the other two as optional .. And also added a small description to each file because I think for newbies presented with all three files as "Main files", its confusing as to which file to grab on Nexus. Small descriptions now read as follows :

Main Files

Wrye Bash 307 Beta2 - Installer
Executable automated Installer. This will by default install just the Standalone Wrye Bash. It can also install all requirements for a full Python setup if you have any plans to join in with development.

Optional Files

Wrye Bash 307 Beta2 - Python Source
This is a manual installation of Wrye Bash Python files, requiring the full Python setup files to also be manually installed first.

Wrye Bash 307 Beta2 - Standalone Executable
This is a manual installation of the Wrye Bash Standalone files.

Old files

File boneyard of previous versions

 

Edit : On Oblivion Nexus Optional Files also includes :

Wrye Bash 291 - Python Source
This file is the last version of Wrye Bash to support Bash Mon, none of the more recent or future versions will support it. If you have a need for Bash Mon, then this is the version you need to use. See issue #369 for more information. If you dont know what Bash Mon is .. You dont need this version.

Link to comment
Share on other sites

Added a new sticky note on Skyrim Special Edition Wrye Bash .. :) Sounds like your new warning ref record version 44 is catching a few, to which I am redirecting them to the special "Mod Author" and Arthmoors converting to SSE topic.

-----------------------

On Oblivion Nexus, anonimousone has been having trouble with some of his BAINs being grey. He understands what that means, but cannot figure out why, so got him to produce a BashBugDump :

Wrye Bash starting
Using Wrye Bash Version 307.201612302300 (Standalone)
OS info: Windows-7-6.1.7600-SP0
Python version: 2.7.12
wxPython version: 2.8.12.1 (msw-unicode)
input encoding: None; output encoding: None; locale: ('it_IT', 'cp1252')
filesystem encoding: mbcs 
Searching for game to manage:
bush.pyo   71 _supportedGames: Detected the following supported games via Windows Registry:
bush.pyo   73 _supportedGames:  oblivion: d:\Oblivion
bush.pyo  131 _detectGames: Detecting games via the -o argument, bash.ini and relative path:
bush.pyo  137 _detectGames: Set game mode to oblivion found in parent directory of Mopy:  d:\Oblivion
bush.pyo  172 setGame: No preferred game specified.
bush.pyo  152 __setGame:  Using oblivion game: d:\Oblivion
mods_metadata.pyo   40 <module>: Failed to import the loot_api module: (MemoryLoadLibrary failed loading loot_api.pyd)
testing UAC
Traceback (most recent call last):
  File "bash\balt.pyo", line 2435, in __Execute
  File "bash\balt.pyo", line 1563, in _conversation_wrapper
  File "bash\basher\installer_links.pyo", line 571, in Execute
  File "bash\bosh\__init__.pyo", line 4968, in listSource
  File "bash\bosh\__init__.pyo", line 5180, in _list_package
  File "bash\archives.pyo", line 194, in list_archive
  File "subprocess.pyo", line 703, in __init__
  File "subprocess.pyo", line 870, in _get_handles
  File "subprocess.pyo", line 885, in _make_inheritable
WindowsError: [Error 6] Handle non valido
Traceback (most recent call last):
  File "bash\balt.pyo", line 2435, in __Execute
  File "bash\balt.pyo", line 1563, in _conversation_wrapper
  File "bash\basher\installer_links.pyo", line 571, in Execute
  File "bash\bosh\__init__.pyo", line 4968, in listSource
  File "bash\bosh\__init__.pyo", line 5180, in _list_package
  File "bash\archives.pyo", line 194, in list_archive
  File "subprocess.pyo", line 703, in __init__
  File "subprocess.pyo", line 870, in _get_handles
  File "subprocess.pyo", line 885, in _make_inheritable
WindowsError: [Error 6] Handle non valido
Traceback (most recent call last):
  File "bash\balt.pyo", line 2435, in __Execute
  File "bash\balt.pyo", line 1563, in _conversation_wrapper
  File "bash\basher\installer_links.pyo", line 571, in Execute
  File "bash\bosh\__init__.pyo", line 4968, in listSource
  File "bash\bosh\__init__.pyo", line 5180, in _list_package
  File "bash\archives.pyo", line 194, in list_archive
  File "subprocess.pyo", line 703, in __init__
  File "subprocess.pyo", line 870, in _get_handles
  File "subprocess.pyo", line 885, in _make_inheritable
WindowsError: [Error 6] Handle non valido
Traceback (most recent call last):
  File "bash\balt.pyo", line 2435, in __Execute
  File "bash\balt.pyo", line 1563, in _conversation_wrapper
  File "bash\basher\installer_links.pyo", line 571, in Execute
  File "bash\bosh\__init__.pyo", line 4968, in listSource
  File "bash\bosh\__init__.pyo", line 5180, in _list_package
  File "bash\archives.pyo", line 194, in list_archive
  File "subprocess.pyo", line 703, in __init__
  File "subprocess.pyo", line 870, in _get_handles
  File "subprocess.pyo", line 885, in _make_inheritable
WindowsError: [Error 6] Handle non valido
 

-----------------------

The only other problem is Hallodria on Old Skyrim Nexus, who cant get Wrye Bash to start (for a long while, so is understandably a bit frustrated), but is not providing much in the way of detailled info to help us help him / her. I suspect the help in the sticky post has not been quite adhered to, and the problem could be solved easily if I was sitting in Hallodria's seat.

Link to comment
Share on other sites

I can't help Hallodria I guess more than you can - anonimousone: he should try the python version, it seems that subprocess may fail with py2exe we use for the standalone. The failure is when he is trying to list the archives - now why they are grey... probably just invalid ?

Thanks !

 

EDIT: or wait till I compile a new standalone - err, next year :D

Link to comment
Share on other sites

@Utumno

I've pulled your latest beta 2 commits into my local repository, rebased locally, then force push to my remote warmfrost85-250-profiles branch. Unfortunately the force rebase commit doesn't have your commit and line comments. That makes sense but I didn't realize that would happen. I don't think there is a way to avoid this because of the rebase.

I can still see your commit comments in the issue 250 tracker log but that's not a good way to handle it. https://github.com/warmfrost85/wrye-bash/commit/dad8e05de8080e11a49e16bcb7a0d4d742ed0a82  I removed the '#250' wording from the commit message to avoid cluttering the issue 250 log history every time I do a rebase, but apparently there are advantages to recording there. :)

Is this an unavoidable price of rebasing?

 

  • Like 2
Link to comment
Share on other sites

Wow so I'm gone for a couple of days and so much has happened! I'm going to use my supply of exclamation points up.

First of all Merry Christmas everyone!

@Utumno Congratulations on the release!

@alt3rn1ty I'm going to make a post on reddit in a moment announcing the release with some cursory release highlights.  I'll probably add this to the Nexus page as well.

@warmfrost85 Wow the profile system is super impressive. Kudos for this. I'm assuming by junction points you mean the NTFS version of soft links or symlinks rather than hard links, correct?

 

Link to comment
Share on other sites

Thanks everybody - will be off a couple days now :P

6 hours ago, warmfrost85 said:

@Utumno

I've pulled your latest beta 2 commits into my local repository, rebased locally, then force push to my remote warmfrost85-250-profiles branch. Unfortunately the force rebase commit doesn't have your commit and line comments. That makes sense but I didn't realize that would happen. I don't think there is a way to avoid this because of the rebase.

I can still see your commit comments in the issue 250 tracker log but that's not a good way to handle it. https://github.com/warmfrost85/wrye-bash/commit/dad8e05de8080e11a49e16bcb7a0d4d742ed0a82  I removed the '#250' wording from the commit message to avoid cluttering the issue 250 log history every time I do a rebase, but apparently there are advantages to recording there. :)

Is this an unavoidable price of rebasing?

 

Yep it's a different commit (as in sha).

The neatest solution is:

- discuss technical stuff in the related issue, avoid commit comments.

- when writing a commit comment, mention the issue - I would guess that github anchors the commit if it has a comment (I really tried to find docs)

- when writing a commit message (in my local branches, usually utumno-wip, that I keep rebasing and force pushing), what I do is I leave a space - so "Under # 250" - I 99% remember to edit the message before final merge

Link to comment
Share on other sites

1 hour ago, Beermotor said:

 

@warmfrost85 Wow the profile system is super impressive. Kudos for this. I'm assuming by junction points you mean the NTFS version of soft links or symlinks rather than hard links, correct?

 

@Beermotor My current version only uses hard linked files. It doesn't use NTFS junctions. That tool makes it easy to hard link all files in a folder and its subfolders... done in one command. The ideal version uses junctions to reference folders containing hard linked files! This allows the user to create a nearly unlimited number of profiles with almost no impact on disk storage. When you can create a snapshot of your profile (mod setup) at any point in time (without worrying about filling your hard drive), it alters how you approach modding. In fact if you can create a 'backup' profile at any time is there a need for the settings backup and restore functionality? I'm sure I just riled up some one with that statement. :) After all a backup is just a snapshot of the user's setup at a given point in time. That can be achieved using a settings backup archive or a backup profile, depending on the definition of profile. To restore a profile from 'backup' is just a matter of switching to the backup profile. Here I mean a profile with a suffix " (backup)" or some other descriptive wording. I am well aware that I have (and always will have) very limited knowledge of WB and can easily waste peoples time discussing things that are not possible. However, I view my lack of knowledge of WB as an asset as I'll ask questions about things that others may take for granted.  There are always legacy issues, conversion issues, etc. so this may not be possible. Although it works great for my needs and I plan to keep it for my personal use.

Since hard links only work for files on a single drive, setups where the game and save folders are on different drives may be the roadblock to adopting a single profile system. It might be possible to work around that limitation but I didn't explore a solution since I was concentrating on MY needs. :)

Utumno is correct. We need a definition of "profile".  I make a distinction between data associated with a profile and data associated with the game (warning, gray areas ahead). For example, to me cli arguments are game data not profile data. After Christmas, I'll submit my definition of 'Profile'.  What's your definition/characteristics/properties of the ideal profile?

Link to comment
Share on other sites

Just stopping by to say congrats to @Utumno and everyone involved in the development of Wrye Bash for the release.

Also! Happy Holidays, of course! It's been a great few weeks for modding and things are only looking better.

Link to comment
Share on other sites

Congrats on the release! I look forward to the future improvements as well. I've been loving using it as a mod manager for fallout 4 and Skyrim. Is there any chance that support could be added for Fallout 4 VR in the future?

Link to comment
Share on other sites

7 minutes ago, warmfrost85 said:

@Beermotor My current version only uses hard linked files. It doesn't use NTFS junctions. That tool makes it easy to hard link all files in a folder and its subfolders... done in one command. The ideal version uses junctions to reference folders containing hard linked files! This allows the user to create a nearly unlimited number of profiles with almost no impact on disk storage

Awesome thank you for clarifying. I was just curious after skimming over the code and reading the commit comments. It looks very compelling and sounds promising. 

20 minutes ago, warmfrost85 said:

I am well aware that I have (and always will have) very limited knowledge of WB and can easily waste peoples time discussing things that are not possible. However, I view my lack of knowledge of WB as an asset as I'll ask questions about things that others may take for granted.  There are always legacy issues, conversion issues, etc. so this may not be possible. Although it works great for my needs and I plan to keep it for my personal use.

I'm in the same boat:  I know how Bash works from using it for almost a decade, but I still haven't completely wrapped my head around the entire codebase, then again my Python-fu is fairly weak compared to you guys.  I still have to pester Utumno now and again to explain things. I agree with you that is not really a detriment as it doesn't cloud your vision and engenders new ideas.  You can write the code to explore those ideas so all the better.

26 minutes ago, warmfrost85 said:

Since hard links only work for files on a single drive, setups where the game and save folders are on different drives may be the roadblock to adopting a single profile system. It might be possible to work around that limitation but I didn't explore a solution since I was concentrating on MY needs. :)

The way I look at it is you want to encourage having the assets loaded by the game on the same drive as the executable. This is one of the myriad Achilles Heels of the MO/Vortex approach. Performance is going to suck if your mod assets are on another drive (even if that other drive is a SSD) due to overhead from the I/O operations having to interrupt the CPU and then transfer all of that data across the bus. You could kind of negate that by having two NVMe drives, but I'm digressing.

To wax philosophical for a moment, no need to apologize for making software suit your needs. Especially if you think it will be helpful for other people. That's the entire point of open source. :)

 

43 minutes ago, warmfrost85 said:

Utumno is correct. We need a definition of "profile".  I make a distinction between data associated with a profile and data associated with the game (warning, gray areas ahead). For example, to me cli arguments are game data not profile data. After Christmas, I'll submit my definition of 'Profile'.  What's your definition/characteristics/properties of the ideal profile?

 Yes I agree completely.  I've put some thought into it in the past but want to think about it some more. I think input from the community would be fantastic as well.

Keep up the good work. I really think you're on to something good. :beer:

Link to comment
Share on other sites

Another concern that just occurred to me it's that actually editors, more often than not, do not respect hard links, and windows does not propagate file attributes correctly (oh yes, the oblivion CK for certain deleted and recreated the file, breaking the not so hard link : https://superuser.com/questions/306548/windows-7-mklink-hard-links-attributes-expected-contents-do-not-affect ). I do am very very reluctant to use filesystem operations like those without at least a well defined test suite, yes tests. Another thing that does worry me is non portability - and it worries me as an engineer, non portable code tends to be hacky, always.

To the problem number one, junctions are immune provided the junctioned folder has real files, hard link issues apply inside the folder. And the non propagation of modification time/ size will jeopardize Bash caches - subtle bugs, show stopper kind.

 

Re backup/restore - I tackle them cause from my point of view (and I do hope that despite what you say you won't always have very limited knowledge of WB) that's step number one to approach "profiles" issue - including defining what profile means. And a word of caution: I have seen many, many head on attacks on Bash - trying to replace existing components from scratch - they all failed. Hey, its 100k lines of python and some of it is really dense.

Having rewritten most of it, not only I know the code (not all mind you) but I have the experience of how 100k can operate together, and be maintainable and extendible - unfortunately this is knowledge that I can't convey, except by coding. There were and still are many hacks in the code that initially (seemed to) work well, but I kept fixing for years and years after

Don't get me wrong, I am interested in the idea, not riled at all and people, do try those out. Once the dust has settled and I find some time myself we can continue the discussion in the tracker with specific code issues to solve.

Btw re: fresh view that's indeed a plus at times - I plan some backwards incompatible settings changes for 308 and that's an opportunity to refactor the whole settings (aka profiles) thing

Meanwhile would you like to contribute in specific tasks  @warmfrost85 ? There's for all tastes but I can give you stuff that's related to game handling and settings handling - this way you will get a clearer view of what Bash settings are and how they operate.

Link to comment
Share on other sites

Yes, during play testing rarely, but occasionally, I did experience what seemed like a hard linked file timestamp not updating so WB continued to think the file was "mismatched".  Rather than spend a lot of time investigating I just copied the file so it was "new" then reinstalled it via WB to continue my game!! Those kind of issues can be a real sink hole, difficult to support with different versions of windows, file editors, etc. Doing something for personal use is much easier than for general use. I also agree some editors change a file (preserving the hard link) and others save the file as new (breaking the hard link). Breaking the hard link is not issue for me. Not updating a file timestamp is a big deal and outside of my control. This type of profiling may not be compatible for general use which would be a big shame because it's very helpful. I'm not giving up yet. :)

Because of those hard linked issues I consider it an advanced feature only for someone willing to spend the time on it. Once such files are identified, the user can just unlink the affected files and all is well. Maybe the question is: Should WB support two profiling systems: one for the general user (current system) and one for advanced/adventurous users (hard links)?

A junction pointing to a folder of real files would solve the above issues but it would be VERY slow to create a new profile from an existing profile (mod list) and consume A LOT of disk space so that seems to be a no go.

My motivation in contacting the team was to share some code which might be helpful. I would work on it until it was considered viable (not just dump it on you @Utumno).  It works so well for my personal needs that I wanted to share with others who may find value in it.  I've been exclusively using WB for over a year now and have fallen in love with the control it offers. Although to be honest I'm starting to get Skyrim burnout.

Yes, if you have tasks that relate to profiles, I'll take a look and then decide.

 

BTW: I've started looking into creating a FOMOD Installer, well actually an FOMOD Unpacker for personal use. From what I read in the forums, there's probably a greater need for a FOMOD Unpacker than profiles. A Fomod archive is easy to install  using NMM. There was no alternative for the profiling system I wanted so I had to write my own.  I'll continue poking around with a fomod unpacker but that's a BIG task which I may never finish. :(  Just curious, is a fomod unpacker on the road map anywhere?  Don't want to step on more toes. haha

  • Like 1
  • Thanks 2
Link to comment
Share on other sites

You're not stepping on anybody's toes and as I said I am curious about your code (any real Bash code for that matter) and feedback from users.

The timestamp and hardlink issues are essential however - as you said doing something for personal use is much easier than for general use. Imagine now people coming to forums with such WB continued to think the file was "mismatched"  impossible to repro bugs - end of show. Handling forum bug reports is a full time job and I am really grateful people here have taken it off me - would be impossible for me to attend to anyway.

That's an interesting coding problem to me (and an order of magnitude harder than FOMODs, which I have not looked at ever - but yes a nice module that could do that, with tests, would be a great addition) - I fully understand the pros and cons of hardlinks vs copying - I am not satisfied with either approach. Needs thought

 

Meanwhile please have a look at #390 - that's the restore settings issue. I will post some pre alpha code that shows how settings are loaded and mention you

 

EDIT: we have something similar that's not to imitate exactly for OMODS: https://github.com/wrye-bash/wrye-bash/blob/aedcca199fcb0bf96299ce7f36842df6f74e98f3/Mopy/bash/bosh/omods.py

 

 

Link to comment
Share on other sites

1 hour ago, warmfrost85 said:

BTW: I've started looking into creating a FOMOD Installer, well actually an FOMOD Unpacker for personal use. From what I read in the forums, there's probably a greater need for a FOMOD Unpacker than profiles. A Fomod archive is easy to install  using NMM. There was no alternative for the profiling system I wanted so I had to write my own.  I'll continue poking around with a fomod unpacker but that's a BIG task which I may never finish. :(  Just curious, is a fomod unpacker on the road map anywhere?  Don't want to step on more toes. haha

FOMOD support is actually something that we've seen a massive amount of requests for. Fortunately there are already resources available out there from @Ganda with his PyFOMOD module (API docs are here) and some of the other tools he has on Github. He has been looking at CBash lately but I'm sure he would be happy to work with you if you wanted to take a crack at it. 

  • Like 2
Link to comment
Share on other sites

19 minutes ago, Beermotor said:

FOMOD support is actually something that we've seen a massive amount of requests for. Fortunately there are already resources available out there from @Ganda with his PyFOMOD module (API docs are here) and some of the other tools he has on Github. He has been looking at CBash lately but I'm sure he would be happy to work with you if you wanted to take a crack at it. 

Of course, I'm more than happy to spare someone else the nightmare that is working with fomod (although I hope I made it slightly better). And as I've told Beermotor, I'd rather not get involved with ui (I'm pretty terrible at it), so no toe stepping :P 

I'm quite busy until mid-february, so replies might take a while, but aside from that I'll help out!

  • Like 3
Link to comment
Share on other sites

Wow, looks like Santa has given away some of his helpers this year, prospects for the New Year and development has just gone up a few notches, bugs beware :run:

--------------------

17 hours ago, Lazybob1 said:

Congrats on the release! I look forward to the future improvements as well. I've been loving using it as a mod manager for fallout 4 and Skyrim. Is there any chance that support could be added for Fallout 4 VR in the future?

Probably needs people with the hardware to test on, so it would depend upon developers buying into the idea.

Link to comment
Share on other sites

On 12/24/2017 at 11:48 AM, Utumno said:

I can't help Hallodria I guess more than you can - anonimousone: he should try the python version, it seems that subprocess may fail with py2exe we use for the standalone. The failure is when he is trying to list the archives - now why they are grey... probably just invalid ?

Thanks !

 

EDIT: or wait till I compile a new standalone - err, next year :D

 

OK, but just FYI 4 more reports of people who cant start Oldrim with a Bashed Patch enabled. The thought that the bashed patch after building it and patching mods has been eliminated because after telling them to delete the bashed patch and get Bash to generate a new one, ff7legend has reported back with it still being a problem, and he is quite experienced using Wrye Bash.

 

I have asked them to try the suggestions in the sticky post, and generate BashBugDump.logs if Wrye Bash can do that before the Splash screen and the CTD they are getting at that point, meanwhile I am in the process of installing Oldrim again to see if I can repro any of this.

Its only 5 people (also including Hallodria from yesterday) out of a thousand unique downloads at the moment for oldrim, so I suspect probable issues with old database / settings which the troubleshooting should sort out for them if they follow through correctly on the instructions.

 

Edit : I cant repro the Splash screen CTD after installing standalone for all games now including Oldrim

oJTIvi7.png

 

Told them in the comments on Nexus, so hopefully its just a case of old settings or something we cant see remotely.

Link to comment
Share on other sites

@Beermotor Your summary of the Beta2 update post is nice .. It will roll off the front page of comments eventually so I have copied it into a spoiler in the sticky on each Nexus page for Wrye Bash.

Right, back to the turkey, savoury sausage stuffing and sprouts preparation for me before Mrs Alt kicks my arris :)

Edit : @Utumno Should we put Beermotor on the Admin access list for Nexus pages ?

If yes, @Beermotor We will need your Nexus ID number - Go to your Name top right on nexus when logged in, from the drop down choose "User Area". On the User area, next to your name top middle will be your ID number, which we can add to the Mod which would give you editing / uploading access and sticky post editing abilities .. Just guard against having your account taken over on Nexus by keeping abreast of warnings on changing passwords as necessary ( its also possible by downloading specially crafted files from Nexus, so also watch for First time Uploaders and whether you trust their files or not )

Link to comment
Share on other sites

2 minutes ago, alt3rn1ty said:

@Beermotor Your summary of the Beta2 update post is nice .. It will roll off the front page of comments eventually so I have copied it into a spoiler in the sticky on each Nexus page for Wrye Bash.

Right, back to the turkey, savoury sausage stuffing and sprouts preparation for me before Mrs Alt kicks my arris :)

Excellent. I wasn't sure if you could make my post sticky or not so that's fantastic. I posted the same message to Reddit /r/skyrimmods and I'm about to post on S.T.E.P. and LL as well. I probably need to hit a couple of others as well.

I'm waiting on video to import to YouTube for the in-laws so I've ducked out for a few moments myself. We're having breakfast of pastries, eggs,  and Conecuh sausage. :)

The brown-sugar glazed ham is later.

Happy Christmas!

Link to comment
Share on other sites

3 hours ago, alt3rn1ty said:

If yes, @Beermotor We will need your Nexus ID number - Go to your Name top right on nexus when logged in, from the drop down choose "User Area". On the User area, next to your name top middle will be your ID number, which we can add to the Mod which would give you editing / uploading access and sticky post editing abilities .. Just guard against having your account taken over on Nexus by keeping abreast of warnings on changing passwords as necessary ( its also possible by downloading specially crafted files from Nexus, so also watch for First time Uploaders and whether you trust their files or not )

Gotcha. I remember you telling me a little about this previous but good to know. I'll PM it to you.  :highfive:

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...