[WIPz] TES5Edit

[DayDreamer]

We've known for a long time that the Finalize can also make gratuitous changes to the 4 cells surrounding the current cell, in a plus (+) shape. This is the most common problem causing adjacent plugins to conflict with each other....
 

Now that we have decoded the link table, it is very clear that these are dirty edits. We need to have a script that sorts the dirty edit back into its master order, so that the cleaning filter will recognize them as identical -- or we need to program the filter to recognize them as identical.
 
To write a sorter, how do I ask for the next level down? Assume we run the script on the plugin, so Process(e) is giving me the plugin's NAVM. What is the inverse function of WinningOverride()?

[zilav]

It is a style of modern Delphi, not mine

There are several functions to work with overrides

Master(e) - returns a master record for override or nil if a record is already a master

MasterOrSelf(e) - same as previous but returns itself instead of nil

WinningOverride(e) - get the last override or itself

IsMaster(e) - true if master

IsWinningOverride - true if last override

OverrideCount(e) and OverrideByIndex(e, idx) - returns overriding record by index for a master record, must be called on MASTER record only, so usage of MasterOrSelf is mandatory. Calling them on override does nothing.

 

As you can see, there are no functions to get next or previous override from the current record, only the first one (master) and the last one (winning). Anything in between is accessed by override index.

[Arthmoor]

I've been wiping out a lot of those finalization issues in OCS lately, but somewhere along the way there does appear to be a limit to what the CK will allow you to get away with. I've got two cells that persistently come back and start complaining about navmesh bounds being missing even though there's no indication in the error about WHICH navmesh is missing bounds. Finalize the cell, it goes away. Run a cleaning, remove ITMs, it's gone one load but then mysteriously comes back the next.

 

I think there must be some data somewhere in the NAVM record that's not entirely decoded yet that Edit isn't seeing but the CK gets confused by if it's not there. Or something. All I know for sure is that if I finalize the cell and leave it alone, the error never comes back.

 

Another issue that came up with Live Another Life when I relocated the farmhouse is that if you're fixing a navmesh after having edited it and part of that included cutting a triangle on the cell border, you are forced to take on the border cell on the other side because you've modified the triangle indexes. Even if the cell on the other side of your border is never edited in any other way. The mismatch will lead to the entire cell border being broken and NPCs will not be able to cross. I ended up just stopping short of the cell border and leaving all of the border triangles alone. When I finalized and later cleaned the file, the border remained intact.

[zilav]

NAVM is fully decoded, maybe something in NAVI?

[Arthmoor]

There's still an "Unknown" block down at the bottom of the NAVM record. You sure there's not something in there that might be important?

[zilav]

The only unknowns in NAVM are ONAM, PNAM and NNAM subrecords, but they are quite rare and possibly just a leftovers from early development stages.

[DayDreamer]

As you can see, there are no functions to get next or previous override from the current record, only the first one (master) and the last one (winning). Anything in between is accessed by override index.

This seems quite thorough, and I'll see what I can do with them. Also answers my earlier question "how do I test whether the table I'm accessing is in the plugin or falling through to the master?" Thanks!

 

Is there somewhere all the functions are documented?

[DayDreamer]

... if you're fixing a navmesh after having edited it and part of that included cutting a triangle on the cell border, you are forced to take on the border cell on the other side because you've modified the triangle indexes. Even if the cell on the other side of your border is never edited in any other way. ...

This also can happen anytime you delete a triangle. The largest numbered triangle will be renumbered as the triangle number you just deleted (filling the hole in the table). If that renumbered triangle is on the border, the link from the other side of the border is now bad. It can only be fixed by finalizing.

If you ever delete a triangle or a vertex, you have to re-Finalize.

This can be avoided by adding more triangles before deleting any. And it is always better to shift vertices or add a triangle than delete a triangle.

Where is the navmesh best practices? It should be either here or Bethesda forums (or both)!

NAVM is fully decoded, maybe something in NAVI?

Yes! Although we understand NAVI fields, we don't understand how it's used.

[zilav]

Is there somewhere all the functions are documented?

Some info here

http://www.creationkit.com/TES5Edit_Scripting_Functions

 

However it was not written by me, doesn't have all the functions, and some of the exsiting information and parameter types are wrong

I didn't write any documentation myself because I'm an awful lazy dude... j/k. I'm just too bad at english to write walls of text.

[DayDreamer]

This also can happen anytime you delete a triangle. The largest numbered triangle will be renumbered as the triangle number you just deleted (filling the hole in the table). If that renumbered triangle is on the border, the link from the other side of the border is now bad. It can only be fixed by finalizing.

If you ever delete a triangle or a vertex, you have to re-Finalize.

This can be avoided by adding more triangles before deleting any. And it is always better to shift vertices or add a triangle than delete a triangle.

To illustrate how hard some of these are to fix properly, the navmeshCheckEdges script first entries are:

No match!
Referenced triangle 47
^^^ Error triangle 43 in [NAVM:000E6DA7] (in GRUP Cell Temporary Children of JaphetsFollyIceStart [CELL:00094B4A] (in JaphetsFollyWorld "Japhet's Folly" [WRLD:00094B35] at -1,1))
No match!
Referenced triangle 69
^^^ Error triangle 5 in [NAVM:0010F860] (in GRUP Cell Temporary Children of JaphetsFollyShip [CELL:00094B39] (in JaphetsFollyWorld "Japhet's Folly" [WRLD:00094B35] at -2,2))
No match!
Referenced triangle 123
No match!
Referenced triangle 130
^^^ Error triangle 211 in [NAVM:0010F860] (in GRUP Cell Temporary Children of JaphetsFollyShip [CELL:00094B39] (in JaphetsFollyWorld "Japhet's Folly" [WRLD:00094B35] at -2,2))

I've not been able to fix JaphetsFollyIceStart yet. It splits into 2 pieces on CK save. Apparently they've navmeshed on ice floes and the sea bottom, all in the same navmesh, without interconnection. Skipping....

 

NB: Edge Links are called "portals" in CK error messages, as in:

Navmesh 0x0010f865, Tri 46, Edge mismatch, portals are:

 

JaphetsFollyShip has several bad triangles missing a side. Worse, 211 and 300 in the same place, sharing the same vertices. Need to remove 300. Using CK to remove and fix any of these error triangles causes a mess.

 

Because the last triangle 319 is also Edge Linked. So deleting 300 (or any of the others to be fixed) causes 319 to be renumbered, and need to Finalize, upon which all hell breaks loose!

 

Apparently, the prow deck navmesh slides under the mid-ship deck navmesh, but isn't linked together. Really an island. Also, the starboard navmesh doesn't connect at the stern. So deleting 300 and saving causes multiple islands (and a pile of warnings) to appear. Need a strategy to avoid these problems:

 

First, slide the vertices of the midship deck off of the prow.

 

Second, add new triangles between the prow and midship, connecting them properly. I made 3.

 

Third, Q merge 2 vertices at the starboard connecting it to the stern -- eats 1st of the new triangle numbers.

 

Fourth, Find and remove # 300 -- eats 2nd of the new triangle numbers.

 

Save and quit. No warnings, because the navmesh is still in one piece!

 

Now use TES5Edit to repair the listed 69, 123, and 130. This avoids the CK delete and create technique. They stay the same numbers, so no need to finalize. Works perfectly!

[Arthmoor]

If you ever delete a triangle or a vertex, you have to re-Finalize.

This can be avoided by adding more triangles before deleting any. And it is always better to shift vertices or add a triangle than delete a triangle.

Never had any trouble with this except when the triangle is on the border somewhere, but then I'd expect to have to finalize when deleting stuff at the borders.

I've deleted, added, split, recombined, and done pretty much every bad or good thing you can think of to a navmesh away from the borders and have never been presented with the need to refinalize. It's only when I'm *ON* the borders that it gets ugly.

[DayDreamer]

I've deleted, added, split, recombined, and done pretty much every bad or good thing you can think of to a navmesh away from the borders and have never been presented with the need to refinalize. It's only when I'm *ON* the borders that it gets ugly.

Anytime you delete anything, it renumbers to fill the hole in the sequence. Thus, it can renumber a triangle on the border at any time. Unless you pay obsessive attention to where the highest number was before you deleted, you'll have to finalize.

 

The kind of junk I'm now fixing, say there's a border link on one side, but none on the other. That happened because somebody deleted a triangle (say 111), and CK renumbered the highest numbered triangle (say 222) to 111 to fill the hole in the table. So the cell on the other side of the border is still pointing at 222, and CK won't fix it until you finalize.

 

Moreover, it's clear the CK used to do even less. That's how the data got so screwed up. The very first thing out of the scanner, Japhet's Folly, has several cells where the navmesh is completely discontiguous. Nowadays, CK automatically splits and renumbers them.

 

In the cell where you start with the boat, as soon as you edit and save, CK will split the navmesh into 3 parts (was 7 parts before the trial and error steps to preserve the deck navmesh described earlier). The deck is the main part, the water surrounding the boat gets a new navmesh number, and the ice floe gets another navmesh number. You have to finalize to re-do the borders (it complains until you do). The water is now an "island", and the boat has no link to the ice floe. You have to re-build the jump to the floe, and from the lowest point of the floe to the water.

 

CK isn't smart enough to re-build jumps after they are renumbered. In fact, it also has problems on borders. Apparently they tightened the constraints, and vertices that were close enough previously aren't close enough anymore. I've had to slide them around until they link.

 

In another cell, the other half of the navmesh is on the other side of the cell, underwater, with no possible connections! There's some navmesh fragment that was clearly designed to connect the cave door to the water, but it is another disconnected "island". (I've rebuilt the connections.)

 

Over by the dock, there must have been a stair down to the water. Now, there's a big cliff, which the navmesh descends into. There's a single triangle navmesh "island" embedded in the middle of the main navmesh, blocking the dock (now Z -32,767 and replaced). I have no idea how such a thing was even possible. CK won't make one these days.

 

UESP has a bug note that the pirates jump into the water and cannot get out. Yep. That's because links are missing or broken. And most of the water triangles weren't flagged as water. (Hundreds of fixes.)

 

I've fixed the links, and navmeshed under the dock to the slope on the other side to replace the former stair. Pirates (and followers) shouldn't get stuck as much outside. (Didn't even look inside.)

 

Whew! That fixes the first 3 things found by the script. Only 300 or so more to go....

[Arthmoor]

Anytime you delete anything, it renumbers to fill the hole in the sequence. Thus, it can renumber a triangle on the border at any time. Unless you pay obsessive attention to where the highest number was before you deleted, you'll have to finalize.

That's not necessarily true though. I have countless navmesh edits in OCS and also two village mods that have aggressively changed navmeshes in the cells they occupy. All without having to finalize any more than is absolutely necessary. Up to and including the manual removal of many cells which were added to the ESPs simply because the CK thought they needed to be there. The USKP has plenty of situations like this as well and if this were truly as big a problem as you're describing, we'd have heard about it by now.

I've made no effort whatsoever to obsessively track down the highest numbered triangles. Shit just seems to work - provided the work isn't being done ON the border.

 

Also, I'm aware that there are numerous problems with Skyrim.esm. I get hit regularly with errors in navmeshes next to cells I'm editing because the CK decided it wants those cells in the file. It's like the ESM flag hides all of these problems or something.

[Arthmoor]

User reported this in the USKP thread today:

[00:00] Checking for Errors in [02] Unofficial Skyrim Patch.esp
[00:00] Reference -> Target is not persistent
[00:00] Above errors were found in :CTDA -
[00:00] Above errors were found in :Condition
[00:00] Above errors were found in :Conditions
[00:00] Above errors were found in :RiverwoodKidsSceneNode [sMQN:00035102]
[00:00] Above errors were found in :GRUP Top "SMQN"
[00:17] Above errors were found in :[02] Unofficial Skyrim Patch.esp
[00:17] All Done!

He got this with USKP 2.0.4.

 

The error is false. When I set the scene condition up for this, the CK did not set the persistent flag on the dog. Nor does it want to if I force it to redo this. So apparently targets in the scene nodes do not require being persistent.

 

There's no way to correct this in the CK even if we wanted to. They took away the checkbox to manually set a persistent ref, since we no longer have any need to do that. The CK decides when it's needed.

[zilav]

The error is false. When I set the scene condition up for this, the CK did not set the persistent flag on the dog. Nor does it want to if I force it to redo this. So apparently targets in the scene nodes do not require being persistent.

Can you please tell me the name of condition function there?

It can be fixed per function separately, or more precisely per function parameter type.

[Arthmoor]

It's the "GetDead" condition in this screen:

 

 

Stump is the dog in Riverwood that has been made a target to check.

[zilav]

This is already flagged to not require persistence. He is either using an old version of TES5Edit, or the change was inroduced in 3.0.32

[Arthmoor]

Looks like a change 3.0.32 introduced. He's using 3.0.31. Another good reason to get an update out - it's starting to get hard to know what's caused by people using 3.0.31 now

[DayDreamer]

Zilav:

 

The meaning of NAVM flag Unknown 12 is "tested". In almost all of the bad edges, Unknown 12 isn't set. As soon as I fix the error, Unknown 12 is set on save. It's apparently cleared (and set) by the PATHFINDING logic used by the navmesh Find errors command. It might be used internally to mean "I've already been here" during testing.

 

I'd run the script you gave me weeks ago (SearchForNAVMTriangleFlags) trying to figure out meanings. Now I know I can use that same script to find "untested" triangles in cells. That's going to be really handy!!!

 

Could you shorten the "preferred path" to just "preferred"? A triangle isn't a path, per se. It's the travel logic that calls it a preferred path, but that's the whole path. The CK edit menu just says "Toggle preferred".

 

UPDATE: and with those tiny changes, I'd agree it's time for .32 release. Also, the navmeshCheckEdges is working really well for me -- it probably will also be useful for modders to check their plugins.

[hlp]

We have a known issue with FNVEdit cleaning to solve, after that a publication should happen.

 

(or two, or three... )

[Sheson]

I just wanted to confirm that speeds are back to normal with r1579. thx!

[DayDreamer]

Occasionally, I've found tiny islands of navmesh in the middle of the main mesh, usually sharing vertices with good mesh, but edges not connected. Neither the CK nor my search algorithm find these, I've just happened to notice.

 

Since they block travel, I want to delete them, and fill in the hole. But I remember deleting them is bad. So I want to do whatever TES5Edit does with undeleted navmesh?

 

I've tried simply setting all its vertices Z -32767, but that seems to leave a little yellow squiggle in the center of the screen that follows the pan. What's the correct method?

[zilav]

I don't know, TES5Edit doesn't undelete navmeshes since we could not find the correct working method for it.

[Arthmoor]

The only viable method is to move those triangles out of play, but if they're somehow connected only on one edge, that won't be possible without ruining the rest of the navmesh. How the hell did Beth's code allow that?

[zilav]

Uploaded the latest version r1593.

If Hlp will say "yes", it will become the release one for 3.0.32