This is a project to fix bugs in Morrowind that just aren't possible to do with scripting alone. It comes in the form of a patch to the Morrowind program. Currently in beta.

Here is a summary of what's in so far:

- Month fix (missing Morning Star)
- Enchanted item display bug
- Mercantile bug
- Transparent clothes in the inventory
- Unarmored bug
- Spellmaking limits
- Enchanted item value increase
- Merchant equipping
- Restore attributes issue
- Stealing from knocked out NPCs
- Reflect fix
- NPC health bar change
- StreamMusic fix
- Calm spells fix
- Vampire stats fix
- ESP load order fix

See the previous threads for more detail:
Repairing those Cogs #1
Repairing those Cogs #2

Here's what I'm working on now:

- Local ref bug
Description: You've heard of savegame corruption. Morrowind does it differently. It does loadgame corruption.
Status: Almost done.

- Fatigue spells
Description: Damage and absorb fatigue cannot drain fatigue below 0 and cause the enemy to collapse.
Status: Damage fatigue now caps at -1 fatigue. Does not drive fatigue lower so the NPC can get up straight after the spell expires instead of much later. Absorb fatigue doesn't seem to like that, you start to absorb negative fatigue.

- Map
Description: Need a bigger map.
Status: Beta-ish.


Getting it:

This is still in beta. To start testing, you will need the patch program in release 1, and the latest patch file released. Read the readme, it's important.
Note, if this crashes your Morrowind, please fill in this crash report posted here.

Files: http://www.tesnexus.com/downloads/file.php?id=19510

Last release notes:
Release 6
- ESP load order fix. The mod loader can now identify inserted and deleted mods properly, by matching their filenames. You can now change your load order without problems. Adding mods to your savegame will now cause much less issues. It is still recommended to use Wrye Mash to update or delete mods.
- NPC health bar change. Your pet Ogrim should no longer peg its health bar to the screen. Only restore health spells cast by you will displayed, previously any regeneration ability may have trigged it.
- Transparent clothes in the inventory. More changes, the background is transparent again and drawing errors should be reduced.
- Vampire stats fix. The previous fix unintentionally changed birth sign stat boost abilities. Only new characters were affected. Now working properly.

Hi Hrnchamd,

I wrapped up the Morrowind Code Patch Showcase mod I mentioned in a previous post. Have a look (plug it in and enter the trap door between Sellus Gravius' office and Vodunus Nucchius' house in Seyda Neen):
Morrowind Code Patch Showcase Mod

(Edit: Updated mod to v0.3)

I thought it might make sense to have something like this for several reasons:
- Easy, quick demonstration of the code patch's features to interested users
- Easier testing during continued patch development
- If you're making the patch modular in the end version, such a mod will allow the user to control easily whether the game works as he wants it.

What do you think?

Btw, I'm continuing to test the local ref patch (hunting for the missing Marshmarrow at the moment). Will probably take a while since on this installation, every savegame takes 15+ minutes to load and initialize. I'll work at it.

After a few tests, I realized that it is not connected to scripts, you are right. But a problem still exists, it happens when you enter a tomb. Try this: lower the music volume, than enter any tomb, and go out. You should notice that the music volume is immediately set back to maximum in the game settings (and ear it!).

And for the pitch value, it is not used in the game but a few modders wanted to use it (ASE for example). But for sure it is not a serious bug (compare to many others...).

Music has never changed in volume when I enter a tomb. You need to do some work and check which mod is causing it first.

Still testing. The Marshmerrow continues to be elusive. I can reproduce the missing ref message with 100% certainty when entering the cell while running the patched executable, and it never occurs when entering the cell while running the regular Morrowind exe. However, when I only load the mod in question (Complete Trade Fix.esp), there's no message under any condition. So there must be some interaction causing the message. I'm trying to narrow it down now, removing / re-adding plugins until I get the minimal configuration in which the message appears.

Also ran a second test on another large installation - this one is special though, it consists of only 4 esps, but these four add up to 100+ MB. (They consist of hundreds of mods merged together). This should be easier for the patched exe, because there's actually less replacement and deletion going on. Again I ran into *one* missing ref message in 30 minutes of testing. The id started with max_ashtree, the "max" might hint at the "Maximal Estate" mod. Coincidentally (or perhaps not), I ran into this message when visiting the Altmer town Iliandria, which was placed in the cell where *usually* Maximal Estate is. However, the person who had merged the mods together apparently had moved Maximal Estate to another location. Now I'm wondering whether this movement could have left some residuum which now causes the message. That's still shooting into the blue, but at least I'm collecting more data.

I also spent the better part of an hour with trying to figure out why the patched exe wouldn't work together with MGE. Tried several settings and changes, then it occurred to me that MGE might require that the process is named "Morrowind" exactly, and that "MorrowindLocalRefBetaFive" might just not cut it for MGE. That turned out to be the reason. Gnah. Not a problem of the code patch in any way. PEBKAC.

I'm also checking for differences between loadig from main menu and loading from in-game. I've found no differences so far aside from the data leaking out of the current cell, which I described earlier. It's difficult to test systematically though, since I don't know which kind of errors to look for specifically. However, many people know (or at least firmly believe) that in-game loading is bugged, so perhaps it would be best to skip the ingame loading procedure altogether and try to call the main menu loading procedure instead, if that's possible? Memory would need to be freed before though ...

Very instructive, I do like it. The vampire thing is great.
- Add some souls to those soul gems
- The room blocks that make up the underwater part overlap causing flickery z-fighting on the floor. Move one of them down slightly.
- Mr. Reflector's bounding box overlaps the button making it difficult to click on, could you mount it on the wall.


Try testing a pair of mods, one that deletes something from the master and one that moves/replaces the same item.

The reloader seems to only reload objects with the same physical ID, but they are in the correct order for replacement already. Try making a mod that deletes IDs 1-3 from another mod while placing 3 new items in the same cell.

Glad you like it.
- Soul gems: Thought I added souls, might have been lost in a CS crash. Will check / change.
- z-fighting fix: Yep, meant to do that for a while now.
- relocate button: Aye. I mounted the button on the bench so that the player could easily reach it when knocked down, but it turned out that you cannot activate anything when knocked down anyway.


Will do. That might be a reason why I don't see any problems in my testbed in Teruise's house. There, I *have* refs which are deleted by one mod, and then moved by another. These don't cause errors, and they remain deleted. But I don't have a condition where the deleting mod adds something else. Will be easy to add though.

Something I noticed when I was working on Improved Positioning..... Not sure if you can do anything about it though.

Items "migrate" from list to list. This was working with Morrowind Script Extender..... it has commands to call up items off of lists, and get them in order. Funny thing was.... items would not STAY on the list they should. I was having trouble with my scripts, that I simply could NOT explain, so, I wrote a test esp to simply write the items from each list, into a file. (Items, and statics, I think.... been a while) First run thru, everything was good. Leave the cell, come back to the cell, and items had migrated lists. It was consistent at least, so I could work around it, but, it was HIGHLY annoying......

There are three lists in each cell that contain the cell contents. Possibly one is for deleted items. The reloader does replace references in-place, but the cell data is changed when you visit it and it will be overwritten again. The reference loader when searching to replace items, searches two of the lists without preference, so I would not count on objects being in the same list all the time.

The NPC Health bar now seems to be working properly. It wasn't pegged to any companion. It appeared when I cast a healing spell on a companion and shortly after disappeared. It even worked on my pack guar.

Many thanks for the amazing work you are doing.

First of all, thank you for all the work so far!

It yesterday struck me that modifying the executable would probably also be the best way to create a mod which uncaps (or, at least, puts a higher limit on) skills and attributes. Would you perhaps consider adding an 'uncapper' as an option at some stage?

I realise that skill/attribute caps are not a bug, and that it may be too late for any new requests - but in view of all you've achieved so far you seemed by far the best person to ask. :-)

Best,

Swiveller

This doesn't work for me But I want it!!!

Just checked this out for you. No, I'm still working on the map and this is just one of many crash bugs I've seen related to it. The crash is caused by the game trying to draw the yellow location box/marker on the map, but it fails because its location is outside the texture, and the bounds checker isn't catching it.


Thanks. I might get round to it during the next holiday. The caps are scattered around the code, and there are a lot of places to check in the UI that might not display the stats correctly once they are uncapped, so it's not trivial.

I understand that Hrnchamd (blessed be the cut-and-paste) wants to release a final patch before starting to worry about local distributions, and I won't certainly complain about that . However, I'd like to know whether anybody has been able to use this with an European version of MW. I have an European version that is fully patched (and in English, if you were wondering), but the program still considers it wrong and refuses to patch it.

Apparently it works on French and German versions. If the translation was done by another company then the code may be different again, I would have to check everything by hand.

Thank you for considering doing this. :-)

* Swiveller *

Another local ref beta available.

- Fixed an issue with non-persistent ref matching, I typo'd the register the replace-target search variable was in.
This may fix your marshmerrow.

- Changed behaviour for deleted mods. All references from deleted mods are now not loaded at all, no matching takes place. All references that were changed by the mod will reset to default. Removing mods which change original game NPCs should cause those NPCs to reset. Original Morrowind tries to merge the changes, which may seem like a better idea for NPCs, I'm not sure. Wrye Mash does the reset thing when removing mods. The general problem is the replace-target information is no longer available from the mod so the merger could be overwriting anything.
This may fix the weird behaviour when removing a mod.

As well as the previous stuff, could you test
- Change game NPCs and objects with a mod, interact with/delete the objects and remove the mod.
- MVRFs. They are written to the original cell when an object is moved to another cell. Use the positionCell command to port people around. Just want to make sure they are written with the correct modIndex.

Will check. I've just managed to further isolate the missing marshmerrow ref; it's an interaction of the Farmer Mod 4.3 and the Complete Trade Fix 1.6. I was just about to check which specific reference was causing the issue and see whether it had some strange data in it. I'll try it with the new beta now.


Will check. I don't think anybody has ever complained about Mash's logic when removing mods, so falling back to it doesn't sound like a bad idea, especially when it's less error-prone than Morrowind's original way. I also think that it's actually preferable to have the game default to a clean and secure method of mod removal - if people *do* want to keep changes of a mod, they can change the esp (e.g. taking everything out of it except the NPC data and some items) and then use Mash's "Updater" function.


On my way.

First report: The Marshmerrow issue seems to be fixed now. I consistently *did* get a missing ref error for it with LocalRefBeta5 (the previous one), and I consistently *don't* get it with LocalRefBeta6 (the new one). The same goes for the max_ashtree issue in the other installation.

Will do further tests now, and update this post with results (or write new ones when the thread has grown)

Second Report: Cups&Plates testbed
Working fine on first (basic) test. LocalRef bug cntinues to be fixed. The doormarker issue after mod removal is gone now.

Third report: Moving Things Around testbed / Scrolls testbed
Behavior now identical to original Morrowind: For persistent references, all changes are applied. For non-persistent refs, only the first change is applied, all others are ignored. "Ori" behavior is back to regular Morrowind too: If two esps try to move the same ref, then the ref will be at the position set in the first mod, but "ori" wll list the name of the second mod as the origin of this data. Is this intended? (I guess it *could* be changed because the highest byte of virtidx still holds the modindex of the mod which actually "won". I.e., in the example above, the highest byte of virtidx would hold the modindex of the first mod, for both references. On the other hand, the current way of displaying info might be handy for debugging of mod conflicts, since you have the modindex of the "winning" mod and the filename of the "losing" mod available in-game ... although I guess this info could be gleaned from TESPCD as well ... hmm, have to think about it.)

I still wonder if it wouldn't be more consistent to treat persistent and non-persisten references the same, i.e. always let the last change win. Not sure about side-effects though.

Fourth Report: Did a stress test on the new esp loader: loaded several esps with persistent and non-persistent references, took/manipulated some of the items in-game, then saved, deleted one esp and changed the load order of others. Worked fine with LocalRefBeta6: The references of the removed mods were correctly deleted (no side-effects like the doormarker thing), and the references of the mods which had changed their load order were correctly matched. Ori information was correct. No problems visible in the savegame.


Fifth Report: Did some intensive testing of MVRF handling, as requested. Results looked well for LocalRefBeta6. Here are the Details (identical for regular Morrowind and the patched exe unless specified otherwise):

- created a couple of NPC Argonians's, placed them in Foryn's shack with a mod that had modindex 4.
- sent them (and Foryn too) to the lighthouse
- left the hut, saved the game

This created MVRF subrecords for all moved NPCs. Each of these was created with a modindex of 4. In every case, the MVRF subrecord was followed by the FRMR subrecord of the moved NPC.

I then inserted Plates.esp and an unrelated other mod on modindex 4 and 5, pushing the NPCs.esp to modindex 6. With the beta-patched exe, the FRMR's objectindices were correctly updated to 6. The MVRF's objectindices remained at 4. The plates were present in Foryn's hut, and all NPCs were at the spot where I moved them. Made a second save.

With regular Morrowind, the objectindices of the MVRF *and* FRMR subrecords remained at 4. No plates were present in the hut. However, the NPCs I had moved reappeared in the hut. In the lighthouse, however, they appeared too - and there, they inherited the "scaling to half size" from the plates.esp that I inserted. So this was the rsult of the engine mixing up the plate records from the new mod#4 with the saved records of the moved NPCs from fromer mod #4. Looked funny. Made a second save for this condition too.

I then removed the esp that i had inserted on modifndex 5, reloaded the second saves, and resaved. With the beta-patched exe, the objectinidices in the FRMR subrecords were correctly updated to 5. The objectindex in the MVRF subrecord stayed at 4. With regular Morrowind, both stayed at 4 (which is consistent with the known bugs in the engine).

Altogether, the beta-patched exe performed very well in this testbed. WhatÄs odd is that the onjectindex in the MVRF subrecord never changes. However, this doesn't seem to be a problem. Apparently the engine simply ignores this information anyway and uses the data from the immediately following FRMR subrecord instead, and this one gets updated correctly.


Sixth report: Did some thorough checking on the effects of removing esps that change references from esms, as requested. Results:

With the beta-patched exe, after removing the esp ...
- references of misc objects (bottles, plates etc.) get reset, i.e. they reappear.
- containers get reset, i.e. their content is generated new.
- NPCs do not get reset if the esp only changed their object data (like their race)
- NPCs *do* get reset if the esp changed their reference data (like changing their x/y/z coordinates). This includes reviving killed NPCs.

With regular Morrowind:
- references of misc objects do not get reset
- containers do not get reset
- NPCs do not get reset.

As expected, mod removal with Wrye Mash (unticking the mod, then "sync to load order" yielded the same effect as with the beta-patched exe, even when the savegame was loaded by regular Morrowind.

Conclusion: I prefer the clean removal of the patch / Wrye mash, but I think that people's preferences on this will differ ... Depending on the risks that keeping the Morrowind method has, is it possible to offer patch users both options?

I'll see what I can do. The ori filename is incorrect because the game seems to store the filename somewhere else again, I'll have to find why it changes it without overwriting anything else.


Surprised it works. When you load the save where the MVRF and FRMR are different, you don't get a MovedRef mismatch error? This may cause issues when removing mods. What happens when you delete the mod with the new NPCs? Do the MVRFs remain in the save or not.


Going back to the exact Morrowind method will require a lot of duplicating functions, and will cause the old bug to appear for objects that weren't replacers. It really needs the replace-target information from the mod, but the mod isn't loaded anymore. I can make it assume the replacement target is in Morrowind.esm, but assuming stuff is dangerous. I'm sure there will be a lot of complaints if I use the clean removal method by default too.

I don't think I have enough technical understanding of any of this - but would it be possible to provide a choice to "clean save" before a mod is removed? Hope this makes sense.

Hmm, I Can't get the patcher to work.

Description: Morrowind
Fileversion: 1.6.1820
Productversion: 1.6.1820
Language: English (USA)

I recon it should work for this version and this language no?

I'm unsure about the implications myself, but it would be nice to have it as an option so that I could test it. It's by no means necessary though.


Neither with regular Morrowind, nor with the beta-patched exe. However I found a way to produce it (see below).


They get removed together with the rest of the data of the respective NPC. I've tried that with with the beta-patched exe, and for both with simply removing the esp (which happened to be at the end of the load list) and with replacing it with another esp. In both cases the MVRF were removed correctly.

My working hypothesis at the moment is that the modindex in MVRF is never actually used by the engine, which instead uses the FRMR which always follows the MVRF immediately. I've hacked modindex values of 0 and 250 in with EE and still didn't have any problems. The game doesn#T complain, it doesn't show any bugs either. It just diligently reads the number I hack in, and writes it into the next save too.

I also hacked an *objectindex* of 9999 into the MVRF subrecords - and *now* I'm getting MovedRef mismatch errors. So what LocalRefBeta6 apparently does is this:
- Check whether the MVRF is legal by comparing its objectindex to that of the following FRMR
- if equal, use the objectindex and modindex of the FRMR for further calculations
- if not equal, them throw a MovedRef mismatch error and discard the record.


Imho, in that case the patch should default to the safer behavior (clean removal). You said you want to make the individual patches optional, so you can inform the user about the effects of applying this specific patch and give him the opportunity to skip it if he doesn't like them. From what you're saying, I think there are better areas to work on than to provide an alternative here. As I said, I don't think anybody ever complained over Wrye Mash's clean removal function.

Did you use any other programs on the Morrowind.exe? Currently the Code Patch requires the official Morrowind.exe patched to 1.6.1820. It probably won't work if you've used exe-optimizer, fpu2sse, a no-CD patch, or anything else on it. Those have to be applied afterwards, if at all.

I do own the game. But i tested it and it works fine with a no-cd patch.

I reverted back to before using the exe-opt and fpu2sse. Tried it both with and without no-cd patch. I'll make a new, clean installation now to try to try to see if it works better.

Slight correction: Savegame analysis revealed that the whole record does not get discarded when there's a MovedRef mismatch. Instead, the *MVFR* gets discarded. This means that the NPC will reappear in its original cell. However, its DATA_24 (position and orientation) does not get reset, which is only semi-smart.

I teleported NPCs from Foryn's shack to the lighthouse at coordinates 0/0/0, from there they fell on the upper platform which is at about z-level -160. Then I hacked in the mismatching objectindex in MVRF. As a result, MVRF got discarded and the NPC reappeared in Foryn's shack - under the floor, because it had fallen to a z-coordinate of of about -160 in the lighthouse, and the floor in the shack is at about z-level 0.

I'm still seeing no evidence that the modindex in the MVFR is used for anything apart from loading it into memory and storing it into the savegame though.

Making new installation, patching it, using that exe in old install. Works. Sorry for asking without testing this out first.

Thanks for the info. This will probably come up a lot when the code patch gains popularity, so it's good to know that it wasn't a patch incompatibility in your case.

There are probably lots of people who once ran fpu2sse or something like that on their exe and then simply forgot about it, or have a backup of an "original" exe that has actually been processed through fpu2sse. We weren't actually *expecting* someone to storm the scene with a codepatch, so there was little reason to keep backups of the original.

Okay, so the non-persistent override was a two byte fix. There's code specifically for skipping over loading them when it's a objectIndex-only match. Of course it would play havoc with original Morrowind, two mods adding a few objects to the same cell would write all over each other with the partial matches; the non-persistent loader isn't as sophisticated as the persistent loader.

Thankfully the MVRF matcher only ever matched the lower bits, confirmed in code.

I may be able to hijack one of the console commands so that on the next load it tries to merge objects with a certain mod index, and the merge targets will be available too.

Ah, I see. So the "first change wins" rule for non-persistent references wasn't a design goal, but a necessity due to the way the engine loads and matches references. Btw, can you see *why* the loader for non-persistent refs is less sophisticated?


Hmm, does that mean that there is a one-in-x chance of a wrong match due to not comparing all bits?


Sounds amazing - but also very difficult, I guess?

Reporting that I had no oddities playing for an hour or so with the patched exe. I haven't been looking for any of the fixes in particular but at least it has not introduced any error into the game which I can see. It all seems good and well. Haven't tried the local ref patch yet... I doubt i could be of much help, I would mostly be reinventing the wheel to catch up to Psyringe who's doing an super excellent work already.

Aww thanks - I appreciate the compliment, but I'm really only (hopefully ) helping the project along. Hrnchamd is the guy to laud, 'cause without him this project wouldn't even exist.

Regarding this ...:

In case you want to, have a look at the showcase mod I linked to in post #2. Just updated it (fixed a few quirks and slightly improved one script). It's kind of a guided tour to the features of the patch.

The persistent ref loader probably had more work done to it because all savegame refs are persistent. It may have started with a cut and paste job and got worse over time.

The MVRF check isn't a matcher, it just checks if it's really the same object.

Taking over a command is easy, the problem is conflicts with MWE or MWSE which patch the script VM too. MWE in particular hijacks certain opcodes, I have to read all the docs for it.

Regarding the music turning back up to full volume after exiting a tomb: I entered and exited that tomb west of Seyda Neen (the one with Lord Byrne's ashes or however his name was spelled) and the music slider didn't budge from where I had set it about three clicks from the right. I'm guessing the person that mentioned it has another issue causing the music slider to move because it hasn't moved at all for me unless I manually do it.

Okay, I've set up a hijack slot. How should the script command work? Technically it requires the choice of mod to merge back, then you need to save the game, quit Morrowind and unselect the mod, and load the game.

Choosing the mod could either be by modIndex number, or it could take the index from the object you've currently selected. A list of mods isn't too useful due to the lack of scrolling and 200 mod installs.

Interesting. Does this apply to several other exterior cells, then, or just that one? And does this mean a fix is possible? 'Cos that'd be very nice.

Thanks for looking in to it!

Even the Modders for morrowind are better than oblivion. Keep up the good work Hrnchamd(thats a hard name to remember, does it stand for something?)

Been using the patch for some time now and everything seems to be running smoothly with MWE and MWSE in r5. No major problems. However, some mods are going to have to be tweaked now that some bugs have been fixed. Especially the Unarmored bug.

It probably would be writing all over memory and causing corruption, but it's easily fixable.

Fixed: (*, -28) is right up against the edge of the minimap, and the game was drawing a few pixels off the edge into the murky stuff. Will be part of the next release.

Could you give me the cell bounds and dimensions of the TR map so I can size everything appropriately.

The patcher reports wrong exe even when I have reinstalled MW GOTY

Sometimes the GOTY version isn't the latest. Try installing the Bloodmoon patch first and then this patch; that should clear up the error.

You can see the co-ords of the cells faintly on the border of this map. Hope that's usable.

Well done on the fix, that's really great news!

Installed official patch 1.6.1820 US. I already had this version on the cd. Still having same report from hrnpatch

Have you ever used any of the above things before? Try a fresh reinstall (you only need the exe if your afraid of rewriting everything. So you can install it in a new place, and just copy-paste the new exe). Then download r1 and follow the steps in the readme. Then download the latest release.

If its still not working then something might be wrong. o.O I tested it with a no-CD patch and it worked fine. Someone else got it to work as well. I suppose its possible that some no-cd patches might not be compatible with Hrnchamd's patcher?

But anyway, need a bit more information. If you just want to take a shortcut and get past the "whats wrong part", you can do the first thing I mentioned with the fresh install. That should bypass any problems with your exe.

You hurt my feelings

j/k

^^" Guess I didn't take into account oblivion modders that frequent the morrowind mods. ;P But I'd consider you close enough to a morrowind modder. Your in this part of the forum often enough.

@Ljusa-I just reread your post about it not working. You said you already reinstalled GoTY. Well I'm stumped. Try downloading the bloodmoon patch off PES, perhaps the one on the CD is still out of date?

I mentioned it, but after a few tests, I discovered that this bug was only happening with a (probably) corrupted install of Morrowind (corruption caused by a script using streammusic command... I don't know how it could stay in the "memory" or in the saved game, while I completely disabled the mod and began a new game, with patched Morrowind). Now I must reinstall this mod on a clean MW install and see if the music volume is moving when using streammusic command. It should not, I guess.

Did everything as you said already, I have the original exe, but the patcher does not confirm that. File size is 3.973.120 bytes, right? Can I give you my exe to try it yourself?

Just poking my head in... (I'm too retired to contribute anything ). Awesome effort! Nice to see my knowledge being surpassed (and I'm pretty rusty on Morrowind anyway.)

Another bug: CharGenMenus. See UESP: Morrowind Form Change: CharGen Menus.

I imagine this is a low priority for you guys, but it might be a low hanging fruit to fix.

Perhaps another person would try doing it with your exe though? ^^" Not me though, sorry. No reason in particular. Just not up to testing it.

Heya Wrye. Your documentation is still useful after all these years.

Could probably stop the menus changing state, but the spell lists are a bit weird. I'll think about it.

Currently working on the map, I've scaled it down to 5x5 pixels per cell instead of 9x9, and some optional anti-aliasing too. However the way Morrowind renders to textures is excruciatingly slow, it may be a big part of the pause during cell transitions.

Hey, that actually makes sense. ^^ Are you capable of optimizing the texture-stuff?

Oh, it would be wonderful if that could happen! I know that came up recently (very briefly) in the MGE thread, maybe the folks working on it can give some useful input regarding how the engine renders things for cell transitions, and what MW might be doing that takes it so long to load the cells.

The saved games keep a lot of information, so it wouldnt' surprise me if this bug stuck around as well. Anyways, I've only tested in on the one tomb, so others may give me that issue. I'm about to test it on another one and I'll see what the music slider does.

If anyone is interested in helping me, please pm me to stop spamming this thread. Thanks

One thing about something related to cell loading. Take note at how long it takes Morrowind to reload the same exact cell compared to other games - it is as if it actually reloads everything.

Just tested this with a different tomb and the results are the same: the slider doesn't move.


Actually, I think it does...


I hope you're working on the map, Hrnchamd, because it's blocky now. The local map looks like it's split into squares. The world map doesn't really look any different, though.

Further reports on LocalRefBeta6:

I've been testing it on my "humongous installation" condition now. As usual, this slows testing down due to excessive loading and initialization times (which are part of the engine and have nothing to do with the patch).

I've made myself a pair of non-blinding boots of flying speed and zip criss-cross over the landscape, waiting for bugs. None happened so far.

There is an issue with one house in Seyda Neen, but I don't think it falls in the scope of the patch. I'm using "GS Seyda Neen Complete", which (among lots of other things) adds a shop for marksmen, the "Golden Arrow". The house of this shop disappears in this mod setup after saving. This was the case with regular Morrowind, and it's still the case with the LocalRefBeta6. I've never managed to determine the reason, as far as I can tell it's a regular ex_nord_house_01 static. It's not "refs persist" either, so it's unlikely to disappear due to a local ref conflict. However, during one of the LocalRefBetas (the one that created lots of doubling), it reappeared. Still, I don't think that it's an issue connected to the LocalRefBeta - I suspect it's the mod itself doing something strange, wouldn't be the first time (this mod also breaks quests in Veldion and TarMar due to unfiltered dialogue). But I'll investigate anyway. At least this got me thinking about the effects of moved statics and non-persistent refs. I'll run some tests on this.

Altogether, I think that LocalRefBeta6 is solid enough to put it from "special" beta-testing into "general" beta-testing. It should remain in "special" beta testing if you're adding the "choose to merge refs when deleting mods" functionality though, as this will need to be tested thoroughly.

The Golden Arrow is a moved version of one of the houses from Morrowind.esm (physId 0x10bcde), with the door switched. It might be an issue with non-persistent object replacement. I'm going to release the last localref beta with the non-persistent replacement turned on, you'll have to test multiple replacers all over again but for non-persistent refs. It will also have the merge-back feature, right now the command is called 'KeepReplacedRefs' and it takes the mod name from the object you've clicked on, or turns it off if nothing is selected. The merger state isn't cleared on load/save because it affects the saving process, and anyway you should be quitting Morrowind to remove the mod straight away. Will release later today.

Yep, found that out too. This means that in regular Morrowind, the house will not appear in the desired place if another plugin changed the reference before. But I think it's even worse.

There's a lot of moved stuff in this esp. There are even MVRF subrecords inside the esp because references apparently have been moved across cells, something that has been considered illegal or at least inadvisable in the modding docs I've read. This turns the reference into a persistent one, which means that it will now be overwritten by *following* mods hat change the same ref, and it will also make it prone to be disabled due to local ref conflicts.

It's pretty messy, but also a good testbed due to this.


Sounds good, I'll redo the tests for the next version then. In about an hour, I'll have to leave home for 3-5 hours, so don't put yourself in a hurry. I'm not sure how much testing I can squeeze in today, but I'll try.

Did some more tests on what happens when I move refs from one exterior cell to another. Moved Teruise's house from -2,-9 to -2,-10. Regular Morrowind threw a "reference missing in master" error and showed the house on both locations. LocalRefBeta6 showed the house only on the location I moved it to. It also crashed when I revisited the cell after reloading, but I couldn't reproduce it. This might have happened due to other mods installed. I'll keep an eye on it.

Not sure how important it is to research this, since moving refs between cells is seen as a no-no anyways, but collecting data about it won't hurt, I guess.

Have to leave now, will be back in a couple of hours.

Hopefully this is the last localref patch for testing. I've turned on non-persistent overrides, run it through the gauntlet. And the merge-back feature. It turns out I hardly needed to write any new code for it at all! Using merge-target matching in the loader means, if I just write the merge-target index instead of the normal index on save it will be automatically rematched on load. And the loader uses the modIndex from the last replacing ESP instead of the savegame already to allow for mod insertions. You no longer need to specify which mod(s) to merge, the good behaviour falls out of previous design.
Just turn merging on or off.

I've taken over the ToggleCollisionGrid opcode, MWE doesn't seem to use it so it appears safe.
Using it: Type KeepReplacedRefs or KRR in the console. You should see a message that it's on. Entering it again will turn it off. It does not reset on load or save, it does reset when you quit Morrowind. KeepReplacedRefs affects saving a game.

Test saving with and without it. Exit Morrowind, turn off the overriding mods and observe the changes on load.

With the KeepReplacedRefs off:
I've noted that indirect references by name (SPLM->NPCC->ref etc) do not get cleaned silently by Morrowind on load, it just throws a few unrelated-sounding errors (failed to load blah) then resets the indirect references. In particular 'Failed to load spell x' where x is a racial ability is caused by spell x not having a target reference, not because the spell has disappeared anywhere. It would be nice to clean these silently but these load before the ref loader so it's not possible to check whether they are valid or not. It's not perfect but the expected behaviour is still consistent.

I think that's about it, good luck!

About the map: When you enter a cell it renders the cell (land and statics only) to a texture, does a readback, then locks and writes textures 2-4 times to draw the local map, main map, fog of war and who knows what else. It's a lot of work for one frame, and I expect it's a partial cause of the pauses. I would have to look at texture and vertex buffer uploads too, though. It does do this every time you enter a cell, because a mod could completely change the look of the cell with new statics, land mass changes, activator changes, giant spider attacks etc. As usual I'm going to turn off everything and see if anything catches fire or performance improves.

Good timing - just arrived back home. I should have a few hours before I fall asleep on my keyboard again, so I better start testing.

Your a wizard man, keep the magic alive!

awesome news!

About the map: I don't remember seeing it looking blocky before I installed your patches, Hrnchamd, so I didn't know if it was something you could fix. My memory could be bad, but it's not that big of a deal to me. It could also be the seam fix mod I'm running that's causing the seems to show up in the mini map. Anyways, I can live with it if necessary, I'm just happy to have a more stable game now.

First report of LocalRefBeta7: Cups&Plates testbed

Everything okay. Local ref bug continues to be fixed. No doormarker issue.


Second report: Moving things around testbed / scrolls testbed

Behavior as expected: Now for non persistent refs the "last mod wins" rule applies too. Regardless of whether it's a change to the object data or reference data, persistent or non-persistent, esm or esp, the last change wins (but see below). No error messages. All ori info correct.

Point of note: The engine applies the changes sequentially, and it doesn't reset the data before applying the changes of a nw mod. In practive this amounts to a basic form of auto-merging of changes: If mod 1 scales a ref to double size, and mod 2 moves it, then in-game it will be moved and have double size. Position and orientation will always be overwritten because every ref saved with the CS will have position data. I guess that you could delete it with EE and the ref would then inherit the position that the previous mod had set.
(EDIT: No, that doesn't work. The engine expects position/orientation information for every ref. If it's missing, it throws an "abnormal ref termination" error and refuses to load the save. So the "merging" apparently only works for non-mandatory subrecords like XSCL.)
(EDIT2: Yes, it DOES work. The engine throws an error message when reading the plugin, but works on the data it could read correctly. I'll run further tests on that and write a new post about it, this one's getting messy.)

Point of note 2: If any mod in the chain deletes the ref, then it stays deleted. This may be a result of the same logic: One mod adds a deletion marker to the list of subrecords, and the following mods inherit it (in this case, they don't have a chance to undo the deletion since the DELETE subrecord cannot be set to a value that disables it, can it?)

That's quite fascinating. I'll have to play around with it a bit more.


Third report: Moving houses across cells testbed

Hmm, LocalRefBeta7 appears to crash to desktop everytime it encounters a cell with a non-persistent ref that has been moved across cells. This results in a MVRF subrecord in the ESP, which the loader apparently can't handle. While moving refs across cells is known to cause problems, regular Morrowind doesn't crash for me in this testbed, so I think we'll need to improve the LocalRefBeta in that regard. Otherwise it would be impossible to play (for example) the Seyda Neen Complete mod, which moves at least one house across cell boundaries.

It's probably my patch, the inventory figure keeps leaking render state into other parts of the game, since I've changed that in practically every patch it needs testing every time. I bet it's alpha testing this time.


It uses the DATA (pos/orient) subrecord as an end-of-ref marker as well, you'll notice it's the last subrecord in a group. That's the reason for the errors. Applying changes at the subrecord level allows proper partial overrides, I wouldn't call it merging.


Correct, there is a deletion flag stored in the reference. I have no idea what the value in the DELE subrecord is for though, the autogenerated stack frame pointers are too hard to trace through the whole save function.

Yep. The interesting part is that the information nevertheless does get processed correctly. This has interesting implications ... although actual *applications* will be limited due to the error messages and the fact that refs don't store that much interesting data.

When I have the time, I'll poke a bit around with it ... compare values in different files, edit them, etc. Not a priority right now though, testing the new features is more important. Have you seen the last report about the crash issue? I wrote it when you were writing your reply. I think we'll need to address this.

Yeah, I'm looking at it. The persistent loader has a decent amount of code dedicated to MVRFs while the non-persistent one doesn't. It should just create a new reference but instead it goes down the match references path that would have skipped loading it in vanilla Morrowind. Well, if two mods move things to two different cells, hah, too bad.

Okay, I've found an inline loop where the MVRFs are matched... researching it properly now.

By the way, I've now realised why reloading from within the same cell doesn't exhibit so many bugs. It's not data leakage, in fact Morrowind cleans up okay. The non-persistent ref loader only gets run when you enter a cell, loading not counted.

Ah. I guess we should be careful what we wish for, eh? No worries. If you can get it fixed, fine. If not, I'll deal with a seamed mini-map.

The crash bug was a result of the fix I made to get non-persistent refs to override properly. I re-used the same code path with a tweak without realising the MVRF process also needed a way to skip over loading the original (non-persistent) ref because it's been loaded already. I just need to duplicate a few lines of code and restore the original, it'll be fine. Two byte fixes can be dangerous as well as elegant.

There, done and uploaded. Back to tuning the map.

I'm back from another period of involuntary unconsciousness. Gotta cure that sometimes, it's slowing things down. Who need sleep anyway?

Downloaded new LocalRefBeta8. Confirmed that it doesn't crash in the "moving house across cells" testbed. Confirmed that the first two reports from LocalRefBeta7 above still apply. Will run further tests now.

First, thanks for all the work you're doing. It's good to see these things fixed.

Now, would it be possible to add new spell effects? This may be beyond the point of the project, but it would be nice to see bound pauldrons and greaves, and even the other helms and towershield. Also, a bound version of the rest of the Daedric weapons. And the other creatures made summonable, like the ogrims and the dremora lord, etc. I always felt they should be in the game. Eve it it would be a separate release.

I just thought this may be doable as you're messing with the code.

Highly unlikely, if I remember rightly the amount of spelleffects was hardcoded even prior to Tribunal and Bloodmoon. It'd probably require a ground up rewrite.

It actually most commonly happens in cells set to activate music via the StreamMusic command (this most notably happened in the Balmora Expansion and Mournhold Expansion mods in particular, where there were lots of these cells in-game).

I borrowed the terminology from TESTool, which does the same (overrides at subrecord level) when merging objects (it doesn't touch references though). But I see your point.

The ability of the engine to read partial records and process them correctly has some interesting implications. For example, I made four mods, each ofwhich changing Galbedir's Grad soulgem. Mod A scaled it to double size, mod B changed the soul to that of Dahrk Mezalf, mod C switched ownership to Ajira, and mod D moved it to the edge of the desk. With EE I then removed all subrecords from the mods that I hadn't changed in the mod in question. Then I loaded all four into my game. The result was an upscaled soulgem with Dahrk Mezalf's soul at the edge of the desk, owned by Ajira. No errors were thrown (not even for the missing DATA subrecord).

This means that theoretically, we could make super-compatible mods which (for example) only change ownership of items, but don't touch anything else. As I said, practical applications will be limited though because there aren't that many subrecords for references. Nevertheless it's fascinating to have this functionality.

I also tried what happens if the engine reads partial *object* records - if these were dealt with the same way, then we could (for example) make rebalancing mods that work in any language without changing the object names, or replacers which really only replace the artwork without touching anything else. This doesn't work however. The engine doesn't throw an error, but replaces the missing records and fills them with zeroes. I.e. if I change a weapon's artwork entries and delete its WPDT entry, then the engine will read it without complaining, but the resulting object will have zero damage etc.

I'll run some tests on removing mods now, to test the new functionality of the LocalRefBetaPatch.

It's very good news. If you can throw it to background, maybe it doesn't pause the game anymore. If this means disabling the inventory when cell changes occur, it's a sacrifice completely understandable for me. I am all excited again, sorry.

Tried the new reference beta. New character, but also building a mod at the same time. The mod adds a rug and a man to Balmora that has animation wander set to a custom lie down animation. The idea is to get some bums, beggars, tramps into balmora.

In my first test when I didn't have the rug added, and only the man everything worked fine ( duh, it's a new character, new savegame ). But when I changed the mod, it failed to show the change ingame. that is repositioning the NPC and put in the rug.

Dunno if it's actually related I just thought it might be.

EDIT: Test2, new game, get to balmora, character has moved but still no rug o_O. might be because any change to balmora cell by my mod which is later in the loadorder then valitys balmora mod would override the list of meshes in the cell ? Interesting thing to note is that npc did move on the new game.

Did you see what happens when you use vanilla morrowind?

EDITED above, but to make sure, no, I have a pretty graphically modded morrowind. ( suspected cause in my prev post ).

Well, I only meant switching exe's (Rename the current exe NEWmorrowind.exe and rename Morrowind.ORIGINAL to Morrowind.exe).

Wow, no I totally forgot to check. But I'm pretty sure normal MW would clamp it all together, like the conflicts you get when using to many house mods in the same place.

Should let you know for sure if its a problem with the patch, the mod, or morrowind being morrowind.

*Updating* a mod (i.e., making 1st version, playing with it, saving, then making a 2nd version of the mod, and loading the previous savegame) has problems in vanilla Morrowind (usually leads to doubling and/or vanishing of references). This type of problems cannot be caught by the localref beta (some might, as a positive side effect of another change, but the engine will probably still confuse references if it doesn't recognize that the the mod has changed).

However, the rug definitely should appear when ou start a new game. Try starting a new game with only your mod active and see whether the rug is there.

is this rug a new reference, created by your mod, or is it a changed reference from one of the master files?

It's a rug from normal, unmodded morrowind.

How did you put it into the game world?

If you dragged it from the list of statics into the render window, then it's a new reference, and shouldn't conflict with anything.

If you took a rug that was already placed in a cell and moved it to the place where you wanted, then you changed an original reference, which means that other mods which change the same reference could override your changes.

Btw, feel free to upload or mail me the mod, I'd check some things myself then and try to reproduce the problem.

Maybe I will send it to you, thanks for all of your effort man. I dragged it from the list so it's a new reference.

Just trying to following every lead to a possible bug. The Morrowind engine is a pretty complex beast, we're juggling with a dozen of variables, and it's always possible that there's some non-working combination which I neglected in my tests. Hence I'm very interested in clearing up any unexpected behavior of the local ref patch.

Hrnchamd, Psyringe, I'd like to thank you.

You two and your constant local ref debugging have taught me more than I otherwise would have ever known about Morrowind. ^^ And Knowing How It Works (it being anything Ihave an interest in) really helps me when doing things with it -- mods make more sense now, and so do in-game glitches. ^^

Now, I have an idea about the map. Could you split each cell into a ton of tiny "subcells" so to speak so it continuously renders small amounts of stuff in the background? This provides a much better way to map the world, too, it'll be more seamless and less "Oh, so now I'll have this (relatively) massive area of map magically appear on screen."

Preliminary report on merge-back feature testing:

I'm getting inconsistent results. When saving with KRR, sometimes references get reset, sometimes not. When saving without KRR (which is the default method), they always reset, which is correct.

I'm currently unsure what's producing the inconsistencies when savin with KRR. It might also be just me goofing up the testing procedure. I'll redo my tests more thoroughly.

In the meantime, one question: KRR stores all the necessary information inside the savegame, right? So when I save a game with KRR, and remove mods, then reloading the save should yield the same results no matter what I do in between as long as I do not overwrite the save, correct?

I'll keep you updated when I finished my tests with a more thorough testing procedure.

Update: Okay, I'm in the process of tracking it down. The following bug is reproducible:

Setup: Mw -> Trib -> BM -> Test73

Test73 is a mod which moves two barrels, one jug, and Fine-Mouth himself in Fine-Mouth's shack in Seyda Neen. Fine-Mouth gets a secret master mortar&pestle stuffed in his inventory for compensation. the mod also places a cheat amulet in the hut which has a freshly created enchantment (several boosts to facilitate easy stealing and combat).

Procedure:
- start Morrowind LocalRefBeta8
- enter hut, take and wear amulet, take jug, take everything out of both barrels, rob Fine-Mouth blind, leave hut
- save
- exit game
- restart game, load the game just saved
- reenter hut -> the jug has reappeared. both barrels are still empty though.

I didn't switch KRR on anywhere in the process (this was the detail what kept confusing me and sending me down the wrong tracks. I always interpreted the reappearance of the jug as "KRR not working", but since the jug also reappears when KRR is switched off during the save, the state of the jug can not be used to determine whether or not KRR is working as intended.)

Possible reasons:
- the game fails to match the jug in the mod to its master
- the game fails to match the jug in the mod to the savegame record which should tell the game to DELEte it.

Without further testig,I can't tell whether the problem resides in the saving orloading routines. I suspect the saving routine, but will have to run further tests. I'll analyze the savegames now.

Feel free to suggest any specific test or procedure that might be helpful.

EDIT: Found it. In the savegame (after taking the jug), the FRMR of the jug has the modindex of Test73 (correct), but the onjectindex of the jug in the *master*. When loading the save, the game creates the jug as told by the master, moves it as told by Test73.esp, and then the save tells it to delete reference #119,895 of mod #4. Since mod #4 doesn't have such a reference, the command is ignored, and the jug as created by Test73.esp remains.

Hrnchamd, can you confirm this or did you recode the values in FRMR so that the values I see in the save are actually correct?

Maybe pick up the jug then put it back down.

For the game it doesn't matter whether the jug is in my inventory or whether I put it down. As soon as I take it, the game deletes the original jug, and puts a copy of it in my inventory. This copy has modindex 0, i.e. it originates from the save. Whatever I do to this copy will not effect the game's treatment of the original reference in any way. If I put it down and save the game, then the info that goes into the savegame amounts to "delete object #x in mod #y (the jug), and create the following object new: jug, position x/y/z, etc.

Edit: found the reason, see post 92.

I just read up a bit. It sounds like when this is done, load order won't be a science unto itself. >>

Did some further testing on this:


It appears to affect all non-persistent references that are saved, but only those. I'll update this post with further data in a minute ...

Edit: Yep, confirmed. All non-persistent refs that are written to the savegame have in their FRMR their objectindex in the mod that first created them, not the objectindex in the mod that last manipulated them. Persistent refs are saved correctly, their FRMR holds their objectindex in the mod that last manipulated them. *Modindex* is always saved correctly, under all conditions. But the game saves the wrong objectindex for non-persistent refs.

Sigh ... after further checking, I have to revoke my analysis again.

Turns out that the info in the FRMR (objectindex in master, modindex of last manipulating plugin) is actually correct, at least it's the way regular Morrowind stores such data too.

So, new hypothesis: The problem is *not* in the saving routine, but in the loader for non-persistent refs, which apparently fails to match the record in the savegame to the correct ref in the game world.

I'll do some further research on this ... tough beastie, this one, and I'm not making things easier by making wrong assumptions about the game's inner workings. But I'll get through eventually.

Edit:Went back through former iterations of the LocalRefPatch. It turns out that we introuced this bug between v3 and v4 ... I've missed that in all my tests because all my interaction-based testbeds were created with *new* references, but this bug shows up when interacting with references that have been created by one mod and then manipulated by another.

LocalRefBeta4 (the on that introduced this bug) was the one that introduced the recoded virtidx/physidx.

Wait a minute ...

Got to test something.

Edit2: Okay, this might help. I revived the old testbed in Balyn Omarel's house. There are two plates on his table, one is persistent, the other is not. PlateTest1.esp moves both plates up in the air, PlateTest2.esp scales them to double size.

I go in, take both plates, leave, save, reload, re-enter.

Result: the persistent plate has correctly vanished. The non-persistent plate is shown in the position where PlateTest1.esp put it - but with the double size that PlateTest2.esp introduced. So there's a mixup of records.

Checking modindex and objectindex in the savegame shows that both are correct, so perhaps there's a bug somewhere in the matching routine? Buth then the plate would've shown up in the position PlateTest2.esp put it ... so the esp loader seems to have something to do with it? Gah ... I need to understand this better. I'm making too many mistakes.

Sorry for not finding this earlier ... I really should have.

Maybe sleep is not such a bother after all.

sieboldii

It would have taken me three times as long to run all the tests myself, you have proven invaluable so far. I'm grateful for any knowledgeable discussion. The non-persistent loader still needs work, it hasn't been the centre of my attention with all the savegame-centred bugs we've found so far. See what I can do.

Other news. First screenshots of the new map, at the images tab on the download page. Still a work in progress, the location marker is next, then variable zoom (which may not be possible). Using the map with an exisiting save requires an extra program to rescale the map that's saved in your savegame.

Like the map Hrnchamd ^^

EDIT: and I think zoom would be approaching the "feature bloat" level. MW players have done without zoom since the beginning of the game's days, why should that change now? I've never heard a complaint about the map, at least.

Certainly debatable, but not really relevant for the matter at hand. My mistake was to perform tests for save game loading and for esps that manipulate existing references in separate testbeds. Therefore I missed this bug, which relies on an interaction of both, a condition I hadn't tested for. This was a conceptual mistake I made at quite an early step, sleep deprivation is very unlikely to have had an impact.

Anyway, talking about *me* certainly won't bring this project forward. Trying to isolate and capture this bug will, so I'll get back to testing instead.

That map fix looks incredibly awesome! I promise you, many will celebrate you for this.

I've run a couple of tests on the last bug I reported. Here's the wrap-up:

- This bug affects non-persistent refs, which have been created by one esm/esp and then manipulated by another, and then taken by the player (hence saved to the savegame with a DELE subrecord).

- Affected items will reappear in the state that the second-last plugin that manipulated them put them in. If only one plugin manipulated them, then they will appear in the state this plugin put them in.

- Persistent references are not affected.

- References that never get manipulated by other plugins are not affected.

- References that get deleted by plugins are not affected, these get correctly removed without any previous versions of them popping up. The bug only affects references read from savegames.

- FRMR info of affected references in the savegame is correct.

- One report I made earlier (that the reappearing object had somehow retroactively inherited an XSCL subrecord that a *following* plugin changed) could not be reproduced with other objects; I think this was just an optical illusion. The plate was lifted into the air and hence *appeared* scaled without actually being so.

Okay ... I hope I've narrowed it down a bit.

The good news is that the presence of this bug invalidates my previous findings where I concluded that the KRR feature didn't work. So I can retest the KRR feature now, making certain that I only test it with types of references that are not affected by the bug described above. I'll have to redo these tests once this bug is fixed, but at least I can collect some credible data about how well the KRR feature works under the conditions that are unaffected by the bug, i.e. whether or not it introduces more bugs of its own.

Since you're working on the map right now (looking great btw!), I reckon I may be finished with this before you returned to the ref loaders - in that case I'll use the time to construct a better testbed which will focus on interactions that I hadn't tested before.

Of course it's awesome.

To clarify how loading works:
ESMs and ESPs are merged on starting Morrowind, but only persistent objects.
The savegame, when it loads data, it always uses the persistent loader to load changes to objects, even non-persistent ones. I assume deletes just set a flag in a reference. Note that save games have no merge-target information in them.
When you enter a cell, non-persistent objects are loaded and merged using the non-persistent loader.

When I test the plates example, the non-persistent plate which should have been deleted has the source data from the first mod. What may be happening is it marks the reference from the second mod for deletion, but it can only mark items from the modIndex it has. When the cell loader goes to load things, it does the first mod as normal (no matching modIndex, no merge target to match) but skips over the second mod because of the delete marker. Pow, bug.

Edit: Oops, the non-persistent refs are merged at cell load time only, unless there's an induced non-persistent->persistent.

my head just exploded

keep up the great work! this is truly a new golden age for morrowind

The map is fixed?

Oh.

My.

Gawd.

Yep, makes sense.

This would also explain why the bug did not appear in regular Morrowind. In RM, only the first change to any non-persistent reference got through. Hence, the savegame always had the modindex of the one mod that controlled the reference. When entering the cell, RM creates the reference, sees the mod that manipulates it, sees the matching deletion record, and removes the reference from the cell.

So what do persistent objects do differently that the bug doesn't happen to them? AFAIR (correct me if I'm wrong), persistent references are held in memory all the time, so could thiis help to backtrack the mising merge-target information? Or does the engine do some extra work on persistent refs when reading in a cell?

It's because the game applies the information out of order. First the original from the ESM, then the deletion reference from the savegame (possibly as a new reference), then the non-persistent refs on cell load. Either the deletion could match the ESM or the first non-persistent change could match the deletion. Now there's the possibility of changes slipping in-between.

You, sir, have my blessing.

If you want me to test the map fix, I'd be glad to. I'm a TR Modder, so I can test it with a lot of added landed.

A version of release 6 with the map and location markers working should be out tomorrow. Still have to:
- Adjust the map centre to capture all of TR
- Fix the location marker and centre on marker behaviour
- Fix the tooltip offset, it's a bit off in normal Morrowind as well, a smaller map just makes it worse

However, if these were the reason for the bug, then we should've introduced it in LocalRefBeta7, where we turned on non-persistent ref overriding. The bug however can be traced back to LocalRefBeta4, where you introduced the new physidx/virtidx system. Hmmm ... *scratches head*

Yes, but remember the ref override still replaced the modIndex even when it didn't change anything? At least one of the betas fixed that then it changed again.

Hum, okay. There are two refs floating around for the same target, deleted and non-deleted, and the reloader is having issues with that. There will be different behaviour loading with and without restarting Morrowind.

Yep, but I'd have to track back which of this betas this was ... what I see here is that the bug occurs consistently since Beta4 (introduction of new physidx/virtidx), and has since just changed form. In v5 and v6, the reappearing references appeared at the place where the *first* mod that manipulated them had placed them. In v7 and v8, since we switched on non-persistent ref overriding, they reappear at the position where the *second-last* mod that manipulated them placed them. Hence my suspicion that switching on non-persistent ref override just changed the way the bug shows itself, while the bug itself is located somewhere deeper. Not sure whether I'm making sense though.

Anyway, I'll try to have a proper testbed prepared for tomorrow, so we can tackle this one as soon as you have the time.


Hmm, haven't seen this yet ... but will continue to keep an eye on it.

I may have a solution. First, try this out:

Go to Balyn's, take the plates, turn KeepReplacedRefs on and save. This will write merge-target IDs for everything. In the savegame the modIndex should be of the original object. Restart Morrowind and load the save. It should be working. Go outside and reload. The plate reappears.

Looks like I have to fix the reloader first, and second I may have to compromise the clean saving and write merge-target information for deleted non-persistent objects. I wonder what happens when you turn off one of the mods in this situation.

It's a bit like Christmas, you never know what you might get. Maybe something you like or don't like...

Confirmed. This works the same way in my "moving things around" testbed. When saved with KRR, the main menu loader works correctly. The reloader doesn't, it makes the moved refs reappear, and they reappear in the positions where the *first* mod that manipulates them places them. Removing mods in-between does not affect this behavior (tested with 3 mods removed, but will need to be retested with refs that offer further methods of interaction apart from just taking/removing them)

It's interesting that the refs reappear on the second-last modified position when saved without KRR, and reappear on the first modified position when saved with KRR and loaded through the reloader.

Hrnchamd, does the map stay at that distance at all times, even without TR being on the map?

I can't believe the extended map is planned to come out so soon. I'm so happy. ^^

He's mentioned "variable zoom" in post 99m but said it might not be possible. I'm not sure whether this meant in-game zooming or to the possibility of specifying a zoom level when patching though.

There's no function to find out how big all of the combined land is before it gets loaded, so the map has to be at a fixed scale. It might be possible to zoom into the texture but I have no idea how the UI code works, it looks pretty impenetrable. You're probably going to have a on/off switch for the map scale in the patcher. It does auto-upgrade savegame maps to the smaller scale, but not the other way round.

Ah. Did not think about that second one. I was hoping that it would let you have two choices. Out to show all landmasses, and In to show the local landmass. Though Im guessing thats not what it is. ^^" Can dream though right?

Edit: Oh, question answered up above. Any chance it could read a map size from the a file? Perhaps the ini file?

That was the idea but it's hard to modify the in-game code.

I really gotta stop watching tv when i'm trying to post. ^^'

As announced, I've redone my tests on the KRR feature, now taking into account the knowledge of the last bug we found, i.e. reappearances of references due to this bug won't spoil the evaluation of the KRR feature any more.

KRR appears to be working correctly.

Tested on: persistent misc object (plate), non-persistent misc object (jug), NPC, door (locked/trapped)

Procedure:
- Entered Fine-Mouth's hut, emptied barrels, took jug, took plate, killed Fine-Mouth, unlocked door, teleported out
- saved game
- turned KRR on, saved game again
- quit
- removed the esps that changed refeences in the shack
- restarted, loaded non-KRR save, re-entered hut, checked situation, quit
- restarted again, loaded KRR save, re-entered hut, checked situation, quit

All references that I interacted with had been reset in the non-KRR condition. This included reviving Fine-Mouth. All had retained their status in the KRR condition. This included the status of the door (unlocked, but trapped), the scale of the door (which the removed plugin had increased), and container contents.

Due to the problems in the reloader, I quit Morrowind before loading anything, and didn't test conditions with loading in-game.

I'll redo these tests when the other bug has been fixed, but for now it's good to know that KRR seems to work as intended.

Just a quick question - will the eventual patch mean that Wyre Mash won't work? I'm kinda used to relying on it to make the mods I choose to play with compatible and am wondering if mash will conflict with your work?

First off, many of the patched features are independent from anything that Mash does, like the patches that fix the music bug, the restore attributes bug, the missing month bug, etc. As far as I understand Hrnchamd, the plan is to make each patch feature optional, so even if a patch feature did conflict with Mash, you could simply turn that off when patching, and still use all other features.

However, there isn't really a conflict between Mash and the Morrowind Code Patch, rather an overlap of functions in some parts:

- Mash is an excellent tool to prevent mod incompatibilities due to the local ref bug (that's what the "renumber refs" function is for). Hrnchamd is trying to patch this bug in-game. If he succeeds, then renumbering refs with Wrye Mash won't be necessary any more, because the bug that is prevented by doing so has been squashed. You could still do it though, it won't break anything. Plugins with renumbered references, and savegames which depend on them, can still be used. However, patching the bug will be even safer (and definitely quicker) than using Mash for this purpose.

- Mash also offers safe methods of adding and removing mods, again circumventing bugs in the engine by preparing the savegame in a way that the engine's bugs won't be triggered. Again, Hrnchamd is trying to patch these bugs in the code. If he succeeds, then you can change your load order, add mods, and remove mods, without needing Mash to prevent bugs by preparing the save game. This should prevent lots of bugs that appear when changing load order in unpatched Morrowind, like doubling, or crashes. Again, you could still use Mash for this purpose if you want to.

- Mash has a feature to redraw the game's map. Hrnchamd is currently working on increasing the scale of Morrowind's map. This will probably mean that Mash's map redraw function is not directly compatible any more, but this will have to be checked. If things go wrong in-game (like map corruption), you should be able to switch the map patch off, redraw the map with Mash, and then switch the map patch back on again. (Assuming that I understood correctly how Hrnchamd wants to implement the patch.)

- Mash also has many features that the code patch won't or can't touch. For example, Mash has an "Updater", which you should use if one of your mods has changed. I think it would be extremely hard to implement such a function in the code patch, so that's something that we'll continue to rely on Mash for. The same goes for many other useful functions of Mash.

In short: Mash and the code patch should be compatible, perhaps with the exception of the "update map" feature. Some functions of Mash won't be necessary any more if the patch succeeds in squashing the bugs they circumvent.

Thanks Psyringe - that's a really helpful explanation I may have to keep it a s a readme later

Heyas, just had to implement zoom, Vvardenfell looked so tiny. It took around 4 hours to dig my way up from the DirectX level through the scenegraph to the UI. There's a solitary screenshot of it on tesnexus, but it's a very neat change which will work with the normal scale map too. The zoom feature won't be out today, there's no way to accept user input (the scenegraph isn't connected to DirectInput), but the rest of the changes will be out this evening.

Another request, please continue testing release 6, I need to know about the enchant items balance, and how everything else works.

Localref beta? Another release tomorrow.

Standing ready. Will have an improved testing environment finished by tomorrow.

Map continues to look great, I bet this will be one of the most popular features of the patch.

Looks very nice.

Does the zoom have any of the issues experienced with the 'vanilla' map (forced snap to center, cut off land when you move too far etc.), or does it work 'as you'd expect'? I just don't quite get how it works, as an 'alternative' to the 'zoomed out' map. Is it for all intents and purposes just as fixed, and it's just a matter of preference as to whether you zoom or don't, or are there advantages/disadvantages to both forms?

Hmm, is the zoom (which would be great!) really that blurry?

With magnification on you can pan around the whole map, as usual. The lack of detail is because there's about four times the land area in the same texture. Increasing the texture size does break save game compatibility and also requires finding out how the map texture is handled internally. The magnification is fully variable so you may have 1.5x, 0.75x etc.

So, currently (or when the new version arrives), we choose the level of zoom we want at the point of installation, and it's all compatible with savegames, and all levels of zoom are equally 'fixed', it's just an aesthetic choice (one that we can't change our minds on once the patch is installed?)?

But then if we choose we want 'zoomed in', and then also want bigger textures (which presumably you're checking to see if we can do?), it will cause problems with savegames?



Anyway, just to give you another bug to chew on:

NPCs set up to be vampires in the CS, but are done so on 'fresh' NPCs rather than 'copies' of pre-existing Beth vampires, have a face selected in the 'face choice menu' (unlike Beth's vamps, who have no face selected, and in the CS appear with vampiric faces - custom vamps appear with normal faces). While in-game, they are given a vampiric face, but unlike Beth's vamps, upon death their face reverts to the one chosen in the CS, when it should remain vampiric.

If that makes any sense, that's another little bug that you may or may not be able to fix. Just thought I'd give you something new to toy with.


EDIT: Ooh yeah, and does the map fix now fix the previous problem with that cell causing crashes?

There's two components.

1. Being able to record land further out than in vanilla. More land -> less detailed texture. Requires you to make a choice before you load Morrowind.
2. Changing the map magnification. Will be able to changed while playing the game, doesn't affect anything. Nice for people that play at high res.

I've rolled the crash fix into all patches I release. Named cells on top of the south or east border of the map in vanilla will cause crashes, so you know what to test for.

The map test is ready. Read the readme.

Will test once I get home in the morning. Also I want to say that the local ref seems to work really well for me. It would seem my saves corrupt far less and I would say, all in all the patch seems to have made MW more stable.

Awesome!

Tried to test the map, seems to work like a charm.

Hrnchamd, don't if this issue been addressed or if it can be fixed: The game can CTD if three or more message boxes appear on the screen at once in the same frame. I don't know if this is because of the way they're coded or because of CPU overload.

[edit]: I removed the 34th variable because I think that's outside of what's possible to fix via this patch, plus it would be redundant because most scripters work around the issue.

This is an issue with the CS. If you use MWEdit you can select the proper vampire head from the list.


I think this will be easily fixed.

If you can get the issue with the messageboxes crashing the game, that would deffinantly be great.

Heartily seconded. There are a number of crashes that people seem unable to prevent, and messagebox problems seem to be the most likely candidate to cause them. Jac, do we have a script that produces these crashes consistently? When I played around with that (a while ago, and I didn't dig very deep), the script would sometimes crash, sometimes not, which would make testing difficult.

I can't get it to crash with 64 messageboxes per frame, with variable substitutions and talking to people so it gets redirected to dialogue as well. Tell me if you find a reliable way to do it.

I actually think it has to do with the script doing too much stuff per frame, or too many scripts being called every frame.

GCD can cause it if you train and don't wait a few seconds between sessions. I think the crashes related to that result from the scripts the mod runs plus it throwing up message boxes whenever one of your skills increases as well as an attribute.

Might be relevant whether the messageboxes get triggered by the same script or by different ones? Have we tried both?

I just opened a general CTD topic on the Hardware & Software forum. I thought about asking Hrnchamd to eliminate all CTD issues, but I figured he would direct me to a magical pony dealer (or somesuch). But if he would like to stress test some aspects of the engine, and if this resulted in some elegant patches that reduced some of the more flagrant CTD issues, then I would revere him as the fourth member of the Tribunal.

What do you mean "would"? He IS the fourth member of the Tribunal!

But, that would make it the Quadunal.

Tho, he could be that invisible influence behind the scenes which never really gets the proper recognition deserved, the other three just figureheads for the masses to have something tangible to praise.

edit: And of course we're talking about Triads instead of Tribunals - a tribunal is an acting body made up of any number of parts, were as a triad is three distinct parts.

I tried my hand at a reproducable messagebox crash but had no luck yet. Current setup:

- I created 33 global scripts (based on my hypothesis that individual scripts might be needed to trigger the crash). Each of these tests for the same global variable, and if it's set to 1, then it displays a messagebox.
- I have one activator that starts all the scripts
- I have one activator that toggles the global variable

As soon as I start the scripts and toggle the variable, FPS go down from 53 to 18, but I don't get a crash.

It might be that we have to stress the nmachine a bit more until the crash happens. Might also be that I'm on the wrong track altogether though.

I'll postpone this in favor of other tests I wanted to do. If anybody wants to have the mod (setting up 30+ scripts is a bit tedious ), just ask.

Alright, I've found what the deal with the reappearing plate is, I didn't anticipate a master file ref getting loaded after the save game back in release 4 so missed filling in the merge-target information for it. The load now works properly with merged deletes, I'm just trying to narrow down the merge on save to non-persistent objects before it's released.

Great.

I've set up another testbed now which should allow me to test for a lot of ref manipulation and deletion at a glance, so testing should be quick once it's released.

The map works great =) Thanks!

I have never seen this. I've always thought it was dependent on some part of people's setup. e.g. Only with MWSE, MGE, MWA, FPSopt or some combination I don't use.

You may be chasing something that isn't the fault of the Morroind exe for once!

It's not that, Symon. It IS in Morrowind. I'll crash running no external applications when GCD decides to add a lot of stuff to me.

OH OH I HAVE IT!

Using GCD, make a script that will ModSkillofchoiceone 1 and ModSkillofchoicetwo 1 every full second. You'll crash, fast. I almost guarantee it.

Losing savegame compatibility wouldn't be too much of a bother (people can keep multiple copies of morrowind.exe after all, one for new characters and one for old savegames; and it could be possible to make a workaround for it with Mash) so if later down the line you have the time to figure out how it's handled and allow the option to have a bigger map still at a 9²-pixel-per-cell resolution it would be even more awesome than it already is.

Hrnchamd I know this will sound a bit weird. but is it possible to add more "types" of the kind, Humans and beasts. Perhaps a second beast type or a second human type?

The idea is basically to be able to assign another set of animations to another type of human set. Since the animations between the beast and human type can be different in accordance to the base animation file they use, shouldn't that be possible as well?

I'm not sure if it's an .EXE thing tho.

Do I make sense?

This would allow npcs of a certain type, say human_x to have different animations from human.

This seems to crash in the script system somewhere random, it's not apparent what's causing it though.


Only if there's enough interest.


The base animation types and the mapping to race and sex are hardcoded. I'm not going to rewrite the animation system, but I will check if there's any undocumented animation override functions.


New localref beta is out. It includes the fix for the reappearing plates (multiple non-persistent overrides). Please check it out. And release 6 still needs test reports! Enchanted item value balance needs to be checked out, noone's discussing it.

On my way.

That explains a lot. I'm one of the people who isn't too disatisfied with Mprrowind's character advancement.

Sounds like you are on the way to a solution again!

Now, about bones and gloss maps.... (grin).

It's not just GCD acting up. There are crashes which can only be traced back to scripts that are legal, yet they sometimes produce crashes, and the number of messageboxes displayed is a common factor among them. For example, here are some crash experiences from my last installation:

- Creatures6 has Ice Worms which tunnel out of the ground. When doing so they throw rocks around. these rocks are actually NPCs (due to technical reasons). Due to a GMST issue in my game, these rocks/NPCs shouted "Oww!" when falling to the ground, each one. this resulted in lots of messages and frequent crashes. Now the problem *could* have been the script itself, but when I cured my GMST issue (and the messageboxes went away), the crashes went away too.

- From time to time I crash when walking into crowds. The only common factor among these crashes is that many NPCs are greeting me at once, hence - many mesages.

- Other people report crashes when GCD, Morrowind Crafting, or NoM display messageboxes. Of course it *could* be that each of these mods has badly written scripts which make the game crash. However, these are mods which have beeen developed and scrutinized for long times, and although nobody is infallible, I find it hard to believe that all these authors managed to botch up their scripts.

However, as said before, it's very hard to pinpoint these crashes, or to even reproduce them consistently. Hence it will be hard to solve this issue.

First report on LocalRefBeta9: Cups&Plates testbed

Local ref bug continues to be fixed. No doormarker issues. All ori information correct.


Second report: Moving things around testbed

The problem of the reappearing non-persistent refs appears to be fixed. Nothing broken any more in this testbed; what worked before still works.


Third report: Plates in Balyn's house

Same as above; issue seems to be fixed. The reloader still seems to have issues though. Un this testbed, I have one esp which lifts the two plates in Balyn's house, and another which scales them up. Procedure:

- started LocalRefBeta8 (the previous one), entered Balyn's house, took plates, left, saved "save8", quit
- started LocalRefBeta9, entered Balyn's house, took plates, left, saved "save9".
- loaded "save9", entered house, checked: no reappearing plates. left house.
- loaded "save8", entered house, checked: one reappearing plate. left house.
- loaded "save9", entered house, checked: now the other plate reappears.

Apparently, some data from save8 remains in the system. However, I had difficulties reproducing the issue, it didn't occur always.

Is there an easy explanation for such an effect? Could this affect people who use the reloader with LocalRefBeta9 and savegames from unpatched Morrowind, or is it something specific to LocalRefBeta8 saves?

Personally, I'm not at all averse to turning the reloader off and, if that's possible, freeing the memory and calling the main menu loader instead. I think the reloader was one of Bethesda's attempts to optimize loading times, but it seems to be causing more issues than the faster loading time is worth.


Fourth Report: Moved House

Checked the Plugin where I moved a house across cells (MVRF record in the plugin). Works; the house vanishes at its original position, and appears at the new one. Still no reoccurence of the one single CTD I had when going near the door in LocalRefBeta7 (?).


Fifth Report: new Testbed: reference Overrides

There seems to be an inconsistency when an esm deletes a reference from a master, and another esm moves this reference afterwards. In that case, persistent references show up, while non-persistent ones don't. Not sure whether this qualifies as a bug, but I though I'd report it.

Overrides seem to be working okay apart from this. I'm investigating into an NPC that didn't show up under two conditions where he should, but this might be a bug in this new testbed, will have to double-check.


Interaction-based tests are next, and then it's KRR-time again.

The reloader is fine in my opinion, it does strip non-persistent references immediately already. It was the loader in beta 8 that was not matching properly, because the merge target actually got loaded after the save game. Did I mention I changed the ref saving just now so it always merges deleted items? You really need the merge target to make this work. Beta 8 saves will still exhibit the bug because of that.

Report 5, both references should disappear. Checking it out.

This is unrelaed to the changes in LocalRefBeta9 btw. The previous version shows the same behavior. I didn't trace it further back (but can do if it helps). It's another bug that my testbed in Teruise's house didn't catch, I caught it now due to the more organized ref override testbed. I'll put this testbed up for download once I determined whether it's working correctly in the case of the two vanishing NPCs; it might be helpful.

Edit: regarding the reloader: The problem was that after reloading a LocalRefBeta8 save, loading the LocalRefBeta9 save would exhibit a bug as well. But if that can be explained by the way LocalRefBeta8 saved games, then the reloader might be working correctly ... but I'll keep an eye on it.

This interests me. As in my mod i'm working on. I tried moving an NPC that was already placed ingame by me earlier. He did not move, eventually i figured out it was the save game info that kept him in place even if i saved the mod with the npc changed and loaded it up.

Maybe similar thing happening here ? start new test save game?

Unlikely - I start all my tests from a savegame that was saved just after character generation, with no mods installed. If strange bugs occur I also start a new game from scratch, but so far this never changed anything (the only thing that's different between my save and starting from scratch is that engine has to add the mods with my testbeds, and adding of mods to a previously unmodded save seems to be very solid). But your analysis is correct; that's why I always start from an unmodded save.

Just a thought... since the sub-titles were enabled to give you those NPC message boxes, could the sub-title system be responsible?

Actually, with the delete and move, original Morrowind restores the permanent ref from the pits of Oblivion as well. I bet it's to stop NPC conflicts.

Seems like Wrye Mash doesn't like me saving merged refs in the savegame and cleans them out. Going to bite the bullet and unlink and free all previous matches instead of just overwriting. This is going to be quite a bit of code.

Remind me to put a big warning not to clean KRR'd saves, they're only for immediate reloading.

Indeed. Thing is I have an un-released script that does fire off three message boxes in very quick succession in the same frame. It has never crashed. In fact I've never seen this bug. If I'd heard of it at the time I wouldn't have written this script in this way, but as it happens, no issues with it at all. Worse, I always run with 200+ mods. However I don't use any of the capability extension mods such as MWSE or MGE.

It'll be interesting to see what does induce this.

Yep, that's why I called it an inconsistency instead of a bug. There is no obvious "desired" behavior, just two different ways to solve a conflict. It might be "cleaner" to have persistent and non-persistent refs treated the same here, however it's a rare case anyway.


I'll be here and test anything you throw at me.


Ideally, such a warning would be displayed right after the "KRR turned on" message in the console - not everyone will read the readme, and people will forget parts of it over time. Not sure whether that's possible though, I reckon the space is limited and you don't want to change string lengths or pointers.

Okay, I've narrowed the case of the two missing NPCs in my testbed down to two possibilities: Either they fell through the ground because of bad positioning when I moved them around, or there's some weird interaction with another part of the testbed which was thought to research possibilities for de-isolation techniques (esps dependent on esps) later on. I repositioned the NPCs and removed the de-isolation part, so in any case it's solved now, and I can send you the testbed:

Reference Override Testbed

This testbed has meshes and textures. You can uninstall both easily by deleting the "psyr" folders in "Textures" and "Meshes".

Extract to Data Files and activate the eight files that start with "ROT", four of them are esms and four are esps. Then enter the trapdoor to the wilderness which appeared right in front of Census and Excise in Seyda Neen. You'll land on a barren island. A few steps in front of you is a ring floating in the air, wear it if you want levitation to get a better view.

The testbed is a grid of 26 columns (labeled A-Z, but ignore T-Z for now as these are not implemented yet) and 8 rows (labeled 0-7). Each column is one test run. Have a look at the spot at A0, you'll see four things there: a bottle (persistent and take-able), a cup (non-persistent and take-able), an NPC (persistent and interact-able), and a container (non-persistent and interact-able). They are placed so that the non-persistent refs are in front, and the persistent refs behind them. They are also placed so that the refs you can interact with are on the right, while those that you can only take are on the left.

Every column, i.e. every test run, starts with these four references.

Every row denotes a certain action performed on these references. In row 0, they get created by ROT0.esm. In row 1, they get moved by ROT1.esm. In row 6, they get deleted by ROT6.esp, and so on. Rows 0-3 show ESM actions while rows 4-7 show ESP actions.

Not every test run (i.e. column) will have all actions performed on the references. Those in the first column only get created, nothing else happens to them. Those in the second column only get created and then moved. When an action is performed in a given column, then a marker will be placed in the respective row in that column. These markers have symbols on them: a star for creation, an arrow for movement, and a crossed circle for deletion. They are also activators, which gives them mouse-over info. You can walk down a column, hover your mouse over the markers, and the marker names will tell you exactly which actions have been performed in this column.

The markers are color-coded. Green markers are end-points where all four references should appear. Red markers are end points where no references should appear. Yellow markers are intermediate steps where no references should appear either. Blue markers are end points where there is no clear "correct" behavior, as in cases where a plugin tries to move a reference after it has been deleted by another plugin.

The testbed allows checks for different combinations of reference repositioning and deletion, and you cn see at a glance whether everything worked correctly. If it did, then every green marker should have all four references standing on it, red and yellow markers should be free from them, and blue markers are up to you to decide. If something went wrong, you can see whether it affected persistent or non-persistent refs. You can also see exactly what happened in which sequence in the test run that failed, by walking down the column and looking at the markers.

Finally, you can take the cups / bottles, interact with the NPC / container, and see whether they get correctly rematched when loading a save, or correctly resetted or merged after removing a mod.

I mainly contructed the testbed for my own testing, but thought it might be useful for you, so feel free to try it and to suggest changes or additions.

Sixth report: Saving after interactions

I went to the Ref Override testbed, took everything, emptied the vases, and bought the ingredients from the NPCs. Then made a normal save, then made a KRR save, and quit.

Reloading the normal save showed correct rematching for the persistent bottle, the non-persistent cup, and the NPC, but not the container. Containers had an "echo", they appeared on their current location as well as on their previous location in this testbed. When they appeared on their previous location, their contents had resetted.

Reloading the KRR save showed that all container that had any plugin manipulating them had reset, their contents reappeared.

I then removed seven of the eight splugins of this testbed, only let ROT0.esm active.

Loading the non-KRR save showed somewhat odd behavior:
- NPCs and containers were always reset
- Cups (non-persistent) were only reset if one of the removed plugins had a command to delete them.
- Bottles (persistent) were only reset if the *last loading* of the removed plugins had a command to delete them.

Loading the KRRed save showed the following behavior:
- NPCs and bottles (i,e, the two persistent references) were reset if the last-loading of the removed mods had a command to delete them. They retained their data otherwise (for bottles this meant to remain deleted).
- Cups (non-persisrtent) were reset of one of the removed plugins, had a command to delete them
- Vases (non-persistent container) had their position always reset. their contents were reset when any of the removed plugins had a command to delete them.

That's a professional test station you have.

Over here, just unlinking the previous refs means partial overrides aren't propagated correctly. I have to relink it into the persistent ref list and merge the flags. It will take a while. I'm going to remove the merge on delete behaviour from clean saves as part of this work. However the container behaviour is pretty weird. What do you think of the KRR behaviour, apart from the containers, is it reasonable enough for public use?

Reading these last posts I just have to ask: what is a KRR-save and what does it have to do with the console? If this is something that concerns esoteric testing purposes only, then don't feel compelled to give a detailed answer.

I have retraced my steps and gone back to some of the nudist merchants I visited between patches 4 and 5. There were some in Balmora, a few in Caldera, and a heck of a lot in Ald'ruhn. I either switched to the original .exe and did a minor transaction, or if that wasn't possible (b/c I'm running Service Requirements), I just opened the console and gave them a ring. In either case, they covered up their nakedness, and learned some d*mn modesty! Thanks for the tip!

I would like to answer on the question of enchant values, but it a misc skill for my character, and I have never gotten far enough in the game anyways to explore/exploit the enchanting system. Have you done any work to balance the economy after fixing the mercantile bug? If not, do you expect any progress in this area to be the work of modders? I wouldn't play without something like Pirate Lord's Trade Enhancements; would this be enough of a rebalance? I feel like I ought to help this project in some small way, just to show a little appreciation, since I have been following it from the beginning.

By default the localref beta does clean saves, where removing a mod resets all the objects it changes, you pick up something it changed and you have the original to go back to. Wrye Mash does this when you remove a mod with it. Thing is, if the mod just moved the object across the room, and you picked it up, it will appear again because the mod changed it.

What happens when an NPC gets changed? Upon removing the mod they forget about you, disposition is reset, their script gets reset, etc. Not always what you want. Sometimes it is. Morrowind merges by default, but you know the problems it has with that. KeepReplacedRefs [KRR] turns on merging, and we're working to make it as bug free as possible with all this testing.

No-one's given me any feedback on the economy changes, so I can't balance it for community preferences.. the enchanting value addition could be any formula, someone needs to think a bit and come up with something reasonable. The current formula is (base item value) + 0.5 * (soul size) * (enchantment points used).


Stuff:

This is caused by containers migrating from the non-persistent list before being used to the persistent list after, same thing as deletes do. The next release will fix that.


The persistent bottles oddness will go away, and the cups should always reset with the next release. The KRR behaviour is as I expect, it merges player actions only.

Very useful stuff.

Well, I promised to help you to the best of my ability if you were to tackle the local ref bug - I got to adhere to that.


If it's only the partial overrides (like what I did with the soul gem example) not working, then people could certainly live with that. It was a bonus feature so far - fascinating, and certainly nice to have, but not essential.

How does your mid-term time schedule look like, if I may ask? You said you only had limited time to work on this project, so I reckon we might have to wrap up the working parts at one point and live with some remaining problems. Although your project management so far was so professional that I probably shouldn't worry.


I think we should solve the KRR-related inconsistencies listed in post #171 first, otherwise it would be a bit confusing since some types of references show correct behavior, and others don't. Seems you're working on that right now. (Edit: Just read your "is working as I expect" line, I may be misunderstanding something about KRR functionality, will think about it further.)

Edit: Rethought it, you're correct. There are no inconsistencies in KRR behavior. I *thought* there was one because persistent and non-persistent refs were treated differently when they originally had been deleted and then moved. But this inconsistency is unrelated to KRR functionally, instead it follows logically from the way the engine currently treats deleted-then-moved refs. This is the step where the difference between persistent and non.persistent refs is introduced. This meant that deleted-then-moved persistent refs were there for the player to interact with (and hence went into the savegame), while deleted-then-moved nion-persistent refs weren't. KRR then just propagates this difference because its function is to merge player actions back back into the game world.

Try this new localref patch, I didn't manage to check if it works for containers though, I'm going out now. Back to work tomorrow, just going to intergrate up r7, map, localref and the UI installer for release, no new features, no new fixes.

Preliminary tests on LocalRefBeta10 looked good apart from one bug: Non-persistent items reappear after loading a savegame if they have been moved by exactly one plugin. They don't reappear if they are moved by two or more plugins, and they also don't reappear if no plugin ever touches them. It also appears to only affect non-persistent refs which get saved with DELE subrecords. Containers contents, for example, don't show this bug, they are saved correctly.

See attached screenshot: The problem (cup reappearing in lower left corner) occurs on plots B1 and H5 (where only one plugin changes the reference), but not on A0, E3, K7, or N5 (where two or no plugins change the reference): Screenshot

I'll add the other reports as I go along.


First report on LocalRefBeta10: Cups&Plates testbed

Local ref bug continues to be fixed. No doormarker issues. All ori information correct.


Second report: Moving things around testbed

non-persistent refs which were moved by exactly one plugin reappeared after saving -> quitting -> restarting -> loading, as reported above. No problems otherwise. All ori information correct.


Third report: Plates in Balyn's house

No problems any more. This testbed can probably be deprecated in favor of the Referebce Override Testbed, but I'll keep it in the catalogue for the time being as a control for the former.


Fourth Report: Moved House

Checked the Plugin where I moved a house across cells (MVRF record in the plugin). Works; the house vanishes at its original position, and appears at the new one. Still no reoccurence of the one single CTD I had when going near the door in a former LocalRefBeta.


Fifth Report: Reference Override Testbed

Initial display of all references correct. Persistent and non-persistent refs keep getting different treatment if being moved by a plugin after another plugin has deleted them; but we already established that this is a minor inconsistency and not a real problem.


Sixth Report: Reference Override Testbed - Interactions

Went to the testbed, took all cups and bottles, emptied all containers, bought all ingredients from the NPCs. Saved, quit, restarted, loaded.

Result: Cups that have been moved by exactly one plugin reappear (see top of post). No problems otherwise. the container "echoes" have gone. All ori information correct. Ori data for the reappearing cups was correct for the place they reappeared in.


Seventh report: Reference Override Testbed - Removing mods (no KRR)

When removing mods, and loading the non-KRR'ed save, all references got reset correctly.


Eighth report: Reference Override Testbed - Removing mods (with KRR)

After removing the mods and loading the KRR save, all items had been merged back correctly, including container contents and NPC inventory.

There is one oddity: container's position was reset, NPC's position wasn't, although I interacted with both before saving. I investigated three possible reasons for that:

Hypothesis H1: They get treated differently because NPCs are persistent and containers aren't. I'll redo tests with persistent containers.

Hypothesis H2: "Container information isn't saved to the savegame." Savegame analysis however revealed that DATA subrecords for the containers are present. The coordinates in the save matched the coordinates of the *moved* containers, yet the game displayed them at their *original* position after loading the KRR'ed save. Discarded H2.

Hypothesis H3: NPCs are influenced by gravity, containers are not. This might lead to a situation where container position are discarded as "unchanged", whereas NPC locations are kept because they have changed. Tried to investigate by entering the cell with clipping turned off. Behavior of containers and NPCs didn't change.

Ninth report on LocalRefBeta10: KRR'ed containers

Okay, I'm further investigating container. There's definitely something odd, here's what I have now:

- When loading a KRR'ed save, non-empty containers reset their contents. This happens regardless of whether the mod that changed them was removed or not. It also doesn't matter whether I simply didn't take everything out of the container or whether I took all and then put something back in. If the container isn't empty, it respawns its content when loading a KRR'ed save.

- If containers were moved by a plugin, and this plugin gets removed, then the containers will be displayed at their origonal position after loading a krr'ed save, although the moved position is stored in the save (this may be working as designed depending on how your merge-back algorithm works).

- Persistent containers cannot be moved by a plugin, they simply remain at their original position.

The above is not 100% secure as I'm still researching it (will update this post when I find something new), but there's definitey something strange going on with cntainers.

I did a cursory check on doors, these seem to work okay. Depending on what you see in the code, would you recommend testing other object types as well, or are containers a special case?

Should we test KRR effects on spawn points? Documentaton on those is scarce ... from what I know about them, I wouldn't expect them to be treated different than other objects by our changes, but now the container oddities got me thinking.

Hi Hrnchamd,

since you're preparing a release candidate (if I understood you correctly), here are some suggestions of things that might be good to include in the readme:

- KRR feature: After using KRR, *immediately* save and quit the game. (You told me to remind you about this. )

- After removing mods: As with regular Morrowind, you'll get error messages about missing objects, spells, etc. This is normal.

- Compatibility: The Morrowind Code Patch should be compatible with external programs that hook into Morrowind, like MWE, MWSE, or MGE.

- Compatibility 2: Check post 126 about Mash compatibility, feel free to lift anything from it if you think it makes sense. For that matter, feel free to include anything I ever said or did if you think it helps, this goes for posts, testbeds, the showcase mod, etc. Also, feel free to modify any things you're including in any way you see fit, I'm decidedly non-touchy in this regard.

- Testing: I think we can safely say that the patch has been tested extensively, however due to the complexity of the game it's not possible to test every aspect and every interaction. If there are bugs left, please mention them at (link to thread). (Giving a mails address might also be good since the thread will eventially get pruned from the forums.)

I only wish Hrnchamd can make the patch program more compatible with all kind of Morrowind.
get rid of "your version not correct" chance.

Very minor feature request: creating a logfile of the output. I know about running it in a command window to see the output; but running it under Vista, if the UAC isn't already disabled, it's not enough -- you'll get the warning message about it accessing the computer, and after you okay it it'll run in a new window which will be insta-closed when it's over, not in the command window.

Sounds like a good idea to me. Also handy for the people who are having issues but are not very good with computers and don't know how to run it in a command window.

Also for the people who are packrats and will happily keep logs of EVERYTHING if such logs are offered. ...Like me.

The installer will have a GUI soon, it will remember which features you've selected and stuff. Could people with a European / Russian edition of Morrowind contact me?



Found it, containers take a different code path through the save ref routine, KRR missed reindexing them. Container contents aren't stored in refs. Spawn points have a very simple system, LVCR just adds new points with zero matching, and DELE deletes them.

The single replace issue is caused by another load ordering again, I'll write a more general rematcher in.

Everything seems to be working fine up to release 6. I do have some requests though. The first is that is that you put the spellmaking duration back to the original 1440 seconds. I like making spells that last longer than 180 seconds. The second, is if you are able to make the elemental shields increase AR like thier description says, then could you?

It's a bit late for any changes to spell effects now, you should have asked quite a while ago. I'll change the spell maker duration to 600, okay with that?

Why is it too late? And I DID mention it some time ago in thread 2 where you said you'd look into it. Link. I almost forgot that I did find that the Elemental shields decrease damage... Sorry about that. I'd like to know if you find the same results though. However there is still the issue of the AR not updating that I mentioned, if anything can be done about that.

Well, of course 600 is much nicer. I just feel that taking something away from the game isn't fair. Not that it would be easy or practical to make a spell last that long without cheating, but just to know that the option is there. Having it 2880 would be fun just to have an "all day spell".

Would different versions/configurations be possible? You mentioned a GUI that could remember different feartures...

Arsuru, I believe that is the idea. Its been mentioned a few times already, but the "repairs" are supposed to be modular. You can add in or take out fixes.

As for it being too late, Hrnchamd mentioned in the previous thread that he was not taking anymore bug requests because he had real life things coming up very soon. Actually...wasn't all these real life things supposed to start this week?

I see, glad you found it. I'll see whether I can test for something else that may have taken a different path too ... tested doors already (these seem to be okay, their locked/trapped status gets merged back correctly with KRR, also their position and scale). I doubt that armor, clothing etc. are treated differently from the misc items I've already tested extensively, but I haven't run tests on activators yet, so that's something I'll have a look at now.


Okay, good. I don't know much about them (documentation is scarce), but read something about spawnpoint refs wrapping around creature refs in the savegame, and I wasn't sure whether that might require special attention. Thanks for the explanation.


Great. I'm here, in case you want some tests to be run - testing procedures should be really quick now with the new testbed.

One more localref beta. Please test:

- KRR on containers
- does persistent container behaviour match up with normal Morrowind
- reloading pre-modification saves always resets things
- reloading post-modification saves always propagates changes

--------

If anyone has an Italian or Russian version of Morrowind please PM me.

I'm currently integrating the map so it only rescales if there's landmass outside the original map. It will auto-rescale savegame maps too.

I recall that, I just hadn't realized it had been so much time already. Well, I hope he can continue at a latter date, or someone else can. Though, I'm sure it's beyond my current understanding, I'd certainly be willing to learn how all this was done as I'd like to see as many bugs fixed as possible.

So does this mean the map won't have seams on it anymore?

Reports on LocalRefBeta11:

Containers continue to be a tough nut to crack, but apart from them everything else seems to run smoothly. The cup echoes from LocalRefBeta10 have been fixed.

Current container behavior:

a ) Non-persistent containers that were changed by at least one mod reset their contents on every save (not good). Containers that aren't changed by any mods keep their contents.

b ) After removing a mod that changed them, and loading a non-KRR'ed save from before, non-persistent containers reset their contents and position (good)

c ) After removing a mod that changed them, and loading a *KRR'ed* save from before, non-persistent containers remain empty if they were before (good), reset their content if the weren't empty (not good), and reset position (unclear if desired or not).

d) Persistent containers cannot be moved by plugins. Caution: This may be an artifact of my testing method. I took one of my testbeds and set two barrel types to persistent in the same mod 1280 that changed these barrels. This works in regular Morrowind, it might lead to problems in the LocalRefBeta. I think the findings e)-g) make more sense when interpreted as "setting non-persistent objects to persistent with a plugin causes the new reference to be ignored", so it might be a problem of changing persistence with a mod, not a problem of persistent containers. I'll investigate further (see further below).

e ) Persistent containers reset their contents on every save (bad) unless they were empty (good).

f ) After removing a mod that changed them, and loading a non-KRR'ed save from before, *persistent* containers again (like in (e) ) reset their contents unless they were empty.

g ) After removing a mod that changed them, and loading a *KRR'ed* save from before, *persistent* containers again (like in (e) ) reset their contents unless they were empty.


All other tests run on LocalRefBeta11 had positive results:

- Local ref bug continues to be fixed

- No more "echoes" and doubling of references

- no references reappearing when they shouldn't

- correct display of references that were moved avross cells

- correct reset of reference after removing mods and loading non-KRR'ed saves

- correct merging of ref data (except for containers and heir contents) after removing mods and loading KRR'ed saves


I'll set up a specific container testbed for more in-depth tests, especially to get safer data on persistent containers. If I find anything new, I'll update this post or write a new one.

Edit: Also, at least non-persistent containers (haven't tested persistent yet) apparently relock themselves after loading a save. I'll investigate further.

After a few tests in-game with the map patch: when using a mod with landmass far from the original map, and when loading a saved game, the map does not display the name of cities when the mouse cursor is on them. The cursor showing the player's position does not appear any more.

Hrnchamd you are a god among dwemer. And although I don't have the patients to read the reference bug feedback from Psyringe, you too are obviously a god.
P.S. - Good luck on your mid-terms.

Hrnchamd, Any chance you can include a no-cd patch option? I've gotten so used to not having to have the disc in drive, I'm spoiled.

You can add a normal no-CD patch after his, if I'm not mistaken.

Refer to the stickies as to the discussion of illigal activities.

In short: No.

New thread.