As you may be aware, in Oblivion esps are completely isolated from each other, which leads the inability to cause mods to cooperate with each other, or create specialized mods like merged leveled lists. This is in contrast to Morrowind, where these things were all possible. For more information/background, see UESP: Mod Integration. (Closest CS Wiki page is: Sharing Information between Two Plugins.)

Well, I was beating my head against the leveled list problem over the weekend, and essentially in the end decided that it was impossible. Which made me think what Bethsoft would have to fix in order for it make it work. Which was simple, all the engine had to do was... XXXX. If it did that, then problem solved. At which point, I realized that I wasn't absolutely sure that the engine didn't do that already. Well, guess what... (Bear in mind that this was one quick test. Killer problems may yet emerge. In fact, given the problems with esm'ifying mods, I kind of expect killer problems to emerge. Anyway...)

Experiment:
1) I created two simple esps that each defined a trivial object (a soul gem).
2) Converted these two into esms using Wrye Bash.
3) Created a third esp and made it dependent on oblivion.esm and two new esms. Created a leveled list in it that included the two items from the first two esms.
4) Loaded all those up in game and tested the leveled list by giving myself the leveled list, which resolves into randomly giving me one of the items. This worked fine. No big suprise, but wanted to make sure first.
5) Quit, went back to Bash and edited the third esm and changed the mod's master names so that it pointed to the esp versions instead of the esm versions.
6) Reloaded. No errors on load. Gave myself the leveled list items as before, and... It worked! No CTD, no weird landscape errors, etc!

So, in short... even though the construction set won't save dependencies on an esp, apparently the Oblivion engine accepts them and treats them just as if they were esm dependencies!

Which means that you can build leveled list mergers and stuff like that. But again, the caveat... I just did the one quick test, and given the earlier problems with adding esms (which is what this sort of does), I pretty strongly expect similar problems.

Anyway, I'll be busy with RL for a while, so I'm hoping that other people will explore this further. (If I know it doesn't work, I can avoid coding up a leveled list merger for Bash.) Again, procedure is pretty simple...
1) Create an esm clone of mod you want to become dependent on.
2) Use TESCS to build on top of clone.
3) Edit changed mod so that it depends on the original esps instead of the esm.
4) When playing, be sure to include the master esps in the load list (and not include the esms).

The main use that I'm thinking of for this is a leveled list merger. But you might also use it to link houses from different mods, patch official mods, etc.

Note that this does not rule out using esms directly, which I still think should be good for signalling between mods (though apparently the new esm should avoid modifying anything in Oblivion.esm).

Your timing is excellent, Wrye.

I've also tested a similar approach using ESMs (they don't cause problems if they're clean).

I've recently started a project to do exactly what you suggest, but by automatically building clean ESMs. Here's my basic premise.

ScripterRon has contributed his excellent TES4 Plugin Utility Java source code to the project. I got his source working in Eclipse earlier today and started building out the stubs for the first part (splitting). Most of the foundation is already there in his code. Anything you can contribute would be awesome! I'd love to see this in Wrye Bash at some point.

Here's a tutorial I wrote (with help from Martigen) about how to create clean masters.

Interesting. I've added a comment on your Dirty Esms topic. At the end, I note some problems that you might run into. (Though of course, ScripterRon is more of an expert in this area than I am, and will be able to give better advice.)

Thanks! I'll check it out!

So, basically, the Construction Set won't allow ESP dependencies, but the game itself will?

Edit: Misinterpretable attempt at humor removed.

It appears that way.

Ok... I have just done a little test with the method Wrye lists above, and have had some success.

I use Oscuro's_Oblivion_Overhaul, and was looking through it in the CS the other day. I noticed 4 weapons that have 0 count and 0 users, but still have .nif's and meshes etc. meaning they are in the .esp, but not used anywhere in the game. I decided to get them into my game (2 in particular), and did a little test editing my OOO.esp. The 2 swords work fine after a little jigging (they are set for 1hand blade when the .nifs are 2hand blade... easy fix). So, I remembered this post, and decided to try out Wrye's method above.

Heres what I did.

1) Took a fresh copy of Oscuro's_Oblivion_Overhaul.esp (mine has a lot of personal tweaks, mergers etc). Renamed it OOOesmtest.esp.
2) Converted the OOOesmtest.esp to OOOesmtest.esm, then renamed it to Oscuro's_Oblivion_Overhaul.esm
3) Loaded up the CS with Oblivion.esm and Oscuro's_Oblivion_Overhaul.esm, and nothing active.
4) Changed the 1hand to 2hand for the two swords and placed them in the world (next to Hermaeus Mora's shrine). I got some warnings when opening the entries for these two swords, but I figured they where because they where missing icons.
5) Saved as a new OOOesmtest.esp. Got a leaking shader warning, but ignored it.
6) Loaded up Wrye Bash and changed the dependency of OOOesmtest.esp from Oscuro's_Oblivion_Overhaul.esm to Oscuro's_Oblivion_Overhaul.esp.
7) Loaded up the game, making sure that Oscuro's_Oblivion_Overhaul.esm was not checked, while Oscuro's_Oblivion_Overhaul.esp and OOOesmtest.esp where.
8) Went to Hermaeus Mora's shrine, checked and both swords where there and working fine. Except for their missing icons.
9) Quit and came here to tell my tale.

So, in principle, my OOOesmtest.esp makes changes to and uses Objects defined in Oscuro's_Oblivion_Overhaul.esp, and as such, is no longer isolated. I think it's also promising that OOO.esp is so big AND the Oscuro's_Oblivion_Overhaul.esm would have been making changes to Oblivion.esm, yet it still works. In addition, the Oscuro's_Oblivion_overhaul.esp that I loaded the game up with was NOT the same version as the .esp I converted to .esm and based the OOOesmtest.esp, but is the one I made the OOOesmtest.esp dependant upon in WryeBash (my OOO.esp, as I say, has a lot of mods manually and PIU merged into it, as well as some other tweaks).

It does, therefore, appear that the engine handles the dependencies fine, jsut the CS doesn't.

Admittedly I haven't done extensive testing.... there where no apparent crazyness in landscapes etc, or any funny behaviour. But I think it would need more than a 30 second test. Which is what I'm away to do now.

If any one can see some fault in my method, or anything else, please let me know. I'm pretty familiar with the CS, but don't know much about the whole isolation problem, except that its there.

Anyways... Thank you Wrye, both for your bash and for this. This is a great discovery thats going to bring some avenues and abilities back into TES modding.

Awesome! That's very encouraging, Malbulga.

If this ends up being a stable solution, it may end up being one of the biggest developments for modders yet. I'll see about doing some more testing of this with Open Cities this weekend .

I've got at least one mod that was built like this months ago. (One of the Vampire mods, I think ParasiteX's.) The only issue with it is that if you want to alter this mod, you first have to edit it to point to an esm and convert the "master" esp into a real esm again. Takes a few minutes, but it works. I only learned of the dependency because I did want to edit it, and thought "How clever!".

Anyway, it works. That's a popular mod, has been around a good long time, and one version of it used that method to interact nicely with a previous OOO build.

Fantastic! That's very good news!

I think this is a great short-term solution, especially for mods that don't alter a lot of stuff in Oblivion.esm.

KomodoDave and I will continue on our mod-splitter project to make clean masters, though, because it will be a lot better in the long run.

Yeah... I'm finding the switching dependency for your .esp everytime you want to jump between OB and the CS a little tedious (although Wryebash is very user friendly). Its maybe because i'm "tired", and jumping back and forth every minute or so (I'm learning the intricacies of custom summons... once you get them to appear, and stay there, you've got to find a way to stop them attacking you on sight... I've done that, but still can't get them to attack my enemies... but thats for a whole different thread).

Anyways... I've got good news and bad news. Good news is that the method is definitely working. I've got an .esp that lets you summon an Auroran (from KotN)... just he's not a very helpful Auroran yet.

Bad news... when saving my new .esp I'm getting alot of assertion errors, of the type:


and


Hitting Ignore and continuing gets past them... and the .esp saves OK, and seems to work fine enough (though thats just from testing the functionality of the new .esp, and not to see if any glitches have popped up). I'm sure i've seen these messages before though, when trying to save an .esp dependant on another .esp... but I don't really know what to make of them.

Also, there appears to be more in my new .esp than I actually wanted... Looking at it in details on the CS I'm getting cell changes (interestingly enough the same cells as in the error messages 'Failed to CreateGroupData for CELL') and changes to DIAL Group, GREETING/HELLO/GOODBYE/INFOGENERAL/Attack/Hit/Question... when I haven't made any changes to Dialogue (I created a copy of the Auroran and put a script on it, made a new script for the summon and a new scripted spell to call the script, and a new book to add the spell to the player). So I can't figure out where the DIAL changes are coming from... I only noticed them because I ran the conflict detector in OBMM and the came up... but Knights.esp (which my new .esp is dependant upon) doesn't show the same conflicts... so it isn't as if my.esp is just copying something from Knights.esp.

I was figuring I could just use TESSnip to get rid of these "dirty" entries from the .esp... then I might try and find some guinea pigs willing to give it a whirl (once the Auroran starts being a good Auroran, and not a waxwork Auroran).

Anyways... just wanted to let you guys know how its going, and see if anyone could nail down where the errors on save may be coming from, and the additional changes in the .esp.

If you're building on top of knights.esp, you must have done the byte swap (using Bash or something else) to make TESCS think that knights.esp is a 0.8 file instead of a 1.0 file. But that "conversion" doesn't actually change any file formatting. Perhaps the errors you're running into are TESC choking on 1.0 file formatting.

In short, it would be interesting to see if similar problems arise when modifying non-1.0 mods.

I've just released Wrye Bash 0.35 which adds a new mod file command: "Esp/Esmify Masters". Pretty much does what it says. This should make the esp/esm bait and switch go a little faster. (Note this is a command on the mod context menu, NOT on the masters list for the mod.

Just downloaded it... that espify/esmify masters thing is a nice touch.

Had one little problem extracting it. Got an error with...

c:\...\mopy\Images\Thumbs.db
Error # 23235
Error opening file.
CRC Error, Could not be extracted correctly.

I was installing over the previous version, so I'm assuming I'll already have that file, and it could just be a problem on my end, rather than with the archive. Just wanted to let you know though.

Malbulga, when you're working in the CS you're using knights.esm, right?

Yes. Am using Knights.esm in the CS.

Like I say, I recognised the assert errors from trying to make an .esp dependant upon another .esp in the CS... only, when I did that the saved .esp was corrupted and unuseable. But the .esp I'm getting just now has some 'junk' but is perfectly useable. I figure I'll just have to cut the 'junk' away with TESSnip, but I get the errors everytime I save the .esp, so would figure I'd get the junk everytime. So I'm waiting till the mods all finished before cleaning it and just keeping my saves clean by not saving.

Just to let any one who is interested know...

I released the Summon Auroran mod earlier today. It is dependant upon Knights of the Nine (knights.esp) and has been tested and works fine. Release thread can be found Here. Thanks to you guys this was possible.

Reminders for those folks who don't use Bash that much:

Copy to Esp/Esm: Bash has a "Copy to Esm/Esp" command that will copy/convert any mod to the opposing type (esp to esm, esm to esp). (E.g., right-click on "Yoda.esp" and select "Copy to Esp". This will create a "Yoda.esm" file.) This works nicely with the espify/esmify masters command.

Snapshots: If you're working on a large mod and going through successive iterations, you may want to create snapshots of the mod on a regular basis -- just in case something bizarre happens and the mod quits working for no apparent reason (I had this happen several times for my Morrowind mods). To take a snapshot, right click on mod and select "File:Snapshot...". You can later revert to any snapshot using the Revert to Snapshot command. Or if you want both available at once, just go grab the snapshot out of the Data\Bash\Snapshots directory.

Changelog: In a related note, you can also use Bash to keep track of changes associated with a mod you're working on by co-opting the readme viewer/editor feature. Note that you can leave the file in Edit mode permanently -- changes will be autosaved whenever you leave the doc. If you want notes not associated with any particular mod, then just createa dummy esp and attach a "Readme" doc to it. See Bash docs for more info.

This technique is very interesting. I'll have to see if it can work for Morrowind too -- you can't delete or move with an esp a reference in a cell that's from another esp, so it could be useful for compatibility patches for mods that aren't esm.

FYI, the latest version of FranOOOMMM! makes use of this technique to merge creature spawn lists from the OOO.esp, the MMM.esm, and Fran's.esm. It's still just a beta release right now, but it appears to be working.

And bringing it back to the beginning... Bash now (v 0.36) has a leveled list merging feature.

Back on topic.... We haven't seen any major problems yet, which nice and a bit surprising. Perhaps Mod De-Isolation will just be a normal part of life soon and this topic will eventually keel over from disuse...

I think that's the way it's looking.

FYI: I've posted a draft De-Isolation Tutorial on the CS Wiki.

Looks good. I've been away for a while and just got back today... good to see people are still working on this problem, as recently as yesterday as well. Keep up the good work everyone.

Ah, I was looking for this [bump]

Hmm... still can't find what I was looking for. Has anyone had success with making an .esm dependent on another .esm?

While I was working on the UOMP, I was wondering who first discovered ESP dependency/mastering and thus made it possible. I figured that this info would be lost and I wouldn't be able to say thanks to whoever it was.

Thanks Wrye! Brilliant work... as usual.

Hehe. Yep. In fact, I think phoenixamon noted that ParasiteX had made a de-isolated mod some time earlier, but as often happens, the news didn't reach the right ears, so we all continued on in ignorance until Wrye started this.

I suppose it's also fortunate that there were a few of us around to pounce on the concept this time -- you can tell by the dearth of comments that most folks had a hard time grasping the significance -- although I'm sure Wrye would've forged ahead anyway. In the end it really has made a huge difference in the kinds of things we can accomplish, so three cheers for Wrye!

Thanks y'all. (Damn, I'm good! )

ParasiteX reappeared a while back and I asked him about it. As I recall, he was trying to make his mod compatible with OOO, tweaked the mod with a tool or two, got it to work and went on his way, apparently not realizing that this was a really huge thing. Got released in the mod but apparently none of the users realized it was a really huge deal either. Probably if anyone noticed it a little, they figured it was known technique.

So I reinvented in November after getting annoyed at lack of list mergers. In fact, as you'll see at the beginning of the topic, I figured it out after giving up on solving the problem! (I remember that weekend. I was thinking on the problem fairly intensely for about 12 hours over a couple of days trying to solve it before finally giving up. Then I was still annoyed enough to spend the time to figure out what Bethesda would have to do to allow us to do it. And that's when I got it.)

Combining this and a couple of other experiences regarding innovation:
* It's not enough to find a new technique/fact -- you also need to be able to understand how important that technique/fact is.
* Marketing is very important -- you have to tell people -- more, prosletize it a little, get the right group of other people to notice. Developing it into a useful tool, releasing practical usable demos helps a lot, as does documenting at some findable place (e.g. a wiki).
* Knowledge can sometimes be a bit constraining -- you "know" some stuff that you never re-examine, and that prevents you from seeing new possibilities. (Of course counter to that, you need knowledge to do anything. Hence the advantage of competing researchers/developers -- one developer's ignorance is another's gold.)

Probably none of those realizations is new -- but it's interesting to learn/see them in the context of game modding.

PS: Naturally, I did an endzone dance when I realized the damned thing worked.

Hehe. Good story there, man.

Yeah, this is yet another great example of how preconceived notions about "reality" hinder our progress -- but that's another topic entirely.

[bump] and a question:

Are there at least some objects that can be associated with another .esp, without the bait-switching and other techniques? I'm looking at mmmpld's Keychain mod at the moment. To make another mod work with it, you load up both in the CS with the add-on as active, associate one world object from mmmpld's original (a container, though it's base is from vanilla Oblivion) to a script in the add-on, and save. Works perfectly in-game, and re-loading in the CS.

I think that if you express that question in terms of formid access you'll have your answer.

The reason that works is because the object in question came from Oblivion.esm. If you were trying to use a new object from the keychain mod then you'd have a problem.

MS-DOS rides again?

[bump]

Speaking of which, can we fix this?

The thing I find so awesome is that I used Wrye to fix a problem like this a bit, the mod is no longer part of the other other mod. I was having a problem where when I was modding them I needed to what seemed like de-isolate them, but then when I wanted to play test I had to go in and re isolate them...

The re-isolate part (at least how I see it was to fix a cannot find master, even though the data had it checked), as you can tell I am a bit new at modding, but I still am interested in learning how to do my big plan. I know that I have a lot to do to get it realized, however I know I'll eventually have to get over the hurdle, and this whole issue seems to be it. Right now I am starting to understand the jargon associated with this, but no where near enough. However if you would want a Newbie's help. I'll be happy to assist, because that's one why I can understand this stuff better.

Sincerely

A Mar

I think I learned today something new about mod isolation and de-isolation.

Situation: I gave a translation helper some .esp files dependent on another .esp file, made with standard mod deisolation. As I got the files back I discovered a problem, he has forgotten to esmify to main .esp, so the dependent .esps were borked.

Now I didn't want to do all translations again by hand, so I've readded their missing masters by using Wrye Bash. The files work again!

So I think that's what happening when saving an .esp dependent .esp file with the CS: Everything is saved correctly (including data from masters), except the master list itself. Therefore it's possible to "repair" a file broken by the CS's mod isolation using Wrye Bash's add master function - as long as the broken file isn't opened / saved in the CS again.


Oh and what I really wished to say: Bump, great topic!

Am I the only one who finds it funny that the sub-title of this thread is "Maybe"

Does this have anything to do with the problem I am currently having? I made a worldspace and had been working on it awhile, decided it was eating a lot of memory so ESMified it and continued to work on the landscape in a new ESP with the intention of merging the two once the new ESP was eating up too much memory, and repeat the process. Except when I try to merge these two ESP files, I end up with an ESP that is dependent on my original esp (the one I had ESMified), which is obviously Not A Good Thing.