ES Forum Archive > Morrowind Mods > Benchmarking Morrowind and Mods
I'm in the process of setting up a new game. Since I know that I will have to forgo some mods in order to maintain sufficient FPS, I wanted to know which of the mods I plan to use are especially heavy on the FPS. Hence I did some benchmarking. Since the results may be somewhat interesting for other people, I'm posting them here.
Personally, I think the benchmarking was worth the effort, since it answered some questions I had. For example, I now know that texture replacers don't affect FPS at all, that Balmora Expansion Lite is indeed quite a bit faster than the original mod, and how to find my personal sweet spot between quality and performance in MGE. If my results help someone else answering *their* questions, all the better.
1. Testing Environment
Benchmarking is highly dependent on the enviroment - if your machine is very different from mine, then my results will be of limited value for you. Fortunately, my current hard- and software is/was rather popular:
The most important part for Morrowind's performance is the CPU, so if yours is better or worse, your performance will differ from mine accordingly. The graphics card is less important (since Morrowind doesn't make much use of it by itself). If yours is significantly slower than mine, then you may have the slowdown with high-quality texture mods that didn't show up in my tests. Having more than 2 GB of memory won't improve performance, having less might reduce performance, although I doubt that there's a significant difference between having 1 GB or 2 GB. The type of hard disk doesn't matter for FPS, but will influence loading times (which I didn't measure). Windows XP should be just as fast as Windows 2000, but I have no idea about Vista.
2. Testing Procedure
FPS were measured with different settings and different mods loaded. FPS were measured with the in-built Morrowind FPS counter. If FPS went below 10, I used MGE's counter instead if possible, because Morrowind's counter doesn't display values lower than 5.
Unless mentioned otherwise, all measurements were taken at a screen resolution of 1600x1200, with maximum view distance, maximum AI distance, and no shadows. The FPS cap (in the Morroind.ini) was set to 250 in order to prevent it from interfering with the measuring.
For each different setup, FPS were measured in 7 different places, which were chosen to represent different game situations:
SN: Seyda Neen. Load savegame, appear in Sellus Gravius' office, walk out the door to Seyda Neen, turn left, focus on (i.e. move crosshair to) the brown spot of wall above the door to Arrille's storage room, measure FPS. (A very popular and - with mods - very crowded place, prone to exhibit FPS problems.)
Ba: Balmora. Enter "coe -3,-2" into console, turn right, focus on the highest point of Dura gra-Bol's house, measure FPS. (Another very popular spot which often has very low FPS. Note that you'll be appearing inside a house, but since you can look through the walls, this is not a problem.)
SM: Sadrith Mora. Enter "coe 18,4", turn right, descend to the street, turn left, follow the street through the wooden arc with two lanterns. Right next to the arc, on the right side, is some kind of huge mushroom. Stand next to this mushroom so that you just can't see it any more on the screen, then focus on the glowing part of Amruin's forge. (Note: This was originally thought of as a popular place that was less FPS-consuming than Balmora, but it turned out to be even more so, probably the many round organic structures use more polygons than Balmora's flat ones)
Gn: Gnisis. "coe -11,11", turn right, focus on the middle arch of the Madach trade house. (A spot in a smaller town that traditionally receives little love from modders. Again you'll appear inside a (transparent) building, but for testing purposes this is not a problem.)
MT: Mountain Top. "coe 5,11", wait until sliding stops, move the view a little bit until the casting icon intersects the horizontal and the vertical stone in the lower left corner. (A spot on Red Mountain, just outside the Ghostfence, from where you can look over the whole northern half of the island, if you have distant land activated - intended as a test for exactly that, distant land drawing)
Wi: Wilderness. "coe 12,9", turn right, focus on the forked tree that is halfway greyed out. (A spot in the wilderness, no civilization around)
Oc: Ocean. "coe -11,0". (A spot underwater with very little land in the vicinity.)
No interior locations were chosen since interiors usually don't exhibit any FPS problems.
Since the method of measurement wasn't terribly exact to begin with (see below), I didn't control the environment any further. For example, I didn't reboot my PC or restart Morrowind for every cycle of measurements.
3. Measuring Accuracy
Measuring accuracy turned out to be less than optimal. Under identical situations (reboot PC, start Morrowind with identical configuration, load the same savegame, step out into Seyda Neen), my FPS varied between 42 in one test, and 48 in another. Hence, I recommend not to interpret FPS differences of less than 5%, because they might just be artifacts of the measuring system.
Also, Morrowind's FPS counter has the unfortunate habit to change rapidly. It usually fluctuates within a range of numbers, so measurement was performed by observing these fluctuations and then estimating the average value.
One word about measuring FPS in general: When comparing values, please have in mind that FPS are not a linear scale. A mod that reduces FPS by 10 when FPS originally were at 50, will reduce FPS by less when they originally were at 20. This is due to the nature of the scale. So we cannot say that "a mod uses up 10 FPS", because even on the same hardware, how much FPS a mod uses up is dependent on how much else is going on in the game. I will nevertheless use the term "mod x uses up y FPS" to illustrate differences, but please keep in mind that these values apply only to this specific situation, and will scale even on the same machine when the base FPS are different.
We can, however, observe differences in the impact of certain mods, i.e. we can determine which mods (or options) use up more FPS than others, even though the exact number of FPS used up depends on the circumstances.
4. Results
I had several questions I wanted to answer with these tests, so I'll order the results accordingly.
a) Baseline
Without any changes, the setup described in (2) (screen resolution of 1600x1200, maximum view distance, maximum AI distance, no shadows) led to the following FPS values:
This is my Morrowind baseline for this testing environment, and I'll frequently refer to it.
b ) Morrowind Options: screen resolution has no effect
Which options affect Morrowind's performance the most? I tested minimizing view distance, minimizing AI distance, maximizing the number of shadows, and and reducing the screen resolution to 640x480 (the lowest setting available).
As can be seen, minimizing AI distance had a slight positive effect (about 10% FPS gain) in towns. The effect of minimizing view distance was substantially larger, which was to be expected, since we already know that Morrowind spends most of its time drawing polygons, so reducing the amount of data to be drawn had to improve FPS considerably. Maximizing shadows led to a slight reduction in FPS. The effect of maximizing shadows is about as large as the effect of maximizing AI distance.
A reassuring result is that changing the screen resolution had almost no effect on FPS (the effects shown are small and inconsistent, hence I'm inclined to write them off as measuring artifacts). So, in the given environment, there's no reason not to use the highest resolution available. This is important because with high screen resolution, we can forgo Anti-Aliasing, which would cost FPS.
c) FPS optimizer: Increased view distance at no FPS cost
Next, I tested the FPS Optimizer 1.96. I did not use any optimization feature, but only doubled the view distance, which is the most common usage of this program. This had no effect on the FPS, so users of the FPS-Optimizer can double their view distance at no cost.
d) MGE: configurable between quality and performance
Next on my list was MGE. I used version 3.5.5 beta and played around with the options. The "standard" setting is what MGE chooses when you click on "Morrowind defaults" and activate distant land and distant statics; distant statics were generated with the default values that MGE suggested.
As expected, using MGE reduced FPS a bit, but thanks to the continuous improvement of this tool, the impact isn't excessive. We'll see later on that using MGE with these standard settings has about the same impact as one of the more resource-hungry mods. The MT (Mountain Top) location demonstrates this very well: Without MGE, Morrowind basically just renders white fog, and achieves 130 FPS doing that. MGE renders a huge portion of the northern half of Vvardenfell, embellishes it with distant statics, and still achieves 83 FPS.
In the "land only" setting, I used the same settings, but with distant statics switched off. This brought performance back to the baseline level (i.e. the performance without MGE installed) in most cases, and in the cases where it had some impact, FPS were very high to begin with, so a slight reduction won't hurt. This means that distant land in MGE comes at no FPS cost (in the environment where these tests were performed).
"MGE+FPSOptimizer" is a test whether the FPS optimizer (which incurred no FPS cost before) would do so when interacting with MGE. This was not the case, the FPS optimizer remained costless.
In the "low worldtex" setting, I recreated the world texture with the lowest setting available (128, standard is 2048). This did not lead to any FPS gains, not even in the MT location where a lot of distant land is generated, so there's no reason to use it (in environments such as this one). I wanted to try a higher world texture setting too, but couldn't - whenever I increased it above the default value of 2048, trying to create the texture resulted in an error.
In the "high worldmesh" setting, I use a "very high" detail world mesh (standard is "very low"). This did not have any measurable impact on the FPS except at the MT location, where lots of distant land get rendered. There, it brought FPS from 83 down to 68, which is still higher than in any town I tested. Hence, using a higher detail world mesh might be an improvement for people who appreciate the difference (personally, I rarely notice it in-game unless I concentrate on it).
"best quality" is the setting that you get when you click on the "best quality" button in MGE. This brought FPS down a bit more, but still not critically so.
In "all statics", I reverted back to the standard settings, but created distant statics for statics of *all* sizes (i.e., I set the smallest size that should still be drawn in the distance to 0). This sliced FPS in half compared to the standard MGE settings, and to about 1/3 of the Morrowind baseline in towns. Given that this is an unmodded setting and that mods would eat up a substantial amount of FPS too, I'd regard this setting as not playable on this hardware. It looks beautiful though, especially from the MT location.
In the "max" setting, I used the "all statics" setting together with the "best quality" values. This totally slaughtered FPS (and looked even nicer
).
During testing, I noticed that I didn't have a good location to measure the impact of MGE's water reflection. None of the locations I chose had mainly water, except the last one, but there the player was under the surface, so there wasn't any reflection. Therefore, I made a quick additional test in Seyda Neen (stepping out of Sellus Gravius' office, turning right, walking to the shoreline, measuring FPS with different water settings while keeping everything else at my "standard" level). FPS were 54 with non-reflective water, 44 with reflective water, and 34 with reflecting near statics. (With reflecting distant statics the value remained at 34, but I think the spot I chose barely has distant statics that could reflect, so I'd disregard this last value.)
In short, MGE leaves a very good impression. It has no FPS cost if it's only used to enable distant land, and the display of distant statics can be fine-tuned to wherever the user has his personal sweet spot between quality and performance. This tuning can be done on games in progress without any problem, so it's possible to change the settings when they turn out to be suboptimal later on.
e) Balmora Expanded: Light is actually faster
Next, I started installing mods. Before doing so, I had to decide whether to peform my tests with the "regular" version of Balmora Expanded, or with the "light" version that had been tuned for improved performance.
Installing both, I measured 19 FPS at the "Ba" location with regular BE, and 31 FPS with BE light (down from 42 in my Morrowind baseline). That's an impressive improvement for the light version, and I decided to stick with that one, especially since I'll undoubtedly add more Balmora-affecting mods later on. Hence, every bit of FPS I can save in Balmora (while retaining all the features of the mods) is a good thing.
Later on, I tested all mods mentioned in (f) together with both versions of BE. Measured FPS were 14 with BE light, and 10 with BE regular. Again, the light version was about 50% faster than the regular version.
f) Mods and their impact on FPS
Next, I started to test the FPS impact of certain mods. I first chose four popular mods that are known to be somewhat resource-hungry and measured the FPS impact of each individually, and then of all of them combined (with Balmora Expanded light thrown in too).
"The Wilderness mod 2.1" is a popular addition that (among other things) adds lots of farm animals to the cities of Morrowind. this mod turned out to have a surprisingly large impact on FPS, comparable to adding a whole city expansion, or to the effect of adding MGE with distant statics.
"Children of Morrowind" is another very popular mod. It adds NPCs to cities, which means more polygons to draw, more AI to calculate, and more scripts to process. Its impact varied, probably according to the amount of additions done to the locations tested.
"MCA 5.2" is similar, it too adds NPCs to cities (among other things). It was generally a bit easier on the FPS than Children of Morrowind, except of the location SN, where it adds more NPCs.
"Seyda Neen Complete" is a collection of lots of Seyda Neen mods, and it has a huge impact on performance.
While the FPS impact of each individual mod isn't critical, using all of them together brings FPS down to the range of 14-20 FPS in cities. This is already touching the performance ranges where some players don't regard the game as enjoyable any more. (Naturally this threshold is different for every player; some can't tolerate less than 25 FPS, other can go down to 10 without problems. The majority seems to set this threshold somewhere around 15 FPS. Since FPS will go down further when more mods are added, and also when a game has progressed a bit, more global scripts have started, and the player is carrying around more scripted items, many players will have to make choices between these mods (if playing in a similar environment as the one tested here).
IF MGE is thrown in, FPS go down by another 1-5 frames.
g) Texture replacer has no impact on FPS
It's common community knowledge that high quality texture replacers have an impact on FPS. But is this still true? It definitely was true for the hardware that Morrowind was released for, but since then, graphics cards have evolved a lot.
To test this, I installed the three "Visual Packs" (2.11, 2.2 and XT), a popular texture replacer that was known to be resource-hungry, and measured its FPS impact. And there was none. No matter which setup I tried (no other mods installed, all other mods installed, with or without FPS Optimizer, with or without MGE), there was no difference in performance between the Visual Pack textures and Morrowind's stock textures. In the following table "ST" denotes a setting with stock textures, and "VP" denotes the same setting with Visual Pack textures:
Since a texture replacer that had a reputation as an FPS killer turned out to have no FPS impact whatsoever on modern hardware, it seems that we don't have to worry so much about texture replacers when trying to improve performance.
h) Body / head replacers: Again no impact
Another (and perhaps the most popular) category of replacers are body and head replacers. These do not only replace textures, but also meshes. However, modders have usually taken a lot of care not to use more polygons than Bethesda did in their original meshes (which weren't terribly efficient). Hence, adding these mods should not impact performance.
I tested this with "Better Bodies 2.2" and "THE Facepack Compilation", and it turned out to be true, neither of them had an impact on FPS:
5. Conclusion
Looking at the data collected, the following conclusions may be drawn (again, perhaps limited for the hardware on which the tests were performed):
Personally, I think the benchmarking was worth the effort, since it answered some questions I had. For example, I now know that texture replacers don't affect FPS at all, that Balmora Expansion Lite is indeed quite a bit faster than the original mod, and how to find my personal sweet spot between quality and performance in MGE. If my results help someone else answering *their* questions, all the better.
1. Testing Environment
Benchmarking is highly dependent on the enviroment - if your machine is very different from mine, then my results will be of limited value for you. Fortunately, my current hard- and software is/was rather popular:
- CPU: Athlon64 3500+ (Venice)
- Graphics: nVidia 7600 GT (256 MB)
- Memory: 2 GB PC-3200 RAM (3-4-4-8, rather slow timing), 400 MHz x2 (dual channel)
- SATA1 hard disk
- Windows 2000 SP 4
- DirectX 9.0c Aug 2008
- Morrowind Bloodmoon 1.6.1820
- nothing else running except Firewall, Antivirus
The most important part for Morrowind's performance is the CPU, so if yours is better or worse, your performance will differ from mine accordingly. The graphics card is less important (since Morrowind doesn't make much use of it by itself). If yours is significantly slower than mine, then you may have the slowdown with high-quality texture mods that didn't show up in my tests. Having more than 2 GB of memory won't improve performance, having less might reduce performance, although I doubt that there's a significant difference between having 1 GB or 2 GB. The type of hard disk doesn't matter for FPS, but will influence loading times (which I didn't measure). Windows XP should be just as fast as Windows 2000, but I have no idea about Vista.
2. Testing Procedure
FPS were measured with different settings and different mods loaded. FPS were measured with the in-built Morrowind FPS counter. If FPS went below 10, I used MGE's counter instead if possible, because Morrowind's counter doesn't display values lower than 5.
Unless mentioned otherwise, all measurements were taken at a screen resolution of 1600x1200, with maximum view distance, maximum AI distance, and no shadows. The FPS cap (in the Morroind.ini) was set to 250 in order to prevent it from interfering with the measuring.
For each different setup, FPS were measured in 7 different places, which were chosen to represent different game situations:
SN: Seyda Neen. Load savegame, appear in Sellus Gravius' office, walk out the door to Seyda Neen, turn left, focus on (i.e. move crosshair to) the brown spot of wall above the door to Arrille's storage room, measure FPS. (A very popular and - with mods - very crowded place, prone to exhibit FPS problems.)
Ba: Balmora. Enter "coe -3,-2" into console, turn right, focus on the highest point of Dura gra-Bol's house, measure FPS. (Another very popular spot which often has very low FPS. Note that you'll be appearing inside a house, but since you can look through the walls, this is not a problem.)
SM: Sadrith Mora. Enter "coe 18,4", turn right, descend to the street, turn left, follow the street through the wooden arc with two lanterns. Right next to the arc, on the right side, is some kind of huge mushroom. Stand next to this mushroom so that you just can't see it any more on the screen, then focus on the glowing part of Amruin's forge. (Note: This was originally thought of as a popular place that was less FPS-consuming than Balmora, but it turned out to be even more so, probably the many round organic structures use more polygons than Balmora's flat ones)
Gn: Gnisis. "coe -11,11", turn right, focus on the middle arch of the Madach trade house. (A spot in a smaller town that traditionally receives little love from modders. Again you'll appear inside a (transparent) building, but for testing purposes this is not a problem.)
MT: Mountain Top. "coe 5,11", wait until sliding stops, move the view a little bit until the casting icon intersects the horizontal and the vertical stone in the lower left corner. (A spot on Red Mountain, just outside the Ghostfence, from where you can look over the whole northern half of the island, if you have distant land activated - intended as a test for exactly that, distant land drawing)
Wi: Wilderness. "coe 12,9", turn right, focus on the forked tree that is halfway greyed out. (A spot in the wilderness, no civilization around)
Oc: Ocean. "coe -11,0". (A spot underwater with very little land in the vicinity.)
No interior locations were chosen since interiors usually don't exhibit any FPS problems.
Since the method of measurement wasn't terribly exact to begin with (see below), I didn't control the environment any further. For example, I didn't reboot my PC or restart Morrowind for every cycle of measurements.
3. Measuring Accuracy
Measuring accuracy turned out to be less than optimal. Under identical situations (reboot PC, start Morrowind with identical configuration, load the same savegame, step out into Seyda Neen), my FPS varied between 42 in one test, and 48 in another. Hence, I recommend not to interpret FPS differences of less than 5%, because they might just be artifacts of the measuring system.
Also, Morrowind's FPS counter has the unfortunate habit to change rapidly. It usually fluctuates within a range of numbers, so measurement was performed by observing these fluctuations and then estimating the average value.
One word about measuring FPS in general: When comparing values, please have in mind that FPS are not a linear scale. A mod that reduces FPS by 10 when FPS originally were at 50, will reduce FPS by less when they originally were at 20. This is due to the nature of the scale. So we cannot say that "a mod uses up 10 FPS", because even on the same hardware, how much FPS a mod uses up is dependent on how much else is going on in the game. I will nevertheless use the term "mod x uses up y FPS" to illustrate differences, but please keep in mind that these values apply only to this specific situation, and will scale even on the same machine when the base FPS are different.
We can, however, observe differences in the impact of certain mods, i.e. we can determine which mods (or options) use up more FPS than others, even though the exact number of FPS used up depends on the circumstances.
4. Results
I had several questions I wanted to answer with these tests, so I'll order the results accordingly.
a) Baseline
Without any changes, the setup described in (2) (screen resolution of 1600x1200, maximum view distance, maximum AI distance, no shadows) led to the following FPS values:
CODE
Location | SN | Ba | SM | Gn | MT | Wi | Oc
Baseline | 42 | 42 | 34 | 42 | 130 | 73 | 130
Baseline | 42 | 42 | 34 | 42 | 130 | 73 | 130
This is my Morrowind baseline for this testing environment, and I'll frequently refer to it.
b ) Morrowind Options: screen resolution has no effect
Which options affect Morrowind's performance the most? I tested minimizing view distance, minimizing AI distance, maximizing the number of shadows, and and reducing the screen resolution to 640x480 (the lowest setting available).
As can be seen, minimizing AI distance had a slight positive effect (about 10% FPS gain) in towns. The effect of minimizing view distance was substantially larger, which was to be expected, since we already know that Morrowind spends most of its time drawing polygons, so reducing the amount of data to be drawn had to improve FPS considerably. Maximizing shadows led to a slight reduction in FPS. The effect of maximizing shadows is about as large as the effect of maximizing AI distance.
A reassuring result is that changing the screen resolution had almost no effect on FPS (the effects shown are small and inconsistent, hence I'm inclined to write them off as measuring artifacts). So, in the given environment, there's no reason not to use the highest resolution available. This is important because with high screen resolution, we can forgo Anti-Aliasing, which would cost FPS.
CODE
Location |SN | Ba | SM | Gn | MT | Wi | Oc
Baseline | 42 | 42 | 34 | 42 | 130 | 73 | 130
min AI distance | 52 | 45 | 39 | 45 | 150 | 73 | 125
min view distnc | 68 | 70 | 54 | 77 | 150 |150 | 230
max shadows | 38 | 36 | 29 | 37 | 150 | 72 | 135
640x480 | 47 | 41 | 31 | 43 | 150 | 69 | 135
Baseline | 42 | 42 | 34 | 42 | 130 | 73 | 130
min AI distance | 52 | 45 | 39 | 45 | 150 | 73 | 125
min view distnc | 68 | 70 | 54 | 77 | 150 |150 | 230
max shadows | 38 | 36 | 29 | 37 | 150 | 72 | 135
640x480 | 47 | 41 | 31 | 43 | 150 | 69 | 135
c) FPS optimizer: Increased view distance at no FPS cost
Next, I tested the FPS Optimizer 1.96. I did not use any optimization feature, but only doubled the view distance, which is the most common usage of this program. This had no effect on the FPS, so users of the FPS-Optimizer can double their view distance at no cost.
CODE
Location | SN | Ba | SM | Gn | MT | Wi | Oc
Baseline | 42 | 42 | 34 | 42 | 130 | 73 | 130
FPS optimizer | 43 | 41 | 33 | 43 | 150 | 69 | 145
Baseline | 42 | 42 | 34 | 42 | 130 | 73 | 130
FPS optimizer | 43 | 41 | 33 | 43 | 150 | 69 | 145
d) MGE: configurable between quality and performance
Next on my list was MGE. I used version 3.5.5 beta and played around with the options. The "standard" setting is what MGE chooses when you click on "Morrowind defaults" and activate distant land and distant statics; distant statics were generated with the default values that MGE suggested.
As expected, using MGE reduced FPS a bit, but thanks to the continuous improvement of this tool, the impact isn't excessive. We'll see later on that using MGE with these standard settings has about the same impact as one of the more resource-hungry mods. The MT (Mountain Top) location demonstrates this very well: Without MGE, Morrowind basically just renders white fog, and achieves 130 FPS doing that. MGE renders a huge portion of the northern half of Vvardenfell, embellishes it with distant statics, and still achieves 83 FPS.
In the "land only" setting, I used the same settings, but with distant statics switched off. This brought performance back to the baseline level (i.e. the performance without MGE installed) in most cases, and in the cases where it had some impact, FPS were very high to begin with, so a slight reduction won't hurt. This means that distant land in MGE comes at no FPS cost (in the environment where these tests were performed).
"MGE+FPSOptimizer" is a test whether the FPS optimizer (which incurred no FPS cost before) would do so when interacting with MGE. This was not the case, the FPS optimizer remained costless.
In the "low worldtex" setting, I recreated the world texture with the lowest setting available (128, standard is 2048). This did not lead to any FPS gains, not even in the MT location where a lot of distant land is generated, so there's no reason to use it (in environments such as this one). I wanted to try a higher world texture setting too, but couldn't - whenever I increased it above the default value of 2048, trying to create the texture resulted in an error.
In the "high worldmesh" setting, I use a "very high" detail world mesh (standard is "very low"). This did not have any measurable impact on the FPS except at the MT location, where lots of distant land get rendered. There, it brought FPS from 83 down to 68, which is still higher than in any town I tested. Hence, using a higher detail world mesh might be an improvement for people who appreciate the difference (personally, I rarely notice it in-game unless I concentrate on it).
"best quality" is the setting that you get when you click on the "best quality" button in MGE. This brought FPS down a bit more, but still not critically so.
In "all statics", I reverted back to the standard settings, but created distant statics for statics of *all* sizes (i.e., I set the smallest size that should still be drawn in the distance to 0). This sliced FPS in half compared to the standard MGE settings, and to about 1/3 of the Morrowind baseline in towns. Given that this is an unmodded setting and that mods would eat up a substantial amount of FPS too, I'd regard this setting as not playable on this hardware. It looks beautiful though, especially from the MT location.
In the "max" setting, I used the "all statics" setting together with the "best quality" values. This totally slaughtered FPS (and looked even nicer
).CODE
Location |SN | Ba | SM | Gn | MT | Wi | Oc
----------------------------------------------------
Baseline | 42 | 42 | 34 | 42 | 130 | 73 | 130
MGE standard | 32 | 32 | 28 | 30 | 83 | 56 | 43
----------------------------------------------------
MGE land only | 43 | 42 | 33 | 43 | 111 | 65 | 83
MGE+FPSOptimizer| 34 | 31 | 26 | 30 | 86 | 57 | 42
MGE low worldtex| 34 | 31 | 26 | 31 | 85 | 56 | 43
MGE hi worldmesh| 33 | 32 | 27 | 30 | 68 | 54 | 39
MGE best quality| 26 | 29 | 28 | 26 | 45 | 38 | 29
MGE all statics | 16 | 11 | 13 | 14 | 25 | 17 | 20
MGE max | 4 | 3 | 5 | 5 | 8 | 8 | 12
----------------------------------------------------
Baseline | 42 | 42 | 34 | 42 | 130 | 73 | 130
MGE standard | 32 | 32 | 28 | 30 | 83 | 56 | 43
----------------------------------------------------
MGE land only | 43 | 42 | 33 | 43 | 111 | 65 | 83
MGE+FPSOptimizer| 34 | 31 | 26 | 30 | 86 | 57 | 42
MGE low worldtex| 34 | 31 | 26 | 31 | 85 | 56 | 43
MGE hi worldmesh| 33 | 32 | 27 | 30 | 68 | 54 | 39
MGE best quality| 26 | 29 | 28 | 26 | 45 | 38 | 29
MGE all statics | 16 | 11 | 13 | 14 | 25 | 17 | 20
MGE max | 4 | 3 | 5 | 5 | 8 | 8 | 12
During testing, I noticed that I didn't have a good location to measure the impact of MGE's water reflection. None of the locations I chose had mainly water, except the last one, but there the player was under the surface, so there wasn't any reflection. Therefore, I made a quick additional test in Seyda Neen (stepping out of Sellus Gravius' office, turning right, walking to the shoreline, measuring FPS with different water settings while keeping everything else at my "standard" level). FPS were 54 with non-reflective water, 44 with reflective water, and 34 with reflecting near statics. (With reflecting distant statics the value remained at 34, but I think the spot I chose barely has distant statics that could reflect, so I'd disregard this last value.)
In short, MGE leaves a very good impression. It has no FPS cost if it's only used to enable distant land, and the display of distant statics can be fine-tuned to wherever the user has his personal sweet spot between quality and performance. This tuning can be done on games in progress without any problem, so it's possible to change the settings when they turn out to be suboptimal later on.
e) Balmora Expanded: Light is actually faster
Next, I started installing mods. Before doing so, I had to decide whether to peform my tests with the "regular" version of Balmora Expanded, or with the "light" version that had been tuned for improved performance.
Installing both, I measured 19 FPS at the "Ba" location with regular BE, and 31 FPS with BE light (down from 42 in my Morrowind baseline). That's an impressive improvement for the light version, and I decided to stick with that one, especially since I'll undoubtedly add more Balmora-affecting mods later on. Hence, every bit of FPS I can save in Balmora (while retaining all the features of the mods) is a good thing.
Later on, I tested all mods mentioned in (f) together with both versions of BE. Measured FPS were 14 with BE light, and 10 with BE regular. Again, the light version was about 50% faster than the regular version.
f) Mods and their impact on FPS
Next, I started to test the FPS impact of certain mods. I first chose four popular mods that are known to be somewhat resource-hungry and measured the FPS impact of each individually, and then of all of them combined (with Balmora Expanded light thrown in too).
"The Wilderness mod 2.1" is a popular addition that (among other things) adds lots of farm animals to the cities of Morrowind. this mod turned out to have a surprisingly large impact on FPS, comparable to adding a whole city expansion, or to the effect of adding MGE with distant statics.
"Children of Morrowind" is another very popular mod. It adds NPCs to cities, which means more polygons to draw, more AI to calculate, and more scripts to process. Its impact varied, probably according to the amount of additions done to the locations tested.
"MCA 5.2" is similar, it too adds NPCs to cities (among other things). It was generally a bit easier on the FPS than Children of Morrowind, except of the location SN, where it adds more NPCs.
"Seyda Neen Complete" is a collection of lots of Seyda Neen mods, and it has a huge impact on performance.
While the FPS impact of each individual mod isn't critical, using all of them together brings FPS down to the range of 14-20 FPS in cities. This is already touching the performance ranges where some players don't regard the game as enjoyable any more. (Naturally this threshold is different for every player; some can't tolerate less than 25 FPS, other can go down to 10 without problems. The majority seems to set this threshold somewhere around 15 FPS. Since FPS will go down further when more mods are added, and also when a game has progressed a bit, more global scripts have started, and the player is carrying around more scripted items, many players will have to make choices between these mods (if playing in a similar environment as the one tested here).
IF MGE is thrown in, FPS go down by another 1-5 frames.
CODE
Location | SN | Ba | SM | Gn | MT | Wi | Oc
Baseline | 42 | 42 | 34 | 42 | 130 | 73 | 130
---------------------------------------------------
Wilderness Mod | 30 | 23 | 22 | 31 | 150 | 64 | 140
Children of MW | 41 | 32 | 27 | 29 | 125 | 73 | 145
MCA 5.2 | 37 | 37 | 27 | 36 | 150 | 73 | 145
SN Complete | 23 | | | | | |
All | 15 | 14 | 18 | 20 | 135 | 62 | 125
All + MGE std. | 13 | 13 | 16 | 17 | 83 | 48 | 43
Baseline | 42 | 42 | 34 | 42 | 130 | 73 | 130
---------------------------------------------------
Wilderness Mod | 30 | 23 | 22 | 31 | 150 | 64 | 140
Children of MW | 41 | 32 | 27 | 29 | 125 | 73 | 145
MCA 5.2 | 37 | 37 | 27 | 36 | 150 | 73 | 145
SN Complete | 23 | | | | | |
All | 15 | 14 | 18 | 20 | 135 | 62 | 125
All + MGE std. | 13 | 13 | 16 | 17 | 83 | 48 | 43
g) Texture replacer has no impact on FPS
It's common community knowledge that high quality texture replacers have an impact on FPS. But is this still true? It definitely was true for the hardware that Morrowind was released for, but since then, graphics cards have evolved a lot.
To test this, I installed the three "Visual Packs" (2.11, 2.2 and XT), a popular texture replacer that was known to be resource-hungry, and measured its FPS impact. And there was none. No matter which setup I tried (no other mods installed, all other mods installed, with or without FPS Optimizer, with or without MGE), there was no difference in performance between the Visual Pack textures and Morrowind's stock textures. In the following table "ST" denotes a setting with stock textures, and "VP" denotes the same setting with Visual Pack textures:
CODE
Location | SN | Ba | SM | Gn | MT | Wi | Oc
---------------------------------------------------
Baseline (ST) | 42 | 42 | 34 | 42 | 130 | 73 | 130
VP | 47 | 43 | 33 | 43 | 150 | 75 | 150
---------------------------------------------------
ST + all mods | 15 | 14 | 18 | 20 | 135 | 62 | 125
VP + mods | 14 | 14 | 18 | 20 | 150 | 61 | 125
---------------------------------------------------
ST + FPSO | 43 | 41 | 33 | 43 | 150 | 69 | 145
VP + FPSO | 46 | 44 | 32 | 43 | 160 | 75 | 150
---------------------------------------------------
ST + MGE | 32 | 32 | 28 | 30 | 83 | 56 | 43
VP + MGE | 33 | 31 | 27 | 30 | 83 | 57 | 43
---------------------------------------------------
ST + mods + MGE| 13 | 13 | 16 | 17 | 83 | 48 | 43
VP + mods + MGE| 13 | 12 | 16 | 17 | 83 | 49 | 43
---------------------------------------------------
Baseline (ST) | 42 | 42 | 34 | 42 | 130 | 73 | 130
VP | 47 | 43 | 33 | 43 | 150 | 75 | 150
---------------------------------------------------
ST + all mods | 15 | 14 | 18 | 20 | 135 | 62 | 125
VP + mods | 14 | 14 | 18 | 20 | 150 | 61 | 125
---------------------------------------------------
ST + FPSO | 43 | 41 | 33 | 43 | 150 | 69 | 145
VP + FPSO | 46 | 44 | 32 | 43 | 160 | 75 | 150
---------------------------------------------------
ST + MGE | 32 | 32 | 28 | 30 | 83 | 56 | 43
VP + MGE | 33 | 31 | 27 | 30 | 83 | 57 | 43
---------------------------------------------------
ST + mods + MGE| 13 | 13 | 16 | 17 | 83 | 48 | 43
VP + mods + MGE| 13 | 12 | 16 | 17 | 83 | 49 | 43
Since a texture replacer that had a reputation as an FPS killer turned out to have no FPS impact whatsoever on modern hardware, it seems that we don't have to worry so much about texture replacers when trying to improve performance.
h) Body / head replacers: Again no impact
Another (and perhaps the most popular) category of replacers are body and head replacers. These do not only replace textures, but also meshes. However, modders have usually taken a lot of care not to use more polygons than Bethesda did in their original meshes (which weren't terribly efficient). Hence, adding these mods should not impact performance.
I tested this with "Better Bodies 2.2" and "THE Facepack Compilation", and it turned out to be true, neither of them had an impact on FPS:
CODE
Location | SN | Ba | SM | Gn | MT | Wi | Oc
Baseline | 42 | 42 | 34 | 42 | 130 | 73 | 130
---------------------------------------------------
Better Bodies | 46 | 42 | 34 | 44 | 150 | 75 | 150
THE | 49 | 43 | 32 | 43 | 150 | 71 | 150
all mods+BB+THE| 15 | 15 | 19 | 22 | 145 | 64 | 130
Baseline | 42 | 42 | 34 | 42 | 130 | 73 | 130
---------------------------------------------------
Better Bodies | 46 | 42 | 34 | 44 | 150 | 75 | 150
THE | 49 | 43 | 32 | 43 | 150 | 71 | 150
all mods+BB+THE| 15 | 15 | 19 | 22 | 145 | 64 | 130
5. Conclusion
Looking at the data collected, the following conclusions may be drawn (again, perhaps limited for the hardware on which the tests were performed):
- Higher screen resolution does not reduce FPS; higher view distance and AI distance (as well as more shadows) do.
- Texture replacers, the FPS optimizer, and current body and head replacers do not reduce FPS.
- MGE distant land does not reduce FPS except in areas where a huge amount of land gets rendered.
- MGE distant statics reduce FPS, impact can be fine-tuned from extremely heavy to almost nil.
- Several popular mods reduce FPS enough so that some players won't want to use all of them together.
Interesting and well done. Thanks.
Wow. Great job! 
But... MGE results are one other thing that makes me want to cry. You say that you get 83 FPS on the top of the Red Mountain looking in the distance... Well whenever I'm "just looking in the distance" I never get above 25! >:|
But... MGE results are one other thing that makes me want to cry. You say that you get 83 FPS on the top of the Red Mountain looking in the distance... Well whenever I'm "just looking in the distance" I never get above 25! >:|
Very impressive comparison, both in method and results. It must have taken quite some time, too. Thanks you for doing this, Psyringe - it will sure help many. It sure willl help me, for one, but now I have to study and can't get into the tweaking madness 
. However, this is sure a topic to bookmark.
Personally, I had some suspects about texture replacers and resolution, but was just incredulous until seeing the cold hard data. The results with the Optimizer are just heart-warming, while the effect of MGE is not that surprising. By the way, which were your statics settings when generating distant statics? The MGE version I use (which is the same as yours) gives the option to reduce the complexity of some static meshes, and I've found that doing so really helps performance - with little impact look-wise, since the lost complexity is not apparent.
(By the way, if you want to try some serious FPS eating, download Slof's Goth Shop II. Even though the shop is actually a small room in the Foreign Quarter, it lowers my performance more than a crowded Balmora. I don't know if it's the heavy scripting or the enormous detail, but I think the latter is more important, since the slowing happens in menu mode too. Luckily, the awesome clothes sold there do not seem to affect FPS when outside the shop, so it is not a big problem, just something I thought I'd point).
. However, this is sure a topic to bookmark.Personally, I had some suspects about texture replacers and resolution, but was just incredulous until seeing the cold hard data. The results with the Optimizer are just heart-warming, while the effect of MGE is not that surprising. By the way, which were your statics settings when generating distant statics? The MGE version I use (which is the same as yours) gives the option to reduce the complexity of some static meshes, and I've found that doing so really helps performance - with little impact look-wise, since the lost complexity is not apparent.
(By the way, if you want to try some serious FPS eating, download Slof's Goth Shop II. Even though the shop is actually a small room in the Foreign Quarter, it lowers my performance more than a crowded Balmora. I don't know if it's the heavy scripting or the enormous detail, but I think the latter is more important, since the slowing happens in menu mode too. Luckily, the awesome clothes sold there do not seem to affect FPS when outside the shop, so it is not a big problem, just something I thought I'd point).
Thanks for the kind words.
Just two days actually. Unmodded Morrowind loads surprisingly fast, and I made a couple of macros to teleport to the selected locations. Also, all locations were selected to be reached easily (usually just one teleport and then slightly changing the view) since I'm actually a terribly lazy person.
You mean the percebtage setting? Hmm, I always left that at "Full" - I ought to play around a bit with that setting.
QUOTE(Knef @ Sep 10 2008, 10:19 AM) 
Very impressive comparison, both in method and results. It must have taken quite some time, too.
Just two days actually. Unmodded Morrowind loads surprisingly fast, and I made a couple of macros to teleport to the selected locations. Also, all locations were selected to be reached easily (usually just one teleport and then slightly changing the view) since I'm actually a terribly lazy person.
QUOTE(Knef @ Sep 10 2008, 10:19 AM)
By the way, which were your statics settings when generating distant statics? The MGE version I use (which is the same as yours) gives the option to reduce the complexity of some static meshes, and I've found that doing so really helps performance - with little impact look-wise, since the lost complexity is not apparent.
You mean the percebtage setting? Hmm, I always left that at "Full" - I ought to play around a bit with that setting.
Wow, some research there! This ought to be a sticky. And the text should be held on to, as too often really useful stuff like this tends to disappear after a year or two.
Very interesting, greak work !
QUOTE(GoddamnHippie @ Sep 10 2008, 05:38 AM)
Wow, some research there! This ought to be a sticky. And the text should be held on to, as too often really useful stuff like this tends to disappear after a year or two.
Don't misunderstand this thread. As Psyringe explains, this is only very useful to people with similar hardware configurations to his *and* who are interested in the mods he tested. The greater the difference between your computer and his, the less useful this information is to you. If your computer is very different from his, or if you're not interested in any of the mods tested, then this is entirely useless to you.
@ Psyringe, if you do repeat similar tests in the future, I highly recommend you use Fraps' benchmarking feature to measure the average FPS you're actually getting. Your best guesses simply cannot be as good as Fraps' measured values.
Keep up the good work, Psyringe. ;^)
QUOTE(GoddamnHippie @ Sep 10 2008, 12:38 PM)
Wow, some research there! This ought to be a sticky. And the text should be held on to, as too often really useful stuff like this tends to disappear after a year or two.
Maybe not a sticky, but it's definitely worth backing up in the ES Forum Archive for future reference.
(I've already made the request
).
QUOTE(Psyringe @ Sep 10 2008, 04:10 AM)
c) FPS optimizer: Increased view distance at no FPS cost
Next, I tested the FPS Optimizer 1.96. I did not use any optimization feature, but only doubled the view distance, which is the most common usage of this program. This had no effect on the FPS, so users of the FPS-Optimizer can double their view distance at no cost.
Next, I tested the FPS Optimizer 1.96. I did not use any optimization feature, but only doubled the view distance, which is the most common usage of this program. This had no effect on the FPS, so users of the FPS-Optimizer can double their view distance at no cost.
Hmm, that's quite strange. I get quite heavy fps hit from 2xView distance and pretty much have to use the auto adjustment settings (decreases view distance when needed to keep certain FPS level == 25-35 in this case) to keep it at playable levels. Comparison shots here (ignore the grass tests, look "Vanilla" vs. "Vanilla 2xView distance"): http://www.bethsoft.com/bgsforums/index.ph...;#entry12433349
Maby it's the higher FOV setting, but even then I don't think the hit should be that high. Or maby it's because of the dual core CPU? I think I have red somewhere that Morrowind doesn't like dual core CPUs much... The graphics card is about the same level as yours. Otherwise my system should be slightly faster.
Anyway, thanks for your great research. Good work!
