Book Texture Inconsistencies (#243 Revisited)

[patmauro]

Intro

I opened #243 last year to mirror a change I pushed to MOULa, which attempts to make some improvements to the way book textures are handled in the game by making them all more internally consistent - but I think I bit off a little more than we could chew with this one: not only is this a difficult one to review, due to changing 22 PRPs at once, but the other big problem is that I came to you with a solution already in mind, without having given any time to actually discuss the problem in the abstract and then come up with a strategy that would make sense for this repo prior to submitting a PR. This is absolutely my bad, and I want to try this a different way this time.

By opening this up as an Issue, I want to give us a space where I can go over my findings here a little more thoroughly and give an opportunity for some brainstorming here so we have an opportunity to settle on a strategy that would make sense for this Repo before I come back with some PRs that would replace #243.

The Issues

1. Book textures are internally inconsistent for the same model across Ages. Specifically: "islmDniBook_side" and "islmDniBook_page" variants are wildly inconsistent across the game's PRPs; ideally, these should all match.
2. "islmDniBook_side" mostly uses a comically undersized 8x64 variant which looks awful in-game, even for 2003. I genuinely think this stands out like a sore thumb compared to the rest of the game, to an extent I actually find somewhat embarrassing, and this undersized texture really deserves to be axed from the game & improved. Several Ages already utilize the full-size 64x512 texture and a few use a medium-sized 32x256 texture.
3. "islmDniBook_page" has several ugly yellow splotches, and in general is undersized as well, but this is less of an issue here. Newer iterations have solved this either by using the texture xbooknewpage*0#0.hsm which is a "fixed" version of islmDniBook_Page, or by replacing islmDniBook_Page with a texture that is essentially the same thing as xbooknewpage.

IslmDniBook_Side

Again, this is my highest priority issue here - the 8x64 texture is unacceptable and the most noticeable issue here.

5 (soon to be 6) Ages already use the full-size 64x512 texture for this - it makes a big difference and I'd argue this should probably just be the standard. The 256 variant is acceptable, but is still inconsistent since many Ages already use 512 (and this number usually increases when new Ages are converted through Korman).

64 Texture (rotated)

64 Example

256 Texture (rotated)

256 Example

512 Texture (rotated)

512 Example

Strategies: Of the two pieces of this puzzle, this is the much simpler piece. My original proposal (and what went into MOULa) involved switching all Ages to 512 for consistency - but since there are texture budget concerns, I think we could re-think this; I could settle for abandoning the idea of making these "consistent" and just bumping the smallest ones up to 256. I could also see this being an entirely separate PR from the "Page" issue I outline below, as this gets a little more complicated.

Age	Side Texture Name	Current Size	Original Proposal from #243:	Notes
Ahnonay	islmdnibook_side*0#0.hsm	8x64 (Smallest)	64x512 (Full)
AhnonayCathedral	islmDniBook_side.dds	64x512 (Full)	No Change	May have benefitted from a prior Korman conversion which cleaned this up.
BaronCityOffice	islmdnibook_side*0#0.hsm	8x64 (Smallest)	64x512 (Full)
City	islmdnibook_side*0#0.hsm	8x64 (Smallest)	64x512 (Full)
Dereno	islmdnibook_side*0#0.hsm	8x64 (Smallest)	64x512 (Full)
EderDelin	islmdnibook_side*0#0.hsm	8x64 (Smallest)	64x512 (Full)
EderTsogal	islmdnibook_side*0#0.hsm	8x64 (Smallest)	64x512 (Full)
Ercana	islmdnibook_side*0#0.hsm	8x64 (Smallest)	64x512 (Full)
ErcanaCitySilo	islmdnibook_side*0#0.hsm	8x64 (Smallest)	64x512 (Full)
Garden	islmdnibook_side*0#0.hsm	8x64 (Smallest)	64x512 (Full)
Garrison	islmdnibook_side*0#0.hsm	64x512 (Full)	No Change	Makes sense that this is used here due to the Maintainer's Nexus.
Gira	islmdnibook_side*0#0.hsm	64x512 (Full)	No Change
GreatTreePub	islmDniBook_side.dds	64x512 (Full)	No Change	May have benefitted from a prior Korman conversion which cleaned this up.
GreatZero	islmdnibook_side*0#0.hsm	32x256 (Medium)	64x512 (Full)
Kadish	islmdnibook_side*0#0.hsm	32x256 (Medium)	64x512 (Full)
Kveer	islmdnibook_side_0_0.dds	8x64 (Smallest)	64x512 (Full)
Myst	N/A	N/A	No Change	NOTE: Myst does have a hidden book model, but it does not use islmdnibook_side and thus is not in scope here.
Negilahn	islmdnibook_side*0#0.hsm	8x64 (Smallest)	64x512 (Full)
Neighborhood	islmdnibook_side*0#0.hsm	8x64 (Smallest)	64x512 (Full)
Neighborhood02	islmdnibook_side*0#0.hsm	8x64 (Smallest)	Superseded by #250	#250 replaces this with a texture named "islmdnibook_side.dds" which is the full size. Thus, no change needed if merged.
Nexus	islmdnibook_side*0#0.hsm	64x512 (Full)	No Change	Makes sense for full size to be used here since the spine goes close to player's face.
Payiferen	islmdnibook_side*0#0.hsm	8x64 (Smallest)	64x512 (Full)
Personal	islmdnibook_side*0#0.hsm	8x64 (Smallest)	64x512 (Full)
PhilRelto	islmdnibook_side*0#0.hsm	8x64 (Smallest)	64x512 (Full)	This is for the hidden books in the hydrants. May not be strictly necessary, but would be good for consistency.
Spyoom	islmdnibook_side_0_0.dds	8x64 (Smallest)	64x512 (Full)
Teledahn	islmdnibook_side*0#0.hsm	32x256 (Medium)	64x512 (Full)
Tetsonot	islmdnibook_side*0#0.hsm	8x64 (Smallest)	64x512 (Full)

IslmDniBook_Page & IslmDniBook_Page1

This is where it gets a little more complicated as there are several sub-factors involved here beyond just size.

First: The difference between "Page" & "Page1" - This is an interesting case, because if you check the source files, you will see the source image is actually the same for both of these, despite being 2 different assets. This is because at full size, these "collapse" into the same image, but at smaller sizes, these diverge and become two separate images - one grainier, and one smoother, as though 2 separate compression mechanisms were used. In game this seems to be used to provide a little visual diversity between how the pages look, however many Ages simply collapse these down into one asset, or better yet, replace both of these with "xbooknewpage" outright, which is always full-size.

Second: In all sizes and variants, there is a series of ugly yellow splotches on the page. This is corrected in the texture xBookNewPage, which many newer Ages either use instead, or have replaced their IslmDniBook_Page instance with a texture that essentially looks the same as xBookNewPage.

Third: In many cases, due to new content being added over time, etc, there is sometimes a duplicate asset for one of the Page textures at a smaller size. There's really no reason these should be two separate assets, and in these cases, ideally the smaller one should be removed in favor of the larger version of this texture already in use in the PRP. But this would probably require a re-export of the PRP that is outside my current ability. This may be due to a relto page in the Age using the higher-quality page texture as the baseline for the relto page rather than the book.

Note: Unlike IslmDniBook_Size, these textures all use a naming convention that DIRECTLY corresponds to their image size (#0 indicates original size, #2 indicates medium size, #3 indicates smallest size). It's one of many reasons I'd be open to not changing the sizes here if we can come up with new textures that address the splotches.

Page & Page1 at 64:

64 Page1 (left) and Page (right) In-Game:

Page & Page1 at 128:

128 Page1 (left) and Page (right) In-Game:

Page & Page1 at 512 (same image):

512 Page1 (left) and Page (right) In-Game:

Page at 512 with yellow smudges corrected:

Splotchless 512 Page1 (left) and Page (right) In-Game:

Strategies: My original proposal (and what went into MOULa) makes all Page textures consistently full sized and splotchless - but this one especially can probably be handled a little smarter. In an ideal world, we would remove all duplicates and size up all textures to the full size. But even in this ideal case, it still poses a small problem, where Page and Page1 are no longer "unique" assets, since at 512 they "become" the same image. Again, in my original solution I just sized everything up to the 512 splotchless variant - but to be frank, I'm less concerned about the size here and more concerned about removing the splotches. Bear in mind, though - keeping the smaller sizes and removing the splotches wouldn't really be possible without introducing image loss, since we don't have the "resized" variants in Sources. So to go this route, we'd have to use xBookNewPage as a starting point and then figure out how to size this down in two different ways, so we can re-create both the "grainy" and "smooth" compression styles respectively. That being said - bumping to at least 128 is still probably warranted in this case. Alternatively: we could go full-size, but merge down all instances of the page texture to a single asset, similar to how Cathedral, Watchers' & Kirel have been handled.

Age	Page Texture Name(s)	Current Size	Has Splotches	Original Proposal from #243:	Notes
Ahnonay	islmdnibook_page*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
Ahnonay	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
Ahnonay	yeeshabookinside*0#2.hsm (OOS)	N/A	N/A	N/A	Note: Seems to be using this to replace the page textures outright instead of the proper Yeesha stamp (but this is out of scope)
AhnonayCathedral	islmDniBook_page1.dds	512x512 (Full)	No	No Change	May have benefitted from a prior Korman conversion which cleaned this up.
BaronCityOffice	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
City	islmdnibook_page*0#0.hsm	512x512 (Full)	Yes	512x512 Splotchless
City	islmdnibook_page*0#3.hsm	64x64 (Smallest)	Yes	512 Splotchless (but remove if possible)	Hypothetically, this is a duplicate asset and ideally could be removed in favor of the larger texture in this same PRP.
City	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
Dereno	islmdnibook_page*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
Dereno	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
EderDelin	islmdnibook_page*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
EderDelin	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
EderTsogal	islmdnibook_page*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
EderTsogal	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
Ercana	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
ErcanaCitySilo	islmdnibook_page*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
ErcanaCitySilo	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
Garden	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
Garrison	N/A	N/A	N/A	N/A	xbookpage is used instead. this still has the splotches, but is out of scope for this change.
Gira	islmdnibook_page*0#0.hsm	512x512 (Full)	Yes	512x512 Splotchless
Gira	islmdnibook_page*0#3.hsm	64x64 (Smallest)	Yes	512 Splotchless (but remove if possible)	Hypothetically, this is a duplicate asset and ideally could be removed in favor of the larger texture in this same PRP.
Gira	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
GreatTreePub	islmDniBook_page1.dds	512x512 (Full)	No	No Change	May have benefitted from a prior Korman conversion which cleaned this up.
GreatZero	islmdnibook_page*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
GreatZero	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
Kadish	islmdnibook_page*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
Kadish	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
Kveer	islmdnibook_page_0_3.dds	64x64 (Smallest)	Yes	512x512 Splotchless
Kveer	islmdnibook_page1_0_3.dds	64x64 (Smallest)	Yes	512x512 Splotchless
Negilahn	islmdnibook_page*0#0.hsm	512x512 (Full)	Yes	512x512 Splotchless
Negilahn	islmdnibook_page*0#3.hsm	64x64 (Smallest)	Yes	512 Splotchless (but remove if possible)	Hypothetically, this is a duplicate asset and ideally could be removed in favor of the larger texture in this same PRP.
Negilahn	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
Neighborhood	islmdnibook_page*0#2.hsm	128x128 (Medium)	Yes	512x512 Splotchless
Neighborhood	islmdnibook_page1*0#2.hsm	128x128 (Medium)	Yes	512x512 Splotchless
Neighborhood	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512 Splotchless (but remove if possible)	Hypothetically, this is a duplicate asset and ideally could be removed in favor of the larger texture in this same PRP.
Neighborhood02	islmdnibook_page*0#3.hsm	64x64 (Smallest)	Yes	Superseded by #250	#250 collapses this and "page1" down to a single texture, xbooknewpage, which is full size and splotchless. Thus, no change needed if merged.
Neighborhood02	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	Superseded by #250	#250 collapses this and "page" down to a single texture, xbooknewpage, which is full size and splotchless. Thus, no change needed if merged.
Nexus	N/A	N/A	N/A	N/A	xbooknewpage is used instead, which is full size and does not have the splotches.
Payiferen	islmdnibook_page*0#0.hsm	512x512 (Full)	Yes	512x512 Splotchless
Payiferen	islmdnibook_page*0#3.hsm	64x64 (Smallest)	Yes	512 Splotchless (but remove if possible)	Hypothetically, this is a duplicate asset and ideally could be removed in favor of the larger texture in this same PRP.
Payiferen	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
Personal	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
PhilRelto	islmdnibook_page*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless	Not sure why Personal only uses Page1 but PhilRelto uses Page and Page1.
PhilRelto	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
Spyroom	islmdnibook_page_0_3.dds	64x64 (Smallest)	Yes	512x512 Splotchless
Spyroom	islmdnibook_page1_0_3.dds	64x64 (Smallest)	Yes	512x512 Splotchless
Teledahn	islmdnibook_page*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
Teledahn	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless
Tetsonot	islmdnibook_page*0#0.hsm	512x512 (Full)	Yes	512x512 Splotchless
Tetsonot	islmdnibook_page*0#3.hsm	64x64 (Smallest)	Yes	512 Splotchless (but remove if possible)	Hypothetically, this is a duplicate asset and ideally could be removed in favor of the larger texture in this same PRP.
Tetsonot	islmdnibook_page1*0#3.hsm	64x64 (Smallest)	Yes	512x512 Splotchless

Conclusion

Based on the info I've provided here, my hope is we can brainstorm together and come up with a strategy to address this that will be more agreeable than my previous attempt and what I originally submitted to MOULa. I'm not looking for a quick solution here - I wanna give this some time to cook - but whatever we land on, I'll want to try to mirror this back to MOULa as well, to prevent the PRPs from diverging too much.

[dpogue]

IslmDniBook_Side:
I think the 256px one is a good size for almost everywhere except the Nexus and Maintainers' Nexus, but I also kinda feel like this is just a bad texture. Even at full 512 size, it doesn't end up looking like pages so much as like lines of colour. Maybe there's an opportunity here to try something a bit different that's designed specifically to look good at 256.

IslmDniBook_Page:
Fixing the yellow splotches should be the main priority here. I think trying to standardize on 128×128 is probably sufficient for everything except the Nexus. There might be cases where 256×256 is warranted, but those should be exceptions. I think starting with the 512 splotch-less version and blurring/adding noise before sizing it down should work to re-create the page/page1 variants. Cyan also used to sharpen their images at smaller sizes to introduce a bit of noise (which is why sometimes DIRT and Choru stuff ends up looking different than MOULa)

[patmauro]

IslmDniBook_Page: Fixing the yellow splotches should be the main priority here. I think trying to standardize on 128×128 is probably sufficient for everything except the Nexus. There might be cases where 256×256 is warranted, but those should be exceptions.

Indeed - something I've worried about when approaching this is that, for _Page, there are clear naming conventions here, and since I'm not re-exporting these PRPs, just touching them up, ideally those naming conventions should still mean something. Page should mean grainy, Page1 should mean smooth, #3 should mean smallest, #2 should mean medium and #0 should mean original size. 128 to 512 is a big jump, but I think introducing a new 256 tier could solve this problem - we just size everything up by one tick. This keeps 128x128 as the standard, but also makes sure that in the very rare instance that the "medium" size is specified (which I think is only the Hood), that distinction is still meaningful.

Which would mean:

becomes
,

becomes
,

and

still swaps out with
.

I think that could work?

[patmauro]

In addition to the size / optimization concerns with the last attempt at this (and it already feels like we've done a good job of finding better alternatives to my original approach so far - note I'm no longer recommending anything be bumped up to full-size 512x), another big concern, as @Hoikas pointed out, is that quite frankly, trying to review a PR that makes as many changes as the original did is just really difficult to review.

I've been thinking about this, and I think it's not just that 22 PRP's is a lot to review, but also that inside each PRP were a lot of different stacked changes every time. I think touching a lot of PRPs is probably unavoidable, but it feels like I could at least make life a little easier here by layering the pull requests over a longer period of time, and ensuring that each pull request only covers one specific type of change.

For example - based on some of the optimizations described above, I could split this into 4 PRs over time, like so:

1. Improve Quality of Book Side Textures: Bumps all 8x64 instances of _Side to 32x256, and that's it - does not change anything else. Would affect 17 PRPs, but would be the exact same change in each case.
2. Cleanup Large page textures: Removes splotches from 512x512 & 256x256 page textures but does not change any texture sizes. This would affect 8 PRPs.
3. Cleanup & Improve Quality of Unique Hood Page Textures: Handles an edge case where Hood books use a unique page size. Removes the splotches and resizes these from 128x128 to 256x256 (a new "medium" tier that would keep the texture's naming convention accurate but would still be much smaller than the full-size 512). Affects only one PRP.
4. Cleanup & Improve Quality of Small Page Textures: All remaining page textures are 64x64. This final PR would bump all 64x64 page textures up to 128x128 (making this the new baseline) and remove the troublesome splotches from each. Affects 21 PRPs but again, would be the same change in every case. (This step could also be done before 2 and 3.)

So, it feels to me like making each of these changes more specific like this will make this easier to review, even if it means the overall process will take a while - but I'd love any input as to whether there might be an even better way of doing this. Again, I still feel bad about the way I handled this the first time, and want to make sure I do this right this time.

[dpogue]

The easiest starting point here might be trying to get things to the intended result in Ages where we currently have source files (i.e., Max files or Blender conversions). Changes in those Ages would have to be made in the source files anyways for consistent on future exports.

[patmauro]

The easiest starting point here might be trying to get things to the intended result in Ages where we currently have source files (i.e., Max files or Blender conversions). Changes in those Ages would have to be made in the source files anyways for consistent on future exports.

My knee-jerk reaction is that this feels like it may be overkill, as I am mainly looking for a better-optimized alternative to the already-merged content/138, where these textures were simply swapped out in PRPShop. That being said - I also recognize that it is 100% my own fault that we are now in a position where there is an unnecessary PRP divergence in the first place, since I pushed those changes to OU without really taking the time to seek proper guidance on this first - so I certainly want to keep an open mind about alternative strategies.

Thus far I've shied away from getting involved with any changes that can't be handled with PRPshop alone, but if fixing the problem I created here the "right" way means finally getting my hands dirty with Max/Blender, then so be it.

[dpogue]

Starting with doing swaps in PrpShop is also fine, but only for Ages that don't have corresponding source files, because otherwise those will be overwritten if future changes are made and the content re-exported (i.e., updates to Kirel, or the ongoing work to make the Nexus more dynamic).