Carnivores HD [NPUB30869] texture loads fail, leading to purple objects ingame

[EdHerdman]

Quick summary

Running recent builds, such as rpcs3-v0.0.29-15479-39bbf17c_win64: In Carnivores HD with Version 1.01 update, on loading any map the log shows a string of messages such as:

E SYS: Stat: 'sys_fs_stat' failed with 0x80010006 : CELL_ENOENT, “/dev_hdd0/game/NPUB30869/USRDIR/tex_env_07_isle_14_b” [x11]
E SYS: Stat: '_sys_lwmutex_lock' failed with 0x80010005 : CELL_ESRCH [x5]
E SYS: Stat: 'sys_fs_stat' failed with 0x80010006 : CELL_ENOENT, “/dev_hdd0/game/NPUB30869/USRDIR/tex_env_08_isle_14_b” [x11]
E SYS: Stat: 'sys_fs_stat' failed with 0x80010006 : CELL_ENOENT, “/dev_hdd0/game/NPUB30869/USRDIR/tex_env_08_isle_02_b” [x8]

Ingame, the failed texture loads usually cause purple items - rocks, foliage, even parts of your drone ship and dinosaur models can be affected. Sometimes the problem resolves by approaching the objects and cycling through available LODs, but usually it is persistent. Quite often there are only a few objects affected despite a log indicating more textures failed to load.

This happens on builds from earlier this week also. It happens in a totally fresh copy of RPCS3 with full default settings or with 200% resolution scale, and with multithreaded RSX and asychronous texture streaming on or off. I've had it happen under both NVIDIA drivers 536.67 and 536.99. All files are hosted on a NVMe SSD.

I've followed the RPCS3 Wiki steps to validate my game files are correct. Apologies for forgetting the RSX capture - I will provide one if needed.

Attach a log file

RPCS3.log.gz
RPCS3.log.gz (this second log corresponds to the attached screenshots)

System configuration

AMD Ryzen 7 3700X, 32 GB RAM
Windows 11 Pro 22H2 OS build 22621.2134
NVIDIA Geforce RTX 4080 with 536.67 and 536.99 drivers
Samsung SSD 970 EVO Plus 2TB

[ChwaraeTeg]

You mention this occurs with recent builds from the past week, does that mean it was okay on older builds? If so then you'd need to track down the exact build that caused the regression.

[EdHerdman]

I simply don't know if it worked at any point before I started testing last week, which is why I didn't tag it as a regression. Still a good suggestion, so I tried a build from a year ago: rpcs3-v0.0.23-14051-829af305_win64. That has the same problem and the log messages look the same to me. I could go further back but compiling the shaders is notably longer...

I also tried the very latest build, rpcs3-v0.0.29-15481-0e23b893_win64, and the problem still exists.

RPCS3.log.gz rpcs3-v0.0.23-14051-829af305_win64
RPCS3.log.gz rpcs3-v0.0.23-14051-829af305_win64

[kd-11]

Are the textures even present on the disk? Could be a pkg issue.
Something to keep in mind is that there are many characters that can be used in filenames in *nix-based systems that are completely illegal on windows (such as the colon) though I doubt that's what's going on here. Not much we can do about the texture if it's truly missing but I think it exists on either the game directory or at least in the pkg file and we're missing it at runtime for some reason.

[EdHerdman]

I'm certain the textures are present on drive and pretty sure they'll load successfully sometimes. Textures that are missing on one run through the map are present on another map load (I'm quite sure it's not variations of the textures responsible). I'm seeing specific textures go missing repeatedly but I haven't done a careful comparison of the missing texture messages in the log. I'm not totally sure if missing textures will load in later - right now, with the latest version, it's often the highest resolution, highest LOD textures will go missing and that's only visible when you get right next to an object. Multithreaded RSX + Async texture loading might affect which exact textures are missing here. I'll try to find some time to play with those settings, individual maps, and older versions to see if I can spot some patterns.

Gory details to address your other concerns: First, I looked for evidence of complaints online about missing or purple textures, and didn't find any. Next, I validated the RAPs (both the original and the update) and they pass. Every time I've tested this on a new version of RPCS3, I've unzipped a copy of RPCS3, installed the PS3 firmware, and then installed the RAPs, to rule out bad files being carried across from version to version. I'm running W11, the game files, and RPCS3 off a Samsung 970 EVO Plus NVMe M.2 2 TB SSD (recent production with recent firmware, like 2022). Perf counters indicate there's plenty of headroom on the CPU/GPU side.

As far as weird filenames, at least according to the log the characters all look to be straightforward English descriptions of the files.

My log is full of things like this, though:

·W 1:30:58.383969 {PPU[0x1000000] Thread (main_thread) [HLE:0x00a908c4, LR:0x0028d430]} cellGem: cellGemGetInfo(info=*0x942700)
·! 1:30:58.443791 {PPU[0x1000000] Thread (main_thread) [libfs: 0x00d60ec0]} sys_fs: sys_fs_fcntl(0xc0000006): /dev_hdd0
·W 1:30:58.443812 {PPU[0x1000000] Thread (main_thread) [libfs: 0x00d61ee4]} sys_fs: sys_fs_stat(path=“/dev_hdd0/game/NPUB30869/USRDIR/tex_mmap_lev_isle_06”, sb=*0xd004d618)
·E 1:30:58.444000 {PPU[0x1000000] Thread (main_thread) [libfs: 0x00d61ee4]} sys_fs: 'sys_fs_stat' failed with 0x80010006 : CELL_ENOENT, “/dev_hdd0/game/NPUB30869/USRDIR/tex_mmap_lev_isle_06” [1]

and example stats:
·! 1:31:13.710222 {Performance Sensor} SYS: CPU Usage: Total: 13.1%, Cores: 65.7%, 7.9%, 42.2%, 20.4%, 28.2%, 15.7%, 9.5%, 15.7%, 0.1%, 1.7%, 0.1%, 1.7%, 0.1%, 0.1%, 0.1%, 0.1%
·W 1:31:13.717699 {PPU[0x1000000] Thread (main_thread) [HLE:0x00a908c4, LR:0x0028d430]} cellGem: cellGemGetInfo(info=*0x942700)

Finally, I would like to reinstall Windows 11 sometime soon to rule out some weird obscure setting in Windows causing problems. But I can't think of any other program throwing up an odd error like this, including some fairly performance demanding ones.

BIOS is recent too. I've got Above 4G decoding and Resizable BAR on, virtualization off. In Windows the only exotic setting I have is Windows DEP is on for everything.

[digant73]

@EdHerdman is that still an issue?

[EdHerdman]

@digant73 and @kd-11, sorry for late response:

kd-11's PR #17666 has improved this game, although I still see broken textures ingame. Summarizing:

1. Unchanged: The console still reports the file system / texture errors. It always appears to be the same list of files. PSN PKG validator passes both my main USA game PKG and the v1.01 update PKG. Maybe it's a developer oversight.

2. Improved bloom effect: Before PR SPU: Update CELL Communication Performance #17646, areas of color bled into other areas and the pattern looked like vertical stripes. Possibly related to the pixel_center_intager shader attribute being implemented? All these screenshots are taken at default settings and resolution:

rpcs3-v0.0.38-18290-0403231a_win64_msvc, note the thumb color bleeding and the edges of straight lines:

rpcs3-v0.0.38-18293-8984c2b3_win64_msvc, note the purple broken texture appeared, but bloom and straight lines are well-behaved now:

3. Changed broken textures: Broken textures affect different objects seemingly at random, and I am fairly sure this is not related to RAM errors on my end. I'm still very early in testing this, but I don't remember ever seeing partially correct textures alongside a strip of incorrect textures before rpcs3-v0.0.38-18309-50eb46d2_win64_msvc. In all my past tests, when a rock has a broken texture, it is entirely covered by the broken texture.

[kd-11]

The broken texture parts look like shader inputs interpreted visually which means this is just a synchronizion issue. Something to try here is to combine read color buffers with write color buffers options.

[EdHerdman]

Another improvement! First, rpcs3-v0.0.38-18310-3df0c8ca_win64_msvc with default settings:

Next, with read and write color buffer settings on. Bloom on objects is more correct and the sky is not totally washed out.

However, this didn't correct broken textures.

So I set read and write depth buffers on. After about 25 minutes of playtime I have yet to see any broken textures. It seems likely this is now fixed up of all the problems I'd spotted. I'll do a bit more testing when I have a chance to see if the later maps have any issues, or if I can get the same results with only writing or reading buffers. Not sure if this is ready to be closed yet.

[kd-11]

If the setting is stable, just describe the settings required so we can update in the wiki and then close the ticket.

[EdHerdman]

Changing only read and write color buffers + read and write depth buffers seems stable. I'll reopen if there are more relevant findings.