fix(scraper): prevent nil pointer dereference in scrapeScene#6857
fix(scraper): prevent nil pointer dereference in scrapeScene#6857dutchdevil-83 wants to merge 4 commits intostashapp:developfrom
Conversation
When scrapeScene finds no results, ret remains nil and is passed to processSceneRelationships, which unconditionally dereferences it at line 89 (ret.Performers = ...), causing a panic. Initialize ret to an empty ScrapedScene so processSceneRelationships always has a valid pointer. This also preserves the intent of stashapp#3953: returning a scene with only relationship fields set when scraped non-relationship data is absent. Fixes panic: runtime error: invalid memory address or nil pointer dereference at pkg/scraper/mapped.go:89 in processSceneRelationships
There was a problem hiding this comment.
Pull request overview
Fixes a panic in the mapped scraper’s scrapeScene flow by ensuring relationship processing always receives a non-nil *models.ScrapedScene, allowing “relationship-only” scraper results (per #3953) to be returned instead of crashing.
Changes:
- Initialize
retto an empty&models.ScrapedScene{}before processing scene relationships. - Preserve existing behavior to return
nilonly when both direct results and relationships are absent.
Comments suppressed due to low confidence (1)
pkg/scraper/mapped.go:200
- This change fixes an important edge case (no scene fields returned but relationships are returned). There doesn't appear to be a unit test covering
scrapeScenereturning a non-nil scene populated only with relationships whenlen(results)==0; adding a regression test for this scenario would help prevent future panics/behavior regressions.
// Initialize ret to a non-nil empty scene so that processSceneRelationships
// can safely populate relationship fields even when no direct results were found.
// This preserves the intent of #3953: returning a scene with only relationships.
ret := &models.ScrapedScene{}
if len(results) > 0 {
ret = results[0].scrapedScene()
}
hasRelationships := s.processSceneRelationships(ctx, q, 0, ret)
// #3953 - process only returns results if the non-relationship fields are
// populated
// only return if we have results or relationships
if len(results) > 0 || hasRelationships {
return ret, nil
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Gykes
left a comment
There was a problem hiding this comment.
I would also recommend adding in some kind of regression test for the relationships-only scrape path.
Something that does the following:
- configures a scene scraper with empty mappedConfig results but a populated Performers/Tags block
- asserts the function does not panic
- asserts the returned *ScrapedScene carries the relationships
|
Agreed — this path is worth covering. The production fix prevents the nil dereference, but a regression test should verify that a scene scraper with no direct scene-field results still returns a non-nil ScrapedScene populated with relationship data. I’d add a test in pkg/scraper/xpath_test.go that clears the scene mappedConfig, keeps the Performers/Tags mappings, asserts scrapeScene does not panic, and verifies the returned performers/tags. |
Signed-off-by: Marco <130363067+dutchdevil-83@users.noreply.github.com>
Description
scrapeScenedeclaresretas a nil*models.ScrapedSceneand only initializes it when results are found. It then unconditionally callsprocessSceneRelationships, which dereferencesretat line 89:This causes a panic when a scraper returns no direct scene fields but the function continues to process relationships.
Root cause
Fix
Initialize
retto an empty&models.ScrapedScene{}soprocessSceneRelationshipsalways receives a valid pointer:This also preserves the intent of #3953: when a scraper yields no non-relationship fields but does yield relationships, the function now correctly returns a scene populated with only those relationships instead of panicking.
Panic trace