fix: reduce SEO index bloat from zero-content profile pages#1347
Open
devanshk404 wants to merge 2 commits intoSuperteamDAO:mainfrom
Open
fix: reduce SEO index bloat from zero-content profile pages#1347devanshk404 wants to merge 2 commits intoSuperteamDAO:mainfrom
devanshk404 wants to merge 2 commits intoSuperteamDAO:mainfrom
Conversation
- Sitemap now only includes leaderboard profiles (earned ≥$1 via TalentRankings), reducing talent URLs from ~50k to ~5k - Profile pages for users with no earnings and no submission in the last 3 months now render noindex/nofollow to avoid wasting crawl budget on low-quality pages Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
@devanshk404 is attempting to deploy a commit to the Superteam Team on Vercel. A member of the Team first needs to authorize it. |
Contributor
WalkthroughAdds a TalentRankings filter to sitemap generation to include only profiles with earnings, and changes talent profile page logic to compute Changes
Sequence Diagram(s)(omitted) Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
src/pages/earn/t/[slug]/index.tsx (3)
814-824: Consider usingawaitfor consistency in async function.Using
.then()inside anasyncfunction is functional but mixes async patterns. Usingawaitwould be more consistent with the rest of the function.Refactor to use await
const isOnLeaderboard = totalWinnings > 0; - const hasRecentSubmission = isOnLeaderboard - ? false - : await prisma.submission - .findFirst({ - where: { - userId: talent.id, - createdAt: { gte: threeMonthsAgo }, - }, - select: { id: true }, - }) - .then((r) => !!r); + let hasRecentSubmission = false; + if (!isOnLeaderboard) { + const recentSubmission = await prisma.submission.findFirst({ + where: { + userId: talent.id, + createdAt: { gte: threeMonthsAgo }, + }, + select: { id: true }, + }); + hasRecentSubmission = recentSubmission !== null; + } const shouldNoIndex = !isOnLeaderboard && !hasRecentSubmission;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/pages/earn/t/`[slug]/index.tsx around lines 814 - 824, Replace the .then() usage with await for consistency: in the computation of hasRecentSubmission (the const named hasRecentSubmission), await the result of prisma.submission.findFirst({ where: { userId: talent.id, createdAt: { gte: threeMonthsAgo } }, select: { id: true } }) and then convert the result to a boolean (e.g., !!result) instead of chaining .then(); keep the same conditions (isOnLeaderboard check) and variable names (talent.id, threeMonthsAgo).
748-749: Consider using date arithmetic that handles month-end edge cases.
setMonth()can produce unexpected results when the source date is near month-end. For example, if today is March 31, subtracting 3 months would try to create "December 31" but could wrap incorrectly on months with fewer days.Safer date calculation
- const threeMonthsAgo = new Date(); - threeMonthsAgo.setMonth(threeMonthsAgo.getMonth() - 3); + const threeMonthsAgo = new Date(); + threeMonthsAgo.setMonth(threeMonthsAgo.getMonth() - 3); + // Clamp to valid day if month overflow occurred + if (threeMonthsAgo.getDate() !== new Date().getDate()) { + threeMonthsAgo.setDate(0); // Last day of previous month + }Alternatively, use a date library like
date-fnsif already available:import { subMonths } from 'date-fns'; const threeMonthsAgo = subMonths(new Date(), 3);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/pages/earn/t/`[slug]/index.tsx around lines 748 - 749, The current calculation for threeMonthsAgo uses Date.prototype.setMonth which can mis-handle month-end dates (the variable threeMonthsAgo and its setMonth call); replace this with a safer approach such as using date-fns' subMonths (import { subMonths } from 'date-fns' and compute threeMonthsAgo = subMonths(new Date(), 3)) or implement manual date arithmetic that preserves day-of-month by adjusting for month length before calling setMonth, ensuring the threeMonthsAgo value is correct for end-of-month dates.
738-738: Add explicit return type togetServerSideProps.Per coding guidelines, top-level module functions should declare return types. While the
GetServerSidePropstype annotation provides some type checking, an explicit return type improves clarity.Add return type
-export const getServerSideProps: GetServerSideProps = async (context) => { +export const getServerSideProps: GetServerSideProps<TalentProps> = async (context) => {Note: This may require adjusting the return for the
talent: nullcase to match the props type, or using a union type.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/pages/earn/t/`[slug]/index.tsx at line 738, Add an explicit return type annotation to the getServerSideProps function (the exported getServerSideProps) to match the props you return (e.g., GetServerSidePropsResult<Props> or GetServerSideProps<{ talent: TalentType | null; ... }>) and update the returned objects so their types align — for example ensure the branch that returns talent: null conforms to the declared Props type (or use a union allowing null). Adjust the Props/Talent type references used in the annotation to match the existing prop shape returned by getServerSideProps.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/pages/earn/t/`[slug]/index.tsx:
- Around line 813-825: The sitemap and profile page are using different data
sources (TalentRankings.totalEarnedInUSD vs live computation in isOnLeaderboard
which sums Submission.rewardInUSD and GrantApplication.approvedAmountInUSD),
causing inconsistent indexing; pick one consistent source and apply it
everywhere: either change the profile page logic around
isOnLeaderboard/totalWinnings and shouldNoIndex to read from TalentRankings (and
document/ensure TalentRankings is updated), or change the sitemap generator to
compute total earnings using the same live prisma queries (prisma.submission and
GrantApplication sums) used by the profile page so both sitemap and profile use
the identical data source. Ensure the chosen approach references the same unique
symbols (TalentRankings, isOnLeaderboard, totalWinnings, shouldNoIndex,
prisma.submission, GrantApplication) and add a comment documenting which source
is authoritative.
---
Nitpick comments:
In `@src/pages/earn/t/`[slug]/index.tsx:
- Around line 814-824: Replace the .then() usage with await for consistency: in
the computation of hasRecentSubmission (the const named hasRecentSubmission),
await the result of prisma.submission.findFirst({ where: { userId: talent.id,
createdAt: { gte: threeMonthsAgo } }, select: { id: true } }) and then convert
the result to a boolean (e.g., !!result) instead of chaining .then(); keep the
same conditions (isOnLeaderboard check) and variable names (talent.id,
threeMonthsAgo).
- Around line 748-749: The current calculation for threeMonthsAgo uses
Date.prototype.setMonth which can mis-handle month-end dates (the variable
threeMonthsAgo and its setMonth call); replace this with a safer approach such
as using date-fns' subMonths (import { subMonths } from 'date-fns' and compute
threeMonthsAgo = subMonths(new Date(), 3)) or implement manual date arithmetic
that preserves day-of-month by adjusting for month length before calling
setMonth, ensuring the threeMonthsAgo value is correct for end-of-month dates.
- Line 738: Add an explicit return type annotation to the getServerSideProps
function (the exported getServerSideProps) to match the props you return (e.g.,
GetServerSidePropsResult<Props> or GetServerSideProps<{ talent: TalentType |
null; ... }>) and update the returned objects so their types align — for example
ensure the branch that returns talent: null conforms to the declared Props type
(or use a union allowing null). Adjust the Props/Talent type references used in
the annotation to match the existing prop shape returned by getServerSideProps.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: c64db095-db8d-4a4e-8711-7b3b5ac744e3
📒 Files selected for processing (2)
src/app/sitemap.tssrc/pages/earn/t/[slug]/index.tsx
Comment on lines
+813
to
+825
| const isOnLeaderboard = totalWinnings > 0; | ||
| const hasRecentSubmission = isOnLeaderboard | ||
| ? false | ||
| : await prisma.submission | ||
| .findFirst({ | ||
| where: { | ||
| userId: talent.id, | ||
| createdAt: { gte: threeMonthsAgo }, | ||
| }, | ||
| select: { id: true }, | ||
| }) | ||
| .then((r) => !!r); | ||
| const shouldNoIndex = !isOnLeaderboard && !hasRecentSubmission; |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Search for TalentRankings update/sync logic
echo "=== Searching for TalentRankings update logic ==="
rg -n -C3 'TalentRankings' --type=ts -g '!*.d.ts' -g '!node_modules/**'
echo ""
echo "=== Checking for scheduled jobs or triggers that sync rankings ==="
rg -n -l 'cron|schedule|job' --type=ts -g '!node_modules/**' | head -20Repository: SuperteamDAO/earn
Length of output: 2580
🏁 Script executed:
#!/bin/bash
# First, let's look at the profile page code in context
echo "=== Viewing profile page lines 800-830 ==="
sed -n '800,830p' src/pages/earn/t/\[slug\]/index.tsx | cat -n
# Let's also check a wider context around those lines
echo ""
echo "=== Looking for totalWinnings definition ==="
rg -n 'totalWinnings' src/pages/earn/t/\[slug\]/index.tsx -B5 -A2Repository: SuperteamDAO/earn
Length of output: 3425
🏁 Script executed:
#!/bin/bash
# Search for TalentRankings creation/update operations
echo "=== Looking for TalentRankings write operations ==="
rg -n 'talentRankings\.(create|update|upsert)' --type=ts -g '!node_modules/**' -A3
# Search for where totalEarnedInUSD is calculated
echo ""
echo "=== Looking for totalEarnedInUSD calculation ==="
rg -n 'totalEarnedInUSD' --type=ts -g '!node_modules/**' -B2 -A2Repository: SuperteamDAO/earn
Length of output: 1487
🏁 Script executed:
#!/bin/bash
# Search for background jobs or sync mechanisms
echo "=== Looking for job/worker files ==="
fd -e ts -e js 'job|worker|queue|sync' . --type=f | grep -v node_modules | head -20
# Check API routes that might update rankings
echo ""
echo "=== Checking submission/grant update endpoints ==="
rg -n 'talentRankings' src/pages/api --type=ts -B3 -A3Repository: SuperteamDAO/earn
Length of output: 297
🏁 Script executed:
#!/bin/bash
# More broad search for TalentRankings mutations
echo "=== Broad search for TalentRankings operations ==="
rg -n 'TalentRankings' --type=ts -g '!*.d.ts' -g '!node_modules/**' --type-add='ts:{*.ts,*.tsx}' | grep -E '(create|update|upsert|delete|prisma)'
# Check prisma schema for TalentRankings
echo ""
echo "=== Checking Prisma schema for TalentRankings ==="
find . -name 'schema.prisma' | head -1 | xargs cat | grep -A20 'model TalentRankings'Repository: SuperteamDAO/earn
Length of output: 1058
🏁 Script executed:
#!/bin/bash
# Check if TalentRankings is only read from (not written)
echo "=== All TalentRankings references ==="
rg -n 'TalentRankings' --type=ts -g '!*.d.ts' -g '!node_modules/**' -A1 -B1 | head -80
# Check for Prisma client usage with talentRankings (lowercase)
echo ""
echo "=== Looking for prisma.talentRankings usage ==="
rg -n 'prisma\.talentRankings' --type=ts -g '!node_modules/**' -A3 -B1Repository: SuperteamDAO/earn
Length of output: 1901
🏁 Script executed:
#!/bin/bash
# Check migrations
echo "=== Looking for migration files ==="
find . -path '*/migrations/*' -name '*.sql' | head -20
# Check for seed files
echo ""
echo "=== Looking for seed/migration scripts ==="
find . -name '*seed*' -o -name '*migrate*' | grep -v node_modules | head -20Repository: SuperteamDAO/earn
Length of output: 140
🏁 Script executed:
#!/bin/bash
# Check for database triggers in schema
echo "=== Looking for database triggers in schema ==="
find . -name 'schema.prisma' | xargs cat | grep -i -E '(trigger|@db\.|@@db\.)'
# Check for raw SQL operations
echo ""
echo "=== Looking for raw SQL queries (executeRaw, queryRaw) ==="
rg -n '(executeRaw|queryRaw)' --type=ts -g '!node_modules/**' -B2 -A4
# Check the job queue files we found
echo ""
echo "=== Checking job queue implementations ==="
head -50 src/features/agents/utils/queueAgent.tsRepository: SuperteamDAO/earn
Length of output: 16057
🏁 Script executed:
#!/bin/bash
# Look for external services or job processors
echo "=== Checking for job queue/worker setup ==="
rg -n 'bull|queue|worker|cron' --type=ts -g '!node_modules/**' | grep -v 'queueAgent\|queueEmail' | head -30
# Check package.json for job queue libraries
echo ""
echo "=== Checking dependencies for job/queue libraries ==="
cat package.json | grep -E '(bull|agenda|kue|node-cron|later)' | head -20Repository: SuperteamDAO/earn
Length of output: 3554
🏁 Script executed:
#!/bin/bash
# Search more broadly for where TalentRankings might be updated
echo "=== Looking for any mutation to earnings/rankings data ==="
rg -n '(createMany|updateMany|deleteMany|raw)' --type=ts -g '!node_modules/**' -B3 -A3 | head -100
# Check if there are any API endpoints that might handle ranking updates
echo ""
echo "=== Checking API routes for ranking/earnings updates ==="
find src/pages/api -name '*.ts' | xargs grep -l 'Ranking\|earning' | head -10Repository: SuperteamDAO/earn
Length of output: 6483
🏁 Script executed:
#!/bin/bash
# Check if TalentRankings might be defined as a VIEW (views might be in migrations)
echo "=== Checking for references to TalentRankings being a view ==="
rg -n 'view\|VIEW' --type=ts -g '!node_modules/**' | grep -i ranking
# Check for any comments or documentation about TalentRankings
echo ""
echo "=== Checking for documentation/comments about TalentRankings ==="
rg -n 'TalentRankings' --type=ts -B2 -A2 | grep -E '(\/\/|\/\*|\*)'
# Look for README or docs
echo ""
echo "=== Searching docs for TalentRankings info ==="
find . -name 'README*' -o -name '*.md' | xargs grep -l 'TalentRankings\|ranking' 2>/dev/null | head -5Repository: SuperteamDAO/earn
Length of output: 237
🏁 Script executed:
#!/bin/bash
# Check full Prisma schema for any computed fields or relations that might auto-populate TalentRankings
echo "=== Checking if TalentRankings has any special Prisma configuration ==="
cat prisma/schema.prisma | grep -A30 'model TalentRankings'
# Check if there's any relation from User that might trigger TalentRankings
echo ""
echo "=== Checking User model for TalentRankings relation ==="
cat prisma/schema.prisma | grep -A100 'model User' | grep -i 'ranking'Repository: SuperteamDAO/earn
Length of output: 1282
Unresolved sitemap and profile page inconsistency in earnings-based indexing.
The concern is confirmed. The sitemap filters talent profiles by TalentRankings.totalEarnedInUSD > 0, but the profile page computes isOnLeaderboard from a live SQL query summing Submission.rewardInUSD and GrantApplication.approvedAmountInUSD.
Investigation reveals no write operations to TalentRankings exist in the codebase—the table is queried for the leaderboard and sitemap but never updated by the application. This means:
- If
TalentRankingsis synced externally or stale, profile pages can be included in the sitemap but noindexed (live data shows $0) - Conversely, profiles can be excluded from sitemap but indexed if the live computation shows earnings
Both the sitemap and profile page should use the same data source for consistency. Either compute both from the live queries or ensure TalentRankings is kept current via a documented and reliable mechanism.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/pages/earn/t/`[slug]/index.tsx around lines 813 - 825, The sitemap and
profile page are using different data sources (TalentRankings.totalEarnedInUSD
vs live computation in isOnLeaderboard which sums Submission.rewardInUSD and
GrantApplication.approvedAmountInUSD), causing inconsistent indexing; pick one
consistent source and apply it everywhere: either change the profile page logic
around isOnLeaderboard/totalWinnings and shouldNoIndex to read from
TalentRankings (and document/ensure TalentRankings is updated), or change the
sitemap generator to compute total earnings using the same live prisma queries
(prisma.submission and GrantApplication sums) used by the profile page so both
sitemap and profile use the identical data source. Ensure the chosen approach
references the same unique symbols (TalentRankings, isOnLeaderboard,
totalWinnings, shouldNoIndex, prisma.submission, GrantApplication) and add a
comment documenting which source is authoritative.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
scutuatua-crypto
approved these changes
Mar 24, 2026
…noindex Active grant applicants with no bounty submissions were incorrectly getting noindexed. Now checks both submission and grantApplication tables within the 3-month window before deciding to noindex a profile. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/pages/earn/t/[slug]/index.tsx (1)
748-749: Month rollover edge case in date calculation.
setMonth(getMonth() - 3)can produce unexpected results on month-end dates. For example, on March 31, subtracting 3 months attempts to create "February 31", which rolls forward to early March.For SEO purposes this variance is likely acceptable, but if precise 3-month windows matter, consider using a date library or clamping to valid dates.
♻️ Optional fix for precise date handling
const threeMonthsAgo = new Date(); -threeMonthsAgo.setMonth(threeMonthsAgo.getMonth() - 3); +threeMonthsAgo.setMonth(threeMonthsAgo.getMonth() - 3); +// Clamp to last day of target month if overflow occurred +if (threeMonthsAgo.getDate() !== new Date().getDate()) { + threeMonthsAgo.setDate(0); // Set to last day of previous month +}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/pages/earn/t/`[slug]/index.tsx around lines 748 - 749, The current calculation for threeMonthsAgo uses Date.setMonth/getMonth which can roll into the wrong day for month-end dates; update the logic that defines the threeMonthsAgo variable in this component (the const threeMonthsAgo assignment) to handle month-rollover safely — either use a date library (e.g., dayjs/Date-fns) to subtract three months (preserving month end behavior) or implement clamping: compute the target month/year, set the day to the minimum of the original day and the last day of the target month before constructing the Date; ensure the replacement updates only the threeMonthsAgo initialization and keeps the variable name the same.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@src/pages/earn/t/`[slug]/index.tsx:
- Around line 748-749: The current calculation for threeMonthsAgo uses
Date.setMonth/getMonth which can roll into the wrong day for month-end dates;
update the logic that defines the threeMonthsAgo variable in this component (the
const threeMonthsAgo assignment) to handle month-rollover safely — either use a
date library (e.g., dayjs/Date-fns) to subtract three months (preserving month
end behavior) or implement clamping: compute the target month/year, set the day
to the minimum of the original day and the last day of the target month before
constructing the Date; ensure the replacement updates only the threeMonthsAgo
initialization and keeps the variable name the same.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 35a258f5-99c0-4378-9e1b-4cfd162ebdd9
📒 Files selected for processing (1)
src/pages/earn/t/[slug]/index.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
~50k+
/earn/t/[username]/profile pages with zero earnings and zero submissions were being indexed by Google and included in the sitemap. This wastes crawl budget and drags down sitewide quality signals, slowing indexing of pages that matter.Changes
Fix 1 — Sitemap (
src/app/sitemap.ts)Talent profile sitemap now filters to leaderboard-only profiles (users with
totalEarnedInUSD > 0inTalentRankings).Fix 2 — Profile page noindex (
src/pages/earn/t/[slug]/index.tsx)Profile pages now serve
noindex, nofollowfor users who are:Rule: index if earned ≥$1 OR submitted at least once in last 3 months.
Performance notes
findFirst(LIMIT 1, hits existing index onSubmission.userId + createdAt)s-maxage=60, stale-while-revalidate=600), so the extra query only runs on cache missesTest plan
noindex, nofollowin<head>totalEarnedInUSD > 0🤖 Generated with Claude Code
Summary by CodeRabbit