Skip to content

Implement GitHub Commit Comments Downloader Function - #381

Open
splimon wants to merge 9 commits into
sailuh:masterfrom
splimon:378-github-commit-comments-downloader-function
Open

Implement GitHub Commit Comments Downloader Function#381
splimon wants to merge 9 commits into
sailuh:masterfrom
splimon:378-github-commit-comments-downloader-function

Conversation

@splimon

@splimon splimon commented Mar 19, 2026

Copy link
Copy Markdown
Collaborator

Purpose

Kaiaulu currently has the functiongithub_api_project_commits to download commits from the GitHub API, but no equivalent function for downloading commit comments.

This new function is needed to transfer sentiment labels attached to commit comments from the GHTorrent database. To map these labels to Kaiaulu-downloaded data, the comment text must be retrieved through Kaiaulu's GitHub API pipeline rather than sourced directly from the static GHTorrent database. The bridge between the sentiment label and the downloaded comment is the commit comment ID.

This was identified and discussed in sailuh/sentiment_github_dataset#1.

Process

Implement a new functionc called github_api_project_commit_comments in R/github.R by:

  • Referencing the existing github_api_project_commits function as a template
  • Updating the endpoint to GET /repos/{owner}/{repo}/comments per the GitHub REST API documentation
  • Updating function parameters to accept owner, repo, and token
  • Verifying that returned comment_id values match those in GHTorrent's commit_comments table

Task List

  • Create new github_api_project_commit_comments function with owner, repo, and token parameters
  • Implement function using existing github_api_project_commits function as a template
  • Update the endpoint to GET /repos/{owner}/{repo}/comments
  • Test endpoint manually in browser (https://api.github.com/repos/{owner}/{repo}/comments) to confirm response format
  • Verify returned comment_id values match GHTorrent's commit_comments table

- Adds `github_api_project_commit_comments` to fetch commit comments from GET /repos/{owner}/{repo}/comments
- Adds `github_parse_project_commit_comments` to convert commit comment API responses into a parsed data.table
- Captures key fields for analysis (comment_id, commit_id, author info, body text, path/line context, and timestamps)
- Enables commit-comment workflows that join external sentiment labels by comment_id
splimon added 4 commits March 20, 2026 03:34
- Adds `get_github_commit_comment_path()` to `config.R` to read the `commit_comments`
  save path from project .yml files
- Adds commit comments download and parse sections to `download_github_events.Rmd`
- Adds `page` and `per_page` parameters to `github_api_project_commit_comments`
  and `github_api_project_pull_request_inline_comments` to allow explicit page requests
- Fixes a bug in `github_api_iterate_pages` where `gh_next()` returns the same page
  repeatedly. Adds duplicate-page detection and an explicit page=N fallback for both
  the commit comments and PR inline comments endpoints
- Fixes missing `save_path_pr_comments` argument in `download_github_pull_request_comments.Rmd`
- Adds `commit_comments` path to `kaiaulu.yml`
- Added `dir.create` function to `download_github_events.Rmd` and `download_github_pull_request_comments.Rmd` to automatically create directories to store extracted PR and commit comments
- Uses `gh` version 1.2.0 to resolve the bug of the same page saving
- Removes previously implemented duplicate-page detection fallback functions created in [commit d93ad20](sailuh@d93ad20)
…kflows

- Add generator script (exec/conf.R) that pulls all projects from the GHTorrent database to produce conf YAMLs for each project with sentiment labels
- Generate YAMLs for main project repos and fork repos with comments that are not accessible from the main repo. Verify via API checks
- Generate GitHub paths for commit/PR comments used in sentiment labeling (e.g., rawdata/github/{owner}/{repo}/commit_comments/ and rawdata/github/{owner}/{repo}/pr_comments/)
- Standardize naming and headers for reproducible config files
@codecov

codecov Bot commented Mar 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 32.11%. Comparing base (2bc8d14) to head (c7a14e1).
⚠️ Report is 10 commits behind head on master.

Files with missing lines Patch % Lines
R/github.R 0.00% 21 Missing ⚠️
R/config.R 0.00% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #381      +/-   ##
==========================================
- Coverage   39.79%   32.11%   -7.69%     
==========================================
  Files          20       21       +1     
  Lines        3091     4151    +1060     
==========================================
+ Hits         1230     1333     +103     
- Misses       1861     2818     +957     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

splimon added 2 commits April 3, 2026 04:31
- Add 82 generated YAML configuration files for main project repositories with sentiment-labeled data from the GHTorrent database
- Include GitHub rawdata paths for sentiment workflow (commit comments and PR inline comments)
- R/github.R: removed NULL columns for cleaner CSV conversion
- vignettes/download_github_events.Rmt: Added ability to save data table for commit comments as a CSV
@splimon splimon linked an issue Apr 6, 2026 that may be closed by this pull request
5 tasks

@phuong808 phuong808 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran the notebook locally and verified that everything executes without issues. The outputs match the results shown in the PR, and I was able to reproduce them successfully. Looks good to me.

@carlosparadis

Copy link
Copy Markdown
Member

@phuong808 don't forget to update the kanban on this PR! You can do from the right pane in this PR.

Some columns in inline_comments contain NULL or list-type values that fwrite() can't handle directly. The added code chunk coerces those columns to character before writing.
@splimon
splimon requested a review from jseto808 April 17, 2026 09:58

@jseto808 jseto808 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The R function additions follow existing naming conventions: github_api_project_commit_comments and github_parse_project_commit_comments follow the github_api_project_* and github_parse_project_* patterns respectively, and get_github_commit_comment_path follows the get_github_*_path pattern. The 82 new project YAML configs are consistent with existing conf files. The notebooks ran without issue. No issues found.

Added name to `DESCRIPTION` as a contributor.

Added `github_api_project_commit_comments()`, `github_parse_project_commit_comments()`, and `get_github_commit_comment_path()` under NEW FEATURES in `NEWS.md`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement GitHub Commit Comments Downloader Function

5 participants