OncoKB Public Website

OncoKB is a precision oncology knowledge base and contains information about the effects and treatment implications of specific cancer gene alterations. Please cite Suehnholz et al., Cancer Discovery 2024 and Chakravarty et al., JCO PO 2017.

This application was generated using JHipster 6.10.3, you can find documentation and help at https://www.jhipster.tech/documentation-archive/v6.10.3.

Status

Info

Development

Running environment

Make sure your running environment is the following:

• Java version: 8
• MySQL version: 5.7.28
• Redis

Setup Redis (optional)

The following steps is one way to set up the redis. As long as you have redis setup and ready to connect, then that would work.

• Install Docker
• Install Kubernetes(k8s) (In Mac versin, you can enable k8s in Docker Preference)
• Install helm
• Install redis helm install oncokb-public-redis bitnami/redis --set auth.password=oncokb-public-redis-password --set replica.replicaCount=1
• Follow the instructions after installing redis. Then you need to proxy the redis out, command looks like kubectl port-forward --namespace default svc/oncokb-public-redis-master 6379:6379

SendGrid mock server (local)

You can run a local mock SendGrid-compatible HTTP endpoint for bulk email testing.

Files:

• src/main/docker/sendgrid-mock/main.go (single-file Go server, no external dependencies)
• src/main/docker/sendgrid-mock/Dockerfile
• src/main/docker/sendgrid-mock.yml

Run it:

docker compose -f src/main/docker/sendgrid-mock.yml up --build

It listens on http://localhost:8086 and logs request method, path, headers, query, and payload to container logs. It returns endpoint-specific responses:

• POST /v3/mail/send -> 202 Accepted
• GET /v3/asm/suppressions/{email} -> 200 OK
• POST /v3/asm/groups/{groupId}/suppressions -> 201 Created
• DELETE /v3/asm/groups/{groupId}/suppressions/{email} -> 204 No Content
• unsupported/invalid requests -> 400 or 404 with a JSON error payload

To point this app to the mock endpoint, configure SendGrid host via environment variables (example):

export APPLICATION_SENDGRID_ENABLED=true
export APPLICATION_SENDGRID_API_KEY=dummy-key
export APPLICATION_SENDGRID_BASE_URL=http://localhost:8086

SendGrid configuration (production)

Configure SendGrid via environment variables (recommended for production):

export APPLICATION_SENDGRID_ENABLED=true
export APPLICATION_SENDGRID_API_KEY=<your-sendgrid-api-key>
export APPLICATION_SENDGRID_BASE_URL=https://api.sendgrid.com

# Dynamic template IDs used for bulk/news emails
export APPLICATION_SENDGRID_NEWS_TEMPLATE=<dynamic-template-id>
export APPLICATION_SENDGRID_NEWS_TEMPLATE_NO_UNSUBSCRIBE=<dynamic-template-id>

# ASM (unsubscribe group) IDs by audience
export APPLICATION_SENDGRID_BULK_EMAIL_TYPES_CUSTOM_ASM_GROUP_ID=<id>
export APPLICATION_SENDGRID_BULK_EMAIL_TYPES_ALL_USERS_ASM_GROUP_ID=<id>
export APPLICATION_SENDGRID_BULK_EMAIL_TYPES_DEVELOPERS_ASM_GROUP_ID=<id>
export APPLICATION_SENDGRID_BULK_EMAIL_TYPES_SCIENTIFIC_NEWS_ASM_GROUP_ID=<id>

Notes:

• Keep APPLICATION_SENDGRID_API_KEY in a secret manager; do not commit it to git.
• APPLICATION_SENDGRID_BASE_URL should remain https://api.sendgrid.com in production.
• If an ASM group ID is omitted for an audience, unsubscribe-group behavior for that audience is disabled.

Required SendGrid API key permissions

This app calls the following SendGrid endpoints:

• POST /v3/mail/send
• GET /v3/asm/suppressions/{email}
• POST /v3/asm/groups/{groupId}/suppressions
• DELETE /v3/asm/groups/{groupId}/suppressions/{email}

Your SendGrid API key therefore needs:

• Mail Send: permission to send mail.
• Suppressions / ASM: read and write permissions for unsubscribe groups and suppressions.

If these permissions are missing, bulk send and email subscription preference updates will fail.

Modify config

/src/main/resources/config/application-dev.yml

1. Confirm database name, username, password under datasource config.
2. Change api-proxy-url to the URL where oncokb running. For example, http://localhost:8888/oncokb
3. Make sure the password for your Redis as same as the password for Redis defined in this file

For webpack frontend development, frontend-only runtime flags are initialized in /src/main/webapp/app/devServerConfig.ts. If a frontend change depends on window.serverConfig in local webpack dev, update that file instead of editing index.html.

Usage metrics schema

This repo also contains a separate Liquibase changelog tree for usage-metrics rollups:

• src/main/resources/config/liquibase-usage/master.xml

This schema is intentionally separate from the main application schema.

Reasoning:

• we want metrics/rollup data separated from the application's main schema
• the metrics schema is for aggregated usage data and pipeline-owned tables, not the raw application data model
• we did not enable automatic local migrations for this schema by default because supporting it cleanly with the existing Liquibase flow would have required too many hacks and environment-specific assumptions

Current behavior:

• the normal application Liquibase flow remains unchanged
• the usage metrics Liquibase runner exists in the application code, but it is disabled by default
• local development will not auto-create or auto-migrate the metrics schema unless you explicitly opt in

To enable it locally:

export APPLICATION_USAGE_LIQUIBASE_ENABLED=true

Optional overrides:

export APPLICATION_USAGE_LIQUIBASE_SCHEMA=oncokb_metrics
export APPLICATION_USAGE_LIQUIBASE_CHANGE_LOG=classpath:config/liquibase-usage/master.xml

The usage Liquibase runner uses separate bookkeeping tables:

• DATABASECHANGELOG_USAGE
• DATABASECHANGELOGLOCK_USAGE

Planned production usage:

• raw token stats continue to be exported by the Java backend to S3
• Airflow ingests those exports
• Airflow populates the metrics schema rollup tables

So the metrics schema is meant for rolled-up usage data and related pipeline state, not as a replacement for the raw token_stats flow.

Local Usage Metrics Schema Migrations

Note

All arguments for liquibase must start with -Dliquibase.. So for example, if you see rollbackCount as an argument, then you must put -Dliquibase.rollbackCount=1. See ./mvnw liquibase:help for a list of commands.

1. Add liquibase migrations src/main/resources/config/liquibase-usage/changelog/XXXXXXXXXXXXXX.xml

2. Add your xml file to the src/main/resources/config/liquibase-usage/master.xml list.

3. Update the command arguments below if you want a different schema name than oncokb_metrics

4. Run the migration.

   ./mvnw liquibase:update \
     -Dliquibase.changeLogFile=src/main/resources/config/liquibase-usage/master.xml \
     -Dliquibase.databaseChangeLogTableName=DATABASECHANGELOG_USAGE \
     -Dliquibase.databaseChangeLogLockTableName=DATABASECHANGELOGLOCK_USAGE \
     -Dliquibase.expressionVars.usageSchemaName=oncokb_metrics

Create and Run a SQL Script for production

1. Rollback create rollback script. (If your local database has the migrations added already)

   [!NOTE] I couldn't find a way to just make a script for a specific tag without doing the rollback. Ideally we shouldn't need to rollback to create the sql script.

   1. Create the script

      # Generates a rollback sql script
      ./mvnw liquibase:rollbackSQL \
        -Dliquibase.rollbackCount=1 \
        -Dliquibase.changeLogFile=src/main/resources/config/liquibase-usage/master.xml \
        -Dliquibase.databaseChangeLogTableName=DATABASECHANGELOG_USAGE \
        -Dliquibase.databaseChangeLogLockTableName=DATABASECHANGELOGLOCK_USAGE \
        -Dliquibase.expressionVars.usageSchemaName=oncokb_metrics

   2. Copy the migration script. The location is target/liquibase/migrate.sql by default.

      # Puts the sql script into your clipboard and you paste it where you need it
      cat ./target/liquibase/migrate.sql | pbcopy

   3. Run the script

2. Make your database changes

   1. Create the script

      # Generates a migration sql script
      ./mvnw liquibase:updateSQL \
        -Dliquibase.changeLogFile=src/main/resources/config/liquibase-usage/master.xml \
        -Dliquibase.databaseChangeLogTableName=DATABASECHANGELOG_USAGE \
        -Dliquibase.databaseChangeLogLockTableName=DATABASECHANGELOGLOCK_USAGE \
        -Dliquibase.expressionVars.usageSchemaName=oncokb_metrics

   2. Copy the migration script. The location is target/liquibase/migrate.sql by default.

      # Puts the sql script into your clipboard and you paste it where you need it
      cat ./target/liquibase/migrate.sql | pbcopy

   3. Run the script.

Notes

• this schema is separate from the main application schema on purpose
• this schema is intended for rolled-up metrics data and pipeline-owned tables, not the raw application data model
• the usage metrics Liquibase runner in the app is disabled by default
• the usage metrics runner uses its own changelog tables:
  • DATABASECHANGELOG_USAGE
  • DATABASECHANGELOGLOCK_USAGE
• we did not make this follow the normal local auto-migration path by default because doing that cleanly would have required too many hacks around the existing schema and migration flow

Building

Before you can build this project, you must install and configure the following dependencies on your machine:

1. Node.js: We use Node to run a development web server and build the project. Depending on your system, you can install Node either from source or as a pre-packaged bundle.

After installing Node, you should be able to run the following command to install development tools. You will only need to run this command when dependencies change in package.json.

yarn install

We use yarn scripts and Webpack as our build system.

Run the following commands in two separate terminals to create a blissful development experience where your browser auto-refreshes when files change on your hard drive.

./mvnw
yarn start

yarn is also used to manage CSS and JavaScript dependencies used in this application. You can upgrade dependencies by specifying a newer version in package.json. You can also run yarn update and yarn install to manage dependencies. Add the help flag on any command to see how you can use it. For example, yarn help update.

The yarn run command will list all of the scripts available to run for this project.

Connect with server side locally

After starting project up locally, you should type the following command in your browser console

localStorage.setItem("localdev", true)

To unset do:

localStorage.removeItem("localdev")

or

localStorage.setItem("localdev", false)

Managing dependencies

For example, to add Leaflet library as a runtime dependency of your application, you would run following command:

yarn add --exact leaflet

To benefit from TypeScript type definitions from DefinitelyTyped repository in development, you would run following command:

yarn add --dev --exact @types/leaflet

Then you would import the JS and CSS files specified in library's installation instructions so that Webpack knows about them: Note: There are still a few other things remaining to do for Leaflet that we won't detail here.

For further instructions on how to develop with JHipster, have a look at Using JHipster in development.

Local Database Migrations

Note

All arguments for liquibase must start with -Dliquibase.. So for example, if you see rollbackCount as an argument, then you must put -Dliquibase.rollbackCount=1. See ./mvnw liquibase:help for a list of commands.

1. Add liquibase migrations (Used an LLM to put together the xml file) src/main/resources/config/liquibase/changelog/XXXXXXXXXXXXXX.xml

2. Add you xml file to the src/main/resources/config/liquibase/master.xml list.

3. Update the pom.xml file with your database configurations

4. Run the migration. Docs for more info

   ./mvnw liquibase:update

Create and Run a SQL Script for production

1. Rollback create rollback script. (If your local database has the migrations added already)

   [!NOTE] I couldn't find a way to just make a script for a specific tag without doing the rollback. Ideally we shouldn't need to rollback to create the sql script.

   1. Create the script Docs for more info

      # Generates a rollback sql script
      ./mvnw liquibase:rollbackSQL -Dliquibase.rollbackCount=1

   2. Copy the migration script. The location is target/liquibase/migrate.sql by default.

      # Puts the sql script into your clipboard and you paste it where you need it
      cat ./target/liquibase/migrate.sql | pbcopy

   3. Run the script

2. Make your database changes

   1. Create the script Docs for more info

      # Generates a migration sql script
      ./mvnw liquibase:updateSQL

   2. Copy the migration script. The location is target/liquibase/migrate.sql by default.

      # Puts the sql script into your clipboard and you paste it where you need it
      cat ./target/liquibase/migrate.sql | pbcopy

   3. Run the script.

Building for production

Packaging as jar

To build the final jar and optimize the oncokb application for production, run:

./mvnw -Pprod clean verify

This will concatenate and minify the client CSS and JavaScript files. It will also modify index.html so it references these new files. To ensure everything worked, run:

java -jar target/*.jar

Then navigate to http://localhost:9090 in your browser.

Refer to Using JHipster in production for more details.

Packaging as war

To package your application as a war in order to deploy it to an application server, run:

./mvnw -Pprod,war clean verify

Packaging with a docker image ready

./mvnw package -Pprod verify jib:dockerBuild -DskipTests

Testing

To launch your application's tests, run:

./mvnw verify

Screenshot tests

Screenshot tests are run by Jest, Puppeteer. They're located in screenshot-test/. Because different dev environments have different systems, which may cause the resulting image doesn’t quite match the expected one from the __baseline_snapshots__ directory saved in source control, we dockerized the test process and can be run with:

yarn run screenshot-test-in-docker

You can also run screenshot-test locally using yarn run screenshot-test

Update screenshot tests

After running the command above, you will see __diff_output__ and __latest_snapshots__ folders created under screenshot-test. These new images indicate potential changes. If these changes are expected, please follow the below steps to update the images under __baseline_snapshots__.

1. Send a pull request to the intended branch
2. The PR will trigger the Screenshot Test GitHub action
3. Using this example, the action will generate two folders(visual-regression-diff-report and visual-regression-screenshots) under the Artifacts section.
4. visual-regression-diff-report folder includes the files should be updated. You can copy over the ones with the same name under visual-regression-screenshots to your pull request.

Using Docker to simplify development (optional)

You can use Docker to improve your JHipster development experience. A number of docker-compose configuration are available in the src/main/docker folder to launch required third party services.

For example, to start a mysql database in a docker container, run:

docker-compose -f src/main/docker/mysql.yml up -d

To start a local Keycloak instance for test or development authentication flows, run:

docker-compose -f src/main/docker/keycloak.yml up -d

This stack imports the oncokb-public realm from src/main/docker/keycloak/oncokb-public-realm.json on startup. The imported OIDC client is:

• realm: oncokb-public
• client-id: web_app
• client-secret: CLIENT_SECRET

The sample realm also creates two local Keycloak users:

• username: dev@mskcc.org
• username: test@myorg.org
• password: password

For the OncoKB app to accept those logins, the same emails must also exist as OncoKB users in your local application database. You can seed both app users and long-lived local bearer tokens with:

scripts/bootstrap-local-test-users.sh

The script prints the generated tokens for:

• dev@mskcc.org
• test@myorg.org

To stop a service and remove its container, run the matching down command, for example:

docker-compose -f src/main/docker/mysql.yml down

You can also fully dockerize your application and all the services that it depends on. To achieve this, first build a docker image of your app by running:

./mvnw -Pprod verify jib:dockerBuild

Then run:

docker-compose -f src/main/docker/app.yml up -d

For more information refer to Using Docker and Docker-Compose, this page also contains information on the docker-compose sub-generator (jhipster docker-compose), which is able to generate docker configurations for one or several JHipster applications.

Testing Slack Integration

1. Ensure you can access the MSK secret server

2. Ensure you can access the Slack API Manager

3. Install ngrok

4. Update the /src/main/resources/config/application-dev.yml file

   • Change spring.mail.username to dev.oncokb@gmail.com
   • Change spring.mail.password to the password for dev.oncokb@gmail.com found in the secret server
   • Change application.slack.user-registration-webhook to the slack hook.
   • Change application.slack.slack-base-url to https://oncokb.slack.com
   • Change application.slack.user-registration-channel-id to C03EASPQ8AZ
   • If you need to use a slack endpoint that requires a token, update application.slack.slack-bot-oauth-token with the token get from this page.

5. Start up ngrok

   ngrok http http://localhost:9090

6. Copy the forwarding URL generated by ngrok.

   • Should be something like https://xxxx-xxx-xxx-xx-xx.ngrok-free.app

7. Update the Request URL in the slack message configuration in the Request URL to https://xxxx-xxx-xxx-xx-xx.ngrok-free.app/api/slack

   • Replace the base URL with the URL generated by ngrok for forwarding

8. Register an account in your local instance of OncoKB public

   • Ensure the email you use is one you have access to

9. Check the slack-app-dev channel for a message and approve it

Update Privacy Notice

If you have a major change you can use pandoc which can be installed with brew

pandoc -f docx -t gfm /path/to/word/doc.docx -o ./src/main/webapp/app/pages/privacyNotice/markdown/PrivacyNotice.md

Don't forget to keep the TableOfContents component in the markdown

Update PrivacyNotice.md and run the following command.

yarn run buildPrivacyNotice

Update News

1. From the News Google doc click to File > Download > Markdown

2. Move the markdown file here src/main/webapp/app/pages/newsPage/markdown/NewsContentMMYYYY.md

3. Add <br/> to force new lines in the markdown. (This is normally needed in the tables)

4. Create News React Components

   yarn run buildNewsPages

5. Add the release date to DATA_RELEASES in the constants.tsx file.

   export const DATA_RELEASES: DataRelease[] = [
     { date: 'DDMMYYYY', version: 'vX.XX' },
     ...
   ];

6. Add NewsList to NewsPage.tsx

   <NewsList date={'MMDDYYYY'} />

7. Add NewContent to NewsPageContent.tsx

   export const NEWS_BY_DATE: { [date: string]: NewsData } = {
     'MMDDYYYY': {
       rawComponent: <NewsContentDDYYYY />,
     },
     ...
   }