Workshop: evaluate pkg quality with the {riskassessment} application

Shiny Gatherings x Pharmaverse, 28 JAN 2025

Slides: 🔗 Follow Along!


On behalf of the R Validation Hub team:

Aaron Clark Arcus Biosciences

Jeff Thompson Arcus Biosciences

Disclaimer




Any opinions expressed in this presentation and on the following slides are solely those of the presenter and do not necessarily reflect those sponsoring the work

👋 Who We Are

The R Validation Hub is a collaboration to support the adoption of R within a biopharmaceutical regulatory setting (pharmaR.org)

  • Grew out of R/Pharma 2018
  • Led by participants from ~10 organizations
  • With frequent involvement from health authorities (primarily the FDA)
  • And subscribers from ~60 organizations spanning multiple industries

👷‍♂️ The R Validation Hub: What We Do

Products

White Paper

Guidance on compliant use of R and management of packages

Repositories

Building a public, validation-ready resource for R packages

Coline Zeballos

Communications

Connecting validation experts across the industry

Jaxon Abercrombie, Anuja Das, Antal Martinecz

{riskmetric}

Gather and report on risk heuristics to support validation decision-making

Eric Milliman

{riskassessment}

A web interface to {riskmetric}, supporting review, annotation and cataloging of decisions

Aaron Clark, Jeff Thompson

📞 Community Meetings


Sign up

🗓️ Agenda

  • 🖥️{riskassessment} App Workstream 5min
  • 🥩 Reviewer POV - The App’s Best Use Case 15min
  • 🥔 Admin POV - Configure for your org 20min
  • 💬 Room Discussion 10min

Latest Features

Two tools: what do they do




is a framework to quantify an R package’s “risk” by assessing several meaningful metrics designed to evaluate package development best practices, code documentation, community engagement, and development sustainability.




is a full-fledged R package containing a shiny front-end that augments the utility of {riskmetric}. The application’s goal is to provide a central hub for an organization to review and assess the risk of R packages, providing handy tools and guide rails along the way.

Quantify risk programmatically

Sometimes “quality” is measurable! Software dev best practices dictate an R-package should have:

  • A license
  • Source code available for browsing
  • An easy to contact maintainer
  • A place to report bugs
  • Evidence that new bugs are being addressed
  • Complete Function documentation
  • Adequate test coverage
  • Community usage

18 total assessments (to date)!

Why create a Shiny app?


Main goal: complement GxP “package inclusion requests” with data.

Why create a Shiny app?


Main goal: complement GxP “package inclusion requests” with data.

Why create a Shiny app?


Main goal: complement GxP “package inclusion requests” with data.

Why create a Shiny app?


Main goal: complement GxP “package inclusion requests” with data.

  • Shifts responsibility of assessing package risk to the person(s) requesting it

Why create a Shiny app?


Main goal: complement GxP “package inclusion requests” with data.

  • Shifts responsibility of assessing package risk to the person(s) requesting it
  • Leverage the app’s risk-based “decision triage” (based on org-defined rules) to output a risk summary report

Why create a Shiny app?


Main goal: complement GxP “package inclusion requests” with data.

  • Shifts responsibility of assessing package risk to the person(s) requesting it
  • Leverage the app’s risk-based “decision triage” (based on org-defined rules) to output a risk summary report
  • Changes communication from “Can we add this?” to a informed “We can add this.”

Why create a Shiny app? (cont’d)

Bonus / Secondary Perks:

  • Provide a platform for package exploration without the need to write any custom {riskmetric}
  • Run {riskmetric} on the same machine with the same environment – creating a central hub for reproducibility
  • Maintain consistent, org-specific settings/options when producing risk outputs
  • Manage who’s involved in the review process via user authentication & role management
  • Facilitate and store user summaries & communication, on packages and/or metrics

🥩 ‘Reviewer’ POV

I want it

Start with {riskassessment}

Live Demo

🥩 Reviewer POV

Submit {riskassessment} Report(s)

🥔 ‘Admin’ POV

Jeff Thompson

💬 Room Discussion

Thank you

thank you to our many contributors

our work is the product of the donated time of many passionate individuals

Slides: 🔗 For your Review

Affiliation

Community Grants & Sponsorships

Over USD $1.4 Million

Organizing Large Scale Collaborative Projects

R Validation Hub, R-Ladies

Co-Host Multidisciplinary Data Science Forums

Stanford Data Institute

Direct Support for Key R Events

R/Medicine, R/Pharma, useR!, LatinR, and more

Direct Worldwide Support for R User Groups