Skip to contents

riskmetric

riskmetric is a collection of risk metrics to evaluate the quality of R packages.

This package is in experimentation. Final considerations about design are being considered, but core concepts are considered final.

Background

The risk of using an R package is evaluated based on a number of metrics meant to evaluate development best practices, code documentation, community engagement and development sustainability. We hope to provide a framework to quantify risk by assessing these metrics. This package serves as a starting point for exploring the heterogeneity of code quality, and begin a broader conversation about the validation of R packages. Primarily, this effort aims to provide some context for validation within regulated industries.

We separate three steps in the workflow to assess the risk of an R package using riskmetric:

  1. Finding a source for package information (installed package or CRAN/git source) pkg_ref()
  2. Assessing the package under validation criteria pkg_assess()
  3. Scoring assessment criteria pkg_score()

The results will be assembled in a dataset of validation criteria containing an overall risk score for each package as shown in the example below.

Installation

You can install riskmetric from CRAN with:

install.packages("riskmetric")

Or from GitHub using devtools with:

devtools::install_github("pharmaR/riskmetric")

Example

Scrape metadata locally or remotely, then assess that metadata and score it to estimate risk. For each package, derive a composite measure of risk, or a collection of individual scores which can be easily used to generate validation reports.

library(dplyr)
library(riskmetric)

pkg_ref(c("riskmetric", "utils", "tools")) %>%
  pkg_assess() %>%
  pkg_score()

The {riskassessment} application

riskassessment is a full-fledged R package containing a shiny front-end that augments the utility of riskmetric. The application’s goal is to provide a central hub for an organization to review and assess the risk of R packages, providing handy tools and guide rails along the way. The app uses a local database to store & display:

  • all riskmetric metrics, including package risk scores over time
  • organization-wide metric weighting, plus rules to automate org decisions (whether to endorse/ prohibit the pkg)
  • package-level user dialogue on the perceived risk, to facilitate communication & notes

To learn more about riskassessment, please browse the user guide or consider taking the demo app for a spin.

Get Involved

We have a bi-weekly sprint meeting for developers to discuss the progress.

riskmetric is centrally a community project. Comfort with a quantification of risk comes via consensus, and for that this project is dependent on close community engagement. There are plenty of ways to help:

  • Share the package
  • File issues when you encounter bugs
  • Weigh in on proposed metrics, or suggest a new one
  • Help us devise the best way to summarize risk into a single score
  • Help us keep documentation up to date
  • Contribute code to tackle the metric backlog