rstudio::conf 2022 recap

Stephen Turner

July 25-28, 2022

Highlights

  • RStudio PBC is now Posit
  • Tidymodels for ML training, evaluation, and deployment
  • Quarto
  • Shiny for Python & Serverless Shiny
  • Miscellaneous

RStudio is now Posit

RStudio PBC is now Posit

RStudio PBC is now Posit

  • 2020: RStudio became a Public Benefit Corporation, recognized as a Certified B Corporation (blog post).

  • Long-term focus on sustainable open-source software development for data science.

    • What will they keynote at rstudio::conf 2122 look like?

    • RStudio is becoming more multilingual (see also Quarto)

    • R, Python, Julia, etc. may be long-dead languages.

  • 2022: RStudio becomes Posit

Tidymodels

Tidymodels: Overview

  • Collection of packages for machine learning using tidyverse principles
  • caret is no longer being developed
  • Tidymodels is ready for production. CRAN 1.0.x releases
  • Extremely well documented. See tidymodels.org
  • Workshop materials: workshops.tidymodels.org

Tidymodels: simple linear regression

Compare base R versus tidymodels for a simple linear regression:

Using base stats::lm():

lmfit <- lm(mpg~., data=mtcars)

Using tidymodels workflows + parsnip:

library(tidymodels)
tmfit <- 
  workflow() %>% 
  add_model(linear_reg()) %>% 
  add_formula(mpg~.) %>% 
  fit(data=mtcars)

Tidymodels: boosted trees

Set up workflow:

# Boosted tree spec 
xgb_spec <-
  boost_tree(
    trees = 500, min_n = tune(), 
    stop_iter = tune(), tree_depth = tune(),
    learn_rate = tune(), loss_reduction = tune()
  ) %>%
  set_mode("classification") %>% 
  set_engine("xgboost")

# Boosted tree recipe
xgb_rec <- 
  recipe(on_goal ~ ., data = nhl_train) %>% 
  step_lencode_mixed(player, outcome=vars(on_goal)) %>% 
  step_dummy(all_nominal_predictors()) %>%
  step_zv(all_predictors())

# Boosted tree workflow
xgb_wflow <- 
  workflow() %>% 
  add_model(xgb_spec) %>% 
  add_recipe(xgb_rec)

Find optimal parameters with cross-validation using a Latin hypercube grid search:

grid <- 
  xgb_wflow %>% 
  extract_parameter_set_dials() %>% 
  grid_latin_hypercube(size = 25)

xgb_res <- 
  xgb_wflow %>% 
  tune_grid(resamples=nhl_val, grid=grid)
xgb_res

Tidymodels: usemodels

Run this:

usemodels::use_ranger(mpg~., data=mtcars)

Copied to the clipboard:

ranger_recipe <- 
  recipe(formula = mpg ~ ., data = mtcars) 

ranger_spec <- 
  rand_forest(mtry = tune(), min_n = tune(), trees = 1000) %>% 
  set_mode("regression") %>% 
  set_engine("ranger") 

ranger_workflow <- 
  workflow() %>% 
  add_recipe(ranger_recipe) %>% 
  add_model(ranger_spec) 

set.seed(38757)
ranger_tune <-
  tune_grid(ranger_workflow, 
            resamples = stop("add your rsample object"), 
            grid = stop("add number of candidate points"))

Tidymodels: usemodels

Run this:

usemodels::use_glmnet(mpg~., data=mtcars)

Copied to the clipboard:

glmnet_recipe <- 
  recipe(formula = mpg ~ ., data = mtcars) %>% 
  step_zv(all_predictors()) %>% 
  step_normalize(all_numeric_predictors()) 

glmnet_spec <- 
  linear_reg(penalty = tune(), mixture = tune()) %>% 
  set_mode("regression") %>% 
  set_engine("glmnet") 

glmnet_workflow <- 
  workflow() %>% 
  add_recipe(glmnet_recipe) %>% 
  add_model(glmnet_spec) 

glmnet_grid <- tidyr::crossing(penalty = 10^seq(-6, -1, length.out = 20), 
                               mixture = c(0.05, 0.2, 0.4, 0.6, 0.8, 1)) 

glmnet_tune <- 
  tune_grid(glmnet_workflow, 
            resamples = stop("add your rsample object"), 
            grid = glmnet_grid)

Tidymodels: usemodels

Run this:

usemodels::use_xgboost(mpg~., data=mtcars)

Copied to the clipboard:

xgboost_recipe <- 
  recipe(formula = mpg ~ ., data = mtcars) %>% 
  step_zv(all_predictors()) 

xgboost_spec <- 
  boost_tree(trees = tune(), min_n = tune(), 
             tree_depth = tune(), learn_rate = tune(), 
             loss_reduction = tune(), sample_size = tune()) %>% 
  set_mode("regression") %>% 
  set_engine("xgboost") 

xgboost_workflow <- 
  workflow() %>% 
  add_recipe(xgboost_recipe) %>% 
  add_model(xgboost_spec) 

set.seed(9270)
xgboost_tune <-
  tune_grid(xgboost_workflow, 
            resamples = stop("add your rsample object"), 
            grid = stop("add number of candidate points"))

Tidymodels + vetiver + plumber = đź”®

Model training and deployment. Do this in one R session.

library(tidymodels)
library(vetiver)
library(plumber)

# Not the way you'd actually split data - for demo only
cars_train <- mtcars[1:24,]
cars_test <- mtcars[25:32,]

# Random forest using C++ implementation
rf_spec <-
  rand_forest(trees=1000) %>%
  set_mode("regression") %>%
  set_engine("ranger")

# Simple workflow
rf_wflow <- workflow(mpg~., rf_spec)

# Fit the model
rf_fit <- fit(rf_wflow, cars_train)

# Create a vetiver model object
v <- vetiver_model(rf_fit, "mtcars_mpg")

# Create a plumber API
pr <- pr() %>% vetiver_api(v)

# Run the API server, open at http://127.0.0.1:5678/
pr_run(pr, port=5678)

Predict from the model endpoint. Do this in a separate session.

library(vetiver)

# Use the same IP/port from above
endpoint <- vetiver_endpoint("http://127.0.0.1:5678/predict")

# same split from above
cars_test <- mtcars[25:32,]

# Predict from the endpoint (returns a 1-col tibble)
predict(endpoint, cars_test)

# Stick this onto the original data, plot, etc.
cbind(cars_test, predict(endpoint, cars_test))

Model workflow

Consider simple prep-processing (e.g., centering/scaling numeric predictors), simple feature engineering (e.g., PCA), followed by vanilla GLM/tree model.

Wrong:

Tidymodels makes this easy:

Quarto

We don’t talk about Quarto

See Alison Hill’s blog post from April

What is Quarto?

  • Quarto is an open-source scientific and technical publishing system built on Pandoc.
  • quarto is a language agnostic command line interface (CLI).
  • “Batteries included” – RStudio now comes with the quarto CLI installed.
  • https://quarto.org/
sturner$ quarto --help
  Usage:   quarto
  Version: 1.0.36

  Options:
    -h, --help     - Show this help.                            
    -V, --version  - Show the version number for this program.  

  Commands:
    render          [input] [args...]   - Render input file(s) to various document types.            
    preview         [file] [args...]    - Render and preview a document or website project.          
    serve           [input]             - Serve a Shiny interactive document.                        
    create-project  [dir]               - Create a project for rendering multiple documents          
    convert         <input>             - Convert documents to alternate representations.            
    pandoc          [args...]           - Run the version of Pandoc embedded within Quarto.          
    run             [script] [args...]  - Run a TypeScript, R, Python, or Lua script.                
    install         <type> [target]     - Installs an extension or global dependency.                
    publish         [provider] [path]   - Publish a document or project. Available providers include:
    check           [target]            - Verify correct functioning of Quarto installation.

What is Quarto?

What is Quarto?

Shiny for Python

Shiny for Python

  • Similar conceptually to Shiny for R, and designed to feel like idiomatic Python

  • Documentation: https://shiny.rstudio.com/py.

  • Deployment options include shinyapps.io, RStudio Connect, Shiny Server Open Source, and on a static server.

Shiny for Python

Minimal example:

from shiny import App, render, ui
app_ui = ui.page_fluid(
    ui.input_slider("n", "N", 0, 100, 20),
    ui.output_text_verbatim("txt"),
)
def server(input, output, session):
    @output
    @render.text
    def txt():
        return f"n*2 is {input.n() * 2}"
app = App(app_ui, server)

Shinylive: Serverless Shiny

  • Shiny for Python running completely in the browser, no server required
  • Powered by WebAssembly and Pyodide
    • WebAssembly (WASM): binary format for compiled programs that can run in a web browser at near-native speeds. webassembly.org
    • Pyodide: Port of CPython to WASM. pyodide.org
  • No installation needed: no shiny, no python.
  • Easy deployment (any static web hosting service)
  • Security: runs in browser sandbox

Shinylive: example

URL: stephenturner.github.io/shinypy-helloworld

Code: github.com/stephenturner/shinypy-helloworld

from shiny import App, render, ui
import numpy as np
import matplotlib.pyplot as plt

app_ui = ui.page_fluid(
    ui.layout_sidebar(
        ui.panel_sidebar(
            ui.input_slider("n", "N", 0, 100, 20),
        ),
        ui.panel_main(
            ui.output_plot("plot"),
        ),
    ),
)

def server(input, output, session):
    @output
    @render.plot(alt="A histogram")
    def plot():
        np.random.seed(19680801)
        x = 100 + 15 * np.random.randn(437)
        fig, ax = plt.subplots()
        ax.hist(x, input.n(), density=True)
        return fig

app = App(app_ui, server, debug=True)

Miscellaneous

What’s new in knitr

  • Lots of new goodies. See knitr release notes for details.
  • Previously you could re-use content of an existing chunk with the ref.label option. New opts.label="prev-chunk" re-uses a labeled chunk’s options (can overwrite in current chunk’s options).
  • New chunk options, {r, file=c("script-1.R")} instead of {r, code=readLines("script.R")} to use the contents of script.R as the content of this chunk.
  • Chunk options

Chunk options can be written inside a code chunk after #|, e.g.,

#| echo = FALSE, fig.width = 10,
#| fig.cap = "This is a long caption."

Or using YAML. Convert old chunk options in a .Rmd to the new syntax with knitr::convert_chunk_header().

#| echo: false
#| fig.width: 10

tidyclust 📦

  • Emil Hvitfeldt (RStudio) introduced the tidyclust package.
  • The goal of tidyclust is to provide a tidy, unified interface to clustering models.
  • Closely modeled after the parsnip package.
  • Documentation at emilhvitfeldt.github.io/tidyclust.
  • Example usage:
library(tidyclust)
kmeans_spec <- 
  k_means(num_clusters = 3) %>%
  set_engine("stats") 
kmeans_spec_fit <- 
  kmeans_spec %>%
  fit(~., data = mtcars)
predict(kmeans_spec_fit, mtcars[1:4, ])
extract_cluster_assignment(kmeans_spec_fit)
extract_centroids(kmeans_spec_fit)

Workboots 📦

Mark Rieke at Memorial Hermann Health System introduced the **workboots** package for generating prediction intervals in the tidymodels ecosystem. Documentation/source: markjrieke.github.io/workboots. Example usage:

Create a workflow with tidymodels:

library(tidymodels)

data("penguins")
penguins <- penguins %>% drop_na()
set.seed(123)
penguins_split <- initial_split(penguins)
penguins_test <- testing(penguins_split)
penguins_train <- training(penguins_split)

penguins_wf <- 
  workflow() %>%
  add_recipe(
    recipe(body_mass_g ~ ., data = penguins_train) %>% 
           step_dummy(all_nominal())) %>%
  add_model(boost_tree("regression"))

Generate predictions from 2000 bootstrap models, and summarize with 95% PI:

library(workboots)
set.seed(345)
penguins_pred_int <-
  penguins_wf %>%
  predict_boots(
    n = 2000,
    training_data = penguins_train,
    new_data = penguins_test
  )
pengins_pred_int %>%
  summarise_predictions()

GSK

  • Becca Krouse from GSK talked about lessons from pharma on R package assessment.

    • There are thousands of R packages - how do you go about vetting & choosing which packages to integrate into your workflow?

    • Things to look for:

      1. Community – how active is the development/user community around the package?

      2. Documentation – how well documented is the package?

      3. Tests – does the package have good built-in tests (testthat)?

      4. Authors/creators – are creators grad students/postdocs who will abandon the project when it’s done? Would you anticipate long-term development and support?

  • Ellis Hughes discussed driving change and efficiency within organizations workflows.
    • Uses the officeverse packages (e.g., flextable, officer) to ingest metadata, processed results files generated by upstream applications, etc., and produce formatted reports, usable by non-programming staff.

UKHSA

  • Isaac Florence from the UK Health Security Agency talked about scaling and automating R workflows with Kubernetes and Airflow.

  • UKHSA using RH OpenShift implementation of k8s to deploy containers at whatever scale on whatever hardware available (cloud, laptop, standard HPC, etc).

  • Airflow (also open source) from Apache is a workflow scheduler and monitoring platform aka “fancy cron.”

  • At UKHSA, each team has own k8s namespace (“project” in OpenShift), which has defined users, resources, and security.

  • Each Airflow DAG is assigned to a project so teams can have multiple DAGs. Teams can see their own projects and DAGs only in Airflow, aka simplicity/security.

  • Every Airflow task creates a new pod from a container image, specifying a terminal command (eg R script). Specs of images/secrets/credentials/etc all defined in k8s - airflow just tells k8s what pod to run and when.

Lightning talks

  • George Stagg (RStudio) discussed WebR: R compiled to webassembly that runs in the browser. Source; Demo.
  • Davis Vaughn (RStudio) talked about the new clock package (clock.r-lib.org) as a package for working with date-time data.

    • Clock is not replacing lubridate.

    • Improved safety with time zones, calendars, etc. E.g., "2022-01-30" + months(1)

    • New date types: year_month_day to year_quarter_day year_week_day, etc.

    • Clock is compatible with the slider package (rolling averages) and ivs (ranges).

  • Beatriz Milz talked about running R Scripts with GitHub Actions. Slides: beamilz.com/talks/en/2022-rstudio-conf.
  • Tan Ho: GitHub for R package development. Beyond all the normal things, tip for using GitHub releases for large file storage. You can attach up to 2GB files to a release without cluttering git history.

Talks I missed

  • Kelly O’Briant (RStudio): Remote Content Execution with RStudio Connect and Kubernetes
  • Mark Sellors (R4Pi.org): The worlds smallest R environment? Running R on a $15 computer
  • Andreas Hofheinz (d-fine): leafdown: Interactive multi-layer maps in Shiny apps (slides)
  • Tanya Cashorali (TCB Analytics): Cross-Industry Anomaly Detection Solutions with R and Shiny (slides)
  • E David Aja (Rstudio): You should be using renv (rstudio.github.io/renv)
  • Aaron Chafetz (USAID): Digging a Pit of Success for Your Organization: Embracing a R-based ecosystem in the US federal government (slides)
  • Kamil Zyla (Appsilon): Rhino: Shiny application framework for enterprise (appsilon.github.io/rhino)
  • Hadley Wickham (RStudio): An introduction to R7 (rconsortium.github.io/OOP-WG)

R/Med 2022: August 23-26 (virtual)

  • Registration ($50): events.linuxfoundation.org/r-medicine
  • Workshops:
    1. Building dashboards with Shiny
    2. Gene expression analysis with R
    3. Geospatial data analysis and visualization
    4. Reproducible Research with Quarto
    5. Tabulating biomedical data with gtsummary
    6. Enterprise-grade Shiny App Development with rhino
    7. Using Public Data and Maps for Powerful Data Visualization
    8. Introduction to R for medical data
  • Keynotes: JJ Allaire, Frank Harrell
  • Much more (link to schedule)

strnr.quarto.pub/rstudioconf2022