Author: Stefano Mangiola
Date: 2026-04-28

# Probabilistic Outlier Identification for RNA Sequencing Generalized Linear Models

Stefano Mangiola

#### 2026-04-28

#### Package

ppcseq 1.20.0

# 1 Introduction

Relative transcript abundance has proven to be a valuable tool for understanding the function of genes in biological systems. For the differential analysis of transcript abundance using RNA sequencing data, the negative binomial model is by far the most frequently adopted. However, common methods that are based on a negative binomial model are not robust to extreme outliers, which we found to be abundant in public datasets. So far, no rigorous and probabilistic methods for detection of outliers have been developed for RNA sequencing data, leaving the identification mostly to visual inspection. Recent advances in Bayesian computation allow large-scale comparison of observed data against its theoretical distribution given in a statistical model. Here we propose ppcseq, a key quality-control tool for identifying transcripts that include outlier data points in differential expression analysis, which do not follow a negative binomial distribution. Applying ppcseq to analyse several publicly available datasets using popular tools, we show that from 3 to 10 percent of differentially abundant transcripts across algorithms and datasets had statistics inflated by the presence of outliers.

# 2 Installation and use

The input data set is a tidy representation of a differential gene transcript abundance analysis

    library(dplyr)
    library(ppcseq)

To install:

Before install, for linux systems, in order to exploit multi-threading, from R write:

    fileConn<-file("~/.R/Makevars")
    writeLines(c( "CXX14FLAGS += -O3","CXX14FLAGS += -DSTAN_THREADS", "CXX14FLAGS += -pthread"), fileConn)
    close(fileConn)

Multi-threading allows the sampling or variational bayes to share the computation on multiple cores.

Then, install with

    if(!requireNamespace("BiocManager", quietly = TRUE))
      install.packages("BiocManager")
    BiocManager::install("ppcseq")

You can get the test dataset with

    data("counts")
    counts

    ## # A tibble: 394,821 × 9
    ##    sample  symbol   logCPM    LR   PValue        FDR value       W Label
    ##    <chr>   <chr>     <dbl> <dbl>    <dbl>      <dbl> <int>   <dbl> <chr>
    ##  1 10922PP SLC16A12   1.39  41.1 1.46e-10 0.00000274   160 -0.129  High
    ##  2 10935PP SLC16A12   1.39  41.1 1.46e-10 0.00000274   150 -0.127  High
    ##  3 10973PP SLC16A12   1.39  41.1 1.46e-10 0.00000274   146 -0.426  High
    ##  4 10976PP SLC16A12   1.39  41.1 1.46e-10 0.00000274   347 -0.0164 High
    ##  5 10985PP SLC16A12   1.39  41.1 1.46e-10 0.00000274   175 -0.135  High
    ##  6 11026PP SLC16A12   1.39  41.1 1.46e-10 0.00000274   244  0.125  High
    ##  7 11045PP SLC16A12   1.39  41.1 1.46e-10 0.00000274   399 -0.0892 High
    ##  8 11082PP SLC16A12   1.39  41.1 1.46e-10 0.00000274   100  0.261  Neoadjuvant
    ##  9 11086PP SLC16A12   1.39  41.1 1.46e-10 0.00000274    37 -0.132  Neoadjuvant
    ## 10 11103PP SLC16A12   1.39  41.1 1.46e-10 0.00000274    73  0.146  Neoadjuvant
    ## # ℹ 394,811 more rows

You can identify anrtefactual calls from your differential transcribt anundance analysis, due to outliers.

    # Import libraries

    if(Sys.info()[['sysname']] == "Linux")
    counts.ppc =
        counts %>%
        mutate(is_significant = FDR < 0.0001) %>%
        identify_outliers(
            formula = ~ Label,
            .sample = sample,
            .transcript = symbol,
            .abundance = value,
            .significance = PValue,
            .do_check = is_significant,
            percent_false_positive_genes = 5,
            approximate_posterior_inference = FALSE,
            cores = 1,

            # This is ONLY for speeding up the Vignette execution
            draws_after_tail = 1
        )

The new posterior predictive check has been added to the original data frame

    if(Sys.info()[['sysname']] == "Linux")
    counts.ppc

The new data frame contains plots for each gene

We can visualise the top five differentially transcribed genes

    if(Sys.info()[['sysname']] == "Linux")
    counts.ppc_plots =
        counts.ppc %>%
        plot_credible_intervals()

    if(Sys.info()[['sysname']] == "Linux")
    counts.ppc_plots %>%
        pull(plot) %>%
        .[seq_len(1)]

    sessionInfo()

    ## R version 4.6.0 Patched (2026-04-24 r89963)
    ## Platform: aarch64-apple-darwin23
    ## Running under: macOS Tahoe 26.3.1
    ##
    ## Matrix products: default
    ## BLAS:   /Library/Frameworks/R.framework/Versions/4.6/Resources/lib/libRblas.0.dylib
    ## LAPACK: /Library/Frameworks/R.framework/Versions/4.6/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1
    ##
    ## locale:
    ## [1] C/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
    ##
    ## time zone: America/New_York
    ## tzcode source: internal
    ##
    ## attached base packages:
    ## [1] stats     graphics  grDevices utils     datasets  methods   base
    ##
    ## other attached packages:
    ## [1] ppcseq_1.20.0       rstan_2.32.7        StanHeaders_2.32.10
    ## [4] dplyr_1.2.1         knitr_1.51          BiocStyle_2.40.0
    ##
    ## loaded via a namespace (and not attached):
    ##  [1] benchmarkmeData_2.0.0 gtable_0.3.6          tensorA_0.36.2.1
    ##  [4] xfun_0.57             bslib_0.10.0          ggplot2_4.0.3
    ##  [7] QuickJSR_1.9.2        inline_0.3.21         lattice_0.22-9
    ## [10] vctrs_0.7.3           tools_4.6.0           generics_0.1.4
    ## [13] stats4_4.6.0          curl_7.1.0            parallel_4.6.0
    ## [16] tibble_3.3.1          pkgconfig_2.0.3       Matrix_1.7-5
    ## [19] checkmate_2.3.4       RColorBrewer_1.1-3    S7_0.2.2
    ## [22] distributional_0.7.0  RcppParallel_5.1.11-2 lifecycle_1.0.5
    ## [25] compiler_4.6.0        farver_2.1.2          statmod_1.5.1
    ## [28] codetools_0.2-20      htmltools_0.5.9       sass_0.4.10
    ## [31] yaml_2.3.12           pillar_1.11.1         jquerylib_0.1.4
    ## [34] tidyr_1.3.2           arrayhelpers_1.1-0    cachem_1.1.0
    ## [37] limma_3.68.0          iterators_1.0.14      foreach_1.5.2
    ## [40] abind_1.4-8           posterior_1.7.0       tidybayes_3.0.7
    ## [43] tidyselect_1.2.1      locfit_1.5-9.12       digest_0.6.39
    ## [46] svUnit_1.0.8          purrr_1.2.2           bookdown_0.46
    ## [49] fastmap_1.2.0         grid_4.6.0            cli_3.6.6
    ## [52] magrittr_2.0.5        loo_2.9.0             utf8_1.2.6
    ## [55] dichromat_2.0-0.1     pkgbuild_1.4.8        edgeR_4.10.0
    ## [58] scales_1.4.0          backports_1.5.1       httr_1.4.8
    ## [61] rmarkdown_2.31        matrixStats_1.5.0     otel_0.2.0
    ## [64] benchmarkme_1.0.8     gridExtra_2.3         coda_0.19-4.1
    ## [67] evaluate_1.0.5        doParallel_1.0.17     V8_8.2.0
    ## [70] ggdist_3.3.3          rstantools_2.6.0      rlang_1.2.0
    ## [73] Rcpp_1.1.1-1.1        glue_1.8.1            BiocManager_1.30.27
    ## [76] jsonlite_2.0.0        R6_2.6.1