---
title: Using scrapper to analyze single-cell data
author: 
- name: Aaron Lun
  email: infinite.monkeys.with.keyboards@gmail.com
date: "Revised: December 26, 2025"
output:
  BiocStyle::html_document
package: scrapper
vignette: >
  %\VignetteIndexEntry{Using scrapper to analyze single-cell data}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}    
---

```{r, echo=FALSE, results="hide", message=FALSE}
require(knitr)
opts_chunk$set(error=FALSE, message=FALSE, warning=FALSE)

library(BiocStyle)
self <- Biocpkg("scrapper")
```

# Overview

`r self` implements bindings to C++ code for analyzing single-cell data, mostly from the [**libscran**](https://github.com/libscran) libraries.
Each function performs an individual analysis step ranging from quality control to clustering and marker detection.
Users can also conveniently perform end-to-end analyses with a single call to the `analyze.se()` function. 

# Basic analysis

Let's fetch a small single-cell RNA-seq dataset for demonstration purposes:

```{r}
library(scRNAseq)
sce.z <- ZeiselBrainData()
sce.z
```

We run it through the `r self` analysis pipeline.
This performs the usual steps - quality control, normalization, selection of highly variable genes, PCA, clustering, visualization and marker detection,
see the [OSCA book](https://bioconductor.org/books/release/OSCA/) for more theoretical details. 

```{r}
library(scrapper)
is.mito <- grep("^mt-", rownames(sce.z))
results <- analyze.se(
    sce.z,
    rna.qc.subsets=list(Mito=is.mito),
    num.threads=2 # limit number of threads to keep R CMD check happy.
)
results$x # most outputs are stored in a copy of the input SingleCellExperiment. 
```

Now we can have a look at some of the results.
For example, we can make a t-SNE plot:

```{r}
library(scater)
plotReducedDim(results$x, "TSNE", colour_by="graph.cluster")
```

We can also look at the markers defining each cluster:

```{r}
# Ordering by the median AUC across pairwise comparisons involving the cluster.
previewMarkers(results$markers$rna[[1]], order.by="auc.median")
```

`analyze.se()` can also be broken down to its component functions for further customization:

```{r}
test <- sce.z
test <- quickRnaQc.se(test, subsets=list(mt=is.mito), num.threads=2)
test <- test[,test$keep]
test <- normalizeRnaCounts.se(test, size.factors=test$sum)
test <- chooseRnaHvgs.se(test, num.threads=2)
test <- runPca.se(test, features=rowData(test)$hvg, num.threads=2)
test <- runAllNeighborSteps.se(test, num.threads=2)
markers <- scoreMarkers.se(test, groups=test$clusters, num.threads=2)
```

# Blocking on batches 

Let's fetch another brain dataset and combine it with the previous one.

```{r}
sce.t <- TasicBrainData()

# Only using the common genes in both datasets.
common <- intersect(rownames(sce.z), rownames(sce.t))
combined <- combineCols(sce.t[common,], sce.z[common,])
block <- rep(c("tasic", "zeisel"), c(ncol(sce.t), ncol(sce.z)))
```

We set `block=` in each step to account for the batch structure.
This ensures that the various calculations are not affected by inter-block differences.
It also uses MNN correction to batch effects in the low-dimensional space prior to further analyses.

```{r}
# No mitochondrial genes in the combined set, so we'll just skip it.
blocked_res <- analyze.se(combined, block=block, num.threads=2)
```

Hopefully, the correction is able to get rid of the batch effect and merge the datasets together. 

```{r}
plotReducedDim(blocked_res$x, "TSNE", colour_by="block")
```

We can also check the distribution of clusters across batches.
The presence of batch-specific clusters might indicate that the batch effect was not fully removed;
or they might represent cell types that are unique to one of the studies, who knows.

```{r}
table(blocked_res$x$graph.cluster, blocked_res$x$block)
```

Finally, some markers for another cluster:

```{r}
previewMarkers(blocked_res$markers$rna[[1]])
```

We can also compute pseudo-bulk profiles for each cluster-dataset combination, e.g., for differential expression analyses.

```{r}
aggregates <- aggregateAcrossCells.se(
    blocked_res$x,
    list(cluster=blocked_res$x$graph.cluster, dataset=blocked_res$x$block)
)
aggregates
```

# Combining multiple modalities

Let's fetch a single-cell dataset with both RNA-seq and CITE-seq data.
To keep the run-time short, we'll only consider the first 5000 cells.

```{r}
sce.s <- StoeckiusHashingData(mode=c("human", "adt1", "adt2"))
sce.s <- sce.s[,1:5000]

# Combining the various ADT-related alternative experiments into a single object.
altExp(sce.s, "all.adts") <- rbind(altExp(sce.s, "adt1"), altExp(sce.s, "adt2"))
altExp(sce.s, "adt1") <- altExp(sce.s, "adt2") <- NULL

sce.s
```

We specify `adt.altexp=` to indicate that one of the alternative experiments contains ADT data.
This instructs the analysis to use data from both modalities during clustering and visualization.

```{r}
is.mito <- grepl("^MT-", rownames(sce.s))
is.igg <- grepl("^IgG", rownames(altExp(sce.s, 'all.adts')))
multi_res <- analyze.se(
    sce.s,
    adt.altexp="all.adts",
    rna.qc.subsets=list(mito=is.mito),
    adt.qc.subsets=list(igg=is.igg),
    num.threads=2
)
```

Here's an obligatory plot:

```{r}
plotReducedDim(multi_res$x, "UMAP", colour_by="graph.cluster")
```

We can inspect the top markers among the ADTs for a cluster:

```{r}
# By default, markers are sorted by the mean Cohen's d.
# We just replace the column name when printing the dataframe here.
previewMarkers(multi_res$markers$adt[[5]], c(effect="cohens.d.mean"))
```

... along with the RNA markers, as before.

```{r}
# Using the delta-detected to focus on silent-to-expressed genes.
previewMarkers(multi_res$markers$rna[[5]], order.by="delta.detected.mean")
```

# Other useful functions

The `runAllNeighborSteps()` fucntion will run `runUmap()`, `runTsne()`, `buildSnnGraph()` and `clusterGraph()` in a single call.
This runs the UMAP/t-SNE iterations and the clustering in parallel to maximize use of multiple threads.

The `scoreGeneSet()` function will compute a gene set score based on the input expression matrix.
This can be used to summarize the activity of pathways into a single per-cell score for visualization.

The `subsampleByNeighbors()` function will deterministically select a representative subset of cells based on their local neighborhood.
This can be used to reduce the compute time of the various steps downstream of the PCA.

For CRISPR data, quality control can be performed using `computeCrisprQcMetrics()`, `suggestCrisprQcThresholds()` and `filterCrisprQcMetrics()`.
To normalize, we use size factors defined by centering the total sum of guide counts for each cell.

# Session information {-}

```{r}
sessionInfo()
```
