# Smoothclust Tutorial

# Introduction

`smoothclust` is a method for identification of spatial domains and spatially-aware clustering in spatial transcriptomics data. The method generates spatial domains with smooth boundaries by smoothing gene expression profiles across neighboring spatial locations, followed by unsupervised clustering. Spatial domains consisting of consistent mixtures of cell types may then be further investigated by applying cell type compositional analyses or differential analyses.

# Installation

The `smoothclust` package can be installed from Bioconductor as follows (using R version 4.4 onwards). This is the recommended installation for most users. Additional details are shown on the Bioconductor package page.

    install.packages("BiocManager")
    BiocManager::install("smoothclust")

The latest development version of the package can also be installed from the devel version of Bioconductor or from GitHub.

# Input data format

Input data can be provided either as a SpatialExperiment object within the Bioconductor framework, or as numeric matrices of expression values and spatial coordinates. See help file (`?smoothclust`) for details.

In the example workflow below, we assume the input is in `SpatialExperiment` format.

# Tutorial

The example workflow in this section demonstrates how to run `smoothclust` to generate spatial domains with smooth boundaries for a dataset from the 10x Genomics Visium platform.

    library(smoothclust)
    library(STexampleData)
    library(scuttle)
    library(scran)
    library(scater)
    library(ggspavis)

Load dataset from `STexampleData` package:

    # load data
    spe <- Visium_humanDLPFC()

    ## see ?STexampleData and browseVignettes('STexampleData') for documentation

    ## downloading 1 resources

    ## retrieving 1 resource

    ## loading from cache

    # keep spots over tissue
    spe <- spe[, colData(spe)$in_tissue == 1]

    dim(spe)

    ## [1] 33538  3639

    assayNames(spe)

    ## [1] "counts"

Run `smoothclust` using default parameter settings, which have been selected to be appropriate for Visium data from human tissue.

The method for smoothing can be specified by providing the `method` argument, with available options `uniform`, `kernel`, and `knn`. Additional arguments can be used to set parameter values, including `bandwidth`, `k,` and `truncate`, depending on the choice of method. For more details, see the function documentation (`?smoothclust`) or the paper (in preparation).

    # run smoothclust using default parameters
    spe <- smoothclust(spe)

The smoothed expression counts are stored in a new assay named `counts_smooth`:

    # check output object
    assayNames(spe)

    ## [1] "counts"        "counts_smooth"

Calculate log-transformed normalized counts (logcounts) on the smoothed expression counts. Here, the argument `assay.type = "counts_smooth"` specifies that we want to calculate logcounts using the smoothed counts from `smoothclust`.

    # calculate logcounts
    spe <- logNormCounts(spe, assay.type = "counts_smooth")

    ## Warning in .library_size_factors(assay(x, assay.type), ...): 'librarySizeFactors' is deprecated.
    ## Use 'scrapper::centerSizeFactors' instead.
    ## See help("Deprecated")

    ## Warning in .local(x, ...): 'normalizeCounts' is deprecated.
    ## Use 'scrapper::normalizeCounts' instead.
    ## See help("Deprecated")

    assayNames(spe)

    ## [1] "counts"        "counts_smooth" "logcounts"

Run clustering. We use a standard clustering workflow from single-cell data, consisting of k-means clustering on the top principal components (PCs) calculated on the set of top highly variable genes (HVGs) with logcounts as the input.

We use a relatively small number of clusters for demonstration purposes in this example:

    # preprocessing steps for clustering

    # remove mitochondrial genes
    is_mito <- grepl("(^mt-)", rowData(spe)$gene_name, ignore.case = TRUE)
    table(is_mito)

    ## is_mito
    ## FALSE  TRUE
    ## 33525    13

    spe <- spe[!is_mito, ]
    dim(spe)

    ## [1] 33525  3639

    # select top highly variable genes (HVGs)
    dec <- modelGeneVar(spe)

    ## Warning in .model_gene_var(x = assay(x, i = assay.type), ...): 'modelGeneVar' is deprecated.
    ## Use 'scrapper::modelGeneVariances' instead.
    ## See help("Deprecated")

    ## Warning in fitTrendVar(fm, fv, ...): 'fitTrendVar' is deprecated.
    ## Use 'scrapper::fitVarianceTrend' instead.
    ## See help("Deprecated")

    ## Warning in combineBlocks(collected, method = method, equiweight = equiweight, : 'combineBlocks' is deprecated.
    ## See help("Deprecated")

    top_hvgs <- getTopHVGs(dec, prop = 0.1)

    ## Warning in getTopHVGs(dec, prop = 0.1): 'getTopHVGs' is deprecated.
    ## Use 'scrapper::chooseHighlyVariableGenes' instead.
    ## See help("Deprecated")

    length(top_hvgs)

    ## [1] 986

    spe <- spe[top_hvgs, ]
    dim(spe)

    ## [1]  986 3639

    # dimensionality reduction

    # compute PCA on top HVGs
    set.seed(123)
    spe <- runPCA(spe)

    # run k-means clustering
    set.seed(123)
    k <- 5
    clust <- kmeans(reducedDim(spe, "PCA"), centers = k)$cluster
    table(clust)

    ## clust
    ##    1    2    3    4    5
    ##  967 1574  492  272  334

    colLabels(spe) <- factor(clust)

Plot clusters / spatial domains generated by the smoothclust workflow:

    # color palettes
    pal8 <- "libd_layer_colors"
    pal36 <- unname(palette.colors(36, "Polychrome 36"))

    # plot clusters / spatial domains
    plotCoords(spe, annotate = "label", pal = pal8)

Plot manually annotated reference labels, which can be used to evaluate the performance of the clustering in this dataset:

    # plot reference labels
    plotCoords(spe, annotate = "ground_truth", pal = pal8)

# Session information

    sessionInfo()

    ## R version 4.6.1 (2026-06-24)
    ## Platform: x86_64-pc-linux-gnu
    ## Running under: Ubuntu 26.04 LTS
    ##
    ## Matrix products: default
    ## BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
    ## LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.32.so;  LAPACK version 3.12.0
    ##
    ## locale:
    ##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
    ##  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
    ##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
    ##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
    ##  [9] LC_ADDRESS=C               LC_TELEPHONE=C
    ## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
    ##
    ## time zone: Etc/UTC
    ## tzcode source: system (glibc)
    ##
    ## attached base packages:
    ## [1] stats4    stats     graphics  grDevices utils     datasets  methods
    ## [8] base
    ##
    ## other attached packages:
    ##  [1] ggspavis_1.19.0             scater_1.41.2
    ##  [3] ggplot2_4.0.3               scran_1.41.1
    ##  [5] scuttle_1.23.1              STexampleData_1.21.0
    ##  [7] SpatialExperiment_1.23.0    SingleCellExperiment_1.35.1
    ##  [9] SummarizedExperiment_1.43.0 Biobase_2.73.1
    ## [11] GenomicRanges_1.65.1        Seqinfo_1.3.0
    ## [13] IRanges_2.47.2              S4Vectors_0.51.5
    ## [15] MatrixGenerics_1.25.0       matrixStats_1.5.0
    ## [17] ExperimentHub_3.3.1         AnnotationHub_4.3.2
    ## [19] BiocFileCache_3.3.0         dbplyr_2.6.0
    ## [21] BiocGenerics_0.59.10        generics_0.1.4
    ## [23] smoothclust_1.9.3           BiocStyle_2.41.0
    ##
    ## loaded via a namespace (and not attached):
    ##  [1] DBI_1.3.0            gridExtra_2.3.1      httr2_1.3.0
    ##  [4] rlang_1.3.0          magrittr_2.0.5       otel_0.2.0
    ##  [7] compiler_4.6.1       RSQLite_3.53.3       png_0.1-9
    ## [10] vctrs_0.7.3          pkgconfig_2.0.3      crayon_1.5.3
    ## [13] fastmap_1.2.0        magick_2.9.1         XVector_0.53.0
    ## [16] labeling_0.4.3       rmarkdown_2.31       ggbeeswarm_0.7.3
    ## [19] purrr_1.2.2          bit_4.6.0            xfun_0.60
    ## [22] bluster_1.23.0       cachem_1.1.0         beachmat_2.29.0
    ## [25] jsonlite_2.0.0       blob_1.3.0           DelayedArray_0.39.3
    ## [28] BiocParallel_1.47.0  irlba_2.3.7          parallel_4.6.1
    ## [31] cluster_2.1.8.2      R6_2.6.1             bslib_0.11.0
    ## [34] RColorBrewer_1.1-3   limma_3.69.2         jquerylib_0.1.4
    ## [37] Rcpp_1.1.2           knitr_1.51           Matrix_1.7-5
    ## [40] igraph_2.3.3         tidyselect_1.2.1     viridis_0.6.5
    ## [43] abind_1.4-8          yaml_2.3.12          codetools_0.2-20
    ## [46] curl_7.1.0           lattice_0.22-9       tibble_3.3.1
    ## [49] withr_3.0.3          KEGGREST_1.53.5      S7_0.2.2
    ## [52] evaluate_1.0.5       Biostrings_2.81.5    pillar_1.11.1
    ## [55] BiocManager_1.30.27  filelock_1.0.3       BiocVersion_3.24.0
    ## [58] scales_1.4.0         glue_1.8.1           metapod_1.21.0
    ## [61] maketools_1.3.2      tools_4.6.1          BiocNeighbors_2.7.2
    ## [64] sys_3.4.3            ScaledMatrix_1.21.0  locfit_1.5-9.12
    ## [67] ggside_0.4.1         buildtools_1.0.0     grid_4.6.1
    ## [70] AnnotationDbi_1.75.0 edgeR_4.11.4         beeswarm_0.4.0
    ## [73] BiocSingular_1.29.0  vipor_0.4.7          cli_3.6.6
    ## [76] rsvd_1.0.5           rappdirs_0.3.4       viridisLite_0.4.3
    ## [79] S4Arrays_1.13.0      dplyr_1.2.1          gtable_0.3.6
    ## [82] sass_0.4.10          digest_0.6.39        ggrepel_0.9.8
    ## [85] SparseArray_1.13.2   dqrng_0.4.1          rjson_0.2.23
    ## [88] farver_2.1.2         memoise_2.0.1        htmltools_0.5.9
    ## [91] lifecycle_1.0.5      httr_1.4.8           statmod_1.5.2
    ## [94] bit64_4.8.2