Confidentiality and the threat model

Depth controls: hiding design structure

Design columns are byte-identical by default, which preserves the experimental layout exactly. A publicly registered trial layout is a fingerprint. To keep the structure but hide the site or block labels, set a design column’s action to alias:

df <- data.frame(
  site = factor(rep(c("Roseworthy", "Minnipa", "Turretfield"), each = 6L)),
  rep = rep(seq_len(3L), 6L),
  yield = rnorm(18L)
)
roles <- propose_roles(df, detect = FALSE)
roles <- set_role(roles, "site", role = "design", action = "alias")
s <- synthetic(mask(df, roles, mode = "collaborate", seed = 1L))
site_names_gone <- !any(as.character(s$site) %in% as.character(df$site))
site_names_gone
#> [1] TRUE
knitr::kable(
  as.data.frame(table(s$site), stringsAsFactors = FALSE),
  col.names = c("aliased site", "plots"),
  caption = "The three-site structure survives even though the labels do not."
)

Aliasing a design column breaks byte-identity, so it is never a default. Names can be hidden the same way: mask(..., alias_names = TRUE) replaces every retained column name with an opaque code that the recipe inverts on the round-trip, and a character vector hides only the names given.

What an alias does and does not hide. The alias vocabulary is fixed and sorted (trt_001, trt_002, …), but which level receives which alias is drawn from a random permutation, so for a six-level column there are 720 equally likely maps. This matters because the vocabularies in field research are routinely public: a variety roster, an N-rate ladder, a site list. The same draw governs the join keys mask_set() shares across tables. A scrambled factor keeps the original level order, so levels() of a clone does not reveal the map.

Clones made with earlier versions can reveal it. In masque 0.13.0 and earlier, a scrambled treatment listed its levels in the scrambled order. In 0.12.0 and 0.13.0, a join key aliased by mask_set() listed its aliases in the original level order. Make such a clone again with 0.14.0.

Two limits go with that, and they are the custodian’s to manage:

• The seed is confidential material. The permutation comes from the seeded stream, so mask(seed = 1L) reproduces the same map, exactly as it reproduces the same synthetic values. Publishing the seed beside the synthetic re-opens the inversion for anyone who can also reproduce the draw. Keep the seed with the recipe, or pass seed = NULL where reproducibility is not required and take a fresh map each call.
• Frequencies are not hidden. Aliasing is a bijection on labels: the count of each level is unchanged by design, because the design footprint is what makes the clone useful. A vocabulary whose level counts are known and distinct is still matchable one level at a time. audit_mask() reports those counts so the exposure is visible rather than implicit.

An alias therefore raises the cost of re-identification. It cannot be relied on against a determined attacker, and none of it is a differential-privacy guarantee.

The conditional clone: preserving the treatment effect

The default numeric synthesis re-simulates every scrambled numeric column from one Gaussian copula with empirical-quantile marginals, fitted on the whole table. That preserves each column’s marginal distribution and the global covariance, which is enough to develop most pipelines. It does not keep the relationship between treatment and outcome. The synthetic outcomes are drawn from the pooled distribution and the treatment labels are relabelled independently, so a causal model fitted on the clone sees no association between an arm and its response. A pipeline whose whole purpose is to estimate a treatment effect would silently give the wrong answer on such a clone.

mask()’s conditional = TRUE argument fixes this. It fits and samples the copula within each treatment-by-design stratum rather than pooling, so a row’s synthetic outcome inherits the location of the treatment that row carries. The treatment effect a causal model estimates is kept, within sampling tolerance.

The contrast is easiest to see on a two-arm trial with a known effect:

set.seed(42)
n <- 600L
arm <- factor(rep(c("ctrl", "treat"), each = n / 2L))
# A real +5-unit effect of the treated arm on yield.
yield <- 10 + 5 * (arm == "treat") + rnorm(n, sd = 2)
trial <- data.frame(genotype = arm, yield = yield)

roles <- propose_roles(trial, detect = FALSE)
roles <- set_role(roles, "genotype", role = "treatment", action = "scramble")
roles <- set_role(roles, "yield", role = "outcome")

true_effect <- coef(lm(yield ~ genotype, trial))[["genotypetreat"]]
true_effect
#> [1] 4.988538

Cloning both ways from the same seed, then re-estimating the effect on each clone:

marg <- mask(trial, roles, mode = "local", seed = 1L, conditional = FALSE)
cond <- mask(trial, roles, mode = "local", seed = 1L, conditional = TRUE)

effect_of <- function(m) {
  coef(lm(yield ~ genotype, synthetic(m)))[["genotypetreat"]]
}
effect_marg <- effect_of(marg)
effect_cond <- effect_of(cond)

knitr::kable(
  data.frame(
    clone = c("marginal (default)", "conditional"),
    estimated_effect = c(effect_marg, effect_cond),
    true_effect = true_effect
  ),
  digits = 3,
  caption = paste0(
    "The treated-arm effect, re-estimated on each clone against the ",
    "true effect fitted on the original trial."
  )
)

The marginal clone collapses the effect toward zero. The conditional clone recovers it. Both clones still match the pooled marginal of the outcome:

knitr::kable(
  data.frame(
    source = c("original", "marginal clone", "conditional clone"),
    mean_yield = c(
      mean(trial$yield), mean(synthetic(marg)$yield),
      mean(synthetic(cond)$yield)
    ),
    sd_yield = c(
      sd(trial$yield), sd(synthetic(marg)$yield),
      sd(synthetic(cond)$yield)
    )
  ),
  digits = 3,
  caption = paste0(
    "Both clones preserve the pooled mean and SD of yield; only the ",
    "conditional clone preserves the arm-to-arm difference."
  )
)

Figure: the effect on each clone

effect_df <- data.frame(
  clone = factor(
    c("marginal (default)", "conditional"),
    levels = c("marginal (default)", "conditional")
  ),
  estimated_effect = c(effect_marg, effect_cond)
)
ggplot2::ggplot(
  effect_df, ggplot2::aes(x = clone, y = estimated_effect, fill = clone)
) +
  ggplot2::geom_col(width = 0.6) +
  ggplot2::geom_hline(yintercept = true_effect, linetype = "dashed") +
  ggplot2::scale_fill_viridis_d(guide = "none") +
  ggplot2::labs(
    x = "clone", y = "estimated treatment effect (yield units)",
    title = "Marginal versus conditional cloning of a known treatment effect"
  ) +
  ggplot2::theme_minimal()

Estimated treatment effect (yield units) on the marginal versus conditional clone, against the true effect fitted on the original trial (dashed line). The marginal clone collapses the effect toward zero; the conditional clone recovers it.

The conditioning columns (the treatment plus any retained design columns) are recorded on the recipe, so the choice is auditable:

recipe(cond)@conditional
#> [1] TRUE
recipe(cond)@conditioning_cols
#> [1] "genotype"
recipe(cond)@conditioning_used
#> [1] "genotype"
recipe(cond)@fallback_frac
#> [1] 0

conditioning_cols is what the call asked for; conditioning_used is the stratum the clone actually got, and fallback_frac is the share of rows that did not get a stratum at all. They differ because a conditional clone needs enough rows per stratum to fit a stratum-local copula, and the finest stratum available is rarely the one that has them. On a replicated factorial of six N rates by three varieties by four replicates, treatment crossed with every retained design column is seventy-two cells of one row. mask() therefore walks a coarsening ladder: it drops design columns until the cells hold at least five rows. Treatment columns are never dropped. Rows still in cells that are too small at the last rung are pooled into one fallback stratum.

The default ladder (ladder = "hierarchy") drops plot coordinates first, then blocking columns, then environment columns, so a site or a county is never given up before the replicates inside it. It carries the main effect of every dropped column into the clone as an additive shift. The county, replicate and block means of the original therefore survive even when the copula is fitted within the treatment alone; the threat table above lists what that discloses. The recipe lists the shifted terms as conditioning_shifted. ladder = "levels" reproduces clones made with masque 0.11.1 to 0.12.0: it drops the column with the most levels first and keeps nothing of it.

Any coarsening, and any residual pooling, raises a classed masque_conditional_degraded warning naming the columns given up and the pooled share, and both facts are written onto the recipe. A pipeline can catch the class if it should stop when its clone is less conditional than it asked for:

withCallingHandlers(
  mask(trial, roles, seed = 1L, conditional = TRUE),
  masque_conditional_degraded = function(w) stop(conditionMessage(w))
)

Conditional cloning composes with both modes and with mask_set() (each table is stratified by its own treatment and design columns). With no treatment or design column to condition on at all, the path degrades to the global copula and raises the same classed warning. It is worth reaching for whenever the development pipeline estimates an effect, not just a distribution.

What the copula carries: monotone, not non-monotone, association

Numeric columns that are kept and re-simulated together share one Gaussian copula, fitted on the normal scores of their ranks. A Gaussian copula holds a single correlation per pair, so it reproduces a monotone association (linear, or any order-preserving curve) but not a dependence that a correlation cannot express. A non-monotone relationship, such as a U-shaped dependence of an outcome on a covariate, reads as near-zero rank correlation and is reproduced as near-independence.

set.seed(7)
n <- 2000L
x_lin <- rnorm(n)
y_lin <- 2 * x_lin + rnorm(n, sd = 0.5)     # monotone (linear)
x_u <- runif(n, -3, 3)
y_u <- x_u^2 + rnorm(n, sd = 0.5)           # non-monotone (U-shaped)
d <- data.frame(x_lin, y_lin, x_u, y_u)

s <- synthetic(mask(d, propose_roles(d, detect = FALSE), seed = 1L))

# R-squared of a quadratic fit captures association of any curvature.
fit_r2 <- function(x, y) summary(lm(y ~ poly(x, 2)))$r.squared
knitr::kable(
  data.frame(
    pair = c("monotone (y = 2x)", "non-monotone (y = x^2)"),
    original = c(fit_r2(x_lin, y_lin), fit_r2(x_u, y_u)),
    clone = c(fit_r2(s$x_lin, s$y_lin), fit_r2(s$x_u, s$y_u))
  ),
  digits = 3,
  caption = paste0(
    "R-squared of a quadratic fit, original versus clone, for a ",
    "monotone and a non-monotone pair."
  )
)

The monotone pair keeps its association almost exactly. The non-monotone pair loses it entirely, so on the clone the covariate carries no information about the outcome. conditional = TRUE does not repair this: it preserves the outcome’s location within each treatment-by-design stratum, not the curvature of the outcome’s dependence on a continuous covariate. A pipeline whose modelling step is non-linear (a smoothing spline, a generalised additive model, a tree, an interaction term) sees on the synthetic only the monotone part of any relationship in the original. A good fit there is not evidence that the step behaves correctly on the real data. Validate such a step by running it on the original through the recipe.

The leakage audit, and a refused write

audit_mask() inspects the synthetic against the original and grades the leakage of each column. In collaborate mode it runs automatically and warns at mask() time. The fixture below is built to trip it: a PII-suspected column the user retains against the default, and a categorical covariate with a frequency-one level.

df <- data.frame(
  rep = rep(seq_len(3L), each = 20L),
  variety = factor(rep(paste0("V", seq_len(6L)), 10L)),
  contact_email = factor(rep(c("a@x", "b@y"), 30L)),
  rare_treatment = factor(c(
    "only_one",
    sample(c("alpha", "beta", "gamma"), 59L, replace = TRUE)
  )),
  yield = rnorm(60L, 5, 1),
  stringsAsFactors = FALSE
)

roles <- propose_roles(df, mode = "collaborate")
knitr::kable(
  roles[, c("col", "role", "action", "pii_suspected")],
  caption = paste0(
    "The proposed plan: `contact_email` is auto-flagged ",
    "`pii_suspected` and dropped."
  )
)

Suppose the custodian overrides the flag, keeps contact_email, and makes the rare column a covariate:

roles <- set_role(roles, "yield", role = "outcome")
roles <- set_role(roles, "contact_email", role = "covariate", action = "keep")
roles <- set_role(roles, "rare_treatment", role = "covariate")
m <- mask(df, roles, mode = "collaborate", seed = 1L)
#> Warning: audit_mask() flagged HIGH leakage on columns: contact_email, rare_treatment
knitr::kable(
  audit_mask(m)[, c("col", "leakage_class", "notes")],
  caption = paste0(
    "The mask-time audit: `contact_email` and `rare_treatment` both ",
    "flag HIGH."
  )
)
#> 
#> ── masque audit (mode = collaborate) ───────────────────────────────────────────────────────────────
#> • 2 HIGH, 0 medium, 3 low across 5 columns
#> • Rows with a globally unique NA pattern: 0.0%
#> 
#> ── HIGH (2) ──
#> 
#> ✖   covariate contact_email                     PII-pattern column name; kept as-is - visible to collaborators
#> ✖   covariate rare_treatment                    levels aliased; rare level (freq = 1)
#> 
#> ── LOW (3) ──
#> 
#> ℹ   design    rep                               exact-match 100.0%
#> ℹ   treatment variety                           levels aliased
#> ℹ   outcome   yield                             ok

contact_email (real values kept across the trust boundary) and rare_treatment (a frequency-one level) are flagged. mask() raised a classed masque_high_leakage warning above and recorded the findings in recipe@warnings; the guided masque() flow never silences it. While a HIGH finding stands, the package will not write the synthetic to disk:

out_dir <- tempfile()
masque(df, roles = roles, out = out_dir, mode = "collaborate", seed = 1L,
       quiet = TRUE)
#> Warning: audit_mask() flagged HIGH leakage on columns: contact_email, rare_treatment
#> Error in `.gate_release()`:
#> ! Write blocked: the audit flagged HIGH leakage on 2 columns.
#> ✖ Flagged: contact_email, rare_treatment.
#> ℹ Re-role, alias, or drop each flagged column, then mask again.
#> ℹ Or pass `allow_high = TRUE` to write anyway after your own review - the override is recorded.
dir.exists(out_dir)
#> [1] FALSE

Nothing is written: the refusal fires before the writer touches disk, the two flagged columns are named in the message, and the remedy is spelled out (re-role, alias, or drop the column, then mask again). A custodian who has reviewed the findings and still wants to proceed can pass allow_high = TRUE; the override is itself raised as a classed masque_high_override warning and recorded in the recipe’s warnings, so the exception stays auditable rather than silent.

Beyond that gate the release decision stays with the custodian. Whether a synthetic table is appropriate for a given collaborator, environment, or jurisdiction depends on context the package cannot see. masque informs that decision. It does not make it.

Multi-table sets

When several tables share a key, mask_set() aliases that key identically across all of them so the synthetic tables still join. A linked key is the join surface, aliased consistently rather than permuted (permuting a key would break the join regardless of masking).

set_dir <- system.file("extdata", "met_set", package = "masque")
ms <- mask_set(set_dir, mode = "collaborate", seed = 1L, quiet = TRUE)
#> Warning: Numeric environment column year remains "keep" in collaborate mode.
#> ℹ This preserves environment structure but may disclose year or other numeric labels; review before
#>   release.
ag <- synthetic(ms)$agronomy
qa <- synthetic(ms)$quality
join_survives <- setequal(unique(ag$gen), unique(qa$gen))
join_survives
#> [1] TRUE

The recipe bundle is private exactly as a single recipe is; write_set() never writes it.

Geographic coordinates

Latitude and longitude are treated as sensitive: propose_roles() flags a column whose name looks like a coordinate (gps, lat/latitude, lon/longitude) as pii_suspected and proposes drop. Dropping is the safest choice when the synthetic does not need locations.

A coordinate is masked unless told otherwise. If a column whose name says coordinate would be written through with action = "keep", the call stops. There are three ways to say what happens to it:

1. Declare the pair to coords, which coarsens it.
2. Give the column drop or scramble, which masks it.
3. Pass allow_unmasked_coords = TRUE after deciding the position is not sensitive. The recipe records this, so the decision is visible later.

A column that looks like a coordinate only by the shape of its values raises a warning instead of stopping, because that detection is a guess and a false positive should not block a legitimate mask.

When the synthetic does need plausible coordinates, to exercise a spatial pipeline say, a plain scramble is the wrong tool: the copula re-simulates each axis and smears a latitude/longitude pair into a continuous cloud that can land in the sea. masque offers two purpose-built alternatives.

jitter_coordinates() coarsens coordinates in place by a geographic-masking jitter. The default donut scheme displaces every point by a random distance drawn uniformly by area from an annulus, in a random direction, and re-draws until the point falls on land, so a coastal site is never pushed offshore. The longitude step is corrected by cos(latitude) so the ground distance matches the requested kilometres at any latitude, and the NA pattern and the latitude/longitude pairing are preserved.

sites <- data.frame(
  site = c("A", "B", "C"),
  lat  = c(-34.9, -35.2, -33.6),
  lon  = c(138.6, 142.0, 148.2)
)
knitr::kable(
  jitter_coordinates(sites, "lat", "lon", min_km = 5, max_km = 20, seed = 1L),
  digits = 4,
  caption = paste0(
    "Three sites, each displaced 5-20 km in a random direction and ",
    "re-drawn until on land."
  )
)

A coordinate belongs to a site, not to a row. One displacement is drawn per site and broadcast to every row of that site, so a table holding many rows per physical place (plots within a trial, samples within a paddock) comes back with one masked coordinate per place, exactly as the source table has one true coordinate per place. By default a site is a set of rows sharing an identical input coordinate, which needs no user action and cannot alter genuinely point-level data. Where a site’s recorded coordinates differ slightly between rows, naming the site columns consolidates the group to its centroid.

plots <- data.frame(
  trial = rep(c("T1", "T2"), each = 4L),
  lat   = rep(c(-34.9, -35.2), each = 4L),
  lon   = rep(c(138.6, 142.0), each = 4L)
)
masked <- jitter_coordinates(plots, "lat", "lon", by = "trial", seed = 1L)
one_coord_per_trial <- nrow(unique(masked[c("lat", "lon")]))
one_coord_per_trial
#> [1] 2

Grouping is a confidentiality control, not only a fidelity one. Donut displacement is isotropic, so the mean of many independent draws around one true site converges on that site. Displacing a 360-row trial row by row leaves its centroid a median 0.58 km from the truth, against the 5 km floor the donut was asked for. One draw per site removes that estimator and leaves the full displacement in place. The unit has to be chosen carefully: it is the finest grouping denoting one physical place at one time, so for a multi-year trial series it is location by year rather than location alone, because trials at one named location in different seasons legitimately sit in different paddocks.

A site that cannot be placed on land within max_tries re-draws is set to NA on both axes, with a warning. masque does not emit a true coordinate inside a table the caller believes is masked.

Declaring the pair to mask() applies the same coarsening as part of masking, so the coordinates never pass through the copula:

m <- mask(df, roles, mode = "collaborate",
          coords = list(list(lat = "GPS_S", lon = "GPS_E", by = "TRIAL")),
          seed = 1L)

The recipe records the grouping each declared pair was masked under, so a clone carries the account of its own coordinate structure. A coordinate declared to coords is retained in the synthetic, so audit_mask() reports it as medium, noting that the column was coarsened in place. A coordinate kept with its real values would be HIGH.

synthesise_geospatial() is the alternative when it is preferable to re-anchor points around fake centroids supplied for each region.

How far to displace. The right magnitude is not a constant. It is calibrated to the density of the entities being protected, so that the masked point is spatially k-anonymous: roughly, at least k comparable entities lie closer to the masked point than the true one (Hampton et al., 2010). Individual-level urban health data is typically masked with a standard deviation of about one kilometre, because cities are dense. Fields and farms are far sparser, so comparable protection needs a much larger displacement; a donut of roughly 5 to 20 km (the default) moves a point across several properties while keeping it in the same agroclimatic region (Zandbergen, 2014). A formal guarantee needs the radii calibrated to the local field density rather than the default.