Every table masque returns is now a data.table: synthetic(),
propose_roles(), set_role(), role_options(), audit_mask(),
apply_recipe(), unmask(), read_set(), the data and report tables
of clean_table() and conform_table(), and the recommended_roles and
per_env tables of detect_design(). tibble is no longer a dependency.
Code that selects columns with x[c("a", "b")] should use
x[, c("a", "b")], or convert with as.data.frame() first.
synthetic(), audit_mask(), set_role(), apply_recipe(),
clean_table() and conform_table() return their own copy, so editing
the result by reference with := never changes the input data or the
masque object.
The package has a logo.
The README states no hard-coded version, carries status badges, and has a Related packages section on synthpop and sdcMicro.
Getting started edits the role table with data.table syntax.
test-coverage workflow (covr, uploaded to Codecov) and lint
workflow (lintr with the package's .lintr; it reports and does not
fail).Confidentiality states what a recipe holds on its own (the real labels, the original column names and the seed) and adds a row for conditional clones: they reproduce the outcome mean of each treatment, block and environment level, so a level with only a few plots shows close to its real mean.
The README, API stability and the other vignettes are reworded more plainly, and the pkgdown site lists the articles in reading order.
levels() of the clone showed the whole map. In 0.12.0 and 0.13.0, a
join key aliased by mask_set() listed its aliases in the original
level order, which undid their random assignment. If you shared such a
clone, make it again with 0.14.0 and share the new one in its place. The
same seed gives the same values. Only the level order changes.apply_recipe() returns an aliased factor with the clone's levels.
It kept the original level order, so a model fitted on the translated
original had a different reference level from the same model fitted on
the clone, and coefficients compared by position did not line up. On
agridat::besag.met in collaborate mode the genotype coefficients of the
two fits correlated at 0.05 by position and 0.92 by name. Predictions
were not affected.Messages say column or columns, site or sites, where they said
column(s).
The vignettes and help pages are reworded more plainly. Recipe anatomy and the round-trip said its coordinate example was jittered as one site; each plot there is its own site.
The conditioning ladder follows the design hierarchy. The ladder
that coarsens the stratum of a conditional clone used to drop design
columns by level count, so on a multi-environment trial it dropped the
environment before the replicate inside it and the clone lost the
environment effect: on agridat::besag.met the county F statistic went
from 987 on the original to about 1 on the clone, and the genotype
ranking went with it. mask(), mask_set() and masque() take a
ladder argument. The new default, "hierarchy", drops plot coordinates
first, then blocking columns, then environment columns, ordering within a
tier by the level-adjusted share of variance a column explains, and
carries the main effect of every dropped column, and of each pair of
dropped blocking or environment columns, into the clone as an additive
shift estimated beside the stratum that was kept. Rows the stratum floor
pools into the fallback keep the stratum columns' main effects the same
way, so a three-replicate variety trial keeps its genotype means. A level
with a single row is pooled with the other singletons before the fit, so
no row's own outcome is ever carried; a term that is noise is shrunk to
nothing and the residual spread is rescaled for the degrees of freedom
the fit removed. Over twenty seeds the county F on besag.met is 952
against 987 and the nested county:rep F 28 against 33; on
agridat::yates.oats the block F is 11.3 against 13.2 where it was 0.8.
The ladder of 0.11.1 to 0.12.0 is available as ladder = "levels", and
a recipe records which one made it.
The recipe gains ladder, conditioning_dropped (in drop order) and
conditioning_shifted; the masque_conditional_degraded warning and the
recipe print name the shifts. The new vignette What a conditional clone
keeps of the design runs eight agridat designs through one stated pass
rule under both ladders.
print() on the records from clean_table() and conform_table()
dispatches from an installed package. The S7 method assignments for
print(), format() and plot() left a copy of each generic in the
namespace, so the two S3 print methods registered on that copy and an
installed masque printed the raw list, attr(,"class") and all. The
assignments now run inside local(). A test installs the package into a
temporary library in a subprocess and prints both records.
plot_design_summary(engine = "ggplot2") returns the plot. It
returned the summary invisibly after printing, so ggplot2::ggsave()
failed and the Getting started vignette's description was wrong. The
ggplot2 engine now returns the ggplot object for every design class;
the split-plot and CRD panels, which had only a base-graphics branch,
gained one. The base engine still draws and returns the summary
invisibly.
detect_design() reads a blocked multi-environment trial as the
incomplete-block design it is. The lattice rule required every block to
hold the same number of treatments and every treatment the same number of
blocks, so one filler plot in agridat::besag.met failed both tests and
each county was reported as CRD with rep and block never mentioned.
Regularity is now the share of blocks at the modal size and of treatments
at the modal replication, floored at 0.9 each; besag.met reads as an
alpha-lattice on rep:block in every county.
conform_table() names the reason a character column is left as text: a
few values that do not parse as numbers, labels that are all distinct, or
a vocabulary wider than max_levels. The last was reported for all three.
mask() and mask_set() check the shape of roles before they read its
mode provenance, so something that is not a roles table is refused for
that reason, with the typed masque_bad_roles_refusal class, and no
warning about a missing mode attribute precedes the refusal.
mask() refuses a missing roles with masque_missing_roles_refusal
instead of R's bare missing-argument error, accepts the record from
conform_table() and masks its data, and takes quiet = TRUE to
silence only the masque_mode_downgrade warning for a plan that is
deliberately shared across both modes.
synthesise_geospatial() refuses a wrong-shaped anchor_centroids (a
data frame, an unnamed list, an element that is not a c(lat, lon)
pair, or a list naming none of the anchor levels) with
masque_bad_anchor_centroids_refusal, instead of warning once per level
and returning every coordinate NA.
agridat exemplars (CRD, RCBD,
Latin square, split-plot, alpha lattice, augmented, blocked MET, repeated
measures): allocation byte-identical, NA mask identical, blocking and
environment F at half the original or better where the original detects
the term, treatment level means correlated at 0.7 or better and within a
factor of two in spread. Three seeds on CRAN, twenty off it.conform_table() takes a cleaned table the last step to an analysable one, and
reports every assumption it makes as a plain sentence. It handles the two jobs
clean_table() deliberately refuses: deciding that "NSW" and "Nsw" are one
category, and deciding that a character column holding six labels is a factor.
Both are off by default (merge_labels = "report", types = "report"), so
the first call names what it would do and changes nothing.
Storage is decided before categories, which is the ordering that matters: a
column of numbers held as text is not a set of categories, and merging "4.2"
into "4.4" because they differ by one character is how an automatic cleaner
destroys data. Only columns that remain categorical are considered for merging,
and a one-character edit between labels shorter than four characters is ignored.
A pair differing only by capitalisation is one category whatever happens, so it is merged onto the more frequent spelling (on a tie, the one that appeared first). A pair differing by one edit might be two real categories, so it is merged only when one spelling is clearly more common, and refused otherwise.
All three vignettes were brought to the orchestra vignette quality bar:
each now opens with the question in the voice of the person asking it
(a custodian or an analyst), follows a fixed Why/What/Do/Read/Limits
shape, carries a ggplot2 figure that shows what its tables cannot
(the conditional-clone treatment-effect recovery, the multi-environment
coverage plot, and the original-versus-clone marginal overlay), and
demonstrates a fail-closed refusal for real in each case (an
incompatible role/action pairing, a blocked HIGH-leakage write, and an
unmapped factor level).
Every orchestra-facing decline in mask(), mask_set(), unmask() and
audit_mask() that was a bare, unclassed cli::cli_abort() now carries a
class ending in _refusal plus the shared orchestra_refusal marker, so
a caller (or a conductoR node) can catch the decline programmatically
instead of string-matching the message. masque_unmasked_coords and
masque_conditional_degraded were already classed and are unchanged.
mask_set() draws the alias map for linked (join-key) columns from the
seeded permutation like every other alias map; it was lexicographic and
seed-invariant, so a public vocabulary could invert it (M-01).
The collaborate-mode alias map is no longer the sort order. Aliases
(trt_001, <col>_L001, ...) were assigned to levels in lexicographic
order and were invariant to seed, so a holder of the synthetic who
could guess the vocabulary -- a public variety roster, an N-rate ladder,
a site list -- inverted the map by sorting a candidate list, with no
recipe. The assignment is now drawn from a uniform random permutation
taken from the seeded stream mask() already uses. The map is recorded
on the recipe exactly as before, so apply_recipe() / unmask() still
invert it, and a fixed seed still reproduces it. Two notes for
callers: the seed is now confidential material (published beside the
synthetic it re-opens the inversion), and level frequencies are
unchanged by design and remain matchable. The confidentiality
vignette states both.
This changes the integer codes of an aliased factor, which are now a
relabelling, not a copy of the original codes. Code that compared
as.integer() of an aliased column against the source should compare the
partition instead (recode both by order of first appearance); the
incidence structure itself is unchanged.
conditional = TRUE no longer collapses silently to the pooled
copula. The conditioning stratum was treatment crossed with every
retained design column, so on a replicated factorial (six N rates by
three varieties by four replicates) every cell held one row, all of them
fell below the minimum stratum size, and the whole numeric block was
pooled -- the marginal clone under a conditional label, with the recipe
still asserting conditional = TRUE. mask() now walks a coarsening
ladder: design columns are dropped, the finest first, until the cells
reach the minimum; treatment columns are never dropped. On a replicated
2 x 3 factorial with a planted effect this restores the treatment
sum-of-squares fraction from 0.004 to inside the stratified-bootstrap
envelope of the source's 0.859.
Degradation of a conditional clone now raises a classed
masque_conditional_degraded warning -- when the ladder gives up a
column, when rows remain in the pooled fallback, and when there is
nothing to condition on at all -- so a caller can catch it instead of
matching on message text.
conditioning_used (the rung of the ladder actually
reached) and fallback_frac (the share of rows pooled at that rung)
alongside the existing conditioning_cols (what was requested).
print() on a recipe names the coarsening and the pooled share.min_stratum at 2, 5 and 20 on a layout whose cells straddle all
three, and synthesise_numeric_local(regularise = FALSE) on a
rank-deficient block -- both previously unexercised at any value.getting_started vignette's environment-overview figure now has a
fig.cap/fig.alt and two sentences reading the panel (what even bar
heights and a single connected component mean, and what a broken MET
would look like there).getting_started mode-comparison
table ("numeric values that may match observed" -> "... the observed
values").recipe_anatomy documents the recipe's coords and
allow_unmasked_coords properties (added in 0.10.0/0.11.0) and shows
recipe(m)@coords on a worked coordinate example.README.md no longer hard-codes a stale version number; points to
NEWS.md instead and flags the 0.10.0/0.11.0 coordinate-handling
changes for readers coming from an older tag.apply_recipe()'s roxygen no longer describes the pre-0.6.0 ignore
role; step 2 now states the current, mode-independent
action == "drop" rule.@examples to read_recipe() and unmask(), the package's
last two exports without a runnable example.± in
internal and roxygen comments to the package's ASCII --/+/-
convention (no rendered-text or behaviour change).A coordinate in a masked table is masked unless you say otherwise.
mask() now stops when a column whose name says coordinate (gps,
lat, lon, easting, northing, utm, wgs84, ...) would be written
through with action = "keep". A position locates the site and often the
operator with it, so the burden is on the caller to say what should happen
to it, and when nothing is said the default is to refuse.
Three ways to state otherwise, all unchanged in spelling:
coords, which coarsens it by the on-land geomask;drop, the default propose_roles()
already proposes for a coordinate-named column, or scramble);allow_unmasked_coords = TRUE, having decided the
coordinate is not sensitive. It is recorded on the recipe.The refusal is the classed condition masque_unmasked_coords.
A column detected as a coordinate only by the shape of its values -- a
numeric pair inside plausible latitude and longitude ranges carrying at
least four decimal places, which is metre-scale precision no temperature or
yield column has -- raises the classed warning masque_coords_suspected
and does not stop. Confidence is lower there, and a false positive should
not block a legitimate mask. Ordinary numeric covariates do not trip it:
whole numbers, low-precision values and unpaired columns are all excluded.
audit_mask() reported a coordinate declared to coords as HIGH leakage
with the note "kept as-is - visible to collaborators". The column had in
fact been coarsened in place by the geomask, displaced by the requested
radius and shared across its site. It now audits as medium in
collaborate mode, with the note "coordinate coarsened in place by the
geomask". The stale HIGH also tripped the package-managed write gate, so
every caller who used coords had to override it to write.
The audit mask() runs for itself in collaborate mode was built on a
temporary recipe that omitted the coordinate record, so it reached the same
wrong conclusion as above even after the record existed.
A coordinate belongs to a site, not to a row. jitter_coordinates() and
mask(coords = ) now draw one displacement per site and broadcast it to
every row of that site, instead of displacing every row independently. A
table holding many rows per physical place -- plots within a trial, samples
within a paddock, observations within a farm -- now comes back with one
masked coordinate per place, as its source table has one true coordinate per
place.
Tables masked by 0.9.2 or earlier carry the old per-row structure. Their coordinate columns assert that observations kilometres apart share a site, and should be treated as invalid for anything spatial until the table is rebuilt under 0.10.0.
The change closes a confidentiality defect as well as a fidelity one. Donut displacement is isotropic, so the mean of many independent draws around one true site converges on that site: averaging the 360 rows of a single trial recovered the true position to a median 0.58 km, against the 5 km floor the donut was asked for. One draw per site removes that estimator and leaves the full displacement in place.
jitter_coordinates() gains by, which chooses how a site is identified.
by = NULL (the default) groups rows sharing an identical input coordinate.
A character vector names site columns, for the case where a site's recorded
coordinates differ slightly between rows; such a group is consolidated to its
row-weighted centroid and the consolidation is reported by a classed
masque_geo_consolidated warning, with masque_geo_wide_group raised when a
named group spans more than the displacement radius. by = FALSE restores
the pre-0.10.0 per-row draw and warns (masque_geo_ungrouped) if the input
carries repeated coordinate pairs. It exists only so a table masked by an
earlier version can be reproduced.
A site that cannot be placed on land within max_tries re-draws is now set
to NA on both axes. Leaving it unchanged shipped
the true coordinate inside a table the caller believed was masked, behind
a warning. The masque_geo_unplaced warning is unchanged in class and still
names the count.
Genuinely point-level input -- every coordinate pair already distinct -- is unaffected by all of the above, and reproduces its 0.9.2 output exactly under the same seed.
One displacement per site holds within a table, not across rebuilds. Each
rebuild with a fresh seed is an independent draw, so a site kept in several
masked versions can be averaged back out: at the 5-20 km default, eight kept
rebuilds recover it to 4.5 km, below the floor. Reuse the seed and the
displacement is identical, so there is nothing to average. Documented under
"Rebuilding, and the displacement budget" in ?jitter_coordinates.
mask(coords = ) accepts by in a coordinate spec, for example
coords = list(list(lat = "GPS_S", lon = "GPS_E", by = "TRIAL")). The
grouping is taken from the original table, so a site column that is
aliased, permuted or dropped in the synthetic still groups correctly.
The recipe records what happened to each declared coordinate pair: the jitter
parameters, the grouping it was masked under, the number of sites that
grouping produced, any consolidated sites, and any site set to NA. It is
shown by print(recipe(m)) and readable at recipe(m)@coords. Recipes
written before 0.10.0 read back with no such record.
mask() gave every declared coordinate pair the same seed, so two
latitude/longitude pairs in one table received identical displacement
vectors. Each pair now draws from its own sub-stream, deterministically
derived from the mask() seed.mask() gains a coords argument to coarsen geographic coordinates as part
of masking. Declare one or more latitude/longitude pairs (for example
coords = list(c(lat = "GPS_S", lon = "GPS_E"))) and each pair is passed
through synthesis untouched and then displaced in place by the on-land
jitter_coordinates() geomask (a 5-20 km donut by default), instead of being
copula-scrambled into implausible locations. A pair may be a bare
c(lat =, lon =) vector or a list(lat =, lon =, min_km =, max_km =, ...)
that also carries jitter parameters; both spellings work (numbers supplied
through the vector form are coerced back from strings). A declared pair always
survives, coarsened; the recipe records that it was coarsened, and
apply_recipe() retargets a pipeline to the real coordinates.propose_roles() now protects a conservatively detected
multi-environment allocation by default. Before 0.9.1, an environment
column not recognised by the name heuristic could remain
covariate/scramble, which moved observations between environments.
High-confidence categorical environment columns now become
design/keep in local mode and design/alias in collaborate mode.
Numeric environment columns remain design/keep and raise a classed
disclosure warning in collaborate mode. Disable the new scope path
explicitly when reproducing the former detector output:
detect_design(df, env = FALSE)
propose_roles(df, detect = FALSE)
detect_design() gains an append-only env argument and an orthogonal
environment-scope contract. Use env = NULL for conservative automatic
resolution, env = c("site", "year") for an explicit composite, or
env = FALSE for the whole-table compatibility path. The returned
design_summary records scope confidence, environment basis, experiment
groups, exact treatment-connectivity components, and advisory
per-environment design summaries without storing the source data.print() on a design_summary now leads with scope and reports bounded,
aggregated MET diagnostics. plot() supports compact base and ggplot2
environment overviews plus one selected field layout through
environment =. Connectivity and inner-design labels are diagnostic and
do not claim to reconstruct the original randomisation.jitter_coordinates() coarsens latitude/longitude in place by an
on-land geographic-masking jitter (donut or Gaussian), so a synthetic table
can carry realistic coordinates without revealing a true field or farm
location. The donut default moves every point 5-20 km in a random direction,
re-drawing until it lands on land (via maps); the NA pattern and the
latitude/longitude pairing are preserved. It complements
synthesise_geospatial(), which instead re-anchors coordinates at
user-supplied fake centroids.Omitting mode from mask() or mask_set() now inherits the mode stored
by propose_roles(); it no longer falls back silently to local defaults.
An explicit collaborate-to-local change raises a classed
masque_mode_downgrade warning. mask_set() rejects role plans carrying
mixed mode provenance.
mask() and mask_set() now warn (classed masque_mode_unset) when no
mode is supplied and the roles table carries no mode provenance, instead
of silently masking in local mode. A data.table() wrap or a saveRDS()
round-trip strips the tibble attribute that records the mode, so the warning
turns a silent local-mode downgrade into a recoverable one.
Site-only environment candidates now require replicated treatment evidence across sites for automatic promotion. A block nested within one farm is left for review and not classified as an environment.
Environment connectivity diagnostics now guard both dense incidence and
environment-adjacency allocations. Oversized problems return an explicit
not_computed diagnostic instead of attempting an unsafe allocation.
Experiment-group diagnostics tolerate a small, recorded treatment overlap while rejecting genuinely interleaved groups. The evidence now reports the confined and shared-treatment fractions.
Explicitly pinned role actions are no longer overwritten when a detected
environment recommendation promotes a column to the design role.
A medium-confidence or ambiguous environment candidate is now preserved as
design/keep in both modes; it no longer falls through to a covariate
default. Previously an unreplicated environment named with a site token
outside the design-name heuristic (for example county, location or
farm) could be silently row-permuted by a default mask(), moving
observations between environments. Such a column is kept byte-identical and
is never auto-aliased; pass env = to enable environment-aware masking.
Invalid (non-syntactic) column names are now legalised in every
clean mode, not only "auto", and the repair is raised as a classed
masque_name_repaired warning and recorded in the recipe. Previously,
under clean = "off" an invalid name like GY_%VARMAX was silently
rewritten by make.names() inside numeric synthesis with no map
recorded, so the synthesised column no longer matched its source: the
original column survived unmasked alongside the synthetic copy --
a leak -- and the round-trip broke. mask() now legalises names up
front in all modes, remaps the roles table, records the map, and
synthesise_numeric_local() no longer rewrites names.
unmask() now restores the original (pre-legalisation) column names,
so a legalised name round-trips symmetrically with apply_recipe().
role_options() renders the two-axis vocabulary as data: every
role and action pair, the storage kinds the pair works for, and the
reason it is constrained when it is. The grid is generated from the
same compatibility rules roles_validate() enforces at mask()
time, so it cannot drift from the validator.
role_options(kind = "factor") filters to the combinations
available for one column's storage kind.set_role(), direct data-frame edits with NA
action re-resolution, the role_options() grid, and hand-flagging
pii_suspected on a sensitively valued column the name heuristic
cannot see.e) no longer destroys the session
when the spreadsheet editor cannot start. utils::edit() on a data
frame requires the X11 dataentry widget - missing on macOS without
XQuartz and on headless systems - and its error previously propagated
out of masque(), losing the proposal and the user's place. The
editor call is now guarded: on failure a dependency-free console
editor takes over (pick a column, then a role and an action from
numbered menus). Every console edit is applied through set_role(),
so vocabulary validation and default-action re-resolution match the
scriptable path exactly, and the roles table's provenance attributes
survive.[e] is read as e).This release corrects a safety-contract defect in the guided workflow.
In 0.5.0 through 0.7.1, masque() and mask_set() wrapped their
internal mask() calls in suppressWarnings(), so the HIGH-leakage
warning raised by the collaborate-mode audit never reached the caller -
and the guided summary could then print sharing language for output the
audit had flagged. Generation must never imply release; the behaviour
now matches the documentation.
masque() and mask_set() no longer suppress warnings from mask().
The HIGH-leakage finding is raised as a classed warning
(masque_high_leakage) so callers can handle it programmatically.masque()'s out and write_set() -
now refuse to write while a HIGH finding stands: nothing is
written, and the flagged columns are listed with the remedy. A new
allow_high argument (default FALSE) overrides the refusal after
the custodian's own review; the override raises a
masque_high_override warning and is recorded in the recipe's
warnings, so the exception stays auditable. Local mode is never gated.BLOCKED block with the next
corrective action when HIGH findings stand, and no longer prints
"share only the synthetic output". print() on a masque object shows
the same tally.warning(). It
forced callers (and this package's own examples) to blanket-suppress,
which is exactly how the HIGH warning was lost - an alarm-fatigue
defect. The reminder is recorded on the recipe and shown by print()
and the guided summary instead. Genuine findings keep the warning
channel to themselves.mask() now errors on unused ... arguments instead of silently
ignoring them, so a misspelled argument (e.g. sedd = 1) can no
longer look like success.save_recipe() does not
encrypt, that generating a synthetic table is not a release decision,
and that masque contributes evidence to Safe Data / Safe Outputs
while the remaining Five Safes are organisational.masque_recipe no longer carries the detect_design() artefact
that propose_roles() stashes on the roles table (attr(roles, "design"), plus the proposed_actions / mode provenance). The
recipe is contractually runtime-minimal, and the design object is not
part of its contract; it inflated the saved recipe and, because its
serialised size varies across R versions, could push save_recipe()
output past the documented small-file budget on R-devel. Saved recipes
are now smaller and deterministic across toolchains. No public API or
round-trip behaviour changes -- recipe(m)@roles keeps every role and
action assignment.A conditional clone mode that preserves the treatment-to-outcome relationship, so a causal model fitted on the synthetic recovers the real treatment effect -- not just the marginal distribution.
mask(), mask_set(), and masque() gain a conditional argument
(default FALSE). With conditional = TRUE, the numeric block is
re-simulated within each treatment-by-design stratum, not from
one global Gaussian copula, so a row's synthetic outcome inherits the
location of the treatment that row carries. The treatment-to-outcome
map -- the quantity a causal model reads -- survives the clone, within
sampling tolerance. The default conditional = FALSE preserves the
exact 0.6.0 behaviour (global copula, marginal and structural fidelity
only), so existing scripts are byte-identical.
The motivation: structural and marginal fidelity alone are not sufficient for causal inference. A clone that matches every marginal and the global covariance can still sever the relationship between an arm and its response, because the outcomes are drawn from the pooled distribution and the treatment labels are relabelled independently. A model fitted on such a clone returns a null effect.
masque_recipe records two new fields, conditional (logical)
and conditioning_cols (the treatment and retained design columns the
clone stratified on), and print(recipe(m)) now reports the clone
fidelity (marginal / structural versus conditional). Recipes written
by masque 0.6.0 and earlier read back with the back-compatible
defaults (conditional = FALSE, no conditioning columns).A two-axis roles step, a hygiene layer, column-name aliasing, a multi-table set layer, and a single guided verb turn masque into an end-to-end tool for confidential tabular data.
role column (what a column
is) and a new action column (what mask() does to it). The role
vocabulary changed -- the old keep and ignore roles become the
keep and drop actions, and new roles date, id, text, and
other join design / treatment / outcome / covariate.mask() no longer requires a column roled outcome.roles_validate() (and therefore by mask()) with a
one-time deprecation warning that preserves the old mode semantics, so
existing scripts keep working. Re-run propose_roles() to silence the
warning and adopt the new schema.masque() is the front door: one call reads the input (a data
frame, a file, a folder, an Excel workbook, or a named list), proposes
column roles, pauses for review in an interactive session, masks,
audits, and - given an out path - writes the result. It dispatches a
single table through mask() and a multi-table input through
mask_set(), and returns the same object the lower-level verbs do, so
it stays fully scriptable: pass an edited roles table to skip the
prompt. (The internal S7 result class previously bound to masque is
now masque_obj; the class name is unchanged.)role (what
a column is) and action (what mask() does to it). role is
one of design, treatment, outcome, covariate, date, id,
text, other; action is one of keep, scramble, alias,
drop. propose_roles() resolves a mode-appropriate default action
for every column, so the table you review is the masking plan that
runs.date role (date/time columns are row-permuted with
class and NA pattern preserved), and the explicit "retain untouched"
and "skip entirely" choices are now the keep and drop actions,
available on any column, where they were once only the keep and
ignore roles.propose_roles() gains a mode argument. The proposed actions
differ between local and collaborate (for example a treatment is
kept locally but aliased for collaboration); the table records the
mode it was prepared for.set_role() helper edits a roles table ergonomically.
Re-assigning a column's role re-resolves its default action; passing
an explicit action
pins the column so a later mode change leaves it alone. Direct
roles$role[...] <- edits still work.mask() no longer requires an outcome column. With none marked,
the Gaussian copula simply re-simulates every scrambled numeric
column jointly.role = "design", action = "alias" is a new opt-out from
byte-identical design preservation: the design structure is kept
but the site / block labels are hidden behind opaque aliases and
restored by unmask(). The default for design columns remains
keep (byte-identical).roles_validate() validates the (role, action, kind) combination and
fails closed on impossible pairings (a scrambled design column, an
aliased numeric, a scrambled id). It returns the validated table with
any NA actions resolved.action column; the
keep / ignore roles; the mask_levels column) are upgraded
automatically with a one-time deprecation warning, preserving the v1
mode semantics exactly.mask_set() masks a whole multi-table dataset at once - a folder
of CSV / TSV / .fst files, a multi-sheet Excel workbook, or a named
list of data frames - returning one synthetic table per input table
and a single private recipe bundle.links
argument, or set links = FALSE to mask each table independently.read_set() ingests the folder / workbook / list into a named
list of data frames (clean rectangles only - a missing header row or
non-rectangular sheet fails with an explanatory error). New
write_set() writes a masked set back out, mirroring the input format
(workbook in, workbook out; folder in, folder of CSVs out). The
private recipe bundle is never written by write_set().synthetic(), recipe(), apply_recipe(), and unmask() all
dispatch on the set: synthetic() returns the named list of tables,
recipe() the bundle, and the round-trip verbs operate table by table.data.table joins Imports (fast fread / fwrite); readxl and
writexl are Suggested for the Excel paths.alias_names argument on mask() hides the column names
themselves - the last identifying surface a kept or design column
exposes. TRUE replaces every retained name with an opaque alias
(col_001, col_002, ...); a character vector aliases just the named
columns. The original-to-alias map lives in the (private) recipe and
is inverted by apply_recipe() and unmask(), so a pipeline written
against the aliased synthetic round-trips. This realises the
long-reserved column_name_map recipe slot. reveal_maps() now also
prints the column-name map.clean_table() verb (and a clean argument on mask(), default
"auto") tidies a dirty table before masking: column names are
legalised (valid, unique R names), and leading / trailing whitespace
is trimmed from names and from character / factor labels. Both fixes
are reported through cli, recorded in the recipe, and re-applied by
apply_recipe() so retargeting still lines up."north" vs
"North") or by a single edit ("Compass" vs "Compas") - are
reported, never merged: deciding whether two similar labels are the
same value is a judgement masque leaves to the user. Set
clean = "report" to preview fixes without applying them, or
clean = "off" to skip hygiene entirely.MASS dependency: the copula now draws latent normals
through a Cholesky factor of the regularised covariance, removing a
hard import.New feature release: joint-treatment masking plus role-table usability
and first-class date handling. mask() and roles_validate() now
accept designs with two or more treatment factors (factorial,
split-plot trials).
keep role: users can now explicitly mark a column for
byte-identical pass-through in both local and collaborate mode. This is
distinct from design, which remains for experimental-design structure,
and from ignore, which is still dropped in collaborate mode.propose_roles() now proposes Date / POSIX / difftime columns as
covariate, where they were ignore. Date/time covariates are row-permuted,
retain their original class, and preserve the cell-level NA mask.keep with a clear note,
avoiding a confusing attempt to synthesise objects masque does not know
how to mask.apply_recipe()
and unmask() round-trip those aliases back to logical values.detect_design() and its candidate proposer now honour user-supplied
keep roles so explicitly-kept columns are not promoted into design
hints.mask() now masks every column flagged treatment, not just one.
Each treatment factor is aliased independently. A single treatment
keeps the historical trt_NNN collaborate-mode alias prefix; with
two or more, the column name is folded into the prefix
(<col>_trt_NNN, e.g. variety_trt_001) so the opaque labels stay
distinct and self-documenting, mirroring the categorical-covariate
<col>_LNN convention. Aliasing each factor separately, not
the treatment combination, preserves the per-factor structure that
factorial models fit, and keeps each column's alias namespace -- and
therefore audit_mask()'s leakage accounting -- unchanged.roles_validate() no longer errors when more than one column is
flagged treatment. The round-trip path (apply_recipe(),
unmask()) already inverted multiple per-column level maps, so
recovery of every treatment factor works unchanged.Maintenance release: contract-sharpening corrections plus the documentation and metadata that were prepared for v0.4.0 but not released. No new public exports. The two behaviour changes below are deliberate fail-closed corrections to existing exports; user code that depended on the silent failure mode will need to be updated.
apply_recipe() and unmask() now error when a non-NA value is
not present in the recipe's level map. Previously the row was
silently coerced to NA, which could quietly poison downstream
model matrices. Schema drift or a new treatment level in the input
now fails closed with the offending values listed.apply_recipe() now verifies that the NA mask of original matches
the recipe's recorded integrity_fp. A mismatch errors with
guidance. New check_integrity = TRUE parameter (default) gives
an escape hatch (check_integrity = FALSE) for workflows where the
missingness has legitimately changed since the recipe was built.unmask(x, rec) now passes through atomic numeric, integer,
logical, and Date / POSIXct vectors unchanged, matching the
documented numeric pass-through contract. Previously these inputs
errored when the recipe held no level maps.audit_mask()'s exact_match_pct now divides by the number of
jointly-observed comparable cells, not by nrow(df). Columns
dominated by NAs no longer underreport leakage. The audit tibble
gains a new comparable_n column for interpretability.synthesise_geospatial() now uses original's NA mask as the
authority for cell-level preservation (previously used synth's
mask, which could let synthesised coordinates leak into rows that
the original had missing). Adds a nrow(synth) == nrow(original)
check.roles_validate() error message for the multiple-treatment case is
refreshed: drops the stale "v0.2 / deferred to v0.3" wording and
guides the user to either edit the roles tibble or call
propose_roles(df, detect = FALSE) for byte-stable v0.2.x
behaviour.mask()'s
roxygen removed.recipe_io.R doc and the recipe_anatomy vignette reword the
include_simulator = TRUE no-op without pinning it to v0.2 / v0.3.roadmap vignette restructured around feature areas. The hard
version pins ("v0.3", "v0.4") are gone -- v0.3 / v0.4 shipped
different features from the prior roadmap, so the pins were stale.getting_started vignette: "vignette('roadmap') -- what's planned
for v0.3+" replaced by "features deliberately deferred from the
current release".test-mask-end-to-end.R,
test-mask-roundtrip-integration.R) call
propose_roles(df, detect = FALSE) so the suite is clean against
the maintainer's local fixtures while the multi-treatment design
decision remains roadmap.expect_warning("HIGH leakage")
so future warning regressions remain visible.unmask(); fail-closed unknown-level handling in
apply_recipe() and unmask(); integrity_fp enforcement
(positive, negative, and the check_integrity = FALSE escape
hatch); synthesise_geospatial() NA-mask source authority and
row-count check.Adds first-class geospatial synthesis. One new export, no breaking changes to the v0.3.0 surface.
synthesise_geospatial(synth, original, anchor_col, lat_col, lon_col, anchor_centroids, site_spread_deg, jitter_deg, seed) --
re-anchors the latitude / longitude columns in a masqued data frame
at user-supplied centroids, while preserving (a) the count of
distinct sites per anchor level, (b) the per-site replication
distribution, and (c) within-site tight clustering with
between-site spread. The original positions are never published;
the function reads them only to count distinct sites. NA pattern
in coordinates is preserved cell-by-cell. RNG hygiene via
withr::local_preserve_seed().
Motivated by the masque release walkthrough, where state-centroid
cran-comments.md for first-submission notes..github/workflows/R-CMD-check.yaml (r-lib standard matrix:
Linux release / devel / oldrel-1, macOS release, Windows release).R CMD check --as-cran reports 0 errors, 0 warnings, 2 NOTEs
(new-submission boilerplate and local HTML Tidy environmental).R/synthesise_geospatial.R carries the full roxygen doc + a
\donttest{} example.Adds automatic experimental-design detection and a sanity-check visualisation. New public surface: 3 exports, 1 vignette.
detect_design(df, roles = NULL, interactive = FALSE, threshold = 0.5, tie_delta = 0.02) -- returns an S7 design_summary with the most
likely design class (CRD, RCBD, IBD/alpha-lattice,
row-column, split-plot, factorial, or none), per-rule scores,
evidence, and a recommended_roles tibble. Rule engine, not ML.design_summary -- S7 class wrapping the detection result.
print() is cli-styled and surfaces top-3 alternates so the user
can see how confident the call was. Slots include class_label,
treatment_col, block_cols, whole_plot_col, sub_plot_col,
spatial_cols, scores, evidence, recommended_roles,
candidates, warnings.plot_design_summary(x, df, engine = c("base", "ggplot2")) -- also
registered as an S7 plot() method. Base-graphics sanity-check
visualisation dispatched per class: replication tile, spatial
layout, factor-nesting tree, treatment-frequency + NA-pattern.propose_roles(df) flips to detect = TRUE by default. The
detected design's recommended_roles are overlaid on the name-based
proposal, promoting structurally-identified treatments and blocks
even when their column names don't match the design / treatment
regexes (e.g., gen in an alpha-lattice). The design_summary is
stashed as attr(roles, "design"). Pass detect = FALSE to
recover the v0.2.x byte-stable behaviour.mask() synthesis behaviour is
unchanged. Only propose_roles() consumes detection output, and
only as role hints.[0, 1] with evidence; the orchestrator picks
the top above threshold, breaking ties in favour of the simpler
design (CRD < RCBD < factorial < IBD < row-column < split-plot).desplot::desplot() or ggplot2-based packages.agridat -- canonical fixtures for tests and the new vignette.ggplot2 -- optional plot engine via engine = "ggplot2"; base
graphics is the default and the fallback.detect = FALSE for toy fixtures.First public release of masque -- a structurally faithful development
surrogate for tabular datasets. Successor to the unreleased synthPR
v0.1.0 (folder-scanning multi-file API), rewritten around a single-file
data-frame-first interface and a round-trippable recipe object.
masque is not an anonymisation or differential-privacy tool. It produces
development surrogates suitable for building and debugging pipelines, and
a private recipe that re-targets a pipeline built against the synthetic
clone back onto the original data. See vignette("confidentiality") for
the threat model.
design, treatment,
outcome, covariate, ignore. Multi-outcome supported. Date /
POSIX columns and PII-pattern column names default to ignore.local -- realistic dev surrogate for the data owner. Column names
and level vocabularies preserved. Treatment-level permutation is
opt-in. Issues a load-time warning when the synthetic is extracted.collaborate -- give the synthetic to a collaborator while keeping
the recipe private. Treatment + categorical-covariate levels are
opaque-aliased (trt_001, <col>_L01). Numeric draws are
jittered within column resolution; integer columns are
stochastically rounded. ignore columns are dropped.
audit_mask() runs automatically and warns on HIGH leakage.propose_roles(df) -- heuristics-driven role tibble; the user edits
and passes to mask().roles_validate(roles, df) -- fail-closed structural + semantic check.mask(df, roles, mode, seed, ...) -- returns an S7 masque object.synthetic(m) / recipe(m) -- accessors that hide S7.apply_recipe(original, recipe) -- forward translate original-namespace
data into the synthetic namespace.unmask(x, recipe, column = NULL) -- inverse on a data frame or atomic
vector; round-trips a pipeline back to the original.save_recipe(rec, path, include_simulator = FALSE) /
read_recipe(path) -- runtime-minimal .rds persistence (under 10 KB
on a 17,000-row, 38-column MET fixture).audit_mask(m, original = NULL, print = TRUE) -- first-class leakage
audit returning the per-column severity tibble.reveal_maps(recipe) -- explicit, banner-fenced unmasked-map reveal
(never automatic; print(recipe) is redacted by default).withr::with_seed / local_preserve_seed);
mask() does not mutate the caller's .Random.seed.recipe is runtime-minimal by default -- no copula matrix or raw
marginals stored. SHA-256 NA-mask fingerprint provided as an
integrity check, not a privacy primitive.print(recipe) redacted by default; reveal_maps() is the only
unmasked path.audit_mask() flags retained PII-pattern columns, unaliased
treatments under collaborate, rare-level leakage, and numeric exact-
match rates above the per-role thresholds.getting_started, confidentiality,
recipe_anatomy, roadmap.inst/extdata/john_alpha.csv -- 72-row, 7-column public fixture
derived from agridat::john.alpha (John 1987, alpha design).Predecessor synthPR v0.1.0 (folder-scanning, multi-file) is archived
at _legacy/synthPR_v0.1.0/ in the development workspace and is not
distributed.