Faker Synthesizer

ydata.synthesizers.FakerSynthesizer

A synthesizer for generating synthetic data based on user-defined configurations.

The FakerSynthesizer allows users to create synthetic tabular data without needing an existing dataset. Instead, it generates data based on user-provided metadata or manually defined column configurations. This approach is useful for:

• Creating mock datasets for testing and development.
• Generating data prototypes before real data is available.
• Ensuring privacy-preserving synthetic data without reference to actual records.

Key Features:

• Metadata-Driven Generation: Generates synthetic data based on predefined Metadata.
• Customizable Column Types: Supports user-defined column structures.
• Multi-Language Support: Uses locale settings to generate realistic names, addresses, etc.

Example Usage:

from ydata.synthesizers import FakerSynthesizer

# Initialize the synthesizer with a specific locale
faker_synth = FakerSynthesizer(locale="en")  # English data generation

faker_synth.fit(metadata)

# Generate synthetic data
synthetic_data = faker_synth.sample(n_samples=1000)

fit(metadata=None, calculated_features=None, scaffold=None, reference_data=None, distributions=None, conditions=None, sequence=None, totals=None, derived_tables=None, table_name='data', errors=None, timeseries=None, *, tables=None, existing_data=None, anchor_tables=None, pool_size=5000)

Configure the FakerSynthesizer using provided metadata.

This method sets up the synthesizer by defining the structure of the synthetic dataset based on the given Metadata. The metadata can either be:

• Computed: Automatically extracted from an existing dataset.
• User-Defined: Manually constructed to specify custom column types and distributions.

Once fit() is called, the synthesizer will use this metadata to generate structured synthetic data that adheres to the defined schema.

Parameters:

Name	Type	Description	Default
metadata	Metadata | None	A metadata object describing the structure of the synthetic dataset, including: - Column names and data types. - Faker-based data generators (e.g., names, addresses, emails). - Value constraints (e.g., numeric ranges, categorical options).	None
calculated_features	list[dict] | None	Optional. Columns computed deterministically from other columns AFTER sampling — exact identities, sign rules, derived flags. Same single-table convention as :class:RegularSynthesizer (bare column names):: [{ "calculated_features": "vat_value", "function": lambda net: (net * 0.20).round(2), "calculated_from": ["net_value"], }] Features are applied in list order (later features may consume earlier outputs) and are excluded from generation. Cross-table reference_keys are not supported by this single-table synthesizer.	None
scaffold	dict | ScaffoldConfig | None	Optional. A declarative structural grid (:class:~ydata.dataset.schemas.mock_schema.ScaffoldConfig or its dict form): the cross product of the declared dimensions (date_range, static values, or from_table sourced from reference_data), optionally filtered by density rules, becomes the table's seed rows. Scaffold columns are emitted verbatim; every other metadata column is generated per seed row, and sample(sample_size=...) is ignored (the scaffold defines the row count). Example:: scaffold={ "dimensions": [ {"column": "product_id", "from_table": "products", "from_column": "product_id"}, {"column": "date", "date_range": {"start": "2026-03-01", "end": "2026-03-31"}}, ], }	None
reference_data	dict[str, DataFrame] | None	Optional. Named DataFrames backing the scaffold's from_table dimensions and density lookups.	None
distributions	dict[str, dict] | None	Optional. Per-column distribution overrides for numerical columns (default: uniform within the declared domain). Pure data, so it can also live in the YAML configuration:: distributions={ "amount": {"distribution": "lognormal", "mean": 3.0, "sigma": 0.5}, "score": {"distribution": "normal", "loc": 50, "scale": 10, "min": 0}, } Supported: uniform, normal, lognormal, exponential, poisson, triangular, beta. min/max clip (falling back to the column's domain); integer columns are rounded.	None
conditions	list[dict] | None	Optional. Conditional generation rules: a column's values are sampled per segment defined by when predicates over one or MORE other columns (equality / is-in, AND semantics; first matching rule wins; unmatched rows use default or the column's base sampling):: conditions=[{ "column": "amount", "rules": [ {"when": {"category": "fuel"}, "distribution": {"distribution": "lognormal", "mean": 8.6, "sigma": 0.55}}, {"when": {"category": ["shop", "commission"], "channel": "online"}, "distribution": {"distribution": "lognormal", "mean": 6.9, "sigma": 0.7}}, ], "default": {"distribution": {"distribution": "uniform", "low": 1, "high": 50}}, }, { "column": "doc_type", "rules": [ {"when": {"category": "fuel"}, "values": {"Invoice": 0.9, "Credit Note": 0.1}}, ], "default": {"values": {"Invoice": 0.7, "Debit Note": 0.2, "Credit Note": 0.1}}, }] when columns may be other metadata columns, scaffold dimensions, or other conditioned columns (ordered automatically; cycles are rejected). Rules carry either a distribution (numerical columns) or values (weighted categorical values, any column type). A when value may also be a CALENDAR PREDICATE over a date column: {"weekday_in": [5, 6]}, {"month_in": [12]}, {"day_in_month": [25]} (multiple predicate keys AND together).	None
sequence	dict | None	Optional. Declares row ordering for temporal semantics:: sequence={"order_by": "settlement_date", "partition_by": ["site_code"]} The output is stably sorted by partition_by + order_by before calculated features run; calculated features with "per_partition": True are applied within each partition, in order — the mechanism for running balances and lag relations (cumsum, shift, rolling inside the callable).	None
totals	list[dict] | None	Optional. Per-group figures the generated data must meet — a ceiling (max) or an exact amount (target) on the sum of a column within each group. Each group is scaled by a single factor, so the mix and spread drawn by the model are kept and only the magnitude moves:: [{"columns": ["net_value", "vat_value", "gross_value"], "measure": "gross_value", "group_by": ["billing_date"], "max": 5000.00, "round": 2}] Applied after calculated features and before derived tables, so aggregates reconcile to the adjusted detail. List every column of a co-dependent family together so their identities survive the rescale.	None
derived_tables	list[dict] | None	Optional. Additional tables computed deterministically FROM the generated table after everything else — aggregates reconcile to the detail by construction:: derived_tables=[ {"name": "daily_settlement", "type": "aggregate", "group_by": ["settlement_date", "site_code"], "aggregations": { "total_value": {"column": "amount", "agg": "sum"}, "n_transactions": {"column": "amount", "agg": "count"}, }}, {"name": "ledger_legs", "type": "custom", "function": my_explode_fn}, # DataFrame -> DataFrame ] Supported aggregations: sum, mean, min, max, count, nunique, first, last (first/last require a sequence). When derived tables are configured, sample() returns a dict of table name to Dataset (including the base table under table_name).	None
table_name	str	Name of the base generated table in the sample() result when derived_tables are configured. Default "data".	'data'
errors	dict | None	Optional. Deterministic error injection, applied as the VERY LAST stage — after calculated features and derived tables, so identities and aggregates are computed on clean data and the errors land only in the delivered output. Keyed by table name (base table under table_name; derived tables may be targeted too — corrupting a derived table breaks reconciliation with the detail, the classic data-quality test scenario):: errors={ "daily_sales": { "columns": { "revenue": { "format_violation": {"rate": 0.03, "styles": ["words", "currency"]}, "missing": 0.02, }, }, "table": {"duplicate_rows": 0.01, "mapping_break": {"columns": ["sku", "department"], "rate": 0.02}}, }, } Column operators: missing, format_violation (styles: words — 1200 becomes "one thousand, two hundred" —, thousands_sep, currency, trailing_space, truncate, case_flip, whitespace; a corrupted column is delivered as a string column — numbers rendered as text next to the corrupted cells — deliberately), out_of_range. missing alone keeps a column's kind (numerics stay numeric). Table operators: duplicate_rows, mapping_break. Reproducible under random_state.	None
timeseries	dict | TimeSeriesConfig | None	Optional. How columns move through time; requires a sequence (partition_by = entity, order_by = time):: timeseries={ "columns": {"deposit_rate": {"seasonality": {"Dec": 1.5, "Jul": 0.8}, "trend": 0.03, "persistence": 0.6}}, "reacts_to": {"withdrawal_rate": [{"column": "competitor_rate_gap", "lag": 1, "window": 2, "effect": 20}]}, "balances": {"balance": {"start": "opening_balance", "growth": "interest_rate", "inflow": "deposit_rate", "outflow": "withdrawal_rate", "flows_as": {"outflow": "withdrawals"}}}, "regime": {"avg_duration": {"stable": 18, "rate_competition": 3}}, "lifecycle": {"start": "random", "avg_length": 36}, } In a schema, seasonality / trend / persistence sit on the column and the other blocks on the table. See :mod:ydata.synthesizers.faker.timeseries.	None

Schema mode — keyword-only arguments: tables: a :class:~ydata.synthesizers.faker.schema.FakerSchema, or a YAML path / dict that :func:~ydata.synthesizers.faker.config.load_faker_schema reads: a database under a top-level tables: key, or a single table (columns plus its blocks). Each table declares its single-table blocks (columns, scaffold, distributions, conditions, sequence, totals, derived_tables, errors) and, when relational, primary_key, key_format, foreign_keys (cardinality, where, unique, context_columns) and table_errors. Mutually exclusive with metadata and with the single-table keyword arguments above, except reference_data for a single table's scaffold. A plain single table is generated exactly as with metadata, and its calculated features may name columns bare; otherwise they use table.column addressing, and cross-table features declare reference_keys. One table -> sample() returns a Dataset (a dict when it has derived tables); several -> a MultiDataset. existing_data: supplied tables (rows kept, keys never rewritten; missing schema columns are generated). Parents they reference are seeded from their foreign key values. anchor_tables: supplied tables that must carry a unique primary key. pool_size: provider/characteristic/regex string columns are drawn from a pool of this many generated values (fast at volume). A positive integer, or None to generate every value individually.

load(path) classmethod

Load a FakerSynthesizer previously saved with save().

Parameters:

Name	Type	Description	Default
path	str	Path to the file written by save().	required

Returns:

Name	Type	Description
FakerSynthesizer		The restored synthesizer, ready to call sample().

Raises:

Type	Description
AssertionError	If the file does not contain a FakerSynthesizer.

sample(sample_size=1000, random_state=None)

Generate a synthetic dataset based on the configured metadata.

This method produces synthetic data according to the schema defined in the fit() step. The generated data adheres to the column types, constraints, and distributions specified in the provided Metadata.

When a scaffold was configured in fit(), the scaffold grid defines both the structural columns (emitted verbatim) and the row count — sample_size is then ignored. Calculated features configured in fit() are computed after sampling, in list order, and are never generated.

Parameters:

Name	Type	Description	Default
sample_size	int | dict[str, int]	The number of synthetic records/rows to generate. Defaults to 1000. Ignored when a scaffold is configured. Schema mode: rows per ROOT table — an int for every root table, or {table: rows}; other tables are sized by their relationships.	1000
random_state	int | None	Seed for reproducible sampling. Seeds NumPy, Python's random and Faker together, so numeric draws, the scaffold's density filtering, regex-generated columns AND Faker-generated columns (characteristics, text) all reproduce. Defaults to None.	None

Sets error_labels_: one row per error injected by this sample (schema mode: table, row, pk_value, column, error_type, original_value, injected_value; single-table mode: the same without pk_value), or None when no errors are configured. Reset by every fit().

Returns:

Type	Description
Dataset | MultiDataset	Dataset | MultiDataset: a Dataset when one table is generated, a MultiDataset when several are (several schema tables, or derived tables — the base table under table_name, default "data", plus one entry per derived table). Index it by table name.

save(path)

Saves the SYNTHESIZER and the model fitted per variable.

Schema

ydata.synthesizers.faker.config.load_faker_schema(source, table_name=DEFAULT_TABLE_NAME)

Load a FakerSynthesizer configuration from a YAML file or a dict.

Accepted forms

• a database: a top-level tables: key mapping table names to table blocks (or that mapping directly, every value with a columns key);
• a single table: columns plus the optional blocks scaffold, distributions, conditions, sequence, totals, derived_tables and errors. Without a columns key, every top-level entry is a column (flat form). It loads as a one-table schema named table_name.

Deep validation (column configurations, conditions, ...) happens at fit().

Parameters:

Name	Type	Description	Default
source	str | Path | dict	path to a YAML file, or an already-loaded configuration dict.	required
table_name	str	name of the table when the configuration is a single table. Default "data", the base table name of a single-table fit().	DEFAULT_TABLE_NAME

Returns:

Name	Type	Description
FakerSchema	FakerSchema	pass it to FakerSynthesizer.fit(tables=...).

Raises:

Type	Description
SynthesizerValueError	if the file cannot be parsed or the configuration is invalid.

ydata.synthesizers.faker.config.faker_schema_from_llm(tables)

Start a Faker configuration from an LLM synthesizer schema.

For the Faker use case when there is only the idea of a schema: an LLM schema (LLMSynthesizer.fit's tables, or generate_schema's output from a plain-English description) describes tables, keys and columns in prose; this turns it into a :class:FakerSchema skeleton to start the Faker configuration from. Tables, primary keys, foreign keys, scaffolds and table errors are copied; dtypes become datatype/vartype; values become uniform categories; prompts become description.

What the LLM schema cannot say — numerical ranges, date ranges and formats — is left for you: :meth:FakerSchema.gaps lists it per column. Fill it in (typically on the YAML from :meth:FakerSchema.to_yaml, adding distributions, conditions or cardinalities as needed), then fit(tables=...)::

schema = faker_schema_from_llm(llm_tables)
schema.gaps()      # {"customers": {"age": ["min", "max"]}}
schema.to_yaml("customers.yaml")
# ... edit customers.yaml ...
FakerSynthesizer().fit(tables="customers.yaml")

Parameters:

Name	Type	Description	Default
tables	dict	the LLM schema, table name -> {"prompt", "columns", ...}.	required

Returns:

Name	Type	Description
FakerSchema	FakerSchema	the skeleton; fit() rejects it until :meth:~FakerSchema.gaps is empty.

Raises:

Type	Description
SynthesizerValueError	if the LLM schema cannot be converted.

ydata.synthesizers.faker.schema.FakerSchema

Bases: RootModel[dict[str, FakerTableSchema]]

A multi-table Faker schema: table name -> :class:FakerTableSchema.

is_single_table property

One table and nothing relational (keys, key format, foreign keys, table errors, Faker providers, helper columns): exactly what a single-table FakerSynthesizer generates, and generated by it.

context_path(table, fk_column, path)

Resolve a context path requested on table.fk_column.

Returns the lookups, in order, as (table, column): the column is read from that table by its primary key. All but the last are foreign keys.

from_any(source, table_name='data') classmethod

Build from a FakerSchema, or anything :func:load_faker_schema reads: a YAML path or a dict, for a database (tables:) or a single table (a one-table schema named table_name).

from_llm_schema(tables) classmethod

Convert an LLMSynthesizer schema into a Faker schema skeleton.

Tables, primary keys, foreign keys, scaffolds and table errors are copied; dtypes are mapped; values become uniform categories; prompts move to description. Columns that still need configuration (numerical min/max, date ranges) are returned as gaps — fill them in before fit(), which validates every non-key column.

Prefer :func:~ydata.synthesizers.faker.config.faker_schema_from_llm, which returns the schema alone; :meth:gaps then says which options each column still needs.

Returns:

Type	Description
(schema, gaps)	gaps maps table name to the columns needing configuration (the keys of :meth:gaps).

gaps()

The generation options each column still needs before fit().

Per datatype: numerical min/max (not with a characteristic), date min/max/format, categorical categories, longtext text_length. Primary and foreign key columns and scaffold columns are structural and never listed. Useful on a skeleton (see :func:~ydata.synthesizers.faker.config.faker_schema_from_llm) and after editing a configuration.

Returns:

Type	Description
dict[str, dict[str, list[str]]]	{table: {column: [missing options]}}; empty when nothing is missing.

to_dict()

This schema as the configuration :func:load_faker_schema reads.

Always the database form ({"tables": {...}}), so table names are kept; a one-table schema loads back as the same single table. Values keep their Python types (integer or boolean category keys stay integers or booleans), so the dict round-trips exactly; the seasonality key on, which YAML 1.1 reads as True, is written back as "on".

to_yaml(path=None)

This schema as a YAML configuration (see :meth:to_dict).

Parameters:

Name	Type	Description	Default
path	str | Path | None	optional file to write; the YAML text is returned either way.	None

Returns:

Name	Type	Description
str	str	the YAML text, loadable with :func:load_faker_schema or fit(tables=path).

ydata.synthesizers.faker.schema.FakerTableSchema

Bases: BaseModel

One table of a multi-table Faker schema.

key_columns property

The primary key as a list of columns (empty without a key).

single_key property

The primary key when it is one column, else None.

context_aliases()

Aliases under which the requested parent context is available.

Every context_columns entry (a path, see the module docstring) is available as <fk column>.<path>. A one-hop entry is also available as <Parent>.<column> when it is the only foreign key to that parent requesting the column.

Returns:

Type	Description
(aliases, ambiguous)	alias -> (fk column, path); and each short
dict[str, list[str]]	<Parent>.<column> requested through several foreign keys -> those
tuple[dict[str, tuple[str, str]], dict[str, list[str]]]	foreign keys (not provided: it does not say which one is meant).

timeseries()

The table's time behaviour, shape-validated; checked against the table's columns when the synthesizer is fitted.