Faker Synthesizer

Overview

YData SDK includes the Faker Synthesizer, a method for generating synthetic data from a schema or metadata only — no real dataset is required. You define column types and characteristics (such as id, name, email, phone, address, zipcode), optional regex patterns, categorical values, and date ranges. The synthesizer then generates rows that match this schema, making it ideal when you need plausible synthetic data without access to any real source data.

You can build the schema with MetadataConfigurationBuilder or a dictionary configuration, and optionally derive metadata from an existing dataset and then generate from it. Locale support allows you to control the language and format of generated values.

Key Features

• No Source Data Required: Generate data purely from a schema or metadata definition.
• Metadata- and Schema-Driven: Define columns by datatype, vartype, and characteristics (id, name, email, phone, address, zipcode, date).
• Flexible Constraints: Use regex patterns for string columns and categorical distributions for controlled variety.
• Date Control: Set min, max, and format for date columns.
• Dual Configuration: Use either MetadataConfigurationBuilder for programmatic setup or a dictionary config.
• Locale Support: Choose a locale (e.g. "en") for language and regional formatting.
• Fit from Data or Scratch: Fit from existing data’s metadata (e.g. infer schema then generate) or from a schema built from scratch.
• Structural Scaffolds: Pin the table's structure with a declarative dimensional grid (dates × entities × slots, with density rules) — coverage and row counts guaranteed by construction.
• Custom Distributions: Choose the statistical shape of numerical columns (normal, lognormal, poisson, ...) instead of uniform-within-domain.
• Conditional Generation: Sample a column differently per segment defined by one or more other columns — first-match rules with distribution or value-probability outcomes.
• Calculated Features (Business Rules): Columns computed exactly from other columns after sampling — arithmetic identities, sign rules, derived flags.

Use Cases

• Prototyping and Demos: Quickly create sample tables for UI demos or product mockups.
• Test Fixtures and Dev Databases: Populate test and development environments with realistic-looking data.
• Schema-Only or Masking Workflows: Generate data that respects a target schema without exposing real data.
• No Real Data Available: Produce synthetic data when data collection is impossible or not yet done.

Best Practices

• Define Clear Datatypes and Characteristics: Use the right characteristic (e.g. email, phone) so generated values match expectations.
• Use Categories and Regex for Control: Categorical distributions and regex keep variety predictable and valid.
• Set Date Ranges Where Needed: Use min/max and format for date columns to avoid invalid or inconsistent values.
• Prefer the Builder for Complex Schemas: MetadataConfigurationBuilder keeps large schemas readable and maintainable.

Worked Examples

Each configuration block below has a runnable, self-contained example that isolates one concept and asserts the property it is supposed to guarantee. Start with the one that matches the problem you have:

Example	The question it answers
Structural Scaffolds	Which rows exist? Grids, calendars, density, and reference attributes carried in by include.
Distributions & Conditions	What shape do the values have, and how does one column depend on another?
Calculated Features	Which columns must be computed rather than sampled, so the arithmetic holds on every row?
Sequence & Per-Partition Features	How do I express "the previous row of the same entity" — running totals, deltas, lags?
Derived Tables & Aggregations	How do I get summaries that reconcile with the detail by construction?
Error Injection	How do I generate data that is realistically wrong, for testing quality pipelines?

Every example lives under examples/synthesizers/fakersynthesizer/ and runs as a plain script.

Advanced Usage

The Faker Synthesizer supports two configuration styles:

• Builder: Use MetadataConfigurationBuilder to add columns with datatype, vartype, characteristic, and optional regex, categories, or date min/max. Then build a Metadata and pass it to fit().
• Dictionary: Pass a dictionary mapping column names to config (datatype, vartype, characteristic, regex, categories, min, max, format, etc.) into MetadataConfigurationBuilder(config) and then into Metadata.

You can fit from scratch (metadata built only from schema) or from existing data by computing Metadata(data) and fitting the synthesizer on that metadata to generate data that matches the same structure.

Structural Scaffolds

Some tables have a structure that must hold exactly — one row per product per day, every day of the month present, quieter weekends. Column-wise generation cannot guarantee that shape; a scaffold can. A scaffold is a declarative dimensional grid: the cross product of its dimensions (a date_range, a static values list, or a from_table column sourced from a reference DataFrame), optionally filtered by density rules, becomes the table's seed rows. Scaffold columns are emitted verbatim; every other metadata column is generated per seed row.

synth = FakerSynthesizer()
synth.fit(
    metadata,
    scaffold={
        "dimensions": [
            {"column": "product_id", "from_table": "products", "from_column": "product_id"},
            {"column": "booking_date",
             "date_range": {"start": "2026-03-01", "end": "2026-03-31", "freq": "D"}},
        ],
        "density": {
            "by": {"table": "products", "column": "category"},
            "rates": {"fuel": 1.0, "beverages": 0.85, "snacks": 0.7},
            "multipliers": [{"when": "weekend", "factor": 1.25, "except": ["fuel"]}],
        },
    },
    reference_data={"products": products_df},   # backs `from_table` dimensions
)
data = synth.sample(random_state=42)            # rows defined by the grid; reproducible

When a scaffold is configured, sample_size is ignored — the grid defines the row count — and random_state makes the grid and the sampled columns reproducible. The same scaffold configuration model is used by the LLM synthesizer, so a structural definition moves between the two engines unchanged.

A scaffold can also be declared in a YAML configuration file alongside the columns, using the reserved top-level scaffold key, and loaded with load_faker_config:

columns:
  quantity:
    vartype: float
    datatype: numerical
    min: 1.0
    max: 60.0

scaffold:
  dimensions:
    - column: product_id
      from_table: products
      from_column: product_id
    - column: booking_date
      date_range: {start: "2026-03-01", end: "2026-03-31", freq: D}

from ydata.synthesizers.faker.config import load_faker_config

config = load_faker_config("faker_config.yaml")
metadata = Metadata(configuration_builder=MetadataConfigurationBuilder(config.columns))
synth.fit(metadata, scaffold=config.scaffold, reference_data={"products": products_df})

Custom Distributions

By default, numerical columns are sampled uniformly within their declared domain. The distributions argument picks the shape instead — supported: uniform, normal, lognormal, exponential, poisson, triangular, beta. Optional min/max clip the values (falling back to the column's domain), and integer columns are rounded:

synth.fit(
    metadata,
    distributions={
        "amount": {"distribution": "lognormal", "mean": 3.0, "sigma": 0.5},
        "n_items": {"distribution": "poisson", "lam": 3.0},
        "score": {"distribution": "normal", "loc": 50, "scale": 10, "min": 0, "max": 100},
    },
)

Being pure data, distributions can also live in the YAML configuration under the reserved distributions key.

Conditional Generation

A column's behavior often depends on other columns: fuel settlements are large, shop settlements small; credit notes are negative; a premium segment buys different products. The conditions argument samples a column per segment, where segments are defined by when predicates over one or more columns (equality or is-in, AND semantics). Rules are evaluated in order — first match wins — and unmatched rows fall back to default or to the column's base sampling:

synth.fit(
    metadata,
    conditions=[
        {   # numerical column: different distribution per segment
            "column": "amount",
            "rules": [
                {"when": {"category": "fuel"},
                 "distribution": {"distribution": "lognormal", "mean": 8.6, "sigma": 0.55}},
                {"when": {"category": ["shop", "commission"], "channel": "online"},
                 "distribution": {"distribution": "lognormal", "mean": 6.9, "sigma": 0.7}},
            ],
            "default": {"distribution": {"distribution": "uniform", "low": 1, "high": 50}},
        },
        {   # categorical column: different value probabilities per segment
            "column": "doc_type",
            "rules": [
                {"when": {"category": "fuel"},
                 "values": {"Invoice": 0.9, "Credit Note": 0.1}},
            ],
            "default": {"values": {"Invoice": 0.7, "Debit Note": 0.2, "Credit Note": 0.1}},
        },
    ],
)

when columns may be other metadata columns, scaffold dimensions (per-product behavior on a grid), or other conditioned columns — dependencies are ordered automatically and cycles rejected at fit() time. Conditions cannot target columns marked unique (per-segment outcomes cannot guarantee uniqueness across segments — rejected at fit()); distribution overrides on unique columns keep the constraint enforced, falling back to duplicates with a warning only when the distribution's support is too small. Conditions compose with the rest of the pipeline in a fixed order: scaffold → column sampling (with distributions) → conditioned columns → calculated features. Like distributions, conditions are pure data and can live in the YAML configuration under the reserved conditions key.

Calculated Features (Business Rules)

Columns bound by exact rules — vat = 20% of net, totals, sign conventions — should be computed, not sampled. The Faker Synthesizer accepts the same single-table calculated-features format as the RegularSynthesizer (bare column names); features are excluded from sampling and applied afterwards, in list order, so later features may consume earlier outputs:

synth.fit(
    metadata,
    calculated_features=[
        {"calculated_features": "gross_amount",
         "function": lambda qty, price: (qty * price).round(2),
         "calculated_from": ["quantity", "unit_price"]},
        {"calculated_features": "vat_value",           # chained on gross_amount
         "function": lambda gross: (gross * 0.20).round(2),
         "calculated_from": ["gross_amount"]},
    ],
)

Calculated features may consume scaffold columns but never overwrite them, and cross-table reference_keys are not supported by this single-table synthesizer. Being Python callables, calculated features cannot be declared in YAML — pass them in code. See the full guide: Calculated Features (Business Rules).

Related Materials

• Faker Synthesizer from source
• Faker Synthesizer from scratch