Skip to content

Calculated Features (Business Rules)

Overview

Calculated features — also known as business rules — are columns computed deterministically from other columns instead of being learned or generated. They exist because some columns are not statistical patterns but identities and rules: a VAT amount that must be exactly 20% of the net value, a total that must equal quantity * unit_price, a sign convention that follows a document type, or a denormalized copy of a value that lives in another table.

Generating such columns statistically (or with an LLM) produces values that are plausible but wrong — a reconciliation check or a unit test will catch a VAT that is off by pennies. Calculated features guarantee these columns are exactly right, by computing them with your own Python function after the rest of the data is produced.

How it works

Across all supported synthesizers the flow is the same:

  1. Calculated columns are excluded from training / generation.
  2. The rest of the table is synthesized as usual.
  3. Your functions are applied to the synthesized data, in the order you declared them, producing the calculated columns.

This means calculated columns never degrade the quality of the generation model — and in the LLM synthesizer they never enter a prompt, so they cost zero tokens.

Benefits

  • Exact identities: arithmetic relationships (totals, taxes, margins) hold to the last decimal, always.
  • Enforced conventions: sign rules, flags, derived statuses are rules, not probabilities.
  • Cross-table consistency: denormalized copies are joined from the source table by key, never invented.
  • Cheaper generation: fewer columns to learn or generate — in the LLM synthesizer, derived columns consume no tokens at all.

Configuration format

A calculated feature is a dictionary with three required keys, addressed as table_name.column_name:

{
    "calculated_features": "dsa.vat_value",                # output column(s)
    "function": lambda fuel, shop: ((fuel + shop) * 0.2).round(2),
    "calculated_from": ["dsa.fuel_sales_value", "dsa.shop_sales_value"],
}
  • calculated_features — one output column, or a list of outputs (all in the same table).
  • function — a Python callable. It receives one pandas.Series per calculated_from entry, positionally, and returns the output column(s): a Series (or 1-D array) for a single output, or a tuple/list of Series, a 2-D array, or a DataFrame for multiple outputs.
  • calculated_from — the input columns, in the order the function expects them.

Cross-table features

When the inputs live in another table, add reference_keys to describe the join between the source table and the feature's table:

{
    "calculated_features": "orders.customer_segment",       # copied into orders
    "function": lambda segment, customer_id: pd.DataFrame(
        {"customer_segment": segment, "customer_id": customer_id}
    ),
    "calculated_from": ["customers.segment"],
    "reference_keys": {
        "source": "customers.customer_id",                  # key on the source table
        "target": "orders.customer_id",                     # key on the feature table
    },
}

The function receives the source columns plus the source key column(s); the result is merged into the feature table by the declared keys — the copy can never disagree with the source.

Chaining

Features are applied in list order, and later features may consume the outputs of earlier ones:

calculated_features = [
    {"calculated_features": "dsa.net_value",
     "function": lambda fuel, shop: fuel + shop,
     "calculated_from": ["dsa.fuel_sales_value", "dsa.shop_sales_value"]},
    {"calculated_features": "dsa.vat_value",                 # consumes net_value
     "function": lambda net: (net * 0.20).round(2),
     "calculated_from": ["dsa.net_value"]},
]

Configurations are validated upfront: unknown input columns, outputs targeting a different table per feature, or invalid chains fail at fit() time with a descriptive error — never mid-generation.

Usage per synthesizer

RegularSynthesizer / TimeSeriesSynthesizer

from ydata.synthesizers import RegularSynthesizer

synth = RegularSynthesizer()
synth.fit(
    dataset,
    metadata=metadata,
    calculated_features=[
        {"calculated_features": "total",
         "function": lambda qty, price: (qty * price).round(2),
         "calculated_from": ["quantity", "unit_price"]},
    ],
)
sample = synth.sample(n_samples=1000)

The calculated columns are dropped before training and recomputed on every sample.

MultiTableSynthesizer

Identical configuration, with table.column addressing and support for cross-table reference_keys:

synth = MultiTableSynthesizer()
synth.fit(multidataset, metadata=metadata, calculated_features=calculated_features)

FakerSynthesizer

Same single-table convention as RegularSynthesizer (bare column names). Features are excluded from sampling and computed afterwards — and they compose with the structural scaffold: scaffold columns may be consumed but never overwritten. Cross-table reference_keys are not supported (single table).

from ydata.synthesizers import FakerSynthesizer

synth = FakerSynthesizer()
synth.fit(
    metadata,
    scaffold=scaffold,                      # optional structural grid
    reference_data={"products": products_df},
    calculated_features=[
        {"calculated_features": "vat_value",
         "function": lambda gross: (gross * 0.20).round(2),
         "calculated_from": ["gross_amount"]},
    ],
)
sample = synth.sample(random_state=42)

LLMSynthesizer

The LLM synthesizer accepts the same format on fit(). Calculated columns are excluded from generation entirely — they never appear in a prompt — and the features are applied after key post-processing, so cross-table joins resolve against final key values (including real keys from anchor tables):

from ydata.synthesizers.llm import LLMSynthesizer

synth = LLMSynthesizer()
synth.fit(
    tables=tables,
    existing_data={"products": products_df},
    anchor_tables={"products": "product_id"},
    calculated_features=[
        # exact identity, computed after generation
        {"calculated_features": "daily_bookings.gross_amount",
         "function": lambda qty, price: (qty * price).round(2),
         "calculated_from": ["daily_bookings.quantity", "daily_bookings.unit_price_applied"]},
        # denormalized copy joined from the (real) anchor table
        {"calculated_features": "daily_bookings.unit_price_applied",
         "function": lambda price, pid: pd.DataFrame(
             {"unit_price_applied": price, "product_id": pid}),
         "calculated_from": ["products.unit_price"],
         "reference_keys": {"source": "products.product_id",
                            "target": "daily_bookings.product_id"}},
    ],
)
data = synth.sample()

Guard rails in the LLM synthesizer

Calculated features may read from anchor tables but never write into them (anchors are immutable), and may not overwrite primary keys, foreign keys, scaffold dimension columns, or columns supplied through existing_data. Violations are rejected at fit() time.

When to use calculated features vs. prompts or training

Use a calculated feature whenever the column has a single correct answer given the other columns: arithmetic identities, sign conventions, mandatory copies, deterministic flags. Use generation (statistical or LLM) when the column carries new information: names, free text, amounts, categories, dates. A useful test — if a validation script could check the column, a calculated feature should produce it.

See also the API reference: Calculated Features API.