Skip to content

Calculated Features (Business Rules)

Overview

Calculated features — also known as business rules — are columns computed deterministically from other columns instead of being learned or generated. They exist because some columns are not statistical patterns but identities and rules: a VAT amount that must be exactly 20% of the net value, a total that must equal quantity * unit_price, a sign convention that follows a document type, or a denormalized copy of a value that lives in another table.

Generating such columns statistically (or with an LLM) produces values that are plausible but wrong — a reconciliation check or a unit test will catch a VAT that is off by pennies. Calculated features guarantee these columns are exactly right, by computing them with your own Python function after the rest of the data is produced.

How it works

Across all supported synthesizers the flow is the same:

  1. Calculated columns are excluded from training / generation.
  2. The rest of the table is synthesized as usual.
  3. Your functions are applied to the synthesized data, in the order you declared them, producing the calculated columns.

This means calculated columns never degrade the quality of the generation model — and in the LLM synthesizer they never enter a prompt, so they cost zero tokens.

Benefits

  • Exact identities: arithmetic relationships (totals, taxes, margins) hold to the last decimal, always.
  • Enforced conventions: sign rules, flags, derived statuses are rules, not probabilities.
  • Cross-table consistency: denormalized copies are joined from the source table by key, never invented.
  • Cheaper generation: fewer columns to learn or generate — in the LLM synthesizer, derived columns consume no tokens at all.

Configuration format

A calculated feature is a dictionary with three required keys, addressed as table_name.column_name:

{
    "calculated_features": "dsa.vat_value",                # output column(s)
    "function": lambda fuel, shop: ((fuel + shop) * 0.2).round(2),
    "calculated_from": ["dsa.fuel_sales_value", "dsa.shop_sales_value"],
}
  • calculated_features — one output column, or a list of outputs (all in the same table).
  • function — a Python callable, or a pre-defined business rule. It receives one pandas.Series per calculated_from entry, positionally, and returns the output column(s): a Series (or 1-D array) for a single output, or a tuple/list of Series, a 2-D array, or a DataFrame for multiple outputs.
  • calculated_from — the input columns, in the order the function expects them.

Cross-table features

When the inputs live in another table, add reference_keys to describe the join between the source table and the feature's table:

{
    "calculated_features": "orders.customer_segment",       # copied into orders
    "function": lambda segment, customer_id: pd.DataFrame(
        {"customer_segment": segment, "customer_id": customer_id}
    ),
    "calculated_from": ["customers.segment"],
    "reference_keys": {
        "source": "customers.customer_id",                  # key on the source table
        "target": "orders.customer_id",                     # key on the feature table
    },
}

The function receives the source columns plus the source key column(s); the result is merged into the feature table by the declared keys — the copy can never disagree with the source.

Chaining

Features are applied in list order, and later features may consume the outputs of earlier ones:

calculated_features = [
    {"calculated_features": "dsa.net_value",
     "function": lambda fuel, shop: fuel + shop,
     "calculated_from": ["dsa.fuel_sales_value", "dsa.shop_sales_value"]},
    {"calculated_features": "dsa.vat_value",                 # consumes net_value
     "function": lambda net: (net * 0.20).round(2),
     "calculated_from": ["dsa.net_value"]},
]

Configurations are validated upfront: unknown input columns, outputs targeting a different table per feature, or invalid chains fail at fit() time with a descriptive error — never mid-generation.

Pre-defined business rules

The arithmetic most finance and accounting tables need is already written. BusinessRule is a catalog of vectorised functions you pass as function, exactly like your own:

from ydata.calculated import BusinessRule

calculated_features = [
    {"calculated_features": "net_amount",               # quantity × unit_price − discount
     "function": BusinessRule.LINE_AMOUNT,
     "calculated_from": ["quantity", "unit_price", "discount"]},
    {"calculated_features": "vat_amount",               # net_amount × vat_rate
     "function": BusinessRule.PRODUCT,
     "calculated_from": ["net_amount", "vat_rate"]},
    {"calculated_features": "gross_amount",             # net_amount + vat_amount
     "function": BusinessRule.SUM,
     "calculated_from": ["net_amount", "vat_amount"]},
]
  • Argument order. Columns in calculated_from follow the rule's signature, as with any function. BusinessRule.LINE_AMOUNT.description prints the formula and its typical uses.
  • Constants. Fix a parameter instead of reading it from a column by calling the rule with it: BusinessRule.GROSS_UP(rate=0.23) with "calculated_from": ["net_amount"].
  • Names. A rule can also be given by name, e.g. "function": "line_amount".
  • Validation. The number of input columns is checked at fit() time — three columns for RATIO, for instance, is rejected before anything is generated.
  • Mixing. Pre-defined rules and your own functions can be combined and chained freely in the same list.
  • Across tables. With reference_keys, a rule is computed on the source table's rows and copied onto the feature table by key — e.g. BusinessRule.GROSS_UP over products.price and products.vat_rate, copied onto orders. The source keys must be unique (one value per source row), and the per-entity rules below cannot be used across tables.

Catalog

Rates are fractions (0.23, not 23).

Rule Formula Used for
Arithmetic
SUM a + b + … gross = net + tax, employer cost, totals of parts
DIFFERENCE a − b − … net = gross − tax, net pay, margin, net book value, available credit
PRODUCT a × b × … tax = net × rate, market value, FX conversion, expected loss
RATIO a / b effective rate, LTV, DTI, utilisation
LINEAR_COMBINATION Σ coefficient × value signed sums, e.g. closing = opening + inflows − outflows with coefficients=[1, 1, -1]
Line amounts and per-unit figures
LINE_AMOUNT quantity × unit_price − Σ discounts invoice and order lines
LINE_AMOUNT_WITH_CHARGES quantity × unit_price + Σ charges trade settlement amounts, lines with surcharges
PROFIT_AND_LOSS quantity × (price − cost) realised and unrealised PnL, gross profit per line
NET_PER_UNIT (amount − deduction) / units straight-line depreciation, NAV per share
Rates
GROSS_UP net × (1 + rate) gross from net and tax rate, price from cost and markup
NET_DOWN gross × (1 − rate) net after withholding or a percentage discount
BACK_OUT_RATE gross / (1 + rate) net from a tax-inclusive amount
Finance formulas
COMPOUND_INTEREST principal × ((1 + rate)^periods − 1) interest earned
ANNUITY_PAYMENT P·r(1 + r)^n / ((1 + r)^n − 1) loan instalments, mortgage payments
SHARE_OF_GROUP value / Σ value within the group portfolio weights, allocation keys
Per entity, in order
CUMULATIVE_SUM opening + cumulative sum of values year-to-date figures, accumulated depreciation
RUNNING_BALANCE opening + cumulative sum of (debit − credit) ledger and account balances
DECLINING_BALANCE opening − cumulative sum of reductions outstanding loan principal, remaining book value
PREVIOUS previous value opening balance = prior period's closing
CHANGE value − previous value balance movements
PCT_CHANGE (value − previous) / previous period returns, growth rates

The per entity rules are computed within each entity (account, loan, position), in order. They need the synthesizer to know the entity and the order — a sequence in the FakerSynthesizer, the entities declared in the Metadata for trained synthesizers — and are computed across all entities at once, with no per-entity Python.

Pre-defined rule or your own function?

Use a business rule when one matches the computation. Write your own function for lookups, conditions, string composition, or any formula outside the catalog — both go in the same list.

Usage per synthesizer

RegularSynthesizer / TimeSeriesSynthesizer

from ydata.calculated import BusinessRule
from ydata.synthesizers import RegularSynthesizer

synth = RegularSynthesizer()
synth.fit(
    dataset,
    metadata=metadata,
    calculated_features=[
        {"calculated_features": "total",
         "function": BusinessRule.PRODUCT,
         "calculated_from": ["quantity", "unit_price"]},
    ],
)
sample = synth.sample(n_samples=1000)

The calculated columns are dropped before training and recomputed on every sample.

MultiTableSynthesizer

Identical configuration, with table.column addressing and support for cross-table reference_keys:

synth = MultiTableSynthesizer()
synth.fit(multidataset, metadata=metadata, calculated_features=calculated_features)

FakerSynthesizer

Same single-table convention as RegularSynthesizer (bare column names). Features are excluded from sampling and computed afterwards — and they compose with the structural scaffold: scaffold columns may be consumed but never overwritten. Cross-table reference_keys are not supported (single table).

from ydata.synthesizers import FakerSynthesizer

synth = FakerSynthesizer()
synth.fit(
    metadata,
    scaffold=scaffold,                      # optional structural grid
    reference_data={"products": products_df},
    calculated_features=[
        {"calculated_features": "vat_value",
         "function": lambda gross: (gross * 0.20).round(2),
         "calculated_from": ["gross_amount"]},
    ],
)
sample = synth.sample(random_state=42)

LLMSynthesizer

The LLM synthesizer accepts the same format on fit(). Calculated columns are excluded from generation entirely — they never appear in a prompt — and the features are applied after key post-processing, so cross-table joins resolve against final key values (including real keys from anchor tables):

from ydata.synthesizers.llm import LLMSynthesizer

synth = LLMSynthesizer()
synth.fit(
    tables=tables,
    existing_data={"products": products_df},
    anchor_tables={"products": "product_id"},
    calculated_features=[
        # exact identity, computed after generation
        {"calculated_features": "daily_bookings.gross_amount",
         "function": lambda qty, price: (qty * price).round(2),
         "calculated_from": ["daily_bookings.quantity", "daily_bookings.unit_price_applied"]},
        # denormalized copy joined from the (real) anchor table
        {"calculated_features": "daily_bookings.unit_price_applied",
         "function": lambda price, pid: pd.DataFrame(
             {"unit_price_applied": price, "product_id": pid}),
         "calculated_from": ["products.unit_price"],
         "reference_keys": {"source": "products.product_id",
                            "target": "daily_bookings.product_id"}},
    ],
)
data = synth.sample()

A runnable end-to-end version of this — a real anchor table, a cross-table copy joined on its real keys, a chained feature, and assertions that both reconcile — is in examples/synthesizers/llm/llm_synthesizer_anchor_database.py.

Guard rails in the LLM synthesizer

Calculated features may read from anchor tables but never write into them (anchors are immutable), and may not overwrite primary keys, foreign keys, scaffold dimension columns, or columns supplied through existing_data. Violations are rejected at fit() time.

When to use calculated features vs. prompts or training

Use a calculated feature whenever the column has a single correct answer given the other columns: arithmetic identities, sign conventions, mandatory copies, deterministic flags. Use generation (statistical or LLM) when the column carries new information: names, free text, amounts, categories, dates. A useful test — if a validation script could check the column, a calculated feature should produce it.

See also the API reference: Calculated Features API.