Skip to content

Text to Dataset (LLM synthesizer)

ydata.synthesizers.llm.LLMSynthesizer(backend=LLMProvider.WORKBENCH, model=WorkbenchModel.GPT5, subscription_key=None, charging_code=None, temperature=1.0, top_p=0.95, n_workers=10, region=None)

Generates tabular or multi-table synthetic data from a prompt-based schema (no source dataset).

Use fit(tables=...) to set the schema, then sample(sample_size=...) to generate.

Example (financial services): >>> from ydata.synthesizers import LLMSynthesizer >>> synth = LLMSynthesizer(model="gpt-5-mini-2025-08-07-dzs-eus2") >>> tables = { ... "transactions": { ... "prompt": "Credit card transactions", ... "columns": { ... "transaction_id": {"prompt": "unique id", "dtype": "string"}, ... "amount": {"prompt": "amount", "dtype": "float"}, ... }, ... } ... } >>> synth.fit(tables=tables) >>> data = synth.sample(sample_size=100)

Parameters:

Name Type Description Default
backend Union[LLMProvider, str]

LLM provider — LLMProvider.WORKBENCH (default), LLMProvider.OPENAI, LLMProvider.ANTHROPIC, LLMProvider.GEMINI, or the equivalent string alias.

 WORKBENCH
model str | None

Model identifier. Uses the backend default when omitted.

 GPT5
subscription_key str | None

API / subscription key. Falls back to the provider's standard environment variable when omitted.

 None
charging_code str | None

Workbench-only.

 None
temperature float

Sampling temperature. Default 1.0.

 1.0
top_p float

Top-p sampling. Default 0.95.

 0.95
n_workers int

Concurrency for LLM calls. Default 10.

 10
region WorkbenchRegion | str | None

Workbench-only. Azure region that serves the request. Defaults to North America (WorkbenchRegion.AMERICA).

 None

fit(tables, existing_data=None, anchor_tables=None, calculated_features=None)

Set the schema used for generation.

Parameters:

Name Type Description Default
tables dict[str, dict]

Map of table name -> {"prompt": str, "columns": {col: {"prompt", "dtype"} or {"dtype": "category", "values": [...]}}. Optional per table: "primary_key", "foreign_keys" (list of {column, referenced_table, prompt}), and "table_errors" (dict with "referential_integrity" list).

Columns may include an optional "pii" dict to guide PII generation style::

   "email": {
       "prompt": "customer email address",
       "dtype": "string",
       "pii": {
           "format": "email",
           "examples": ["john.doe@gmail.com", "alice.smith@yahoo.com"],
           "pattern": "{first}.{last}@{domain}"
       }
   }

Supported pii keys (all optional): - format: lightweight hint (email, name, phone, company, free_text). - examples: list of representative values the LLM should mimic in style. - pattern: soft template string (not enforced as regex).

Columns may include an optional "errors" dict for error injection::

   "merchant_email": {
       "prompt": "merchant contact email address",
       "dtype": "string",
       "errors": {
           "format_violation": 0.08,
           "missing": 0.04,
       }
   }

Supported errors keys (all optional, float in [0, 1]): - format_violation: fraction of rows where the LLM produces malformed values. - missing: fraction of rows set to NaN in post-processing.

Tables may include an optional "table_errors" dict::

   "table_errors": {
       "referential_integrity": [
           {"column": "card_id", "rate": 0.03}
       ]
   }

Each entry in referential_integrity must reference a declared FK column. rate is the fraction of rows whose FK value is replaced with a non-existent parent key.

 required
existing_data dict[str, DataFrame] | None

Optional. If provided, new columns are generated for these rows (e.g. enrich existing transactions).

 None
anchor_tables list[str] | dict[str, str] | None

Optional. Marks tables in existing_data as anchors: real, immutable tables the rest of the database is generated around. Anchor rows, primary keys and foreign key values are never rewritten; children are generated against the anchor's real keys, and parent tables the anchor references are derived from its actual foreign key values (one row per distinct value). Two forms are accepted::

anchor_tables=["products"]                    # PK from `tables` config
anchor_tables={"products": "product_id"}     # PK given here; the
                                             # `tables` entry becomes optional

In prompts, anchor data is reduced to the primary key only unless a foreign key explicitly whitelists columns via context_columns.

 None
calculated_features list[dict] | None

Optional. Columns computed deterministically AFTER generation instead of by the LLM — exact identities (VAT = 20% of net), sign rules, cross-table copies. Same format as MultiTableSynthesizer::

[{
    "calculated_features": "dsa.vat_value",
    "function": lambda fuel, shop: ((fuel + shop) * 0.2).round(2),
    "calculated_from": ["dsa.fuel_sales_value", "dsa.shop_sales_value"],
}]

Features are applied in list order (later features may consume earlier outputs). Calculated columns are excluded from LLM generation entirely — they cost no tokens. Features may read from anchor tables but never write into them, and may not overwrite primary/foreign keys, scaffold dimensions, or supplied existing columns.

 None

Returns:

Type Description
'LLMSynthesizer'

self

sample(sample_size=4, progress_callback=None, anchor_rows=None, random_state=None, checkpoint_dir='tmp_output', run_id=None)

Generate rows from the schema set in fit().

Parameters:

Name Type Description Default
sample_size int | dict[str, int]

Rows per root table. int (same for all) or dict[table_name, int]. Default 4.

 4
progress_callback Callable[..., Awaitable[None]] | None

Optional async callback for progress (e.g. table, rows, percentage).

 None
anchor_rows int | str | dict[str, int | str] | None

Row budget for anchor tables. Children cost one LLM call per anchor row, so anchors are subsampled upfront and the whole database is derived from the subset (parent seeds, children) — integrity always holds within the output. int (same cap for all anchors), "all" (no cap), or dict[table_name, int | "all"]. Default: cap at 1000 rows with a warning.

 None
random_state int | None

Seed for the anchor subsample and error injection, for reproducible runs. Default None.

 None
checkpoint_dir str | Path

Directory the generation progress is saved to. Default "tmp_output" beside the working directory. Each run saves into its own folder there, named after its run id: the date and time it started plus a short suffix (run-20260925-080012-3fa1), printed when the run starts. Every finished table is saved as parquet, and inside a table every batch is saved as soon as the LLM returns it.

A run that stops (an API outage that outlasts the retries, a kernel restart, Ctrl-C) is continued by calling sample() again with the same configuration: finished tables are read back and the table in progress continues from its last saved batch. Any change to the schema, sizes, seed, model settings or supplied data starts a new run instead. A run still generating in another process is never touched. Once the database is delivered, the run's folder is deleted (and checkpoint_dir too, if nothing else is left in it); on failure nothing is deleted.

 'tmp_output'
run_id str | None

The id of a stopped run to continue, as printed when it started and in its error message. Only needed when several runs of the same configuration have stopped; with one, it is continued automatically. A run whose configuration has changed since cannot be continued.

 None

Returns:

Type Description
Dataset | MultiDataset

Dataset if one table, else MultiDataset.

Raises:

Type Description
ValueError

If fit() was not called.