fw.infer_schema

Use fw.infer_schema to start a conversation about column roles. It proposes id, continuous, categorical, or unknown, with the observed cardinality, dtype, and missingness behind each suggestion.

The purpose is review. A proposal does not transform a dataframe, select an analysis on your behalf, or certify that a column is a meaningful identifier.

Jump to the worked example or the argument reference.

Function reference

fw.infer_schema(
    df,
    candidate_keys=None,
    *,
    scope=None,
    missing=None,
    table_id="table",
    progress=None,
    cancel=None,
    timeout=None,
    safe_errors=None,
)

This page describes Fieldwork 0.3.2. Import the package with import fieldwork as fw. View the documented source.

Arguments

Argument	Type / default	Effect
df	pandas.DataFrame; required	The source table. Analysis preserves its values and row order; duplicate column labels are rejected.
candidate_keys	iterable of column names, KeySpec or mappings, or None; None	Optional explicit candidates passed to exact grain analysis; a mapping such as {"name": "part", "columns": ["lot", "sample_id"]} describes a composite key. Adds dependency evidence to proposals; it does not change the role heuristic. None leaves functional-dependency evidence marked not_evaluated.
scope	Scope or None; None	A saved row selection tied to this dataframe; None uses every row. Construct with fw.Scope.from_positions(df, positions). Pass the original dataframe with its scope, rather than a reordered or changed copy.
missing	mapping or None; None	Column names mapped to lists of sentinel values, for example {"mass_g": [-1]}. Native nulls are always missing. Sentinels join the missing level of their column without replacing source values.
table_id	str; 'table'	Nonempty table identity recorded with the evidence. It does not bypass source-fingerprint checks.
progress	bool or callable; None	Silent by default; True displays progress, or pass a synchronous ProgressEvent callback. See runtime controls.
cancel	CancellationToken or None; None	Cooperative cancellation token; cancellation raises AnalysisCancelled without returning a partial result.
timeout	float or None; None	Finite nonnegative seconds from call start; None disables the cooperative deadline. A running pandas/NumPy work item must finish before cancellation.
safe_errors	bool or None; None	True raises any other failure as AnalysisError, naming the operation, phase and column without echoing the original message, which can quote cell values. See runtime controls.

Returns and output choices

Returns a fieldwork.Result with kind="schema_proposal" (schema 2.0). proposals holds each column name, proposed_role, reasons, and fd_evidence. suggested_dimensions and suggested_keys are column names to review. Like every result, it records source, scope, missing_convention, and parameters. Use dictionary access, plaintext, visualization_data, or the SVG and HTML renderers, which draw one card per proposed column. recompute(df) reruns the proposal on the verified source.

Interpretation and boundaries

The stable heuristic checks in order: if a nonempty column has as many distinct encoded levels as rows, propose id; otherwise numeric cardinality greater than min(20, rows / 2) proposes continuous; cardinality at most max(20, int(rows * 0.1)) proposes categorical; otherwise propose unknown. Native missing contributes a distinct encoded level. An ID-like name adds a reason, but does not override these rules.

A unique numeric measurement can therefore be proposed as an ID. Small samples can make accidental uniqueness look convincing. Review the reasons against domain knowledge before supplying a schema to another function. Unknown candidate columns raise KeyError; invalid candidates raise ValueError.

Walkthrough: review a parts-inspection export before grouping it

A small manufacturer sends eight inspected parts from three production lots. Before exploring the export, you need to decide which columns describe groups, which identify parts, and which record measurements. A unique value is not enough to settle that decision: every measured mass happens to differ in this file.

Use fw.infer_schema to make the evidence visible, then write the configuration that reflects the meaning of the columns.

Download the runnable example.

Construct an export small enough to audit

Each row is one inspected part. Lots L10 and L20 have three parts each; L30 has two. Machine A made the parts in L10 and L30, while machine B made those in L20. The timestamps record measurement events, not persistent part identifiers.

import pandas as pd
import fieldwork as fw

parts = pd.DataFrame(
    {
        "sample_id": [f"S{i:02d}" for i in range(1, 9)],
        "lot": ["L10", "L10", "L10", "L20", "L20", "L20", "L30", "L30"],
        "machine": ["A", "A", "A", "B", "B", "B", "A", "A"],
        "mass_g": [
            10.01,
            10.03,
            9.98,
            10.05,
            10.02,
            9.99,
            10.04,
            10.00,
        ],
        "measured_at": pd.date_range(
            "2026-05-04 08:00", periods=8, freq="min"
        ),
    }
)

All eight rows are complete. There are no missing values to account for in the cardinality ratios or the dependency checks below.

Read suggestions alongside their reasons

Start with a table of proposed roles.

proposal = fw.infer_schema(parts)
suggestions = pd.json_normalize(proposal["proposals"])
print(
    suggestions[["column", "proposed_role"]].to_string(
        index=False
    )
)

     column proposed_role
  sample_id            id
        lot   categorical
    machine   categorical
     mass_g            id
measured_at            id

sample_id, mass_g, and measured_at each have eight distinct values in eight rows: a cardinality ratio of 1.0. The uniqueness heuristic therefore proposes id for all three. It does not establish that mass or time is a suitable join key. Another export could contain equal masses or simultaneous measurements.

The categorical suggestions for lot and machine agree with the export’s meaning. The useful outcome is a review list, not a schema to accept wholesale. The result also retains dtype and name-hint reasons, which you can read in each proposal’s reasons list. Without candidate keys, dependency evidence is not_evaluated.

Add candidate keys to ask a more specific question

Does each lot have one machine in this export? Does it have one mass? Supply both the part identifier and the lot as candidates. A candidate here is a proposed determinant to inspect; it need not uniquely identify a row.

with_keys = fw.infer_schema(parts, candidate_keys=["sample_id", "lot"])
by_column = {item["column"]: item for item in with_keys["proposals"]}
print("target | candidate | holds | evaluated groups")
for column in ["machine", "mass_g"]:
    for evidence in by_column[column]["fd_evidence"]["keys"]:
        print(
            f"{column} | {evidence['name']} | {evidence['holds']} | "
            f"{evidence['evaluated_groups']}"
        )

target | candidate | holds | evaluated groups
machine | sample_id | True | 8
machine | lot | True | 3
mass_g | sample_id | True | 8
mass_g | lot | False | 3

These are groups of candidate values, not numbers of matching rows. All three lot groups contain one machine value, so lot → machine holds in the observed data. Each lot contains different masses, so lot → mass_g fails. That variation is expected for measurements of separate parts; it is not evidence of bad data.

The eight sample groups contain one row apiece. Their successful checks cannot test agreement between repeated records of the same sample. Candidate-key evidence adds context without changing any proposed role. Use fw.grain when you need the full dependency results, including violating and singleton-group counts.

Pass a reviewed schema into the next analysis

Keep sample_id as the identifier and treat mass as continuous. This release’s role vocabulary has no temporal role, so leave measured_at as unknown and exclude it from this categorical count. The choice preserves its meaning as a timestamp without pretending it is a part identifier.

reviewed_schema = {
    "sample_id": "id",
    "lot": "categorical",
    "machine": "categorical",
    "mass_g": "continuous",
    "measured_at": "unknown",
}
dimensions = ["lot", "machine"]
counts = fw.levels(parts, features=dimensions, schema=reviewed_schema)
print(fw.render_plaintext(counts))

Fieldwork · levels
Levels (computed)
  lot: 3/3 levels; 8/8 evaluated rows reported
    L10: 3 rows
    L20: 3 rows
    L30: 2 rows
  machine: 2/2 levels; 8/8 evaluated rows reported
    A: 5 rows
    B: 3 rows

Machine A accounts for five of the eight inspected parts, not five lots. Both features use all eight rows, and every observed level is reported. The explicit features selection chooses the columns; passing schema also supplies their reviewed roles. Schema inference itself neither changes the frame nor configures this later analysis.

Before using lot → machine to combine future exports, check a larger file that includes repeated lots across days. This snapshot supports the relationship for three observed lots; it cannot establish that a lot will never move between machines. Keep that operational question separate from deciding that mass is a measurement.

Continue exploring

fw.grain, fw.levels, fw.explore. See results and configuration for serialization and result-method contracts.