fw.profile

Use fw.profile once you know which dimensions matter. It composes four analyses around your chosen order: independent levels, a nested census, optional grain tests of candidate keys, and pairs evidence between the dimensions.

Use explore first when you do not yet know the useful dimensions. Call the composed functions directly when you need only one of them.

Jump to the worked example or the argument reference.

Function reference

fw.profile(
    df,
    dimensions,
    *,
    candidate_keys=None,
    features=None,
    census=None,
    pairs=True,
    top_n_applies_to="census",
    dropna=False,
    scope=None,
    missing=None,
    table_id="table",
    progress=None,
    cancel=None,
    timeout=None,
    safe_errors=None,
)

This page describes Fieldwork 0.3.2. Import the package with import fieldwork as fw. View the documented source.

Arguments

Argument	Type / default	Effect
`df`	pandas.DataFrame; required	The source table. Analysis preserves its values and row order; column labels are identified by their string form, which must be unique.
`dimensions`	iterable of str; required	Nonempty ordered census dimensions. The order controls census prefixes; pairs use the first `max_depth` dimensions.
`candidate_keys`	iterable of str, KeySpec or mapping, or None; `None`	Keys for a grain section; `None` skips grain. A string is a single-column candidate; use `fw.KeySpec("name", ("column_a", "column_b"))` or `{"name": ..., "columns": [...]}` for a composite candidate. Grain tests each candidate against every column.
`features`	iterable of str or None; `None`	Columns for the independent levels section; `None` uses the dimensions.
`census`	mapping or None; `None`	Census options (`fieldwork.typing.CensusOptions`), described below, except `dropna`. Levels share `top_n`, `max_levels`, `min_count`, and `schema`.
`pairs`	mapping or bool; `True`	Pair options (`fieldwork.typing.PairOptions`), described below. `True` uses the defaults and `False` skips the pairs section.
`top_n_applies_to`	“census” or “both”; `"census"`	With `top_n_mode="pre"`, pairs always analyze the census pre-selection; `"both"` makes grain analyze it too. `"both"` requires pre mode and `top_n`. Levels always count the full input.
`dropna`	bool; `False`	`False` counts missing as a value. `True` excludes missing values in every section, each on its own complete cases: per feature in levels, across active dimensions in census, and per key and target or dimension pair in grain and pairs. Section denominators can differ.
`scope`	Scope or None; `None`	A saved row selection tied to this dataframe; `None` uses every row. Every section analyzes these rows.
`missing`	mapping or None; `None`	Additional missing-value sentinels per column, such as `{"score": [-999]}`. The source is not modified.
`table_id`	str; `"table"`	Table name recorded in each section.
`progress`	bool or callable; `None`	Silent by default; `True` displays progress, or pass a synchronous `ProgressEvent` callback. See runtime controls.
`cancel`	CancellationToken or None; `None`	Cooperative cancellation token; cancellation raises `AnalysisCancelled` without returning a partial result.
`timeout`	float or None; `None`	Finite nonnegative seconds from call start; `None` disables the cooperative deadline. A running pandas/NumPy work item must finish before cancellation.
`safe_errors`	bool or None; `None`	`True` raises any other failure as `AnalysisError`, naming the operation, phase and column without echoing the original message, which can quote cell values. See runtime controls.

Census options

Keys of the census mapping, with the same meanings as in census. Omitted keys use these defaults.

Key	Type / default	Effect
`top_n`	positive int or None; `None`	Maximum selected levels per dimension (globally by default, or per parent). `None` disables this selector. Its population effect depends on `top_n_mode`.
`top_n_mode`	“post” or “pre”; `"post"`	`"post"` limits reported branches after counting. `"pre"` first restricts rows to the selected levels, counts that cohort, and shares it with pairs as a Scope named “census top_n cohort”.
`top_n_per_parent`	bool; `False`	`False` selects each dimension’s most frequent levels globally; `True` chooses them within each parent prefix.
`min_retained_fraction`	float in [0, 1]; `0.01`	Warn when pre-selection retains less than this fraction of eligible rows. A warning threshold, not a row filter.
`max_depth`	positive int or None; `None`	Use only this many leading dimensions in census and pairs.
`max_levels`	nonnegative int or None; `100`	Output cap per feature in levels and per parent in census. `None` removes the cap; omitted rows remain in the denominators.
`max_nodes`	nonnegative int or None; `10000`	Maximum emitted non-root census nodes. The root and omitted mass remain; `None` removes the cap.
`min_count`	nonnegative int; `1`	Minimum count for a reported level or census branch. Omitted mass remains recorded.
`schema`	mapping or None; `None`	Caller-supplied column roles: `id`, `categorical`, `continuous`, or `unknown`. Roles annotate evidence; they do not transform values.

Passing dropna in census raises ValueError: use the dropna argument, which applies to every section.

Pair options

Keys of the pairs mapping, with the same meanings as in pairs.

Key	Type / default	Effect
`include_absence`	bool; `False`	Include absence evidence: combinations of observed or declared levels that never occur together.
`reference_domains`	mapping or None; `None`	Declared possible levels per dimension, for absence. Unspecified dimensions use their observed levels.
`pair_contexts`	iterable of mappings or None; `None`	Additional equality-filter contexts, for example `[{"season": "winter"}]`. The global context comes first. Context columns must be disjoint from each analyzed pair.
`limits`	mapping or None; `None`	Work budgets (`fieldwork.typing.PairLimits`): `max_pairs` (`15`) dimension pairs in input order, `max_contexts` (`32`) contexts including the global one, and `max_absence_cells` (`1000`) stored absent cells. `None` values remove a budget.

Returns and output choices

Returns a fieldwork.Result with kind="profile" (schema 2.0). sections holds the levels, census, grain, and pairs results; an unrequested section has status not_requested. profile.section(name) returns one section as a Result. warnings collects the levels and census warnings. The profile records the source, scope, and missing conventions of its census.

Use plaintext for a text report of every section. SVG and HTML draw one section and default to grain; pass section="census" (or another computed section) when grain was not requested. to_dict() retains analytical evidence.

Interpretation and boundaries

Each section keeps its own population record: levels count each feature on its own, census and pairs follow any pre-selection, and each grain test counts its own complete cases. Read a section’s counts against its own scope and population fields, not against another section’s.

Unknown census or pair option keys raise TypeError before any section runs; invalid values or combinations, such as top_n_applies_to="both" without pre-selection, raise ValueError. A profile cannot be recomputed as a whole; recompute its sections.

Walkthrough: which requests could carry a photo?

The municipal operations team from the explore walkthrough found that photos depend on intake channel: three of four app requests have one, the phone desk’s none. Before calling that an upload problem, they want to know whether each channel simply receives different work, and which fields belong to a channel’s own form.

The complete runnable example follows the same steps.

Rebuild the delivery

These are the same twelve synthetic requests as in the explore walkthrough; each row represents one request. Every channel contributes four requests; six concern potholes and six concern lighting. None means the field was not populated in this delivery. It does not tell us whether the source form offered that field or whether a resident chose to leave it empty.

import pandas as pd
import fieldwork as fw

requests = pd.DataFrame(
    [
        ("app", "pothole", "north", "closed", "photo", None),
        ("app", "pothole", "north", "closed", "photo", None),
        ("app", "pothole", "south", "open", "photo", None),
        ("app", "lighting", "south", "open", None, None),
        ("phone", "pothole", "north", "closed", None, "callback"),
        ("phone", "pothole", "south", "open", None, "callback"),
        ("phone", "lighting", "north", "closed", None, "callback"),
        ("phone", "lighting", "south", "open", None, "callback"),
        ("web", "pothole", "north", "closed", "photo", None),
        ("web", "lighting", "north", "closed", None, None),
        ("web", "lighting", "south", "open", None, None),
        ("web", "lighting", "south", "open", None, None),
    ],
    columns=[
        "channel",
        "service",
        "district",
        "status",
        "attachment",
        "callback",
    ],
)
requests.insert(0, "request_id", range(1001, 1013))

attachment records the presence of a photo, while callback records a request for a return call. The repeated values are categories, not file names or phone numbers. The unique request identifier remains in the delivery, but no dimension below splits on it: that would give every request its own branch.

Break the delivery down in a chosen order

Specify channel → service → attachment to compare the service mix inside each intake channel. Offer channel as a candidate key to learn which fields the channel alone determines. Pair evidence is unnecessary for this question, so turn it off.

focused = fw.profile(
    requests,
    ["channel", "service", "attachment"],
    candidate_keys=["channel"],
    pairs=False,
)
print(fw.render_plaintext(focused.section("census")))

Fieldwork · census
Census (computed)
  Input population · 12 evaluated / 12 input rows · 0 missing, 0 restricted exclusions
  path: channel > service > attachment
  total: 12 rows
    channel=app: 4 rows
      service=pothole: 3 rows
        attachment=photo: 3 rows
      service=lighting: 1 row
        attachment=<NA>: 1 row
    channel=phone: 4 rows
      service=lighting: 2 rows
        attachment=<NA>: 2 rows
      service=pothole: 2 rows
        attachment=<NA>: 2 rows
    channel=web: 4 rows
      service=lighting: 3 rows
        attachment=<NA>: 3 rows
      service=pothole: 1 row
        attachment=photo: 1 row

The app has three pothole requests and one lighting request; the web has the reverse mix. Every app and web pothole request includes a photo, and none of their lighting requests does. Their different overall photo fractions therefore coexist with the same pattern within each service in this delivery.

The phone desk contributes two pothole and two lighting requests, all without photos. That is a concrete follow-up for the intake team: check whether the phone workflow can collect or attach photos before describing these four blanks as collection failures.

Ask what the channel alone decides

The grain section tested channel against every column. List the columns it determines exactly:

grain = focused.section("grain")
print(
    [test["target"] for test in grain["dependencies"] if test["holds"]]
)

['callback']

Only callback is fixed by the channel: every phone request asks for a callback, and no other request does. Photos are not: app and web requests vary with the service. The callback field behaves like part of the phone desk’s form, so its blanks elsewhere are expected rather than missing answers.

Draw the breakdown

A profile’s figures default to its grain section. Ask for the census to compare the nested counts:

figure = fw.render_svg(focused, section="census")

The bars show each prefix’s share of all twelve requests. Read the nested labels to compare photos within a channel and service; a bar’s length is not a within-parent percentage. The two kinds of share remain available in the census node records.

See what complete-case filtering would erase

A tempting cleanup is to drop rows with missing dimensions. Run the same profile with dropna=True and read the census population before interpreting it:

complete = fw.profile(
    requests,
    ["channel", "service", "attachment"],
    dropna=True,
    pairs=False,
)
tree = complete.section("census")["tree"]
print("Rows counted:", tree["evaluated_rows"])
print("Rows excluded for missing values:", tree["missing_excluded_rows"])

Rows counted: 4
Rows excluded for missing values: 8

The census now describes four requests, not twelve. All phone requests and all lighting requests have disappeared because their attachment is missing. Reading this tree as the original delivery would hide exactly the intake differences we set out to investigate. The census records the eight excluded rows; the absence of a phone branch here is not evidence that no phone requests arrived.

Keep missing branches in the initial investigation. Next, compare the observed channel-and-service combinations with the actual intake forms, and repeat the same profile on a larger delivery. These twelve rows locate a useful question; they do not establish that the observed pattern is a lasting collection rule.

Save the figure

Save the figure in your working directory:

from pathlib import Path

Path("profile.svg").write_text(figure, encoding="utf-8")

Continue exploring

fw.explore, fw.census, fw.grain, fw.pairs, fw.levels. See results and configuration for serialization and result-method contracts.