Investigate families and exceptions

Start with the dataframe from Getting started. missingness describes availability without changing the data.

availability = fw.missingness(
    df, by=["site"], entity="exam", min_implication=0.75,
    max_pairs=200, example_limit=10,
)
print(availability.to_frame("availability"))
print(availability.to_frame("entities"))
print(availability.to_frame("findings"))

Look for availability_family, similar_availability, presence_implication, and mutually_exclusive patterns. Identical families include always-present or always-missing fields. Similarity uses co-presence Jaccard, so two mostly absent fields do not look similar merely because both are absent.

Choose a finding ID from the table and call availability.inspect(df, finding_id, exceptions=True). This returns the saved representative rows. Its exceptions record reports the total matching count, number omitted, limit and selection method. Use availability.select(df, finding_id, exceptions=True) for a reusable scope containing every matching exception, or inspect(..., all_matches=True) for all matching source rows. Saved example limits never restrict that population.

Declare sentinels deliberately

availability = fw.missingness(df, missing={"image_2": [-999]})

Native missing values are recognized automatically. An arbitrary sentinel is ordinary data until declared. Numeric sentinels match integer and float equivalents; declarations remain in the exported evidence.

Compare row and entity populations

Rows and entities answer different questions. A repeated exam with many rows contributes many row observations but one entity observation. entities records the number of entities with any, all, one, some or no populated rows for each field. Incomplete entity keys are excluded and counted in coverage. one and all overlap for populated singleton entities.

Use by=["site", "modality"] when those columns exist to summarize each joint context. coverage reports how many contexts and pairs were evaluated. A feature outside a pair budget has not been ruled out.

Common inspection errors

A changed or reordered dataframe raises a source-mismatch error. Inspect against the original dataframe or rerun the analysis. Duplicate column names are rejected; duplicate row indexes are supported. Discovery requires string column names and supported scalar cell values.

Select a signature and continue

signature = next(s for s in availability["signatures"] if "image_2" in s["absent"])
selected = availability.select(df, signature["finding_id"], name="missing second image")
paths = fw.suggest_paths(df, scope=selected, features=["site", "exam", "image_2"])
if paths.best is not None:
    tree = paths.best.census(df)

Whole context summaries and entity summaries also provide finding IDs. Context selection recovers every eligible row in that context. Entity pattern selection recovers all scoped rows belonging to matching entities, including rows where the feature is absent. Typed context predicates distinguish, for example, numeric and string labels.

For equal entity weights, use fw.missingness(df, entity="exam", unit="entities", entity_presence="any"). Every populated distinct exam key contributes once. any means at least one populated row; all requires every row. With the default unit="rows", supplying entity adds entity summaries while availability still counts rows.

The overview connects several kinds of evidence. overview.relationships("image_1") returns typed connections and their supporting finding IDs; follow them with overview.inspect or overview.select. Shared names, availability, and value dependencies retain their distinct meanings.