Your first investigation

Fieldwork helps you understand a pandas dataframe: how its rows are grouped, which values are missing, and which records need a closer look. This example follows six image records from two sites.

Install Fieldwork

Use Python 3.11–3.14. Install Fieldwork and its pandas and NumPy dependencies:

pip install fieldwork==0.3.2

In a uv project, use uv add fieldwork==0.3.2. This site documents the alpha release; see Documentation source for the exact version.

Make a small dataset

Each exam has two rows. North has two exams, and South has one. The last row is missing its second image.

import pandas as pd
import fieldwork as fw

df = pd.DataFrame(
    {
        "site": ["North", "North", "North", "North", "South", "South"],
        "exam": [1, 1, 2, 2, 3, 3],
        "image_1": [10, 11, 20, 21, 30, 31],
        "image_2": [12, 13, 22, 23, 32, None],
    }
)

See how the rows are grouped

A census counts rows in nested groups. Put site first, then exam, to see the exams within each site:

tree = fw.census(df, ["site", "exam"])
print(fw.render_plaintext(tree))

Fieldwork · census
Census (computed)
  Input population · 6 evaluated / 6 input rows · 0 missing, 0 restricted exclusions
  path: site > exam
  total: 6 rows
    site=North: 4 rows
      exam=1: 2 rows
      exam=2: 2 rows
    site=South: 2 rows
      exam=3: 2 rows

North has four rows across two exams; South has two rows from one exam. The exam counts describe image records, so each exam contributes two rows. The first line names the kind of result; the second line states which rows were counted.

Find the missing image

Check the two image columns and display their availability counts:

availability = fw.missingness(df, features=["image_1", "image_2"])
counts = availability.to_frame("availability")
print(
    counts[["feature", "populated", "missing"]].to_string(index=False)
)

feature  populated  missing
image_1          6        0
image_2          5        1

The second image is missing once. To retrieve that row through Fieldwork, first display the findings and their IDs:

findings = availability.to_frame("findings")
print(findings[["id", "statement"]].to_string(index=False))

id                               statement
f0         image_2: populated in some rows
f1 Present: image_1, image_2; absent: none
f2       Present: image_1; absent: image_2

In this result, f0 is the finding for image_2 availability. image_1 is populated in every row, so it has no finding of its own. Its exceptions are rows where that image is missing:

missing_images = availability.inspect(
    df, "f0", exceptions=True, all_matches=True
)
print(missing_images.to_string(index=False))

 site  exam  image_1  image_2
South     3       31      NaN

The row belongs to exam 3 at South. You can now check that record against the original image collection. Finding IDs belong to a particular result; read the findings table again when analyzing different data.

Choose what to explore next

Continue with families and exceptions to compare availability across sites and select rows for further analysis. If you do not yet know how to group your data, choose a census path or try the automatic overview in the fw.explore walkthrough.