Refine and save an investigation

Use df and fw from Getting started. We will compare South with the whole dataset, save the full analysis, and create a recipe to use again.

Focus on one site

A scope saves a selection of row positions. South occupies positions 4 and 5 in our six-row dataframe:

south = fw.Scope.from_positions(df, [4, 5], name="South")
full_availability = fw.missingness(df, features=["image_1", "image_2"])
south_availability = fw.missingness(
    df, features=["image_1", "image_2"], scope=south
)
comparison = fw.compare(full_availability, south_availability)
changes = comparison.to_frame("changes")
print(
    changes[["feature", "populated_fraction_delta"]].to_string(
        index=False
    )
)

feature  populated_fraction_delta
image_1                  0.000000
image_2                 -0.333333

The second image is present in five of six rows overall and one of two rows at South. Its populated fraction is about 33 percentage points lower at South. A positive delta would mean the second result has a higher populated fraction.

The comparison keeps each result’s population. Both must use the same counting unit and, for entities, the same key and any/all presence rule. An added or removed field has no comparable delta.

Save and reload the analysis

Convert the result to a dictionary, write it as JSON, then restore it when needed:

import json
from pathlib import Path

evidence = full_availability.to_dict()
json_text = json.dumps(evidence, indent=2, allow_nan=False)
evidence_file = Path("availability.json")
evidence_file.write_text(json_text, encoding="utf-8")

saved_text = evidence_file.read_text(encoding="utf-8")
saved_evidence = json.loads(saved_text)
saved_availability = fw.Result.from_dict(saved_evidence)

The JSON retains the analysis and representative row positions. It does not contain the original dataframe. You can read and render the saved result on its own; retrieving source rows still requires the unchanged df.

Share a report

report = fw.render_html(saved_availability)
Path("availability.html").write_text(report, encoding="utf-8")

Open availability.html in a browser. It works offline, and readers can search included findings, filter patterns or exception rows, and expand their saved evidence. Search coverage and display omissions remain separate; saved example counts do not represent every matching source row. For a static figure, use fw.render_svg(saved_availability) and save the returned string to an .svg file.

Use detail="topology" when you want to show structure without quantities. Labels and scope names remain visible, so this option does not anonymize the report.

To keep rare values out of shared output, pass min_count to levels, census, joint_counts or value_patterns; omissions stay marked. If only shared output may leave your environment, also pass safe_errors=True so an error message cannot quote a cell value (see runtime controls).

Repeat the same analysis

A recipe saves the operation and its options:

recipe = fw.Recipe(
    "missingness",
    {"features": ["image_1", "image_2"]},
    notes="Check image availability",
)
recipe.save("recipe.json")

saved_recipe = fw.Recipe.load("recipe.json")
repeated = saved_recipe.run(df)

For the next delivery, pass its dataframe to saved_recipe.run(...) and compare that result with the original. Recipe parameters must be JSON-compatible values. A scope belongs to one dataframe, so supply it to run(df, scope=south) when needed instead of saving it in the recipe. See results and configuration for overrides and more advanced recipes.