Refine and save an investigation

A scope is a selection of unique source positions. It can be reused across discovery operations and refined to a subset.

scope = fw.Scope.from_positions(df, [2, 3], name="South")
local = fw.missingness(df, scope=scope)
full = fw.missingness(df)
changes = fw.compare(full, local)
print(changes.to_frame("changes"))

Comparison preserves both datasets, missing conventions, scopes, and analysis units. Saved evidence includes before_scope, after_scope, before_analysis_unit, and after_analysis_unit; renderers label both populations. Comparisons require matching counting units, entity keys, and any/all aggregation. A positive delta means the after population has a higher populated fraction. Added/removed fields and empty populations have an undefined delta.

Save evidence without source rows

import json
from pathlib import Path

Path("availability.json").write_text(json.dumps(full.to_dict(), indent=2, allow_nan=False))
saved = fw.InvestigationResult.from_dict(json.loads(Path("availability.json").read_text()))
Path("availability.html").write_text(fw.render_html(saved))
Path("availability.svg").write_text(fw.render_svg(saved))
Path("structure.html").write_text(fw.render_html(saved, detail="topology"))

Full exports include evidence. Topology exports suppress quantitative data and source positions, preserving structural labels, scope names and lineage, context predicates, and qualitative row/entity units with entity keys and presence aggregation. They do not anonymize names. HTML works offline; expand findings to inspect saved measurements. The source dataframe is required only to retrieve original rows.

Save a reusable recipe

recipe = fw.Recipe("missingness", {"entity": "exam", "min_implication": 0.75}, notes="Review paired images")
recipe.save("recipe.json")
repeated = fw.Recipe.load("recipe.json").run(df)

Apply the loaded recipe to the next dataframe delivery and compare its result with the original. Recipe parameters must be strict JSON values. Scopes are source-bound; pass them to run(..., scope=scope) when needed rather than saving them as recipe parameters.

An automatic overview recipe can retain search settings while changing its population:

overview_recipe = fw.Recipe("explore", {"discovery": {"max_candidates": 10}})
selected_overview = overview_recipe.run(df, scope=scope)

Common scope, missing, table_id, and features overrides replace corresponding discovery values while preserving budgets, constraints, context grouping and entity settings. Search-only options belong inside discovery.