A worker kneels at a Python-branded validation machine labeled VALIDATING, sorting boxes from a conveyor belt into two stacks marked PASS and REVIEW alongside wooden crates.

Validating Data With Pointblank in Python

by Rohit Goswami Updated Reading time estimate 30m intermediate best-practices data-science

At 3 a.m., a scheduled data pipeline fails with an AssertionError, and the traceback can’t tell you whether the cause is one stray row or a systematic bleed that should halt the entire run. Pointblank, a Python data-validation library, sharpens that decision: it checks a Polars or pandas DataFrame against rules you declare, and each rule accumulates failures until it crosses a threshold you set.

When a rule trips, you learn its name, the offending column, and the fraction of rows that failed, and Pointblank’s pb command-line tool wires that signal into a continuous integration step.

By the end of this tutorial, you’ll understand that:

  • A validation plan chains col_vals_*() checks onto pb.Validate and runs when you call .interrogate().
  • Thresholds turn a pass/fail check into warning, error, and critical tiers based on the fraction of failing rows.
  • Actions trigger messages or callbacks when a step crosses a threshold, letting a policy decide the outcome.
  • get_sundered_data() routes the passing or failing rows out of an interrogated plan, depending on its type argument.
  • A plan stored as YAML runs through the pb command-line tool, where --fail-on critical returns a nonzero exit code for CI.

Pointblank can also validate SQL tables in DuckDB or PostgreSQL directly, so it complements parse-time schema tools such as Pydantic rather than replacing them. In this tutorial, you’ll see where Pointblank overlaps with Pandera and Great Expectations, two other Python data-validation libraries, and when to choose each one.

To get the most out of this tutorial, you should feel comfortable reading basic Polars or pandas DataFrame code, running Python scripts from the command line, and working in a terminal with a package manager like uv or pip. A virtual environment helps keep those installs isolated.

To follow along, you’ll validate a small built-in dataset, add thresholds and actions, sunder a messier table of atom data, and finally move the whole plan into YAML.

Take the Quiz: Test your knowledge with our interactive “Validating Data With Pointblank in Python” quiz. You’ll receive a score upon completion to help you track your learning progress:


Interactive Quiz

Validating Data With Pointblank in Python

Validate data in Python with Pointblank. Declare quality checks, split clean rows from failing ones, and rerun validation plans from YAML.

Get Started With Pointblank in Python

Start by running one validation plan against a built-in dataset. The examples use uv because it runs self-contained scripts that declare and manage their own dependencies through inline script metadata.

If you don’t already have uv, you can install it using the standalone installer by running the command below:

Language: Windows PowerShell
PS> powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
Language: Shell
$ curl -LsSf https://astral.sh/uv/install.sh | sh

If you don’t have curl installed on your system, then you can use wget as shown below:

Language: Shell
$ wget -qO- https://astral.sh/uv/install.sh | sh

Now that you have uv, the script below declares pointblank[pl] as a dependency. The [pl] suffix is a pip extra that pulls in Polars alongside Pointblank, so uv run pointblank_quickstart.py builds an isolated environment and runs the file in one step.

Pointblank ships a small example table named small_table with thirteen rows and eight columns. Its columns carry deliberately generic names, a through f alongside two date columns, which keeps the focus on the validation mechanics rather than any one domain.

The first script validates three of them, one for each common kind of check. Since small_table has only thirteen rows, you can print the whole thing from the Python REPL and see every value before you run anything.

The scripts in this tutorial declare their own dependencies, but a bare REPL doesn’t, so launch one with Pointblank and Polars already available:

Language: Shell
$ uv run --with 'pointblank[pl]' python

Then load the dataset and print the three columns you’ll validate:

Language: Python
>>> import pointblank as pb
>>> import polars as pl
>>> small_table = pb.load_dataset("small_table", tbl_type="polars")
>>> with pl.Config(tbl_rows=13):
...     print(small_table.select(["c", "d", "f"]))
shape: (13, 3)
┌──────┬─────────┬──────┐
│ c    ┆ d       ┆ f    │
│ ---  ┆ ---     ┆ ---  │
│ i64  ┆ f64     ┆ str  │
╞══════╪═════════╪══════╡
│ 3    ┆ 3423.29 ┆ high │
│ 8    ┆ 9999.99 ┆ low  │
│ 3    ┆ 2343.23 ┆ high │
│ null ┆ 3892.4  ┆ mid  │
│ 7    ┆ 283.94  ┆ low  │
│ 4    ┆ 3291.03 ┆ mid  │
│ 3    ┆ 843.34  ┆ high │
│ 2    ┆ 1035.64 ┆ low  │
│ 9    ┆ 837.93  ┆ high │
│ 9    ┆ 837.93  ┆ high │
│ 7    ┆ 833.98  ┆ low  │
│ 8    ┆ 108.34  ┆ low  │
│ null ┆ 2230.09 ┆ high │
└──────┴─────────┴──────┘

You’ll write one rule for each of these three columns, listed in the same order as the table, with bounds chosen to illustrate the check rather than to describe real measurements:

  • c is an integer column that the plan requires to be non-null, and two of its rows are null.
  • d is a numeric column that the plan requires to fall between 0 and 5000, and one row, 9999.99, falls outside that range.
  • f is a categorical column that the plan limits to the labels low, mid, and high, and every row already matches.

With the whole table in view, the failure counts you’ll see in a moment are no surprise: two null c values, one out-of-range d value, and a clean f column. When you validate your own data, you’d swap these for your real columns and the thresholds your pipeline expects.

The script imports pointblank as pb, the convention you’ll see throughout the tutorial. Each .col_vals_*() call returns the validation object, so the chained calls build up the rule list one method at a time:

Language: Python Filename: pointblank_quickstart.py
# /// script
# requires-python = ">=3.10"
# dependencies = [
#   "pointblank[pl]",
# ]
# ///

import json

import pointblank as pb

def main() -> None:
    validation = (
        pb.Validate(
            data=pb.load_dataset("small_table", tbl_type="polars"),
            tbl_name="small_table",
            label="Quickstart validation",
        )
        .col_vals_between(columns="d", left=0, right=5000)
        .col_vals_in_set(columns="f", set=["low", "mid", "high"])
        .col_vals_not_null(columns="c")
        .interrogate()
    )

    report = json.loads(validation.get_json_report())

    print("Validation summary:\n")
    for step in report:
        print(
            f"{step['assertion_type']:20}"
            f"passed={step['n_passed']:<4}"
            f"failed={step['n_failed']}"
        )

if __name__ == "__main__":
    main()

The script gives you the basic Pointblank workflow:

  • Validate() defines the table and the validation plan.
  • .col_vals_between(), .col_vals_in_set(), and .col_vals_not_null(), the chained methods, add validation rules.
  • .interrogate() runs every rule and collects the results.
  • .get_json_report() returns a JSON string with one entry per step, including assertion_type, n_passed, and n_failed.

Run the script with uv run:

Language: Shell
$ uv run pointblank_quickstart.py
Validation summary:

col_vals_between    passed=12  failed=1
col_vals_in_set     passed=13  failed=0
col_vals_not_null   passed=11  failed=2

In this run, one value in column d sits above the allowed range, and two values in c are null. The report says so directly: which rule failed, how many rows tripped it, and whether the failure is isolated or widespread. A bare assert would have raised only the same opaque 3 a.m. AssertionError from the introduction.

That difference is Pointblank’s declare-run-inspect cycle:

  1. Declare a validation plan once
  2. Run it against a table
  3. Inspect the structured results

Instead of scattering one-off asserts through your pipeline, you describe the checks once and read a report that tells you not just whether the data failed, but what failed and how much.

If you run this validation script in a notebook, the same data renders as an interactive table that you can hand to teammates. Later, you’ll save this style of report with pb run --output-html:

HTML Pointblank validation report for the example table.

The report shows per-step pass and fail counts, severity traffic lights for warning, error, and critical thresholds, and CSV export buttons for the failing rows. Click a step’s CSV button to download only the failed rows for that check. Each row reads clearly enough for a teammate, analyst, or reviewer who doesn’t want to inspect your method chain.

The same validation plan works on pandas. Only the tbl_type="polars" argument ties this quickstart to one backend.

Now that you’ve seen how to declare and inspect checks, the next step is making the validation policy-aware so it can drive automation.

Set Thresholds and Trigger Automated Actions

Binary pass-or-fail stops being enough the moment your data quality policy has more than one severity level. In small_table, for example, one out-of-range value in column d may deserve a warning, while null values in column c and duplicate rows may deserve a harder stop.

Pointblank’s Thresholds() accepts row fractions or absolute counts, so you can tune each check’s cutoff to match the damage a failure implies.

The next script adds three pieces that matter in production:

  1. Thresholds() to declare warning, error, and critical cutoffs
  2. Actions() to attach messages to failing steps
  3. .assert_below_threshold() to trip a continuous integration (CI) gate when the data gets bad enough

After .interrogate(), three status methods let you query the result at different strictness levels. The script below keeps the same small_table dataset and adds a .rows_distinct() check, since duplicate records are often serious enough to stop an automated job. Look back at the full table above, and you’ll spot two identical rows (9, 837.93, high), which is exactly what .rows_distinct() flags:

Language: Python Filename: pointblank_thresholds.py
# /// script
# requires-python = ">=3.10"
# dependencies = [
#   "pointblank[pl]",
# ]
# ///

import pointblank as pb

def main() -> None:
    validation = (
        pb.Validate(
            data=pb.load_dataset("small_table", tbl_type="polars"),
            tbl_name="small_table",
            label="Threshold-driven validation",
            thresholds=pb.Thresholds(warning=0.05, error=0.10, critical=0.15),
            actions=pb.Actions(
                warning=(
                    "Warning: step {step} reached {level} severity during "
                    "{type}."
                ),
                critical=(
                    "Critical: step {step} reached {level} severity during "
                    "{type}."
                ),
            ),
        )
        .col_vals_between(columns="d", left=0, right=5000)
        .col_vals_not_null(columns="c")
        .rows_distinct()
        .interrogate()
    )

    print("All checks passed perfectly:", validation.all_passed())
    print(
        "Anything above the error threshold:",
        validation.above_threshold(level="error"),
    )

    try:
        validation.assert_below_threshold(level="critical")
    except AssertionError as exc:
        print("CI gate tripped:", exc)

if __name__ == "__main__":
    main()

Run the updated script the same way:

Language: Shell
$ uv run pointblank_thresholds.py
Warning: step 1 reached warning severity during col_vals_between.
Critical: step 2 reached critical severity during col_vals_not_null.
Critical: step 3 reached critical severity during rows_distinct.
All checks passed perfectly: False
Anything above the error threshold: True
CI gate tripped: The following steps exceeded the critical threshold level:
Step 2: Expect that all values in `c` should not be Null.
Step 3: Expect entirely distinct rows across all columns.

This example shows the two main status methods you can use after .interrogate():

  • .all_passed() answers the strict question, “Did every validation step pass?”
  • .above_threshold(level="error") answers the policy question, “Did anything get bad enough for intervention?”

The third method, .assert_below_threshold(level="critical"), raises an AssertionError when a critical threshold is crossed, which is the same exception try ... except blocks and CI runners already handle.

In the run above, the range check on column d only reaches warning severity, while the null check on c and the duplicate-row check each cross the critical line.

On thirteen rows, the single out-of-range value is 1/13 ≈ 7.7%, which falls between the 5% warning and 10% error cutoffs. The other two failures land at 2/13 ≈ 15.4%, past the 15% critical cutoff. The first three output lines come from Actions(), and the final assertion gives you the CI-facing failure path.

Move the sliders below to change the failing-row count and the warning, error, and critical cutoffs, and watch which tier the fraction lands in:

Interactive diagram — enable JavaScript to view.

The action string templates use Pointblank placeholders such as {step}, {level}, and {type} plus others like {col} or {column}, {val} or {value}, and {time}.

Route Clean and Dirty Rows With Data Sundering

So far, a failed check has meant stopping the whole batch. Often, you’d rather keep the good rows moving and quarantine only the bad ones for review. Pointblank calls that split sundering, and it happens after .interrogate() through one method.

That method is .get_sundered_data(): it keeps the rows that passed every row-level check on one side and the rows that failed at least one rule on the other.

The following example uses a custom thirteen-row teaching fixture in pointblank_atoms.csv, which you can download from the materials linked to this tutorial. Ten rows are clean. The other three each illustrate a distinct failure mode: an invalid category value, a missing numeric field, and an out-of-range numeric value.

If you work in extract, transform, and load pipelines, analytics, or operations, then you already know those three failure modes.

The columns describe atomic measurements, but you don’t need any physics to follow along. The atom_id column identifies each row, and symbol names the element. The x, y, and z columns carry position coordinates bounded between 0 and 20, and fx, fy, and fz carry the three components of a force vector bounded between -1000 and 1000.

Treat symbol as any categorical column and the numeric columns as any measurements with known bounds.

Load the CSV with Polars and peek at the tail rows where the problems sit:

Language: Python
>>> import polars as pl
>>> atoms = pl.read_csv("pointblank_atoms.csv")
>>> atoms.tail(3)
shape: (3, 8)
┌─────────┬────────┬──────┬─────┬─────┬────────┬─────┬─────┐
│ atom_id ┆ symbol ┆ x    ┆ y   ┆ z   ┆ fx     ┆ fy  ┆ fz  │
│ ---     ┆ ---    ┆ ---  ┆ --- ┆ --- ┆ ---    ┆ --- ┆ --- │
│ i64     ┆ str    ┆ f64  ┆ f64 ┆ f64 ┆ f64    ┆ f64 ┆ f64 │
╞═════════╪════════╪══════╪═════╪═════╪════════╪═════╪═════╡
│ 10      ┆ Zz     ┆ 0.5  ┆ 0.5 ┆ 0.1 ┆ 0.0    ┆ 0.0 ┆ 0.0 │
│ 11      ┆ Cu     ┆ null ┆ 1.5 ┆ 0.2 ┆ 0.0    ┆ 0.0 ┆ 0.0 │
│ 12      ┆ Pt     ┆ 12.1 ┆ 2.5 ┆ 0.3 ┆ 1500.0 ┆ 0.0 ┆ 0.0 │
└─────────┴────────┴──────┴─────┴─────┴────────┴─────┴─────┘

Row 10 carries Zz, which isn’t a known element symbol, row 11 is missing its x coordinate, and row 12 has an fx reading far outside the expected -1000 to 1000 range. Those three rows are exactly the ones sundering will quarantine.

The validation plan that follows encodes one rule per failure mode, then sunders the table into clean and dirty rows:

Language: Python Filename: pointblank_atoms.py
# /// script
# requires-python = ">=3.10"
# dependencies = [
#   "pointblank[pl]",
# ]
# ///

import polars as pl
import pointblank as pb

VALID_ELEMENTS = ["Cu", "Pt"]

def main() -> None:
    atoms = pl.read_csv("pointblank_atoms.csv")

    validation = (
        pb.Validate(
            data=atoms,
            tbl_name="atoms",
            label="Atom data validation",
            thresholds=pb.Thresholds(warning=0.02, error=0.05, critical=0.07),
        )
        .col_vals_in_set(columns="symbol", set=VALID_ELEMENTS)
        .col_vals_not_null(columns=["x", "y", "z"])
        .col_vals_between(columns=["x", "y", "z"], left=0, right=20)
        .col_vals_between(columns=["fx", "fy", "fz"], left=-1000, right=1000)
        .interrogate()
    )

    clean = validation.get_sundered_data(type="pass")
    dirty = validation.get_sundered_data(type="fail")

    print(f"Clean rows: {len(clean)}")
    print(clean.select(["atom_id", "symbol", "x", "fx"]))
    print(f"\nDirty rows: {len(dirty)}")
    print(dirty.select(["atom_id", "symbol", "x", "fx"]))

if __name__ == "__main__":
    main()

The validation plan checks symbol against the known element list, the position columns for nulls, and the position and force vector quantities against their expected ranges.

Each .col_vals_between() call accepts a list of column names, so a single rule covers all three components.

After .interrogate(), .get_sundered_data(type="pass") returns the passing rows, and .get_sundered_data(type="fail") returns the failing rows.

Now, run the script and look at the split:

Language: Shell
$ uv run pointblank_atoms.py
Clean rows: 10
shape: (10, 4)
┌─────────┬────────┬──────┬──────┐
│ atom_id ┆ symbol ┆ x    ┆ fx   │
│ ---     ┆ ---    ┆ ---  ┆ ---  │
│ i64     ┆ str    ┆ f64  ┆ f64  │
╞═════════╪════════╪══════╪══════╡
│ 0       ┆ Cu     ┆ 1.0  ┆ 0.1  │
│ 1       ┆ Pt     ┆ 2.1  ┆ -0.2 │
│ 2       ┆ Cu     ┆ 3.2  ┆ 0.3  │
│ 3       ┆ Pt     ┆ 4.3  ┆ -0.1 │
│ 4       ┆ Cu     ┆ 5.4  ┆ 0.2  │
│ 5       ┆ Pt     ┆ 6.5  ┆ -0.3 │
│ 6       ┆ Cu     ┆ 7.6  ┆ 0.1  │
│ 7       ┆ Pt     ┆ 8.7  ┆ -0.2 │
│ 8       ┆ Cu     ┆ 9.8  ┆ 0.3  │
│ 9       ┆ Pt     ┆ 10.9 ┆ -0.1 │
└─────────┴────────┴──────┴──────┘

Dirty rows: 3
shape: (3, 4)
┌─────────┬────────┬──────┬────────┐
│ atom_id ┆ symbol ┆ x    ┆ fx     │
│ ---     ┆ ---    ┆ ---  ┆ ---    │
│ i64     ┆ str    ┆ f64  ┆ f64    │
╞═════════╪════════╪══════╪════════╡
│ 10      ┆ Zz     ┆ 0.5  ┆ 0.0    │
│ 11      ┆ Cu     ┆ null ┆ 0.0    │
│ 12      ┆ Pt     ┆ 12.1 ┆ 1500.0 │
└─────────┴────────┴──────┴────────┘

The output shows why sundering matters. You now have one branch for clean rows and one branch for quarantined rows. Your downstream pipeline can keep moving the ten clean rows forward while the three dirty ones move to a review queue or a log for manual inspection.

Those three dirty rows are exactly the ones you would want a human to inspect: one invalid category value, one missing position value, and one out-of-range force reading. The validation result can drive the next pipeline step directly. You no longer have to choose between “pass everything” and “fail everything.”

Here, Pointblank does something most schema-first tools don’t provide out of the box. The question is no longer just whether the table passed. It’s which rows can continue, which rows need review, and how much of the batch failed.

Reuse Validation Plans With YAML and the CLI

Python code works well when you’re still shaping the rules. YAML becomes useful when you want to separate the validation plan from the pipeline code that runs it. Pointblank supports Python, YAML, and a command-line interface (CLI), so you can keep the rules in one place and choose how to execute them.

When you want validation rules to live alongside other text-based configuration, such as CI workflows or deployment manifests, YAML is a natural fit. The rules stop looking like application code and start looking like shared configuration.

Profile the Data Before You Write Rules

Before you translate any rules, let the CLI tell you what shape the data has. The pb command bundled with pointblank ships two read-only commands that print summaries to the terminal.

The uv run --no-project --with 'pointblank[pl]' prefix runs the command outside any local project and temporarily installs Pointblank with Polars support, so you can run pb from any directory without setting up an environment first:

Language: Shell
$ uv run --no-project --with 'pointblank[pl]' pb scan pointblank_atoms.csv
✓ Loaded data source: pointblank_atoms.csv
✓ Data scan completed in 1.01s
Use --output-html to save the full interactive scan report.

Column Summary / External source: pointblank_atoms.csv / polars
13 rows / 8 columns

  Column   Type    NA     UQ    Mean      SD     Min    Med     Max    Q₁     Q₃   IQR
 ──────────────────────────────────────────────────────────────────────────────────────
  atom_id  i64      0     13       6    3.89       0      6      12     3      9     6
  symbol   str      0      3       2       0       2      2       2     2      2     0
  x        f64      1     13    6.01    3.90    0.50   5.95    12.1  2.93   8.97  6.05
  y        f64      0     10    4.19    3.07    0.50   3.50    9.50  1.50   6.50     5
  z        f64      0     10    0.47    0.31    0.10   0.40       1  0.20   0.70  0.50
  fx       f64      0      8   115.4   416.0   -0.30      0    1500 -0.10   0.20  0.30
  fy       f64      0      3       0    0.07   -0.10      0    0.10     0      0     0
  fz       f64      0      3       0    0.07   -0.10      0    0.10     0      0     0

The NA column counts nulls, UQ counts unique values, and the rest cover standard summary statistics: the mean, standard deviation, minimum, median, and maximum, the first and third quartiles (Q₁ and Q₃), and the interquartile range (IQR).

Read down the Max column, and you can already spot the bad row: fx peaks at 1500, far outside the -1000 to 1000 range you’ll enforce shortly. Pointblank renders this with Rich, so on a narrow terminal some column labels and values truncate to fit the width.

You can use pb missing to map null values across the table so you can spot gaps before they become failed checks:

Language: Shell
$ uv run --no-project --with 'pointblank[pl]' pb missing pointblank_atoms.csv
✓ Loaded data source: pointblank_atoms.csv

Missing Values / External source: pointblank_atoms.csv / polars
13 rows / 8 columns
                                  Row Sectors
                                  ────────────────────

  Column    Type     1    2    3    4    5    6    7    8    9    10
 ───────────────────────────────────────────────────────────────────
  atom_id   i64      ●    ●    ●    ●    ●    ●    ●    ●    ●     ●
  symbol    str      ●    ●    ●    ●    ●    ●    ●    ●    ●     ●
  x         f64      ●    ●    ●    ●    ●    ●    ●    ●    ●   25%
  y         f64      ●    ●    ●    ●    ●    ●    ●    ●    ●     ●
  z         f64      ●    ●    ●    ●    ●    ●    ●    ●    ●     ●
  fx        f64      ●    ●    ●    ●    ●    ●    ●    ●    ●     ●
  fy        f64      ●    ●    ●    ●    ●    ●    ●    ●    ●     ●
  fz        f64      ●    ●    ●    ●    ●    ●    ●    ●    ●     ●

Symbols: ● = no missing vals in sector, ● = all vals completely missing,
    x% = percentage missing

Under the Row Sectors heading, the pb missing command divides the rows into ten sectors and reports the share of missing values in each, with column 10 covering the final stretch of the table. The 25% cell in the x column flags the single null you saw earlier: one quarter of that final sector is missing.

Those two commands let you skim a new file before you commit to a validation plan, without opening a notebook.

Translate the Python Plan Into YAML

The YAML file below describes the same atom workflow you just ran through pointblank_atoms.py, expressed as configuration instead of code:

Language: YAML Filename: pointblank_atoms.yaml
tbl: pointblank_atoms.csv
df_library: polars
tbl_name: "atoms"
label: "Atom data validation"
thresholds:
  warning: 0.02
  error: 0.05
  critical: 0.07
steps:
  - col_vals_in_set:
      columns: symbol
      set: [Cu, Pt]
  - col_vals_not_null:
      columns: [x, y, z]
  - col_vals_between:
      columns: [x, y, z]
      left: 0
      right: 20
  - col_vals_between:
      columns: [fx, fy, fz]
      left: -1000
      right: 1000

Each top-level entry maps to an argument or method you’ve already seen in Python:

YAML key Python equivalent
tbl and df_library pl.read_csv() plus the DataFrame passed to Validate()
tbl_name and label Validate(tbl_name=..., label=...)
thresholds Thresholds()
steps The chained col_vals_*() calls

The Python-to-YAML translation stays aligned on purpose, so you can move a validation plan you already trust into a config file without rewriting the logic.

Run the Plan as a CI Gate and a Report

With the plan in YAML, the same pb run command gives you a CI gate and a shareable report. First, run the plan as a CI gate:

Language: Shell
$ uv run --no-project \
    --with 'pointblank[pl]' pb run pointblank_atoms.yaml \
    --fail-on critical
✓ Found 1 validation object(s)

Validation Report
Steps: 10 / P: 6 (6 AP) / W: 4 / E: 4 / C: 4 / critical

       Step                Column   Values          Units   Pass      Fail     W   E   C   Ext
 ──────────────────────────────────────────────────────────────────────────────────────────────
  1    col_vals_in_set     symbol   Cu, Pt          13      12/0.92   1/0.08   ●   ●   ●    ✓
  2    col_vals_not_null   x        —               13      12/0.92   1/0.08   ●   ●   ●    ✓
  3    col_vals_not_null   y        —               13      13/1.00   0/0.00   ○   ○   ○    —
  4    col_vals_not_null   z        —               13      13/1.00   0/0.00   ○   ○   ○    —
  5    col_vals_between    x        [0, 20]         13      12/0.92   1/0.08   ●   ●   ●    ✓
  6    col_vals_between    y        [0, 20]         13      13/1.00   0/0.00   ○   ○   ○    —
  7    col_vals_between    z        [0, 20]         13      13/1.00   0/0.00   ○   ○   ○    —
  8    col_vals_between    fx       [-1000, 1000]   13      12/0.92   1/0.08   ●   ●   ●    ✓
  9    col_vals_between    fy       [-1000, 1000]   13      13/1.00   0/0.00   ○   ○   ○    —
  10   col_vals_between    fz       [-1000, 1000]   13      13/1.00   0/0.00   ○   ○   ○    —

╭────────────────────────────────────────────╮
│ ✗ Validation failed with critical severity │
╰────────────────────────────────────────────╯
Exiting with error due to critical validation failures
$ echo $?
1

The header line summarizes the run: ten steps total, six passed (P: 6), and four reached at least warning severity, with all four progressing through error to critical.

Each row below covers one rule against one column, with Pass and Fail counts plus the threshold columns (W, E, C) marking which severities the failing fraction crossed.

The $ echo $? line confirms the exit code: --fail-on critical exits nonzero when any step reaches critical severity, which is exactly what a CI step needs to fail the build. The flag accepts warning, error, critical, or any, so you can match the exit condition to the policy you declared with Thresholds().

Run the HTML variant when you want a report artifact:

Language: Shell
$ uv run --no-project \
    --with 'pointblank[pl]' pb run pointblank_atoms.yaml \
    --output-html pointblank_report.html
✓ Found 1 validation object(s)

Validation Report
Steps: 10 / P: 6 (6 AP) / W: 4 / E: 4 / C: 4 / critical
...
╭────────────────────────────────────────────╮
│ ✗ Validation failed with critical severity │
╰────────────────────────────────────────────╯
✓ HTML report saved to: pointblank_report.html

The --output-html invocation produces the same interactive report you would inspect in a notebook, but as a shareable artifact. Your pipeline can react to the exit status, while a teammate or reviewer can open the report and inspect the failed steps without reading the Python source first.

In a GitHub Actions workflow, the same pb run --fail-on critical step fails the build on critical severity, and you can upload pointblank_report.html as an artifact for the team to inspect.

If you only want the CLI and not a local script, then uvx works too:

Language: Shell
$ uvx --from 'pointblank[pl]' pb scan pointblank_atoms.csv

If you want to keep the YAML file but load it from Python, then .yaml_interrogate("pointblank_atoms.yaml", set_tbl=atoms) provides that bridge so you can point one plan at a different DataFrame each run:

Language: Python
atoms = pl.read_csv("pointblank_atoms.csv")
validation = pb.yaml_interrogate("pointblank_atoms.yaml", set_tbl=atoms)

That same YAML structure adapts to any dataset, which is exactly what the starter template gives you.

Adapt the Starter Template to Your Own Data

The atom YAML you’ve seen so far is specific to one dataset. A more reusable version is stored as pointblank_starter_validation.yaml and keeps the same structure without the domain-specific column names:

Language: YAML Filename: pointblank_starter_validation.yaml
tbl: small_table
df_library: polars
tbl_name: "Starter Validation"
label: "Adapt this template to your data"
thresholds:
  warning: 0.02
  error: 0.05
  critical: 0.10
steps:
  - col_exists:
      columns: [record_id, status, amount]
  - col_vals_not_null:
      columns: record_id
  - col_vals_in_set:
      columns: status
      set: [pending, shipped, delivered]
  - col_vals_gt:
      columns: amount
      value: 0

To adapt it, follow these steps:

  1. Set tbl to your CSV path
  2. List your real columns under col_exists
  3. Pick thresholds values that match your pipeline’s tolerance for failure

The col_exists step catches the case where a column disappears entirely, which makes it a good first failure mode to flag. Run the adapted plan the same way you ran the atom one:

Language: Shell
$ uv run --no-project --with 'pointblank[pl]' \
    pb run pointblank_starter_validation.yaml \
    --data your_data.csv --fail-on critical

The --data flag swaps the tbl target at runtime, which lets the template live in version control while each pipeline job points it at a different file.

You use the two formats for different jobs:

  • Use Python while you’re discovering the right checks and thresholds.
  • Use YAML once the validation plan becomes an artifact that other jobs, teammates, or CI steps should run unchanged.

The same plan isn’t limited to CSV files either. Point tbl at a DuckDB file or another SQL database such as PostgreSQL, and pb run validates the table in place, so the plan scales to batches too large to fit in a DataFrame.

Compare Pointblank With Other Python Validation Tools

Three common tools for validating Python DataFrames are Pointblank, Pandera, and Great Expectations. Pydantic also matters here, but it validates records or payloads before they become a DataFrame. The table below is organized to help you decide, with a cue for when to reach for each tool:

Tool Best for DataFrame backends Trade-offs
Pointblank Stakeholder-friendly HTML reports, threshold-based quality gates (warning/error/critical), row sundering, and YAML-driven rule reuse Polars and pandas (via Narwhals). Also validates DuckDB and PostgreSQL tables No static type checking, so errors surface at runtime only. Not a full observability platform
Pandera Type-safe, declarative schemas with mypy integration and statistical checks pandas and Polars, plus Modin, Dask, and PySpark Primarily binary pass/fail, with no built-in severity tiers, sundering, CLI, or interactive reports
Great Expectations Enterprise-scale observability with versioned expectation suites, checkpoints, and Data Docs pandas, Spark, and SQL, with no native Polars Heavy configuration overhead and setup

Pointblank and Pydantic cover different layers of the pipeline. Pydantic asks whether one incoming record has the right shape at the boundary, while Pointblank asks how a whole batch looks after it has already become a DataFrame.

If you want to get more comfortable with Pydantic before deciding where it fits alongside Pointblank, Real Python’s video course on Using Pydantic to Simplify Python Data Validation walks through schemas, validators, and type coercion step by step.

Limitations to Know About Pointblank

The tool becomes easier to evaluate when you get specific about its limits. These are the ones worth knowing before you commit to it for a production workflow.

  • Checks at runtime: Pointblank gives you no static type-checking story, so schema errors surface only once the data is materialized, and IDE or mypy coverage for column types needs a different layer.
  • Sundering covers a narrower slice than full validation: Row routing reflects only row-level checks, so a .row_count_match() or .col_schema_match() rule never joins the split between clean and dirty rows.
  • Backend parity depends on Narwhals: This compatibility layer helps Pointblank smooth over backend differences, but this tutorial uses Polars to match the upstream Python examples, so test the specific pandas check combinations you rely on rather than assuming identical behavior.
  • The richest report renders interactively: A notebook or saved HTML report shows the full interactive table, while a headless CI run leans on text output or the generated HTML artifact instead.
  • It doesn’t replace a full observability platform: Pointblank covers validation and reporting well, but it isn’t the data observability layer that Great Expectations provides for larger organizations.

Conclusion

Reach for Pointblank when your pipeline already processes a full table in memory and you need to decide whether to continue, warn, or stop. You define checks, interrogate the table, see how many rows failed, choose which threshold matters, and then either halt the job or route the bad rows elsewhere.

Pointblank handles the report-and-route shape of validation that most quality gates need because it surfaces counts, thresholds, and row routing instead of adding validation branches to raw asserts.

In this tutorial, you’ve learned how to:

  • Declare a validation plan with pb.Validate and summarize results with .interrogate()
  • Attach thresholds and actions so warnings, errors, and critical states drive pipeline behavior
  • Split a table into clean and dirty rows with .get_sundered_data()
  • Move the same plan into a YAML artifact and drive it from the pb command-line tool
  • Decide when Pointblank complements record-by-record tools such as Pydantic

If you want to keep going, then revisit Python’s assert statement for the low-level baseline, compare this workflow with Pydantic when you need record-by-record validation, and review the pandas DataFrame if pandas is your main tabular tool.

You can also go deeper with How to Work With Polars LazyFrames and Continuous Integration and Deployment for Python With GitHub Actions when you connect pb run --fail-on critical to a real pipeline.

To build out the automated testing around a gate like this, Real Python’s Testing and Continuous Integration learning path covers writing unit tests and running them in CI, so your validation work joins a fuller quality process.

Next time the 3 a.m. alert fires, you’ll know exactly what failed, how much failed, and what to do next.

Frequently Asked Questions

Now that you have some experience with Pointblank in Python, you can use the questions and answers below to check your understanding and recap what you’ve learned.

These FAQs are related to the most important concepts you’ve covered in this tutorial. Click the Show/Hide toggle beside each question to reveal the answer.

Pointblank is a data-validation library that checks a Polars or pandas DataFrame against rules you declare, then reports how many rows passed or failed each rule. It can also validate SQL tables in databases like DuckDB and PostgreSQL.

Pointblank centers on runtime threshold-based quality gates and reports that teammates can read. Pandera leans on type-safe schemas with static checking, while Great Expectations targets enterprise-scale data observability with heavier configuration.

Yes. Pointblank validates pandas and Polars tables through the same API, so you can switch backends without rewriting your validation plan.

Thresholds set the fraction of failing rows that trips a warning, error, or critical status. They turn an all-or-nothing assertion into a graded policy that tolerates small failure rates and halts on serious ones.

You store the validation plan in a YAML file and run it with the pb command-line tool. Running pb run with --fail-on critical returns a nonzero exit code, which fails the build when the critical threshold is breached.

Take the Quiz: Test your knowledge with our interactive “Validating Data With Pointblank in Python” quiz. You’ll receive a score upon completion to help you track your learning progress:


Interactive Quiz

Validating Data With Pointblank in Python

Validate data in Python with Pointblank. Declare quality checks, split clean rows from failing ones, and rerun validation plans from YAML.

🐍 Python Tricks 💌

Get a short & sweet Python Trick delivered to your inbox every couple of days. No spam ever. Unsubscribe any time. Curated by the Real Python team.

Python Tricks Dictionary Merge

About Rohit Goswami

Rohit Goswami is a researcher and open-source contributor passionate about scientific computing and high-performance Python. He works on F2PY and NumPy, helping bridge Python with Fortran and is an advocate for research software engineering.

» More about Rohit

Each tutorial at Real Python is created by a team of developers so that it meets our high quality standards. The team members who worked on this tutorial are:

Master Real-World Python Skills With Unlimited Access to Real Python

Locked learning resources

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

Master Real-World Python Skills
With Unlimited Access to Real Python

Locked learning resources

Join us and get access to thousands of tutorials, hands-on video courses, and a community of expert Pythonistas:

Level Up Your Python Skills »

What Do You Think?

Rate this article:

What’s your #1 takeaway or favorite thing you learned? How are you going to put your newfound skills to use? Leave a comment below and let us know.

Commenting Tips: The most useful comments are those written with the goal of learning from or helping out other students. Get tips for asking good questions and get answers to common questions in our support portal.


Looking for a real-time conversation? Visit the Real Python Community Chat or join the next “Office Hours” Live Q&A Session. Happy Pythoning!