Filtering Data

Subset DataFrame rows by comparison or predicate using chainable, error-deferred filters

Learn how to subset DataFrame rows in GPandas using boolean comparisons or arbitrary predicates. Both Filter and Where return a chainable, error-deferred builder so conditions can be combined fluently.

Overview

GPandas provides two ways to filter rows:

Operation	Method	Description
Comparison Filter	`Filter()`	Keep rows where a column satisfies a comparison
Predicate Filter	`Where()`	Keep rows for which a custom function returns true

Both methods start a FilterChain that is terminated with Result(), MustResult(), or Err().

Note: Isin, Between, Nlargest, and Nsmallest are steps in the same chain and can be mixed with Filter and Where. See Membership, Range & Top-N.

Filter

Keeps rows where the value in a column satisfies the comparison column <op> value, similar to pandas boolean indexing such as df[df["Age"] > 25].

Function Signature

func (df *DataFrame) Filter(column string, op FilterOp, value any) *FilterChain

FilterOp Constants

Constant	Operator	Keeps rows where
`Equals`	`==`	value equals target
`NotEquals`	`!=`	value differs from target
`GreaterThan`	`>`	value greater than target
`GreaterThanOrEqual`	`>=`	value greater than or equal to target
`LessThan`	`<`	value less than target
`LessThanOrEqual`	`<=`	value less than or equal to target

Comparison Rules

Column Type	Comparison
`float64` / `int64` / `int`	Numeric (cross-type, e.g. an `int` column compares with a `float64` literal)
`string`	Lexicographic
`bool`	`false < true`

Note: Null values never satisfy a comparison and are always excluded from the result, mirroring pandas behaviour where comparisons against NaN are False.

Sample Data

All examples use this employee DataFrame:

Employees DataFrame

Name	Department	Age	Salary
Alice	Engineering	30	95000
Bob	Sales	25	55000
Charlie	Engineering	35	105000
Diana	Sales	28	62000
Eve	Marketing	32	72000
Frank	Engineering	27	88000

Setup Code

package main

import (
    "fmt"
    "log"

    "github.com/apoplexi24/gpandas"
    "github.com/apoplexi24/gpandas/dataframe"
)

func main() {
    gp := gpandas.GoPandas{}

    // Create employee DataFrame
    df, _ := gp.DataFrame(
        []string{"Name", "Department", "Age", "Salary"},
        []gpandas.Column{
            {"Alice", "Bob", "Charlie", "Diana", "Eve", "Frank"},
            {"Engineering", "Sales", "Engineering", "Sales", "Marketing", "Engineering"},
            {int64(30), int64(25), int64(35), int64(28), int64(32), int64(27)},
            {95000.0, 55000.0, 105000.0, 62000.0, 72000.0, 88000.0},
        },
        map[string]any{
            "Name":       gpandas.StringCol{},
            "Department": gpandas.StringCol{},
            "Age":        gpandas.IntCol{},
            "Salary":     gpandas.FloatCol{},
        },
    )

    // Examples follow...
}

Single Comparison

Keep rows where Salary is greater than 80,000:

result, err := df.Filter("Salary", dataframe.GreaterThan, 80000.0).Result()
if err != nil {
    log.Fatalf("Filter failed: %v", err)
}
fmt.Println(result.String())

Output

+---------+-------------+-----+--------+
| Name    | Department  | Age | Salary |
+---------+-------------+-----+--------+
| Alice   | Engineering | 30  | 95000  |
| Charlie | Engineering | 35  | 105000 |
| Frank   | Engineering | 27  | 88000  |
+---------+-------------+-----+--------+
[3 rows x 4 columns]

Chained Filters

Each Filter/Where call returns a FilterChain, so conditions can be combined to form an AND of all predicates. Terminate the chain with Result():

result, err := df.
    Filter("Department", dataframe.Equals, "Engineering").
    Filter("Salary", dataframe.GreaterThan, 90000.0).
    Result()
if err != nil {
    log.Fatalf("Filter failed: %v", err)
}
fmt.Println(result.String())

Output

+---------+-------------+-----+--------+
| Name    | Department  | Age | Salary |
+---------+-------------+-----+--------+
| Alice   | Engineering | 30  | 95000  |
| Charlie | Engineering | 35  | 105000 |
+---------+-------------+-----+--------+
[2 rows x 4 columns]

Chain Evaluation Flow

Where

Keeps rows for which a predicate returns true. The predicate receives a map[string]any for the row (null values are passed as nil), enabling arbitrary multi-column conditions.

Function Signature

func (df *DataFrame) Where(predicate func(row map[string]any) bool) *FilterChain

Example

result, err := df.Where(func(row map[string]any) bool {
    age, _ := row["Age"].(int64)
    return age < 32 && row["Department"] == "Engineering"
}).Result()
if err != nil {
    log.Fatalf("Where failed: %v", err)
}
fmt.Println(result.String())

Output

+-------+-------------+-----+--------+
| Name  | Department  | Age | Salary |
+-------+-------------+-----+--------+
| Alice | Engineering | 30  | 95000  |
| Frank | Engineering | 27  | 88000  |
+-------+-------------+-----+--------+
[2 rows x 4 columns]

Note: Filter and Where can be mixed in the same chain, for example df.Filter(...).Where(...).Result().

Terminating a Chain

A FilterChain is lazy with respect to errors: the first error encountered is carried through and surfaced when the chain is terminated.

Method	Returns	Behaviour
`Result()`	`(*DataFrame, error)`	Returns the result and the first error (if any)
`Err()`	`error`	Returns the first error only
`MustResult()`	`*DataFrame`	Returns the result, panics if the chain holds an error

Error Propagation

If any step fails (for example a missing column), subsequent steps become no-ops and the error is returned by the terminal call:

_, err := df.
    Filter("Department", dataframe.Equals, "Engineering").
    Filter("Missing", dataframe.Equals, 1). // error captured here
    Filter("Age", dataframe.GreaterThan, 0). // skipped
    Result()
if err != nil {
    log.Fatalf("Filter failed: %v", err) // "column 'Missing' not found"
}

MustResult

Use MustResult() when inputs are known to be valid (such as in tests). It panics on error instead of returning one:

adults := df.Filter("Age", dataframe.GreaterThanOrEqual, int64(30)).MustResult()
fmt.Println(adults.String())

Null Values

Comparisons never match null values, so rows with a null in the filtered column are dropped:

// Score column: 10.0, null, 30.0, null, 5.0
result, _ := df.Filter("Score", dataframe.GreaterThanOrEqual, 0.0).Result()
// Only the 3 non-null rows remain (10.0, 30.0, 5.0)

To select rows where a value is null, use Where:

nullsOnly, _ := df.Where(func(row map[string]any) bool {
    return row["Score"] == nil
}).Result()

Error	Cause	Solution
"DataFrame is nil"	Operating on nil DataFrame	Check DataFrame initialization
"unsupported operator"	Invalid `FilterOp` value	Use a defined operator constant
"column 'X' not found"	Invalid column name	Verify the column exists
"predicate must not be nil"	`Where(nil)`	Provide a predicate function
"type mismatch: cannot compare"	Comparing incompatible types	Compare against a value of the column's type

Error Handling Example

result, err := df.
    Filter("Age", dataframe.GreaterThan, int64(25)).
    Filter("City", dataframe.Equals, "NYC").
    Result()
if err != nil {
    switch {
    case strings.Contains(err.Error(), "not found"):
        log.Fatal("Column doesn't exist in DataFrame")
    case strings.Contains(err.Error(), "type mismatch"):
        log.Fatal("Cannot compare values of different types")
    default:
        log.Fatalf("Filter error: %v", err)
    }
}

Index Preservation

Filtering preserves the original index labels of the matching rows:

// Original index: 0, 1, 2, 3
// Filter keeps rows 0 and 2
result, _ := df.Filter("City", dataframe.Equals, "NYC").Result()
fmt.Println(result.Index) // [0 2]

Thread Safety

Filtering operations are thread-safe:

Method	Lock Type	Description
`Filter()`	RLock	Read lock during row evaluation
`Where()`	RLock	Read lock during row evaluation

Each step produces a new DataFrame, so the original is never mutated and concurrent filtering is safe.

Complete Example

package main

import (
    "fmt"
    "log"

    "github.com/apoplexi24/gpandas"
    "github.com/apoplexi24/gpandas/dataframe"
)

func main() {
    gp := gpandas.GoPandas{}

    df, err := gp.Read_csv("employees.csv")
    if err != nil {
        log.Fatalf("Failed to load data: %v", err)
    }

    // Engineers earning more than 90k
    seniorEngineers, err := df.
        Filter("Department", dataframe.Equals, "Engineering").
        Filter("Salary", dataframe.GreaterThan, 90000.0).
        Result()
    if err != nil {
        log.Fatalf("Filter failed: %v", err)
    }

    fmt.Println("Senior engineers:")
    fmt.Println(seniorEngineers.String())

    // Export the subset
    if _, err := seniorEngineers.ToCSV("senior_engineers.csv", ","); err != nil {
        log.Printf("Export warning: %v", err)
    }
}

Filtering Data

Overview

Filter

Function Signature

FilterOp Constants

Comparison Rules

Sample Data

Employees DataFrame

Setup Code

Single Comparison

Output

Chained Filters

Output

Chain Evaluation Flow

Where

Function Signature

Example

Output

Terminating a Chain

Error Propagation

MustResult

Null Values

Filtering Workflow

Error Handling

Common Errors

Error Handling Example

Index Preservation

Thread Safety

Complete Example

See Also

On this page