GPandas
Reshaping & Combining

Merging Data

Join and combine DataFrames using inner, left, right, and full outer merges

Learn how to combine DataFrames using various merge strategies in GPandas, similar to SQL JOIN operations.

Overview

The Merge() function combines two DataFrames based on a common column (key). GPandas supports four merge types:

Merge TypeConstantDescription
InnerInnerMergeOnly matching rows from both DataFrames
LeftLeftMergeAll rows from left, matching from right
RightRightMergeAll rows from right, matching from left
Full OuterFullMergeAll rows from both DataFrames

Function Signature

func (df *DataFrame) Merge(other *DataFrame, on string, how MergeHow) (*DataFrame, error)

Parameters

ParameterTypeDescription
other*DataFrameThe right DataFrame to merge with
onstringColumn name to join on (must exist in both DataFrames)
howMergeHowType of merge: InnerMerge, LeftMerge, RightMerge, FullMerge

Returns

TypeDescription
*DataFrameNew merged DataFrame
errorError if merge fails

Sample Data

All examples use these two DataFrames:

Left DataFrame (df1) - Employees

IDNameDepartment
1AliceEngineering
2BobMarketing
3CharlieEngineering
4DianaSales

Right DataFrame (df2) - Salaries

IDSalaryBonus
1850005000
2720003000
5950008000
6680002000

Setup Code

package main

import (
    "fmt"
    "log"

    "github.com/apoplexi24/gpandas"
    "github.com/apoplexi24/gpandas/dataframe"
)

func main() {
    gp := gpandas.GoPandas{}
    
    // Create employees DataFrame
    df1, _ := gp.DataFrame(
        []string{"ID", "Name", "Department"},
        []gpandas.Column{
            {int64(1), int64(2), int64(3), int64(4)},
            {"Alice", "Bob", "Charlie", "Diana"},
            {"Engineering", "Marketing", "Engineering", "Sales"},
        },
        map[string]any{
            "ID":         gpandas.IntCol{},
            "Name":       gpandas.StringCol{},
            "Department": gpandas.StringCol{},
        },
    )
    
    // Create salaries DataFrame
    df2, _ := gp.DataFrame(
        []string{"ID", "Salary", "Bonus"},
        []gpandas.Column{
            {int64(1), int64(2), int64(5), int64(6)},
            {85000.0, 72000.0, 95000.0, 68000.0},
            {5000.0, 3000.0, 8000.0, 2000.0},
        },
        map[string]any{
            "ID":     gpandas.IntCol{},
            "Salary": gpandas.FloatCol{},
            "Bonus":  gpandas.FloatCol{},
        },
    )
    
    // Examples follow...
}

Inner Merge

Returns only rows where the key exists in both DataFrames.

Inner Merge Example

result, err := df1.Merge(df2, "ID", dataframe.InnerMerge)
if err != nil {
    log.Fatalf("Merge failed: %v", err)
}
fmt.Println(result.String())

Inner Merge Output

+----+-------+-------------+--------+-------+
| ID | Name  | Department  | Salary | Bonus |
+----+-------+-------------+--------+-------+
| 1  | Alice | Engineering | 85000  | 5000  |
| 2  | Bob   | Marketing   | 72000  | 3000  |
+----+-------+-------------+--------+-------+
[2 rows x 5 columns]

Left Merge

Returns all rows from the left DataFrame, with matching data from the right. Non-matching rows have nil for right columns.

Left Merge Example

result, err := df1.Merge(df2, "ID", dataframe.LeftMerge)
if err != nil {
    log.Fatalf("Merge failed: %v", err)
}
fmt.Println(result.String())

Left Merge Output

+----+---------+-------------+--------+-------+
| ID | Name    | Department  | Salary | Bonus |
+----+---------+-------------+--------+-------+
| 1  | Alice   | Engineering | 85000  | 5000  |
| 2  | Bob     | Marketing   | 72000  | 3000  |
| 3  | Charlie | Engineering | <nil>  | <nil> |
| 4  | Diana   | Sales       | <nil>  | <nil> |
+----+---------+-------------+--------+-------+
[4 rows x 5 columns]

Right Merge

Returns all rows from the right DataFrame, with matching data from the left. Non-matching rows have nil for left columns.

Right Merge Example

result, err := df1.Merge(df2, "ID", dataframe.RightMerge)
if err != nil {
    log.Fatalf("Merge failed: %v", err)
}
fmt.Println(result.String())

Right Merge Output

+----+-------+-------------+--------+-------+
| ID | Name  | Department  | Salary | Bonus |
+----+-------+-------------+--------+-------+
| 1  | Alice | Engineering | 85000  | 5000  |
| 2  | Bob   | Marketing   | 72000  | 3000  |
| 5  | <nil> | <nil>       | 95000  | 8000  |
| 6  | <nil> | <nil>       | 68000  | 2000  |
+----+-------+-------------+--------+-------+
[4 rows x 5 columns]

Full Outer Merge

Returns all rows from both DataFrames. Non-matching rows have nil for missing columns.

Full Merge Example

result, err := df1.Merge(df2, "ID", dataframe.FullMerge)
if err != nil {
    log.Fatalf("Merge failed: %v", err)
}
fmt.Println(result.String())

Full Merge Output

+----+---------+-------------+--------+-------+
| ID | Name    | Department  | Salary | Bonus |
+----+---------+-------------+--------+-------+
| 1  | Alice   | Engineering | 85000  | 5000  |
| 2  | Bob     | Marketing   | 72000  | 3000  |
| 3  | Charlie | Engineering | <nil>  | <nil> |
| 4  | Diana   | Sales       | <nil>  | <nil> |
| 5  | <nil>   | <nil>       | 95000  | 8000  |
| 6  | <nil>   | <nil>       | 68000  | 2000  |
+----+---------+-------------+--------+-------+
[6 rows x 5 columns]

Merge Type Comparison

Visual comparison of all merge types:

Merge TypeLeft RowsRight RowsResult Rows (Example)
InnerMatching onlyMatching only2
LeftAll (4)Matching only4
RightMatching onlyAll (4)4
FullAll (4)All (4)6

SQL Equivalent

GPandasSQL Equivalent
InnerMergeINNER JOIN
LeftMergeLEFT OUTER JOIN
RightMergeRIGHT OUTER JOIN
FullMergeFULL OUTER JOIN
-- Inner Merge equivalent
SELECT * FROM employees e
INNER JOIN salaries s ON e.ID = s.ID;

-- Left Merge equivalent
SELECT * FROM employees e
LEFT OUTER JOIN salaries s ON e.ID = s.ID;

-- Right Merge equivalent
SELECT * FROM employees e
RIGHT OUTER JOIN salaries s ON e.ID = s.ID;

-- Full Merge equivalent
SELECT * FROM employees e
FULL OUTER JOIN salaries s ON e.ID = s.ID;

Handling Many-to-Many Relationships

When the key column has duplicate values, the merge produces a Cartesian product of matching rows:

// df1 has duplicate IDs
// ID: 1, 1, 2
// Name: Alice, Alex, Bob

// df2 has duplicate IDs  
// ID: 1, 1, 2
// Salary: 80k, 85k, 72k

// Inner merge produces:
// ID: 1, 1, 1, 1, 2
// Name: Alice, Alice, Alex, Alex, Bob
// Salary: 80k, 85k, 80k, 85k, 72k

Error Handling

Common Errors

ErrorCauseSolution
"both DataFrames must be non-nil"nil DataFrame passedCheck DataFrame initialization
"column 'X' not found in left DataFrame"Key column missingVerify column name
"column 'X' not found in right DataFrame"Key column missingVerify column name
"invalid merge type"Invalid MergeHow valueUse predefined constants

Example Error Handling

result, err := df1.Merge(df2, "ID", dataframe.InnerMerge)
if err != nil {
    switch {
    case strings.Contains(err.Error(), "non-nil"):
        log.Fatal("One of the DataFrames is nil")
    case strings.Contains(err.Error(), "not found"):
        log.Fatal("Key column doesn't exist in one of the DataFrames")
    default:
        log.Fatalf("Merge error: %v", err)
    }
}

Complete Example: Multi-Table Join

package main

import (
    "fmt"
    "log"

    "github.com/apoplexi24/gpandas"
    "github.com/apoplexi24/gpandas/dataframe"
)

func main() {
    gp := gpandas.GoPandas{}
    
    // Load three related tables
    employees, _ := gp.Read_csv("employees.csv")       // ID, Name, DeptID
    departments, _ := gp.Read_csv("departments.csv")   // DeptID, DeptName
    salaries, _ := gp.Read_csv("salaries.csv")         // ID, Salary
    
    // First merge: employees + departments
    empDept, err := employees.Merge(departments, "DeptID", dataframe.LeftMerge)
    if err != nil {
        log.Fatalf("First merge failed: %v", err)
    }
    fmt.Println("Employees with Departments:")
    fmt.Println(empDept.String())
    
    // Second merge: result + salaries
    final, err := empDept.Merge(salaries, "ID", dataframe.LeftMerge)
    if err != nil {
        log.Fatalf("Second merge failed: %v", err)
    }
    fmt.Println("\nComplete Employee Data:")
    fmt.Println(final.String())
    
    // Export final result
    _, err = final.ToCSV("complete_employee_data.csv", ",")
    if err != nil {
        log.Printf("Export warning: %v", err)
    }
}

Merging on Multiple Keys

Merge joins on a single key column. To join on a composite key of two or more columns, use MergeOn, which accepts a slice of key columns and supports the same merge types.

Function Signature

func (df *DataFrame) MergeOn(other *DataFrame, on []string, how MergeHow) (*DataFrame, error)

All key columns in on must exist in both DataFrames. Rows with a null in any key column never match. The result contains the left columns followed by the right columns excluding the join keys.

Example

Given a left DataFrame of sales and a right DataFrame of targets keyed by year and region:

inner, err := left.MergeOn(right, []string{"year", "region"}, dataframe.InnerMerge)
if err != nil {
    log.Fatalf("MergeOn failed: %v", err)
}
fmt.Println(inner.String())
+------+--------+-------+--------+
| year | region | sales | target |
+------+--------+-------+--------+
| 2020 | N      | 10    | 100    |
| 2021 | N      | 30    | 300    |
+------+--------+-------+--------+
[2 rows x 4 columns]

Only rows whose (year, region) pair exists in both DataFrames are kept. A left merge keeps every left row, filling unmatched right columns with null:

left, _ := left.MergeOn(right, []string{"year", "region"}, dataframe.LeftMerge)
fmt.Println(left.String())
+------+--------+-------+--------+
| year | region | sales | target |
+------+--------+-------+--------+
| 2020 | N      | 10    | 100    |
| 2020 | S      | 20    | null   |
| 2021 | N      | 30    | 300    |
+------+--------+-------+--------+
[3 rows x 4 columns]

See Also

Stacking & MultiIndex

Reshape between wide and long with Stack and Unstack, and build composite indexes with SetMultiIndex

Text, Time & Categorical

Work with string, datetime, and categorical column types

On this page

OverviewFunction SignatureParametersReturnsSample DataLeft DataFrame (df1) - EmployeesRight DataFrame (df2) - SalariesSetup CodeInner MergeInner Merge ExampleInner Merge OutputLeft MergeLeft Merge ExampleLeft Merge OutputRight MergeRight Merge ExampleRight Merge OutputFull Outer MergeFull Merge ExampleFull Merge OutputMerge Type ComparisonSQL EquivalentHandling Many-to-Many RelationshipsError HandlingCommon ErrorsExample Error HandlingComplete Example: Multi-Table JoinMerging on Multiple KeysFunction SignatureExampleSee Also