apply, map and writing fast pandas code

map versus apply versus vectorised operations, transform for group-aligned results, pipe for readable chains, and why a row-wise loop is a last resort.

Vectorised first

import numpy as np
import pandas as pd

df["total"] = df["price"] * df["qty"]                 # whole-column arithmetic
df["share"] = df["total"] / df["total"].sum()
df["band"]  = np.where(df["total"] > 100, "high", "low")
df["tier"]  = np.select(
    [df["total"] > 1000, df["total"] > 100],
    ["gold", "silver"],
    default="bronze",
)
df["rank"]  = df["total"].rank(method="dense", ascending=False)
df["team_avg"] = df["total"].groupby(df["team"]).transform("mean")
df["diff"]  = df["total"] - df["team_avg"]
df["tight"] = df["total"].between(50, 150)

Arithmetic, comparison and np.select run in compiled code over the whole column; a Python callback runs once per row.
transform returns a result aligned to the original index, which is exactly what a new column needs.
Vectorised code is not only faster, it is shorter: the intent is visible instead of buried in a lambda body.

map, apply and pipe

df["label"] = df["code"].map({1: "low", 2: "high"})     # dict lookup
df["upper"] = df["name"].map(str.upper)                 # element-wise function
df["len"]   = df["name"].map(len)
df["score"] = df["score"].map({0: np.nan})              # unmapped entries default to NaN

df.map(lambda x: x if not isinstance(x, str) else x.strip())   # whole frame, 2.1+

df.apply(lambda col: col.max() - col.min())             # per column: cheap
df.apply(lambda r: r["price"] * r["qty"] * 1.2, axis=1) # per row: slow, last resort

def clean(d):   return d.assign(name=d["name"].str.strip())
def enrich(d):  return d.assign(total=d["price"] * d["qty"])

report = df.pipe(clean).pipe(enrich).pipe(lambda d: d.describe())

Tool	Reach for it when
Arithmetic / `where` / `np.select`	The rule can be expressed on whole columns
`Series.map`	Element-wise lookup by dictionary or a simple function
`Series.apply`	The transform is genuinely element-wise and hard to vectorise
`groupby.transform`	The result must be the same length as the frame
`DataFrame.apply` over columns	You need one scalar per column
`DataFrame.apply(axis=1)`	Nothing else works - accept the cost and measure
`pipe`	The value is the chain of operations, not any single step

💡

axis=1 builds a Series per row and calls Python for each one - often a hundred times slower than the vectorised equivalent and prone to dtype surprises, because a row of mixed types is promoted to object.

Worked example: same rule, two ways

# slow: a Python call per row, mixed dtypes, no parallelism
def tier(r):
    if r["total"] > 1000:
        return "gold"
    if r["total"] > 100:
        return "silver"
    return "bronze"

df["tier_slow"] = df.apply(tier, axis=1)

# fast: one pass over the column, same result
edges  = [0, 100, 1000, float("inf")]
labels = ["bronze", "silver", "gold"]
df["tier_fast"] = pd.cut(df["total"], bins=edges, labels=labels, right=True)

assert (df["tier_slow"] == df["tier_fast"].astype(str)).all()

The assertion is the point of the example: rewrite the rule, then prove the new version agrees with the old on the real data. Performance work without an equivalence check is guessing.

FAQ

map or apply on a Series?

map is for element-wise transformation, including dictionary lookups, and is usually faster. apply on a Series exists mainly for compatibility; prefer map for new code.

Is an explicit loop ever right?

When each step depends on the previous one - carrying a balance forward, for example. Even then, try shift, cumsum or ewm first; they cover most such cases.

Categoricals, dtypes and memory optimisation Debugging pandas: SettingWithCopy, dtype and index surprises

Last refreshed 2026-09-18.

apply, map and writing fast pandas code

Vectorised first

map, apply and pipe

Worked example: same rule, two ways

FAQ

Related