HowToCodePage

String operations with .str

The .str accessor, regex extraction, contains and replace, splitting columns apart, and cleaning text columns without a Python loop.

The .str accessor

s = df["name"].astype("string")     # nullable string dtype

s.str.strip().str.title()
s.str.len()
s.str.upper()
s.str.contains("ada", case=False, na=False)
s.str.startswith("A").fillna(False)
s.str.zfill(6)
s.str.pad(10, side="right", fillchar=".")
  • Every method works element-wise and returns a Series, so operations chain without a loop.
  • .str requires text; on a mixed column cast first with astype("string") or the accessor raises.
  • Start and end matching with str.startswith is faster than the equivalent regex for simple prefixes.
  • Boolean results inherit NaN from missing input, which breaks filters - pass na=False or .fillna(False).

Extraction and replacement with regex

df["clean"] = df["text"].str.replace(r"\s+", " ", regex=True).str.strip()
df["digits"] = df["code"].str.extract(r"(\d+)", expand=False)

parsed = df["email"].str.extract(r"(?P<user>[^@]+)@(?P<domain>.+)$")
df["user"] = parsed["user"]
df["domain"] = parsed["domain"]

df["tag_list"] = df["tags"].str.findall(r"#(\w+)")
df["has_vip"] = df["tags"].str.contains(r"#vip\b", regex=True, na=False)
df["masked"] = df["phone"].str.replace(r"\d(?=\d{3})", "*", regex=True)
MethodReturns
containsBoolean Series - one flag per row
extractDataFrame of named groups, or a Series with expand=False
extractallOne row per match, keeping the original index
findallA list of all matches per row
replaceThe substituted strings
splitA list, or columns with expand=True
⚠️
A regex that fails to match yields NaN, not an error. Count matches after every extraction - df["digits"].isna().sum() - because a silently unmatched pattern looks exactly like a clean column of missing values.

Worked example: cleaning a name column

names = df["full_name"].astype("string")

parts = names.str.strip().str.replace(r"\s+", " ", regex=True).str.split(" ", n=1, expand=True)
df["first"] = parts[0].str.title()
df["last"]  = parts[1].fillna("").str.title()

titles = r"^(mr|mrs|ms|dr)\.?\s+"
df["first"] = (
    df["first"]
    .str.replace(titles, "", regex=True)
    .str.strip()
    .str.replace(r"[^\p{L}\-']", "", regex=True)   # keep letters, hyphen, apostrophe
)

df["initials"] = (
    df["first"].str[0].fillna("") + df["last"].str[0].fillna("")
).str.upper()

print(df[["first", "last", "initials"]].head())
print(df["first"].isna().sum(), "rows without a first name")

Build the pipeline in small steps and print the intermediate result after each one. Text cleaning fails by producing a plausible-looking column that is subtly wrong, so an eyeball check on real rows matters more here than in numeric work.

FAQ

Why do my boolean string filters drop rows?
A contains result is NaN wherever the input was missing, and NaN is falsy in a mask. Pass na=False.
object or string dtype?
The string dtype is explicit about being text and stores missing values as pd.NA. object is a mixed bag that may hold anything, so prefer string for real text columns.

Debugging pandas: SettingWithCopy, dtype and index surprises Categoricals, dtypes and memory optimisation

Last refreshed 2026-09-18.