Pandas & NumPy

The data-wrangling operations every ML notebook runs on — selection, groupby, joins, reshaping, vectorization.

Selection (the part everyone fumbles)

df.loc[rows, cols]
by LABEL / boolean mask — df.loc[df.x > 5, "y"]
df.iloc[i, j]
by POSITION — iloc[0] first row, iloc[:, -1] last col
Boolean masks
combine with & | ~ and parentheses — never and/or
df.query("x > 5 and y == 'a'")
readable filters for chained conditions
SettingWithCopyWarning
assign via one .loc[mask, col] = val, not chained brackets

Groupby & joins

df.groupby("k")["v"].mean()
split-apply-combine, the core move
.agg(["mean","count"])
multiple stats at once; dict for per-column stats
.transform("mean")
group stat broadcast back to original shape — dedaily-average in one line
pd.merge(a, b, on="k", how="left")
SQL joins; indicator=True debugs match rates
pd.concat([a, b])
stack frames; axis=1 for side-by-side

Reshape & clean

pivot_table / melt
long→wide with aggregation / wide→long (tidy)
df.isna().sum()
the null audit; then fillna / dropna(subset=…)
df.drop_duplicates(subset=…)
dedupe on keys, keep first/last
pd.to_datetime + .dt
parse once, then .dt.date/.dt.dayofweek; resample by .resample("W")
astype("category")
10× memory win on low-cardinality strings

NumPy core

Vectorize, never loop
a * b + c on whole arrays — 100× over Python loops
Broadcasting
shapes align right-to-left; (n,1) × (1,m) → (n,m)
np.where(cond, x, y)
vectorized if/else; chain with np.select
argmax / argsort
indices, not values — top-k = argsort()[-k:][::-1]
@ / np.dot
matrix multiply — cosine sim = a@b / (norm(a)*norm(b))
axis rule
axis=0 collapses rows (per-column), axis=1 collapses columns (per-row)