Pandas & NumPy
The data-wrangling operations every ML notebook runs on — selection, groupby, joins, reshaping, vectorization.
Selection (the part everyone fumbles)
- df.loc[rows, cols]
- by LABEL / boolean mask —
df.loc[df.x > 5, "y"] - df.iloc[i, j]
- by POSITION —
iloc[0]first row,iloc[:, -1]last col - Boolean masks
- combine with
& | ~and parentheses — neverand/or - df.query("x > 5 and y == 'a'")
- readable filters for chained conditions
- SettingWithCopyWarning
- assign via one
.loc[mask, col] = val, not chained brackets
Groupby & joins
- df.groupby("k")["v"].mean()
- split-apply-combine, the core move
- .agg(["mean","count"])
- multiple stats at once; dict for per-column stats
- .transform("mean")
- group stat broadcast back to original shape — dedaily-average in one line
- pd.merge(a, b, on="k", how="left")
- SQL joins;
indicator=Truedebugs match rates - pd.concat([a, b])
- stack frames;
axis=1for side-by-side
Reshape & clean
- pivot_table / melt
- long→wide with aggregation / wide→long (tidy)
- df.isna().sum()
- the null audit; then
fillna/dropna(subset=…) - df.drop_duplicates(subset=…)
- dedupe on keys, keep first/last
- pd.to_datetime + .dt
- parse once, then
.dt.date/.dt.dayofweek; resample by.resample("W") - astype("category")
- 10× memory win on low-cardinality strings
NumPy core
- Vectorize, never loop
a * b + con whole arrays — 100× over Python loops- Broadcasting
- shapes align right-to-left; (n,1) × (1,m) → (n,m)
- np.where(cond, x, y)
- vectorized if/else; chain with
np.select - argmax / argsort
- indices, not values — top-k =
argsort()[-k:][::-1] - @ / np.dot
- matrix multiply — cosine sim =
a@b / (norm(a)*norm(b)) - axis rule
axis=0collapses rows (per-column),axis=1collapses columns (per-row)