Topic: machine-learning
-
SHAP attributions: reference values, output units and causal limits
Explain a particular prediction relative to a declared reference, track the output scale, and avoid interpreting predictive contributions as intervention effects.
-
Early stopping: patience, validation and restoring the best weights
Treat the stopping epoch as a model choice, monitor a separate validation signal, and distinguish the last trained weights from the best observed weights.
-
Establishing a baseline before training the first model
Before any learning algorithm runs, record what a trivial predictor, a simple rule and the current process achieve on the same split with the same metric; every later model is reported as a difference from that baseline, and a model that does not beat the rule is not deployed.
-
Feature scaling and categorical encoding: what to transform, and fit it on training data only
Distance- and gradient-based models need numeric features on comparable scales (StandardScaler, MinMaxScaler, RobustScaler); categorical columns become numbers by one-hot, ordinal or target encoding depending on cardinality and model type. Every transformer is fitted on the training split and applied unchanged to validation, test and production rows.
-
Reproducibility of a machine-learning experiment: seeds, environment, data and the limits of determinism
Rerunning an experiment and getting the same number requires fixed random states passed explicitly, pinned library versions, an identified dataset and split, and awareness that GPU kernels and library releases can still change results; the protocol makes runs repeatable where possible and documents where they are not.
-
Training, validation and test sets: what each split is for and how to cut it
A model is fitted on the training set, choices between models and hyperparameters are made on the validation set (or by cross-validation), and the test set is touched once for the final estimate; how rows are assigned to splits (random, stratified, by group, by time) decides whether the estimate says anything about production.
-
Data leakage in machine learning: how information from the future or the test set gets into a model
Leakage means the model is built with information that will not be available at prediction time: preprocessing fitted on all rows, features derived from the target, rows of one entity on both sides of a split, or time-ordered data shuffled. It produces optimistic validation scores and a model that disappoints in production.
-
Gradient accumulation: effective batches without losing the correct normalization
Accumulate microbatch gradients to reduce activation memory, while preserving loss weighting, optimizer boundaries and clipping order; a larger effective batch is not always equivalent to one physical batch.
-
Label shift: detecting and correcting changing class proportions
Estimate changing class priors only under an explicit label-shift assumption, check identifiability and distinguish prior estimation from calibrated posterior correction.
-
Pool-based active learning: uncertainty sampling without biased evaluation
Select informative examples for annotation, maintain an independent representative evaluation set, and assess benefits against random sampling at the same labeling budget.
-
Knowledge distillation: transferring useful predictions from teacher to student
Train a student against both labels and a teacher’s probability distribution, with explicit temperature, loss weighting and independent evaluation. Compression benefits are conditional on teacher quality, student capacity and deployment behavior.
-
Incremental learning with partial_fit: stable features, class lists and honest evaluation
Incremental estimators update a model from small batches without loading the complete dataset into memory. A reliable workflow also preserves feature meaning, defines the class set and evaluates predictions before learning from their outcomes.
-
Histogram-based gradient boosting: bins, missing values and categorical limits
How histogram boosting reduces split-search work, handles missing and categorical inputs, and trades numerical resolution against training cost.
-
LoRA: low-rank adaptation and what its memory savings actually cover
LoRA learns a low-rank update while keeping base weights frozen. It reduces trainable parameters and their optimizer state, but the base model, activations and deployment compatibility still matter.
-
Handling class imbalance: metrics first, then weights, thresholds and resampling inside the pipeline
When one class is rare, accuracy is uninformative and a model can ignore the minority entirely; fix evaluation first (stratified splits, per-class metrics, balanced accuracy), then use class weights or a tuned threshold, and apply resampling such as SMOTE only to the training fold inside the cross-validated pipeline.
-
Split conformal prediction: intervals with marginal coverage
Convert a fixed regression model into a prediction-interval procedure using held-out residuals, a finite-sample quantile and clearly stated exchangeability assumptions.
-
Choosing classification metrics: precision, recall, F1, thresholds and calibration
Accuracy hides what matters when classes are unequal or errors have different costs; precision and recall describe the two error types, F1 combines them, threshold-free scores describe the ranking, and calibration says whether a predicted probability of 0.8 means 80 percent. Pick the metric from the decision the model supports, before training.
-
Mixed precision training: autocasting, loss scaling and numerical limits
Mixed precision assigns suitable floating-point formats to individual operations. Autocasting and loss scaling address different problems, and performance gains require both numerical checks and measurements on the intended hardware.
-
Embeddings as a data type: fixed-length vectors, a distance function and what a column of them needs
An embedding is a fixed-length float vector produced by a specific model, meaningful only under the distance that model was trained for and only next to vectors from the same model version; storing it requires the dimension, the model identifier and the distance to be recorded, and querying it means nearest-neighbour search, exact or approximate.
-
Contrastive self-supervised learning: pairs, augmentation and collapse
Learn representations by comparing related and contrasting views, choose transformations that preserve useful information, and evaluate representations beyond the training loss.
Machine-readable: JSON