テーマ： data-modelling
-
Designing an append-only time-series table in PostgreSQL
Store measurements in a table partitioned by time range with timestamptz, a composite key of series and time, indexes matched to the query pattern (B-tree per series, BRIN for whole-table time scans) and retention implemented by detaching and dropping partitions instead of DELETE; the choices follow from rows arriving in time order and leaving in whole time slices.
-
Slowly changing dimensions: overwrite, add a row or add a column
When a dimension attribute changes (a customer moves, a product is reclassified), type 1 overwrites and loses history, type 2 adds a new row with effective and expiry dates and a current flag under a new surrogate key, and type 3 keeps the previous value in an extra column. Snapshot tools implement type 2 by comparing an update timestamp or a set of columns on each run.
-
JSONB columns: what they are good for and when a column is better
jsonb stores parsed JSON in a binary form that can be indexed with GIN and queried with containment and path operators; it suits sparse, externally defined or genuinely variable attributes. Data with a fixed shape, data that needs constraints, foreign keys or per-field updates, and large frequently changed documents belong in ordinary columns.
-
Consent and preference records as data: what was chosen, when and through which surface
Model a person's choices as append-only events (subject, purpose, choice, time, source, text version) with a derived current-state view that every consumer reads at the point of use; a boolean on the user row cannot answer what was agreed at the time of a given action or which users a banner bug affected.
-
Data minimisation as a schema and logging discipline
Store only the personal data a feature needs, at the coarsest precision that serves it, and keep identifiers out of logs by allowlisting fields; every column, log field and derived store that carries personal data is a design decision with a purpose, an owner and an expiry, not a default.
-
Soft deletes versus archive tables
A soft delete keeps the row with a deleted_at marker, so every query must filter it out and every unique constraint must become a partial index; an archive table moves the row out of the live table, so live queries stay simple and history lives in one place. Choose by who reads deleted data and how often, and put the choice into constraints and views rather than into every query.
-
Normalising to third normal form and choosing when to denormalise
Normal forms remove repeating groups and facts stored in more than one place; third normal form means every non-key column depends on the key and nothing else. Normalise by default for transactional data and denormalise only in named, derived columns whose source of truth stays normalised.
-
Declarative constraints in PostgreSQL: CHECK, UNIQUE and foreign keys with ON DELETE
Constraints make the database reject invalid states for every writer, not just the application: CHECK for per-row rules, NOT NULL for required values, UNIQUE for identity, and foreign keys with an explicit ON DELETE action. Choosing the referential action and indexing the referencing column are the two decisions most often skipped.
-
Star schema basics: facts, dimensions and declaring the grain
A star schema stores measurements in fact tables and descriptive context in dimension tables linked by keys; the design starts by declaring the grain, what one fact row represents, because every measure and dimension must be consistent with it. Fully additive measures sum across any dimension, semi-additive ones not across time, and ratios must be stored as their components.
-
Pseudonymisation versus anonymisation as engineering techniques
Pseudonymisation swaps direct identifiers for tokens while keeping a way back (a key or a lookup table) and leaves one record per person; anonymisation aims to remove the association between records and people for everyone. Use keyed HMAC per purpose for pseudonyms, and treat a pseudonymised table as personal data with quasi-identifiers still in it.
-
Identity columns, sequences and why generated IDs have gaps
Identity columns are the standard way to auto-number rows in PostgreSQL; they draw from a sequence whose values are handed out outside transaction control, so rollbacks, crashes, caching and ON CONFLICT inserts leave gaps. Gaps are normal; a gapless number needs a separate, serialised counter.
機械可読： JSON