Bit manipulation basics: flags, masks and shifts without surprises

What it is

An integer can carry several independent booleans, one per bit. A flag is a power of two (1 << 0, 1 << 1, ...); a mask is a set of flags. The four operations: set with x | F, clear with x & ~F, toggle with x ^ F, test with (x & F) != 0. A shift multiplies or divides by a power of two and moves fields into place, so (x >> 8) & 0xFF extracts the second byte. The open(2) manual page shows the classic C convention: an access mode combined with flags such as O_CREAT and O_TRUNC using bitwise OR. Python's enum.Flag provides named flags whose members support &, |, ^ and ~, with auto() assigning powers of two.

Why it matters

File modes, socket options, permission bits, protocol headers, feature bitmaps and compact sets all use this representation. Errors are silent: a wrong mask reads a neighbouring field, a wrong test passes for every value.

How to apply

• Name every bit; magic numbers in & and | expressions are unreviewable. Use enum.Flag or IntFlag in Python and named constants elsewhere.
• Test a single flag with x & F against zero; test a multi-bit mask with (x & M) == M, since != 0 is true when only part of the mask is set.
• Update a field in two steps: clear it, then OR in the new value shifted into position and masked to its width.
• Know the width: MDN documents that JavaScript's bitwise operators convert operands to 32-bit integers, so values that do not fit in 32 bits need BigInt. Python integers are unbounded, so ~x equals -x - 1 rather than a fixed-width complement; mask with & 0xFFFFFFFF to emulate 32 bits.
• Use built-ins for counting and finding bits (int.bit_count(), int.bit_length(), bits.OnesCount) instead of loops.

Pitfalls

Precedence: in C-family languages & binds looser than ==, so x & F == 0 means x & (F == 0); parenthesise every test. Right shift of a negative value is arithmetic (sign-extending) with >> in Python and Java, logical with >>> in Java and JavaScript, and implementation-defined in C. Shifting by the operand width or more is undefined in C. ^ is XOR, not exponentiation. A flag defined as zero always passes the (x & M) == M test. Fields that cross byte boundaries in wire formats depend on byte order; pack and unpack with a declared layout rather than ad hoc shifts.