Bytes and encodings

A byte means nothing until two sides agree how to read it

one byte = 8 bits01000001These eight bits never change.The agreement picks the reader.65read as a whole numberAread as texta dark grey dotread as a pixel: brightness 65 of 255inc ecxread as a processor instruction: add 1Wrong agreement: no error message, just a wrong resultbytes C3 A9read as UTF-8: éread as Latin-1: é (the “café” you have seen on web pages)

Bits carry no meaning of their own. A file type, a data type or a protocol is the agreement that says which reader to use.

Bytes and encodings · Going deeper · 1 of 2

The Information stack — bit to bytes-on-the-wire Serialization structure flattened to bytes between programs · JSON, Protobuf Compression same data in fewer bits · lossless · lossy Number Text Image Audio Video int · two's c. float (754) ASCII · Unicode UTF-8 2-D pixel grid color depth samples · time rate · depth frames · time rate · codec interpretations · the agreements that turn bits into meaning Byte 8 bits · the addressable unit 0 / 1

The whole stack, bit to wire

This lesson climbs one stack of agreements: bits, bytes, numbers, text, media, then the tools that work on raw bytes. Most “corrupted file” symptoms are agreement mismatches: a JPEG opened as text, an integer field read as a float.

Bytes and encodings · Going deeper · 2 of 2

One byte read three ways 0 1 0 0 0 0 0 1 As a number 65 unsigned 8-bit int As a letter A ASCII codepoint As a pixel 25% gray (65/255)

One byte, three readings

The agreement lives in file headers, type systems and protocol specifications. The bits 01000001 are identical in every case; only the decoder differs.

Bits and binary

Two states survive noise, and every extra bit doubles what you can say

A bit in real mattera voltage, a magnet, a pit on a discthe line1010110Noise shakes the level, but not across the line.Each extra bit doubles the patterns1 bit22 bits43 bits84 bits165 bits326 bits648 bits256one letter, one shade of red32 bits4.3 billionevery IPv4 address64 bits1.8 × 10¹⁹every memory addressAlmost every design decision in computing is:how many bits do I spend here, and what does that buy?

A bit is the smallest distinction real matter can hold reliably. Stack n of them and you get 2ⁿ patterns.

Bits and binary · Going deeper

A bit holds one of two states 0 1 off · 0 · false on · 1 · true

The substrate does not matter

Voltage above or below a threshold, a magnetic domain pointing one way or the other, an optical pit present or absent: all are bits. Doubling the bit count squares the number of patterns, which is why 64-bit addresses will not run out the way 32-bit ones did.

Integers in binary

A number is the lit columns added up

12864321684210100000164 + 1= 6541hex 0x41: one digit per 4 bitsEight bits can only count this far025565255 + 1 wraps to 0: integer overflow16 bits reach 65,535. 32 bits reach 4,294,967,295. The size is fixed when the program is built.

Binary works like decimal with columns worth 1, 2, 4, 8. The number of columns is fixed, so every integer has a ceiling.

Integers in binary · Going deeper · 1 of 3

The value 65 in four common bases base prefix written as notes decimal binary octal hex (none) 0b 0o 0x 65 0b01000001 0o101 0x41 how humans count how hardware stores 3 bits per digit 4 bits per digit Hex dominates because two hex digits map cleanly to one byte.

One value, four notations

Decimal is for people, binary is what hardware stores, and hex is the readable shorthand because each hex digit is exactly four bits. Two hex digits cover one byte, which is why hex dominates debuggers, colour codes and addresses.

Integers in binary · Going deeper · 2 of 3

4-bit two's complement number line 1000 1001 1010 1011 1100 1101 1110 1111 0000 0001 0010 0011 0100 0101 0110 0111 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 leading bit = 1 → negative leading bit = 0 → non-negative

Negative numbers: make the top column negative

In two’s complement the top column is worth −128 instead of +128, so 1111 1011 is −5. To negate, invert every bit and add 1. The payoff: one adder handles both signs, and a − b is just a + (−b). The range is lopsided, −128 to +127, so negating the lowest value overflows.

Integers in binary · Going deeper · 3 of 3

Bitwise AND, OR, XOR, NOT on 8-bit values a = b = 1 1 0 0 1 1 0 1 1 0 1 0 1 1 1 0 AND OR XOR NOT a 1 0 0 0 1 1 0 0 1 1 1 0 1 1 1 1 0 1 1 0 0 0 1 1 0 0 1 1 0 0 1 0 mask · clear bits not in b set · turn on bits in b toggle · flip bits in b invert · flip every bit No carry between columns: each output bit depends only on the inputs at the same column.

Working on single bits

AND keeps only the bits in a mask, OR sets bits, XOR toggles them, NOT inverts all. They are how packed flags are read and written. Shifting left by one doubles a value in a single cycle, which is why compilers turn x × 2 into a shift.

Floating point

A fraction is stored as the nearest value, not the exact one

A float: 32 bits in three fieldssign: 1where the point sits: 8 bitsthe digits: 23 bits ≈ 7 decimal digits64-bit double: 52 bits ≈ 15 digitsSo only some values exist. Zoomed in near 0.1:0.099999994040.10000000894the 0.1 you typed: cannot be stored0.10000000149what is stored: the nearest tick0.1 + 0.2 == 0.3 → falsethe sum is 0.30000000000000004small errors add up; nothing warns youMoney: count whole cents in an integeror use a decimal typecompare floats with a tolerance, never ==

A float trades exactness for range. Most decimal fractions, 0.1 included, do not exist in binary, so they are rounded on the way in.

Floating point · Going deeper · 1 of 2

IEEE 754 single-precision float layout S EEEEEEEE MMMMMMMMMMMMMMMMMMMMMMM 1 bit 8 bits 23 bits sign exponent mantissa value = (-1)S × 1.M × 2E − bias

IEEE 754, field by field

Single precision is 32 bits split 1, 8 and 23; double is 64 bits split 1, 11 and 52. The stored exponent has a bias of 127 added. A few exponent values are reserved for ±0, ±infinity and NaN, the result of 0 ÷ 0.

Floating point · Going deeper · 2 of 2

Encoding 0.1 in IEEE 754 binary32 — why 0.1 + 0.2 is not 0.3 decimal 0.1 no exact binary form repeating binary fraction 0.0001100110011001100110011001100110011… truncate to 23 mantissa bits, round to nearest 0 01111011 10011001100110011001101 sign exponent (123 = -4 + 127) mantissa (last bit rounded up) decode back to decimal stored value 0.100000001490116119384765625

Where 0.1 goes wrong

0.1 in binary is a repeating fraction, cut off at 23 or 52 bits. The stored float32 is off by about 1.5 × 10⁻⁹, and adding two slightly-off numbers compounds the error. More bits shrink the error; they never remove it.

Check yourself

0.3 as stored0.1 + 0.2 as storedneighbouring values a float can hold

A shop adds 0.10 and 0.20 as floats, and the total fails a check against 0.30. What is the right fix?

Unicode and UTF-8

UTF-8 stores every script in one to four bytes

characterits Unicode numberthe bytes that are stored (UTF-8)AU+0041English letter0100000141éU+00E9accented Latin11000011C310101001A9€U+20ACmost other scripts11100010E2100000108210101100AC😀U+1F600emoji11110000F0100111119F10011000981000000080The purple bits are the agreement: 0 = one byte, 110 = two, 1110 = three, 11110 = four, 10 = “I continue the last one”.Unicode has room for 1,114,112 characters. A plain English file is already valid UTF-8, byte for byte.

Unicode gives every character a number. UTF-8 turns that number into bytes, and the first bits of each byte say how to read it.

Unicode and UTF-8 · Going deeper · 1 of 3

The 17 Unicode planes; the BMP holds most living text Plane 0 — BMP most living scripts U+0000 … U+FFFF supplementary planes (emoji, historic scripts, CJK extensions) 17 × 65,536 = 1,114,112 codepoints (U+0000 … U+10FFFF)

Number first, bytes second

Unicode has 1,114,112 slots in 17 planes, written U+0000 to U+10FFFF. It only assigns numbers; UTF-8, UTF-16 and UTF-32 are separate ways to write them as bytes. Confusing the two is behind half of all encoding bugs.

Unicode and UTF-8 · Going deeper · 2 of 3

Codepoints, glyphs, and grapheme clusters are not the same thing Codepoints Glyphs Grapheme cluster numbers in the standard drawings the font picks what the user sees U+0041 U+0301 LATIN CAPITAL A COMBINING ACUTE A ◌́ font draws each as a separate shape Á user sees one character 2 codepoints 2 glyphs 1 grapheme An emoji like 👨‍👩‍👧‍👦 takes this further: 7 codepoints (zero-width-joined), 7 glyphs, 1 grapheme.

Why string length lies

A codepoint is a number, a glyph is a drawing, and a grapheme is what the reader sees as one character. A family emoji is seven codepoints in one grapheme, so most languages report its length as 7 or 11, not 1.

Unicode and UTF-8 · Going deeper · 3 of 3

NFC versus NFD: two byte sequences for the same word NFC — composed NFD — decomposed 4 codepoints · 5 bytes UTF-8 5 codepoints · 6 bytes UTF-8 c a f é c a f e ◌́ U+0063 U+0061 U+0066 U+00E9 U+0063 U+0061 U+0066 U+0065 U+0301 precomposed é e + combining acute ≠ as bytes Both render identically as "café". Naive byte comparison says they differ.

One word, two byte sequences

“café” can store é as one codepoint or as e plus a combining accent. They look identical and compare unequal. Normalise on the way in (NFC) for filenames, identifiers and database keys; Linux and macOS disagree on the default.

Check yourself

636166C3A9the bytes that were saveda readercafé

A page shows “café” where the author typed “café”. What happened?

Sampling images and sound

Pictures and sound are measurements, and too few of them invent things

Enough samplesToo few samplesjoin the dots and the sound comes backCD: 44,100 measurements a second, 16 bits eachthe dots now trace a slow wave that was never thereno software can remove it afterwardsThe rule: measure at least twice per cycle of the highest pitch you want to keep.Measurements add up fast. One second of raw 1080p video at 60 frames:3 Gbps rawafter a video codec: 100 – 500× smaller. A raw 4K photo is 25 MB before compression.

Sound and light are continuous, so we sample them: measure at fixed steps. What is not captured then can never be recovered.

Sampling images and sound · Going deeper · 1 of 2

Three media as bit-pattern agreements Image Audio Video grid of pixels samples over time frames over time 8 × 8 px e.g. 44,100 samples / s time e.g. 24 frames / s

Three media, one trick

An image is a grid of pixels at 8 bits per colour channel, 16.7 million shades. CD audio is 44,100 samples a second, 16 bits, two channels: 1.41 Mbps. Each extra bit of depth adds about 6 dB of range, so 16 bits give 96 dB. Decode with the wrong depth or channel count and you get noise.

Sampling images and sound · Going deeper · 2 of 2

Adequate sampling versus aliasing above Nyquist — samples reconstruct the wave below Nyquist — the dots fit a slower wave that was never there The aliased reading is permanent — no decoder can undo undersampling.

Aliasing, precisely

Nyquist–Shannon: to keep frequencies up to f you need at least 2f samples a second. 44.1 kHz captures up to about 22 kHz, just above human hearing. Anything higher must be filtered out before sampling, in the analogue domain.

Compression

Compression gives the common things the short codes

ABRACADABRA11 letters. Stored plainly: 8 bits each.letterhow oftennew codebits usedA× 501 × 5 = 5B× 21103 × 2 = 6R× 21113 × 2 = 6C× 11003 × 1 = 3D× 11013 × 1 = 3most common → shortest coderare → longer codeplain88 bitscoded23 bits, and every letter comes backthe floor: 22.4 bits. No lossless code can average lower (Shannon).Lossless (ZIP, PNG) returns every byte. Lossy (JPEG, MP3, video) also drops what eyes and ears do not notice.

Real data repeats. Compression spends fewer bits on what is frequent, down to a floor set by how predictable the data is.

Compression · Going deeper · 1 of 3

Shannon entropy: how many bits per symbol the data really needs uniform distribution skewed distribution all 4 symbols equally likely one symbol dominates A B C D 0.25 0.25 0.25 0.25 A B C D 0.85 0.10 0.04 0.01 H = 2.00 bits H ≈ 0.79 bits incompressible (already maximal) huge headroom for compression H = −Σ p log₂ p the lower bound, in bits per symbol

The floor: entropy

Entropy measures how unpredictable a source is, in bits per symbol. Random bytes need all 8 and cannot shrink. A stream that is 99% one letter carries almost nothing per byte. No lossless coder beats this on average.

Compression · Going deeper · 2 of 3

LZ77 sliding window: re-emit a back-reference instead of repeating bytes already encoded (window) cursor / lookahead t h e c a t a n d t h e d o g "the " appears 12 bytes back; emit a reference instead of the literal (distance = 12, length = 4) A literal byte costs 8+ bits; a back-reference costs the same regardless of run length.

Pointing back instead of repeating

LZ77 slides a window over the input and replaces a repeated string with “go back 12 bytes, copy 4”. gzip, ZIP and PNG run this first, then Huffman codes on what is left. Together they are called DEFLATE.

Compression · Going deeper · 3 of 3

JPEG quality versus file size — diminishing returns above q=80 size quality → q=20 q=50 q=80 q=95 PNG 12 KB 28 KB 68 KB 180 KB 920 KB perception breaks below here visible blocking, ringing indistinguishable from lossless on a phone screen JPEG q=80 is the sweet spot for photos; q=95+ wastes bytes on detail nobody can see.

Lossy: pick the elbow

JPEG turns 8 × 8 pixel blocks into frequencies and drops the fine detail the eye barely sees. The first drop in quality is almost free; the last step to “perfect” costs ten times the bytes. Video codecs add motion between frames for 100 to 500 times.

Error detection and correction

Spare bits are how a machine notices its own mistakes

the data: 8 bitsspare bitArrives intact010000010two 1s: even ✓accepted, and it is correctOne bit flipped010100010three 1s: odd ✗caught: ask for it againTwo bits flipped010110010four 1s: even ✓accepted, and it is wrongThe sender sets the spare bit so the count of 1s is even. More spare bits buy more protection:Parity1 spare bit per bytespots a single flipCRC-3232 spare bits per messagespots bursts, then resendEthernet, ZIP, PNGError-correcting memoryabout 12% more bitsrepairs one flip on the spotservers, space probes

Disks, memory and radio flip bits. Adding redundant bits lets the receiver detect a flip, and with enough of them, repair it.

Error detection and correction · Going deeper

Detection versus correction — both add redundant bits, correction adds more parity / checksum error-correcting code detect → request retransmit detect AND repair in place data check data parity small overhead larger overhead e.g. 32 check bits per packet e.g. 8 check bits per 64-bit word Ethernet, ZIP, PNG, TCP ECC RAM, SSD controllers, QR codes, deep-space radio (Reed–Solomon, LDPC)

Detect or correct?

It depends on whether asking again is cheap. TCP can resend, so it only detects. Memory cannot, and a space probe is light-minutes away, so they correct. ECC memory fixes any single-bit error and detects any double-bit error per word; storage and radio use stronger codes such as Reed–Solomon and LDPC.

Check yourself

sent110100101received100110101parity

A byte is sent with one parity bit. On the way, two of its bits flip. What does the receiver do?

Hashes

A hash is a fixed-size fingerprint of any amount of data

any size inalways 256 bits outhellohello!a 4 GB film34 billion bitsSHA-256stirs every bitinto a fixed state2cf24dba5fb0a30e26e83b2ac5b9…ce06092fb948d9ffac7d1a376e40…still just 256 bitsone character more: nothing in commonSame bytes in, same fingerprint out. It cannot be run backwards.So two fingerprints that match mean the data matches: Git names every commit this way, and a download pagepublishes the hash so you can check the file. Fast, simpler hashes place keys in a hash table in one step.

A hash function turns any input into a short fixed-size value. Change one bit of the input and the whole value changes.

Hashes · Going deeper

A hash function maps arbitrary input to a fixed-size output "hello" "hello!" a 4-GB video file hash function 2cf24dba5fb0a30e… 334d016f755cd6dc… e8c52c1b9e7a0aff… Output is fixed-size (here 256 bits) regardless of input length. One bit of input change → completely different output.

Two families for two jobs

Fast hashes (xxHash, MurmurHash) spread keys evenly for hash tables, which stay O(1) while the table is under about 70% full. Cryptographic hashes (SHA-256, BLAKE3) are slower and built so that nobody can craft two inputs with the same output. Inside, the input is padded, cut into blocks and stirred round after round into a fixed state.

Serialization

To leave a program, data is flattened to bytes both ends agree on

The same record, written two waysJSON20 bytes{"id":150,"ok":true}the field names travel inside the bytes: anyone can read itProtobuf5 bytes0896011001purple = a field number (1, 2). The names live in a schema both sides hold.4× smaller, but unreadable without that schema. Never reuse a field number.Even one number needs an agreement: which byte goes first?writer (x86, ARM)means 305,419,89678563412smallest byte firstreader, other ordergets 2,018,915,34678563412same four bytes, read largest first: wrong, with no errorso every file format and protocol fixes the order in writing

Serialization turns structured data into bytes. The format is a contract: break it and the other side decodes nonsense without an error.

Serialization · Going deeper · 1 of 2

The 32-bit integer 0x12345678 in big-endian and little-endian memory value in a register 0x12345678 big-endian (network) little-endian (x86, ARM) 12 34 56 78 78 56 34 12 addr 0 addr 1 addr 2 addr 3 addr 0 addr 1 addr 2 addr 3 most-significant byte first least-significant byte first Read a little-endian file with a big-endian parser and every multi-byte field is silently scrambled.

Byte order across machines

Big-endian stores the most significant byte first and is the convention on the network; little-endian dominates CPUs (x86, ARM). Inside one machine it is invisible. At the boundary, code converts with htonl and ntohl.

Serialization · Going deeper · 2 of 2

Protobuf field tags survive across schema versions schema v1 schema v2 deployed last quarter today's code id = 1 email = 2 name = 3 (no field 4) id = 1 // reserved 2; name = 3 avatar = 4 field tags 1, 2, 3 stable 2 retired (tombstone), 4 added tag 1 — same tag 3 — same v2 code reads v1 bytes: unknown tag 2 is skipped; missing tag 4 reads as default. Reuse tag 2 for a different type and v1 readers happily decode garbage.

Changing a format without breaking old code

Protobuf identifies fields by number, not name. Add a field with a new number and old readers skip it. Retire one by reserving its number forever. Reuse a number for a different type and old code decodes garbage in production with no error.

Summary

Bits plus an agreement: that is all information is

what it isthe written agreementBitone two-state decision that survives noisea threshold in the hardwareByte8 bits, 256 patterns, no meaning yetthe unit every CPU addressesNumbers · text · pictures · soundthe same bytes, read by an agreed readerIEEE 754 · Unicode, UTF-8 · JPEG, PNGShrink · check · fingerprintwork on any bytes, whatever they meanDEFLATE · CRC-32 · SHA-256Bytes on a wire or diskstructure flattened for another programJSON · Protocol Buffersbuilt on

Every layer here is fixed by a written standard, which is why a file made on one machine opens on another decades later. Next: the machine that moves these bits.

Lesson complete

You finished “Information”

Answer the questions along the way to see your score here.

Open in a new tab