Unit 1: Encoding, hashing and encryption · Lesson 2 of 5 · 16 min
Everything is bytes
Read files as raw bytes with xxd so hashes, keys and ciphertext stop looking mysterious.
mission
- You are
- You are reviewing a bug report from the customer portal.
- Situation
- A customer named Zoë says her name breaks the parcel label printer. The label service counts characters, the printer counts bytes.
- Goal
- Show how text becomes bytes and why the two counts differ.
- Hand in
- The byte count of a word with an accent and the bytes behind that accent.
Letters sit on top of bytes
Dump a small file
xxd shows the offset, the bytes in hex and the printable characters. Crypto tools speak in bytes, so this is the view you will use all course.
xxd hello.txt
00000000: 4869 210a Hi!.
you should see Four bytes: 48 69 21 0a. The last one is the newline.
Look at a character outside ASCII
cat cafe.txt café xxd cafe.txt 00000000: 6361 66c3 a90a caf...
you should see The é takes two bytes in UTF-8: c3 a9.
Count bytes and characters
wc -c counts bytes, wc -m counts characters. Hashes and ciphers always work on bytes.
wc -c cafe.txt 6 cafe.txt wc -m cafe.txt 5 cafe.txt
defender tip When two systems hash the same text and get different results, check the encoding and the trailing newline first.
Turn hex back into bytes
Keys and hashes are often pasted as hex. xxd -r -p turns plain hex back into raw bytes.
echo 48692120 | xxd -r -p; echo Hi! printf 'Hi! ' | xxd -p 48692120
Byte tools
| Command | What it does |
|---|---|
xxd file | Hex dump with offsets and printable text |
xxd -p file | Plain hex, no offsets |
xxd -r -p | Plain hex back to bytes |
wc -c / wc -m | Count bytes / count characters |
What a defender takes away
- 1Text is stored as bytes, and one character can be several bytes.
- 2Hex is two characters per byte, so a 32-byte key is 64 hex characters.
- 3A trailing newline is a byte and changes every hash.
Questions
0 of 2 solved
Answer each one from what you find on your lab machine.