Unit 6: HTTP and TLS on the wire · Lesson 5 of 5 · 20 min
Case file: the certificate nobody owned
Build an expiry report for every certificate, including the one buried in a vendor appliance, and assign owners.
case file
Ericsson Expired Certificate Mobile Outage
Ericsson (O2 UK, SoftBank and others) · 2018
On December 6, 2018, an expired certificate in Ericsson SGSN-MME core network software caused mobile data outages for customers of several operators in multiple countries, including O2 in the UK (and its resellers) and SoftBank in Japan. Millions of O2 customers lost data service, affecting services such as TfL's timetable information. Ericsson apologised and decommissioned the faulty software versions.
What went wrong
A certificate embedded in installed vendor software expired without being tracked or renewed, and the software failed hard when it did.
What would have stopped it
- Inventory all certificates, including those embedded in vendor appliances and software, with owners and expiry dates.
- Automate renewal and alert well ahead of expiry (for example 30/14/7 days).
- Ask vendors to document embedded certificates and design software to warn, not fail, as expiry approaches.
What defenders could have seen
- Certificate-expiry monitoring alerts on core network elements.
- Sudden, simultaneous failure across nodes running the same software version at a specific time.
mission
- You are
- The engineer asked to stop certificate outages for good.
- Situation
- Copies of every certificate corp.example knows about are in
~/certs, with owners inowners.csv. One came built into a core appliance. This lab is modelled on the 2018 Ericsson outage. - Goal
- List every certificate by days left as of 2026-10-14, flag the urgent ones and find those with no owner.
- Hand in
- An expiry report and an owner for every line.
Renewal windows
- 90 daysplan: confirm owner and renewal method
- 30 daysrenew now, alert the owner daily
- 0 daysexpired: outage or silent failure
Read the vendor certificate
Certificates built into appliances are easy to forget because nobody installed them.
ls *.pem core-vendor.pem intranet.pem mail01.pem old-portal.pem vpn-gw.pem www-corp.pem openssl x509 -in core-vendor.pem -noout -subject -issuer -enddate subject=CN = core-node.vendor.example, O = vendor.example issuer=CN = core-node.vendor.example, O = vendor.example notAfter=Dec 6 00:00:00 2026 GMT
you should see The core appliance's certificate expires on 6 December 2026.
Build the expiry report
expiry.py reads each certificate's end date with openssl and counts days from a fixed date, so the whole team sees the same numbers.
python3 expiry.py --as-of 2026-10-14
file expires days state
old-portal.pem 2026-08-31 -44 EXPIRED
vpn-gw.pem 2026-10-30 16 renew now
mail01.pem 2026-11-02 19 renew now
core-vendor.pem 2026-12-06 53 plan
www-corp.pem 2027-01-15 93 ok
intranet.pem 2028-05-01 565 ok
you should see One has already expired, two need renewal within 30 days, and the vendor certificate is next.
Shortlist the next 60 days
This is the list that goes into this week's change plan.
python3 expiry.py --as-of 2026-10-14 --within 60
file expires days state
old-portal.pem 2026-08-31 -44 EXPIRED
vpn-gw.pem 2026-10-30 16 renew now
mail01.pem 2026-11-02 19 renew now
core-vendor.pem 2026-12-06 53 plan
Find certificates without an owner
A certificate with no owner and no renewal method is how an outage happens.
grep -E 'unknown|none' owners.csv
core-vendor.pem,core appliance (built in by vendor),nobody listed,unknown
old-portal.pem,retired portal,it ops,none
you should see The vendor certificate has nobody listed and an unknown renewal method.
defender tip Ask every vendor to list certificates embedded in their products and how they are renewed. Put the answer in your inventory.
What a defender takes away
- 1Inventory every certificate, including ones vendors built in.
- 2Alert at 90, 30 and 7 days, and give each certificate an owner.
- 3Automate renewal wherever you can.
Questions
0 of 3 solved
Answer each one from what you find on your lab machine.