Hook — Why IoT and Embedded Security Matters Right Now
Last week I watched a Mirai‑style botnet recruit 12,000 devices in under an hour — smart bulbs, DVRs, industrial gateways. The common thread? None of them had a verified secure boot chain, and most shipped with hard‑coded credentials that never changed after provisioning. As a researcher who spends evenings pulling firmware apart, I can tell you the attack surface of embedded Linux and RTOS targets is expanding faster than vendors can patch. If you’re building, deploying, or auditing any device that speaks IP, you need a repeatable methodology for firmware extraction, binary analysis, and runtime hardening — today.
Technical Explanation: The Attack Surface of Embedded Devices
Embedded systems differ from general‑purpose servers in three critical ways:
- Immutable storage layouts — flash partitions (bootloader, kernel, rootfs, config) are often mapped 1:1 to physical addresses, making
ddorflashromthe only reliable way to acquire a bit‑perfect image. - Proprietary bootloaders — U‑Boot, Barebox, or vendor‑specific loaders (e.g., Hikvision’s
hikboot) implement custom verification routines that can be bypassed via JTAG/SWD or UART console injection. - Legacy libraries and kernels — many devices run Linux 2.6/3.x or bare‑metal RTOS (FreeRTOS, ThreadX) with no ASLR, no stack canaries, and outdated
glibc/uClibcversions. This makes classic memory‑corruption exploits (stack overflow, heap spray, ROP) trivial once you have a shell.
The threat model therefore shifts from “network service exploitation” to “supply‑chain and physical access” — firmware tampering, insecure OTA updates, and debug interfaces left enabled in production.
Real-World Incidents: When Firmware Fails
In 2022, CVE‑2022‑27255 in TP‑Link Archer routers allowed unauthenticated command injection via the ping CGI endpoint. The root cause? A system() call on user‑supplied input inside a busybox httpd binary compiled without -fstack-protector-strong. Attackers leveraged the same binary to drop a persistent dropbear SSH daemon on the read‑write /tmp partition.
Another case: CVE‑2021‑36260 in Hikvision IP cameras. The vendor’s custom libhikvision.so exposed an unauthenticated GET /SDK/webLanguage endpoint that leaked the device’s AES‑128 key used for firmware encryption. With that key, researchers decrypted the OTA package, injected a reverse shell, and re‑signed the image using the vendor’s own private key (extracted from the bootloader). The result was a full‑device compromise without any network‑level vulnerability.
Both incidents share a pattern: trust boundaries are enforced only in software, never in hardware. When the bootloader accepts any image signed with a key that lives on the same flash, the chain of trust collapses.
Step-by-Step Technical Breakdown: Firmware Extraction and Analysis
Below is the exact workflow I use when a new device lands on my bench. It assumes you have physical access (UART, JTAG, or SPI flash clip) and a Linux analysis host.
1. Acquire a Raw Flash Image
Clip the SPI flash (Winbond 25Q128, 16 MiB) and dump it with flashrom. If the device exposes a UART console, you can also trigger a dd from the bootloader shell.
# Identify the programmer (Bus Pirate, FTDI, CH341A)
flashrom -p buspirate_spi:dev=/dev/ttyUSB0 -r firmware_raw.bin -c "W25Q128.V"
# Verify integrity
sha256sum firmware_raw.bin
# Compare with a second dump to rule out read errors2. Split Partitions with Binwalk
Binwalk’s signature scanner finds U‑Boot, kernel, squashfs, JFFS2, UBIFS, and even custom headers.
binwalk -e firmware_raw.bin
# This creates _firmware_raw.bin.extracted/ with each partition
ls -lh _firmware_raw.bin.extracted/
# Typical output:
# 0x000000 u-boot.bin
# 0x008000 kernel.lzma
# 0x1A0000 rootfs.squashfs
# 0x800000 config.jffs23. Mount and Enumerate the Root Filesystem
SquashFS is read‑only; mount it loopback to inspect binaries, scripts, and configuration.
sudo mount -o loop,ro _firmware_raw.bin.extracted/rootfs.squashfs /mnt/firmware
find /mnt/firmware -type f -name "*.sh" -o -name "*.cgi" -o -name "*httpd*" | head -20
# Look for hard‑coded credentials, suspicious binaries, or world‑writable dirs4. Static Binary Analysis
Pull the main HTTP daemon (often boa, lighttpd, or a custom httpd) and run it through checksec, ropper, and Ghidra.
checksec --file /mnt/firmware/usr/sbin/httpd
# Output example:
# RELRO STACK CANARY NX PIE RPATH RUNPATH Symbols FORTIFY Fortified Fortifiable FILE
# Partial RELRO No canary found NX enabled No PIE No RPATH No RUNPATH 89 No 0 2 /mnt/firmware/usr/sbin/httpd
# ROP gadget hunting
ropper --file /mnt/firmware/usr/sbin/httpd --search "pop rdi; ret" | head -10
# Decompile in Ghidra (headless)
analyzeHeadless /tmp/ghidra_project firmware_httpd /mnt/firmware/usr/sbin/httpd -scriptPath ./scripts -postScript FindHardcodedStrings.java5. Dynamic Analysis with QEMU User‑Mode Emulation
Because most embedded binaries are statically linked against uClibc or musl, you can run them directly under qemu‑arm‑static (or qemu‑mips‑static) with a chroot into the extracted rootfs.
# Install static qemu for the target arch
sudo apt-get install qemu-user-static
# Chroot and run the binary under gdb
sudo chroot /mnt/firmware /bin/bash -c "qemu-arm-static -g 1234 /usr/sbin/httpd -f /etc/httpd.conf"
# In another terminal
gdb-multiarch -ex "set architecture arm" -ex "target remote localhost:1234" \
-ex "break *0x00010420" -ex "continue" /mnt/firmware/usr/sbin/httpd6. Fuzzing the Web Interface
With the binary running under QEMU, point AFL++ or libFuzzer at the CGI entry points. I typically wrap the request parser in a harness that feeds mutated HTTP POST bodies.
# Build a minimal harness (C) that calls the vulnerable function directly
# Compile with AFL instrumentation
afl-clang-fast -O2 -fsanitize=address -o fuzz_httpd harness.c /mnt/firmware/usr/sbin/httpd
# Run
afl-fuzz -i seeds/ -o findings/ ./fuzz_httpd @@--security-opt seccomp=unconfined only for the fuzzer process.Defence and Mitigation Strategies
Hardening embedded Linux is a layered effort. Here are the controls I mandate for every device I ship or audit:
- Hardware Root of Trust — Use a secure element (ATECC608A, STSAFE‑A110) or the SoC’s fused OTP to store the root public key. The bootloader must verify a signed FIT image (U‑Boot
bootmwithverified-boot) before loading the kernel. - Signed, Encrypted OTA — Firmware packages should be AES‑256‑GCM encrypted and signed with ECDSA‑P256. The verification key lives only in the secure element; the decryption key is derived via a device‑unique KDF (e.g., HKDF‑SHA256 with the device UID).
- Disable All Debug Interfaces in Production — JTAG, SWD, UART console, and USB DFU must be permanently fused off (e.g.,
efusebits on ESP32,DBGENon ARM Cortex‑M). If field debugging is required, implement a challenge‑response unlock that requires a short‑lived token from your backend. - Compiler Hardening Flags — Build every userspace binary with
-fstack-protector-strong -D_FORTIFY_SOURCE=2 -pie -fPIE -Wl,-z,relro,-z,now. For kernels, enableCONFIG_HARDENED_USERCOPY,CONFIG_DEBUG_RODATA, andCONFIG_STRICT_DEVMEM. - Least‑Privilege Filesystem Layout — Mount
/read‑only (squashfs),/tmpand/varastmpfswithnoexec,nosuid,nodev. Drop capabilities in systemd units:CapabilityBoundingSet=CAP_NET_BIND_SERVICE. - Runtime Integrity Monitoring — Deploy a lightweight agent (e.g.,
osqueryor custominotifywatcher) that hashes critical binaries on boot and reports anomalies to a centralized log collector.
fwupd with the uefi-capsule plugin even on ARM devices — it gives you a standardized, signed update pipeline and integrates with LVFS for vendor‑wide distribution.Recommended Tools and Further Reading
- Firmware Extraction:
binwalk,firmware-mod-kit,sasquatch(for non‑standard squashfs),ubi_reader(UBIFS). - Static Analysis: Ghidra (headless), IDA Pro with
ida‑pro‑armplugin, Radare2/Cutter,checksec,ropper,pwntoolsfor exploit dev. - Dynamic/Emulation:
qemu-user-static,FirmAE,Firmadyne,IoT‑Inspector(dynamic taint),Frida+Objectionfor runtime hooking. - Hardware Debug:
OpenOCD+JTAGulator,ChipWhispererfor side‑channel,GreatFETfor USB/glitching. - Fuzzing: AFL++, libFuzzer,
boofuzzfor protocol fuzzing,nyxfor hypervisor‑based fuzzing of kernel drivers. - Reading: “Practical Reverse Engineering” (Bruce Dang), “The Hardware Hacker” (Andrew Huang), NIST SP 800‑193 (Platform Firmware Resiliency), and the
OWASP IoT Top 102024 edition.
Key Takeaways
- Embedded devices are not “small servers” — their trust anchors live in silicon, not just OS configuration.
- A repeatable firmware acquisition → extraction → static/dynamic analysis pipeline is the only way to scale vulnerability research across heterogeneous fleets.
- Secure boot, signed OTA, and fused debug ports are non‑negotiable baseline controls; everything else (ASLR, stack canaries) is defense‑in‑depth.
- Tooling has matured:
binwalk+qemu-user-static+Ghidragives you a full‑stack lab on a laptop. - Automate the pipeline (CI/CD for firmware) so every new build passes
checksec,fwupdsigning, and a quick AFL smoke test before release.