Enabling Dynamic Symbolic Execution for TLS Implementation Analysis
2025 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE credits
Student thesisAlternative title
Möjliggörande av dynamisk symbolisk exekvering för analys av TLS-implementationer (Swedish)
Abstract [en]
Ensuring TLS implementations behave as outlined in their specification is essential for secure communication over networks. Security flaws in protocol implementations can have severe consequences, as demonstrated by incidents like the Heartbleed vulnerability and the POODLE attack, leading to data breaches, man-in-the-middle attacks, and unauthorised access to sensitive information. This thesis investigates the use of dynamic symbolic execution to analyse WolfSSL, an open-source implementation of the TLS 1.3 protocol, with a specific focus on its ServerHello message parser. To uncover potential implementation errors, we derive conditions from the TLS 1.3 specification and use them to constrain symbolic execution to program paths where nonconformances may occur. We also investigate how the placement of such constraints, either directly when the message is received, before validation logic, or after message acceptance, affects time overhead, input space coverage, and generalisability. Additionally, we propose an alternative approach to accelerate the testing of multiple requirements by inserting them as logical queries into KQuery files, the intermediate representation of symbolic constraints generated by KLEE, after symbolic execution has completed. This approach eliminates the need to re-run KLEE for each requirement. Our methods are evaluated against a previous test-case–based approach by Wilson and Asplund, showing that our approaches manage to find the same number of nonconformances. The results indicate that the placement of assume statements affects execution time and state space exploration when running KLEE on WolfSSL, and these factors along with generalisability affect how the assume statements should be placed to obtain more desirable results in different scenarios. Our results also indicate that modifying KQuery files can significantly reduce time overhead while still allowing complete exploration of all input cases where a requirement might be violated.
Place, publisher, year, edition, pages
2025. , p. 53
Keywords [en]
Dynamic Symbolic Execution, Symbolic Execution, TLS (Transport Layer Security), TLS 1.3, WolfSSL, Protocol Analysis, Protocol Conformance, KLEE, Security Testing, Software Verification, Automated Testing
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:liu:diva-216024ISRN: LIU-IDA/LITH-EX-A--25/091--SEOAI: oai:DiVA.org:liu-216024DiVA, id: diva2:1981985
External cooperation
Sectra Communications AB
Subject / course
Computer Engineering
Presentation
2025-06-16, Alan Turing, Campus Valla, Linköping, 10:15 (Swedish)
Supervisors
Examiners
2025-11-272025-07-072025-11-27Bibliographically approved