Binary Code Analysis

Completion:classified accreditation
Type of
Hour/semLecturesSem. ExercisesLab. exercisesComp. exercisesOther
Guarantee:Vojnar Tomáš, prof. Ing., Ph.D., DITS
Lecturer:Kozina Stanislav, Mgr., RedHatCZ
Faculty:Faculty of Information Technology BUT
Department:Department of Intelligent Systems FIT BUT
Machine Level Programming (ISU), DITS
Operating Systems (IOS), DITS
Learning objectives:
  The goal is to acquaint students with the operation of modern Unix operating systems on a level close to the binary code and with available tools for observing the behaviour of such systems, including, in particular, their post-mortem analysis.
  This course deepens the knowledge and skill obtained in the course of Machine level programming (ISU) and in the course of Operating systems (IOS) with the main goal of allowing the students to understand the image of a crashed system (the so-called crash dump). Knowledge of programming on the level of assembler and application binary interfaces (ABI) is applied on a real Unix operating system. Within the course, various binary files used in the system are discussed, including their structure and their disassembled contents. The course involves a detailed study of compiler output from the point of view of linkage and run of system code as well as a discussion of differences and relations among various processor architectures, compilers, and application binary interfaces. Next, the course presents selected concepts typical for kernel-level programming whose deep knowledge is necessary for analysing functionality of an operating system kernel. These concepts include, among other, details of interrupt processing, task queues, process/thread synchronisation and memory management inside the kernel, i.e., the so-called SLAB allocator. Knowledge obtained in this way is used as a basis for presenting possibilities of monitoring the behaviour of an operating system on the binary level during its run as well as analysis of images of system memory after a system crash (crash dump analysis). In both cases, usage of tools shipped within common Unix distributions is accented.
Knowledge and skills required for the course:
  Proficiency in C language, x86 assembly code, understanding of operating system principles, practical experience with Unix systems.
Subject specific learning outcomes and competences:
  Practical experience with analysing the image of system memory after a system crash (crash dump analysis). Knowledge of the structure of binary files used in Unix systems (ELF). Understanding differences and relations between processors architectures, compilers, and ABI standards. Students who successfully pass the course will further be able to monitor the run of an operating system on a binary level during its runtime too.
Generic learning outcomes and competences:
  Improved knowledge in the areas of operating systems, machine languages, and debugging and analysis.
Syllabus of lectures:
  1. Computer architectures in general, registers, implicit and explicit stack operations. The x86 and x86_64 computer architectures, common instructions. System V ABI on x86_64 architecture, red zone.
  2. Compilation, linking, and running code. Examples of compiler optimizations, stack optimizations (leaf and tail calls, function inlining), linker script, understanding of the Unix binary file (ELF, objdump).
  3. Linux crash dump analysis, DWARF debug symbols, using the crash(8) tool.
  4. Linux crash dump analysis, understanding kernel Oops, kernel flags, sysrq.
  5. Kernel process and memory management, task_struct, vmas, SLAB allocator.
  6. Interrupt processing, postponing work, bottom halves, softirqs and tasklets, work queues.
  7. Live kernel tracing (SystemTap, ftrace), locks, deadlocks and hangs and their analysis and reconstruction.
Syllabus of computer exercises:
  1. Decomposition of an ELF binary file, decoding its sections, and code disassembling.
  2. Using the crash(1) tool on Linux.
  3. Crash dump analysis of a Linux system on the IA-32 architecture.
  4. Crash dump analysis of a Linux system on the AMD64 architecture.
  5. System tracing using SystemTap and ftrace.
  6. Tracing and analysis of system deadlocks.
Syllabus - others, projects and individual work of students:
  1. Analysis of a crash dump on the IA-32 architecture.
  2. Analysis of a crash dump on the AMD64 architecture.
  3. Monitoring of a running system using SystemTap.
Fundamental literature:
  • Ljubuncic, I.: Linux Kernel Crash Book, 2011.
  • Drake, C., Brown, K.: Panic! UNIX System Crash Dump Analysis, Prentice Hall, 1995.
  • Hofmann, F.: The Solaris Operating System on x86 Platforms, Crashdump Analysis, Operating System Internals, 2005.
Study literature:
  • Intel Corporation: Intel 64 and IA-32 Architectures Software Developer Manuals, 2015.
  • Matz, M., Hubicka, J., Mitchell, M.: System V Application Binary Interface, AMD64 Architecture Processor Supplement, 2013.
Controlled instruction:
  The obtained knowledge of students is examined through three projects focused on rootcausing  problems in an operating system from a provided crash dump (3x 20 points) and through a final written test (40 points).
Progress assessment:
  Projects will be evaluated based on technical reports with a detailed description of the root cause. The reports must be submitted by their deadlines, late submissions will be evaluated by 0 points.
Exam prerequisites:
  To successfully pass the course, a student needs to get at least 50 points in total out of which at least 24 points must be obtained from the projects and at least 16 points from the final test.