IDAClang
Overview
The IDAClang plugin integrates the clang compiler frontend into IDA itself. This allows IDA to parse type information from complex C/C++/Objective-C source code and import it directly into an IDA database.
Libclang
IDAClang utilizes a specialized build of libclang - the opensource C API for the clang compiler. This custom library is also shipped with IDA alongside the plugin itself, so you do not need to worry about it. The plugin will find and load libclang automatically.
Our build of libclang is from Clang v13.0, so it can handle any Objective-C syntax and anything from C++20 and earlier.
Motivation
IDAClang was introduced as a more robust alternative to IDA’s built-in source code parser. The built-in parser can handle simple C source code, but naturally it struggles to handle complex C++ and Objective-C syntax. IDAClang solves this problem by outsourcing all the heavy lifting to a third-party library that can handle the ugly parsing operations. The plugin needs only to parse the abstract syntax tree generated by clang.
As a result, IDAClang should be much more flexible. You can even feed it complete .cpp source files. The plugin will extract whatever useful type information it can find, and ignore the rest.
VTables
One big advantage of using libclang is that we can take advantage of clang’s internal C++ VTable management. For example, when IDAClang parses a C++ class that looks like this:
The following types will be generated in the database:
To create the C_vtbl type, IDAClang traverses clang’s internal VTableLayout data structure. This data structure is the same mechanism that the clang compiler uses during the actual code generation. Thus, we can be very confident that IDAClang is producing correct vtable types - even in much more complex situations. After all, clang knows what it’s doing in this regard.
Moreover, when using IDAClang to generate a type library (see Building Type Libraries with IDAClang below), the plugin will take advantage of clang’s name mangling to populate the symbol table:
Here IDAClang created symbols for the C::func member function, as well as the mangled VTable symbol for the C class.
Templates
Another notable advantage of using libclang is it allows us to gracefully handle C++ templates.
For example, consider the following template declarations:
When clang parses the instance_t declaration, internally it will generate a structure that represents the specialized template S<int, void *>. The IDAClang plugin will then use this internal representation to generate a valid type for S<int, void *> in IDA’s type system:
The type with name S<int, void *> represents the fully resolved structure, with all template arguments replaced. This all happens automatically, and it is especially useful in more complex situations - such as template classes containing virtual methods that depend on template parameters, resulting in specialized VTables.
The IDAClang UI
Enabling the IDAClang Parser
To provide support for third-party parsers, IDA now has a new Source parser field in the Options>Compiler dialog:
To enable the IDAClang parser, select the clang parser from the dropdown menu:
As a quick sanity check, try saving the following declaration in a file called test.h:
Parse the file using menu File>Load file>Parse C header file. IDA should print this to the output window:
The type should now be present in the Local Types view:
Configuring IDAClang
Of course, IDAClang is capable of parsing source code that is much more complex. Often times this requires more detailed configuration of the parser invocations.
To support this, the Compiler>Options dialog provides the Arguments field:
In this field you can provide any argument you would typically provide to the clang compiler when invoking it from the command line. For example:
One of the more important clang arguments is the -target option, which specifies the target architecture and platform. This allows clang to properly configure itself to parse macOS/Windows/Linux system headers. Clang calls this the target "triple" because it is often given in the form of:
Some examples:
The various combinations of supported targets is documented in more detail here.
Note that in the simple test.h example above, we did not specify a target platform. In this case clang will assume that the target platform is the same as the host machine IDA is currently running on. You can print the exact target used by clang by opening Options>Compiler>Parser specific options and enable the following option:
Now when we use IDAClang to parse the test.h file, it will print a message:
Which would be the typical output when IDA is running on macOS. On Windows the default will look something like:
And on Linux:
Such is the default behavior within libclang, but clang supports a wide variety of platforms and architectures. You can almost always specify a target that will match the input binary in the current database.
STL Example
Now let’s try invoking IDAClang on some more real-world source code.
In this example, assume we are analyzing an x64 binary that makes heavy use of the C++ Standard Template Library. Then assume that at some point we want to create a structure that looks like this:
This is the contents of stl/stl_example.h from examples.zip. IDA’s default parser cannot handle such complex C++ syntax, so IDAClang is our only hope of importing this type. The precise configuration of IDAClang will vary between platforms, so we’ll demonstrate them all separately.
To parse stl_example.h on macOS, we’ll have to point IDAClang to the macOS SDK as well as the STL system headers:
Copy the text above into the Options>Compiler>Arguments field.
Note that we point IDAClang to the macOS SDK with the -isysroot option and use the -I option to allow IDAClang to find the proper system headers in the Xcode toolchain. Be wary of the last option (ending with usr/lib/clang/11.0.3/include). This path contains the clang version number, so it might be different on your machine. Also make special note of the -x c++ option. This is used to inform libclang that the input source will not be plain C, which is the default syntax for .h files in libclang.
Now we can use File>Load file>Parse C header file to parse stl_example.h. This will generate a useful type for stl_example_t in our database:
On Windows the configuration is a bit different. If you’re using Visual Studio, libclang is normally able to detect common header paths automatically.
Thus you will likely only need to specify the following arguments in Options>Compiler>Arguments:
Ideally this will be enough to parse stl_example.h and generate some useful type info:
If for whatever reason the heuristics within libclang fail to find the headers on your system, it is very easy to specify the header paths manually. Simply open a Visual Studio x64 Command Prompt and run the following command:
This will print a semicolon-separated list of the header paths used on your system:
This list can be copied directly into the Options>Compiler>Include directories field in IDA. IDAClang will automatically process this list and pass the header paths to clang upon invocation of the parser. This is likely enough to handle most Windows-based source code.
On Linux you can determine the header paths used your system by running the following command:
This will print something like:
You can then use these arguments in the Options>Compiler>Arguments field in IDA:
Then use File>Load file>Parse C header file to parse stl_example.h.
Invoking IDAClang from IDAPython
Like any good IDA feature, IDAClang can also be invoked from an IDAPython script.
IDA 7.7 introduced the ida_srclang module to provide simple support for invoking third-party parsers from IDAPython. Use the following IDAPython commands for an overview of this new module:
The function ida_srclang.parse_decls_with_parser can notably be used to parse source code snippets:
If the is_path argument is False, this function will assume the input argument is a string that represents a source code snippet. Otherwise it will be considered a path to a source file on disk. Also note the til parameter, which will often times be None. This ensures the parsed types are imported directly into the current database.
Examples
IMPORANT NOTE: when libclang parses in-memory strings, it makes no assumptions about the expected syntax. Thus, you must specify the -x option to tell clang which syntax to expect before invoking the parser. Here are the the known syntax directives:
For example, this is how you would use ida_srclang to parse a simple C source string with IDAClang:
STL Example Revisited
We can also handle the same STL example discussed previously, but this time parse stl_example_t as a source snippet:
This should produce an identical result as before when we used File>Load file>Parse C header file for stl_example.h.
Boost Example
In this example we will show how IDAClang can be used in batch mode to improve the analysis of a binary compiled from Boost headers. The experiment will be performed on Debian Linux with gcc 6.3.0.
Consider the following source files from the boost/ directory in examples.zip:
chat_server.cpp
chat_message.hpp
These sources were taken directly from the Boost 1.77 examples, and we’ll use them to compile a test binary. Begin by downloading the Boost 1.77.0 headers, then compile the chat_server application:
Since Boost is a template library, it will generate a bloated binary that contains thousands of instantiated template functions. Thus, IDA’s initial analysis of chat_server.elf will likely not be very pretty. How can IDAClang help us with this? Consider boost/chat_server.py from examples.zip.
This script will configure IDAClang to parse the chat_server.cpp source file and extract any type information it finds, then analyze the input with the imported type info, and saves the resulting database in chat_server.i64. You can run the script like this:
You may have noticed this option:
This option is passed to the IDAClang plugin and it enables CLANG_APPLY_TINFO (see idaclang.cfg for more info).
Now let’s open the resulting database chat_server.i64 in IDA, and try decompiling some functions. Immediately we see that the analysis does benefit from the imported type info. For example chat_session::do_write seems somewhat intelligible after some minor simplifications:
Since IDAClang parsed the chat_session class, we now have a correct prototype for chat_session:do_write, as well as a valid chat_session structure. Note that references to chat_session.write_msgs_ (std::deque<chat_message>) and chat_session.socket (boost::asio::ip::tcp::socket) were correctly resolved.
Granted, this is not the most realistic example. It’s not often we have access to the full source code of the target binary, but hopefully this shows that whenever any relevant source code is available, IDAClang can take full advantage.
Building Type Libraries with IDAClang
The IDAClang plugin is useful for enriching your database with complex type information, but often times the imported types are relevant to more than just one database. In this section we discuss how you can use IDAClang to generate rich, generic type libraries for IDA Pro.
Hex-Rays also provides a command-line version of IDAClang, specifically designed for building custom Type Information Libraries (TILs) that can be loaded into any IDA database.
After downloading the idaclang binary, copy it to the idabin/ directory of your IDA installation (next to the libclang dll).
For an overview of idaclang’s functionality, run:
For a quick demonstration, save the following source in a file named test.h:
You can compile this header into a type library by invoking idaclang the same way you would typically invoke the clang compiler from the command line:
This will generate a file called test.til that contains all types that were parsed in test.h. Try dumping the TIL with the tilib utility.
The tool also provides extra arguments to configure the til generation. They are given the --idaclang- prefix so they can be easily separated from the clang arguments. For example:
This will create the library at /tmp/test2.til, instead of the default location.
Now let’s try building some type libraries from real-world code. The examples in this section will demonstrate the power of IDAClang by creating TILs from many different opensource C++ projects. They cover a large variety of platforms, architectures, and codebases, so it is best to unify the build system using makefiles.
At the top level of examples.zip there should be a makefile named idaclang.mak:
This makefile defines a simple rule for building a TIL using the idaclang command-line utility. It will be used extensively in the following examples.
IDASDK
Hex-Rays publishes an SDK for developing custom IDA plugins, which is comprised mostly of C++ header files. Thus, it is a perfect use case for IDAClang. In this example we will build a type library for IDA itself, using IDA SDK 7.7.
After downloading idasdk77.zip, unzip it into the idasdk subdirectory of examples.zip.
To build this TIL we only need to create a single header file that includes all headers from the IDA SDK, and then parse this file with idaclang. See examples/idasdk/idasdk.h, which contains include directives for all files in idasdk77/include (they happen to be in alphabetical order, but the order shouldn’t matter much):
The IDAClang configuration required to parse idasdk.h is highly platform-dependent, so we provide separate makefiles for each of IDA’s supported platforms.
To demonstrate how we might build idasdk.h on MacOSX, see examples/idasdk/idasdk_mac_x64.mak:
You can build the TIL with:
This will generate a type library named idasdk_mac_x64.til, along with a dump of the til contents in idasdk_mac_x64.til.txt. In the text dump we might notice some familiar types:
It’s worth building a separate til for both x64 and arm64 macOS. IDA’s source code is not very architecture dependent, but many system headers might be. So it’s best to be as precise as possible.
To build this TIL on macOS12 for Apple Silicon, the approach is very similar:
Note that we did not provide the path to the C++ STL headers like we did in idasdk_mac_x64.mak. On macOS12 the C++ headers are shipped within MacOSX12.0.sdk, so there is no need to explicitly tell idaclang where to find them.
To parse idasdk.h on Windows, use examples/idasdk/idasdk_win.mak:
Normally we do not need to specify any include paths, since idaclang can find the Visual Studio headers automatically. If it can’t, you can always explicitly provide include paths with the -I option.
Building idasdk.h on Linux is also fairly straightforward. See idasdk_linux.mak:
You can also include the decompiler types from the hexrays SDK in the type library for idasdk77. Simply copy hexrays.hpp from hexrays_sdk/ in your IDA installation to idasdk77/include/, then add this line to idasdk.h:
Then rebuild the TIL. It will likely yield some useful decompiler types: