You may either start a local debugging session on a new process or start a local debugging session and attach it to an existing process. Both options are accessible through the command line.

idal -rlinux MY_PROGRAM

will start the program, create a temporary database that allows the user to work with the target at once.

The command

idal -rlinux+

will offer you a choice of running processes to connect to.

and we can proceed with our local Linux debugging session.

The IDA Debugger allows you to either start a new process (Run) or attach itself to an existing process (Attach)

Let's select "attach to local win32". What we get is a list of currently running processes to which we can attach with a simple click.

and here is the result once we are attached to the program.

Debugging Mac OSX Applications with IDA Pro

Last updated on March 6, 2021 — v2.0

Overview

IDA Pro fully supports debugging native macOS applications.

Intel x86/64 debugging has been supported since IDA 5.6 (during OSX 10.5 Leopard), but due to IDA’s use of libc++ we can only officially support debugging on OSX 10.9 Mavericks and later. Apple Silicon arm64 debugging for macOS11 is also supported since IDA 7.6.

Note that this task is riddled with gotchas, and often times it demands precise workarounds that are not required for other platforms. In this tutorial we will purposefully throw ourselves into the various pitfalls of debugging on a Mac, in the hopes that learning things the hard way will ultimately lead to a smoother experience overall.

Begin by downloading samples:

which contains the sample applications used in this writeup.

Codesigning & Permissions

It is important to note that a debugger running on macOS requires special permissions in order to function properly. This means that the debugger itself must be codesigned in such a way that MacOS allows it to inspect other processes.

The main IDA Pro application is not codesigned in this way. Later on we’ll discuss why.

To quickly demonstrate this, let’s open a binary in IDA Pro and try to debug it. In this example we’ll be debugging the helloworld app from samples.zip:

on MacOSX 10.15 Catalina using IDA 7.5. Begin by loading the file in IDA:

$ alias ida64="/Applications/IDA\ Pro\ 7.5/ida64.app/Contents/MacOS/ida64"
$ ida64 helloworld

Now go to menu Debugger>Select debugger and select Local Mac OS X Debugger:

Immediately IDA should print a warning message to the Output window:

This program must either be codesigned or run as root to debug mac applications.

This is because IDA is aware that it is not codesigned, and is warning you that attempting to debug the target application will likely fail. Try launching the application with shortcut F9. You will likely get this error message:

Codesigning IDA Pro might resolve this issue, but we have purposefully decided not to do this. Doing so would require refactoring IDA’s internal plugin directory structure so that it abides by Apple’s bundle structure guidelines. This would potentially break existing plugins as well as third-party plugins written by users. We have no plans to inconvenience our users in such a way.

Also note that running IDA as root will allow you to use the Local Mac OS X Debugger without issue, but this is not advisable.

A much better option is to use IDA’s mac debug server - discussed in detail in the next section.

Using the Mac Debug Server

A good workaround for the debugging restrictions on macOS is to use IDA’s debug server - even when debugging local apps on your mac machine. The mac debug server is a standalone application that communicates with IDA Pro via IPC, so we can ship it pre-codesigned and ready for debugging right out of the box:

$ codesign -dvv /Applications/IDA\ Pro\ 7.5/idabin/dbgsrv/mac_server64
Executable=/Applications/IDA Pro 7.5/ida.app/Contents/MacOS/dbgsrv/mac_server64
Identifier=com.hexrays.mac_serverx64
Format=Mach-O thin (x86_64)
CodeDirectory v=20100 size=6090 flags=0x0(none) hashes=186+2 location=embedded
Signature size=9002
Authority=Developer ID Application: Hex-Rays SA (ZP7XF62S2M)
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=May 19, 2020 at 4:13:31 AM

Let’s try launching the server:

$ /Applications/IDA\ Pro\ 7.5/idabin/dbgsrv/mac_server64
IDA Mac OS X 64-bit remote debug server(MT) v7.5.26. Hex-Rays (c) 2004-2020
Listening on 0.0.0.0:23946...

Now go back to IDA and use menu Debugger>Switch debugger to switch to remote debugging:

Now use Debugger>Process options to set the Hostname and Port fields to localhost and 23946.

(Note that the port number was printed by mac_server64 after launching it):

Also be sure to check the option Save network settings as default so IDA will remember this configuration.

Now go to _main in the helloworld disassembly, press F2 to set a breakpoint, then F9 to launch the process. Upon launching the debugger you might receive this prompt from the OS:

macOS is picky about debugging permissions, and despite the fact that mac_server is properly codesigned you still must explicitly grant it permission to take control of another process. Thankfully this only needs to be done once per login session, so macOS should shut up until the next time you log out (we discuss how to disable this prompt entirely in the Debugging Over SSH section below).

After providing your credentials the debugger should start up without issue:

Using a Launch Agent

To simplify using the mac server, save the following XML as com.hexrays.mac_server64.plist in ~/Library/LaunchAgents/:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.hexrays.mac_server64</string>
    <key>ProgramArguments</key>
    <array>
        <string>/Applications/IDA Pro 7.5/dbgsrv/mac_server64</string>
        <string>-i</string>
        <string>localhost</string>
    </array>
    <key>StandardOutPath</key>
    <string>/tmp/mac_server64.log</string>
    <key>StandardErrorPath</key>
    <string>/tmp/mac_server64.log</string>
    <key>KeepAlive</key>
    <true/>
</dict>
</plist>

Now mac_server64 will be launched in the background whenever you log in. You can connect to it from IDA at any time using the Remote Mac OS X Debugger option. Hopefully this will make local debugging on macOS almost as easy as other platforms.

Debugging System Applications

There are some applications that macOS will refuse to allow IDA to debug.

For example, load /System/Applications/Calculator.app/Contents/MacOS/Calculator in IDA and try launching the debugger. You will likely get this error message:

Despite the fact that mac_server64 is codesigned, it still failed to retrieve permission from the OS to debug the target app. This is because Calculator.app and all other apps in /System/Applications/ are protected by System Integrity Protection and they cannot be debugged until SIP is disabled. Note that the error message is a bit misleading because it implies that running mac_server64 as root will resolve the issue - it will not. Not even root can debug apps protected by SIP.

Disabling SIP allows IDA to debug applications like Calculator without issue:

The effects of SIP are also apparent when attaching to an existing process. Try using menu Debugger>Attach to process, with SIP enabled there will likely only be a handful of apps that IDA can debug:

Disabling SIP makes all system apps available for attach:

It is unfortunate that such drastic measures are required to inspect system processes running on your own machine, but this is the reality of MacOS. We advise that you only disable System Integrity Protection when absolutely necessary, or use a virtual machine that can be compromised with impunity.

Debugging System Libraries

With IDA you can debug any system library in /usr/lib/ or any framework in /System/Library/.

This functionality is fully supported, but surprisingly it is one of the hardest problems the mac debugger must handle. To demonstrate this, let’s try debugging the _getaddrinfo function in libsystem_info.dylib.

Consider the getaddrinfo application from samples.zip:

#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <string.h>
#include <stdio.h>

int main(int argc, char **argv)
{
  if ( argc != 2 )
  {
    fprintf(stderr, "usage: %s <hostname>\n", argv[0]);
    return 1;
  }

  struct addrinfo hints;
  memset(&hints, 0, sizeof(hints));

  hints.ai_family = AF_INET;
  hints.ai_flags |= AI_CANONNAME;

  struct addrinfo *result;
  int code = getaddrinfo(argv[1], NULL, &hints, &result);
  if ( code != 0 )
  {
    fprintf(stderr, "failed: %d\n", code);
    return 2;
  }

  struct sockaddr_in *addr_in = (struct sockaddr_in *)result->ai_addr;
  char *ipstr = inet_ntoa(addr_in->sin_addr);
  printf("IP address: %s\n", ipstr);

  return 0;
}

Try testing it out with a few hostnames:

$ ./getaddrinfo localhost
IP address: 127.0.0.1
$ ./getaddrinfo hex-rays.com
IP address: 104.26.10.224
$ ./getaddrinfo foobar
failed: 8

Now load libsystem_info.dylib in IDA and set a breakpoint at _getaddrinfo:

$ ida64 -o/tmp/libsystem_info /usr/lib/system/libsystem_info.dylib

Choose Remote Mac OS X Debugger from the Debugger menu and under Debugger>Process options be sure to provide a hostname in the Parameters field. IDA will pass this argument to the executable when launching it:

Before launching the process, use Ctrl+S to pull up the segment list for libsystem_info.dylib. Pay special attention to the __eh_frame and __nl_symbol_ptr segments. Note that they appear to be next to each other in memory:

This will be important later.

Finally, use F9 to launch the debugger and wait for our breakpoint at _getaddrinfo to be hit. We can now start stepping through the logic:

Everything appears to be working normally, but use Ctrl+S to pull up the segment information again. We can still see __eh_frame, but it looks like __nl_symbol_ptr has gone missing:

It is actually still present, but we find it at a much higher address:

Recall that we opened the file directly from the filesystem (/usr/lib/system/libsystem_info.dylib). However this is actually not the file that macOS loaded into memory. The libsystem_info image in process memory was mapped in from the dyld_shared_cache, and the library’s segment mappings were modified before it was inserted into the cache.

IDA was able to detect this situation and adjust the database so that it matches the layout in process memory. This functionality is fully supported, but it is not trivial. Essentially the debugger must split your database in half, rebase all code segments to one address, then rebase all data segments to a completely different address.

It is worth noting there is another approach that achieves the same result, but without so much complexity.

Debugging Modules in dyld_shared_cache

As an alternative for the above example, note that you can load any module directly from a dyld_shared_cache file and debug it. For example, open the shared cache in IDA:

$ ida64 -o/tmp/libsystem_info2 /var/db/dyld/dyld_shared_cache_x86_64h

When prompted, select the "single module" option:

Then choose the libsystem_info module:

Select the Remote Mac OS X Debugger and for Debugger>Process options use the exact same options as before:

Now set a breakpoint at _getaddrinfo and launch the process with F9.

After launching the debugger you might see this warning:

This is normal. Modules from the dyld_shared_cache will contain tagged pointers, and IDA patched the pointers when loading the file so that analysis would not be hindered by the tags. IDA is warning us that the patches might cause a discrepancy between the database and the process, but in this case we know it’s ok. Check Don’t display this message again and don’t worry about it.

Launching the process should work just like before, and we can start stepping through the function in the shared cache:

This time there was no special logic to map the database into process memory. Since we loaded the module directly from the cache, segment mappings already match what’s expected in the process. Thus only one rebasing operation was required (as apposed to the segment scattering discussed in the previous example).

Both techniques are perfectly viable and IDA goes out of its way to fully support both of them. In the end having multiple solutions to a complex problem is a good thing.

Debugging Objective-C Applications

When debugging macOS applications it is easy to get lost in some obscure Objective-C framework. IDA’s mac debugger provides tools to make debugging Objective-C code a bit less painful.

Consider the bluetooth application from samples.zip:

#import <IOBluetooth/IOBluetooth.h>

int main(void)
{
  NSArray *devices = [IOBluetoothDevice pairedDevices];
  int count = [devices count];
  for ( int i = 0; i < count; i++ )
  {
    IOBluetoothDevice *device = [devices objectAtIndex:i];
    NSLog(@"%@:\n", [device name]);
    NSLog(@"  paired:    %d\n", [device isPaired]);
    NSLog(@"  connected: %d\n", [device isConnected]);
  }
  return 0;
}

The app will print all devices that have been paired with your host via Bluetooth. Try running it:

$ ./bluetooth
2020-05-22 16:27:14.443 bluetooth[17025:15645888] Magic Keyboard:
2020-05-22 16:27:14.443 bluetooth[17025:15645888]   paired:    1
2020-05-22 16:27:14.443 bluetooth[17025:15645888]   connected: 1
2020-05-22 16:27:14.443 bluetooth[17025:15645888] Apple Magic Mouse:
2020-05-22 16:27:14.443 bluetooth[17025:15645888]   paired:    1
2020-05-22 16:27:14.443 bluetooth[17025:15645888]   connected: 1
2020-05-22 16:27:14.443 bluetooth[17025:15645888] iPhone SE:
2020-05-22 16:27:14.443 bluetooth[17025:15645888]   paired:    0
2020-05-22 16:27:14.443 bluetooth[17025:15645888]   connected: 0

Let’s try debugging this app. First consider the call to method +[IOBluetoothDevice pairedDevices]:

If we execute a regular instruction step with F7, IDA will step into the _objc_msgSend function in libobjc.A.dylib, which is probably not what we want here. Instead use shortcut Shift+O. IDA will automatically detect the address of the Objective-C method that is being invoked and break at it:

This module appears to be Objective-C heavy, so it might be a good idea to extract Objective-C type info from the module using right click -> Load debug symbols in the Modules window:

This operation will extract any Objective-C types encoded in the module, which should give us some nice prototypes for the methods we’re stepping in:

Let’s continue to another method call - but this time the code invokes a stub for _objc_msgSend that IDA has not analyzed yet, so its name has not been properly resolved:

In this case Shift+O should still work:

Shift+O is purposefully flexible so that it can be invoked at any point before a direct or indirect call to _objc_msgSend. It will simply intercept execution at the function in libobjc.A.dylib and use the arguments to calculate the target method address.

However, you must be careful. If you use this action in a process that does not call _objc_msgSend, you will lose control of the process. It is best to only use it when you’re certain the code is compiled from Objective-C and an _objc_msgSend call is imminent.

Decompiling Objective-C at Runtime

The Objective-C runtime analysis performed by Load debug symbols will also improve decompilation.

Consider the method -[IOBluetoothDevice isConnected]:

Before we start stepping through this method we might want to peek at the pseudocode to get a sense of how it works. Note that the Objective-C analysis created local types for the IOBluetoothDevice class, as well as many other classes:

This type info results in some sensible pseudocode:

We knew nothing about this method going in - but it’s immediately clear that device connectivity is determined by the state of an io_service_t handle in the IOBluetoothObject superclass, and we’re well on our way.

Debugging Over SSH

In this section we will discuss how to remotely debug an app on a mac machine using only an SSH connection. Naturally, this task introduces some unique complications.

To start, copy the mac_server binaries and the bluetooth app from samples.zip:

to the target machine:

$ scp <IDA install dir>/dbgsrv/mac_server* user@remote:
$ scp bluetooth user@remote:

Now ssh to the target machine and launch the mac_server:

$ ssh user@remote
user@remote:~$ ./mac_server64
IDA Mac OS X 64-bit remote debug server(MT) v7.5.26. Hex-Rays (c) 2004-2020
Listening on 0.0.0.0:23946...

Now open the bluetooth binary on the machine with your IDA installation, select Remote Mac OS X Debugger from the debugger menu, and for Debugger>Process options set the debugging parameters. Be sure to replace <remote user> and <remote ip> with the username and ip address of the target machine:

Try launching the debugger with F9. You might get the following error message:

This happened because debugging requires manual authentication from the user for every login session (via the Take Control prompt discussed under Using the Mac Debug Server, above).

But since we’re logged into the mac via SSH, the OS has no way of prompting you with the authentication window and thus debugging permissions are refused.

Note that mac_server64 might have printed this workaround:

WARNING: The debugger could not acquire the necessary permissions from the OS to
debug mac applications. You will likely have to specify the proper credentials at
process start. To avoid this, you can set the MAC_DEBMOD_USER and MAC_DEBMOD_PASS
environment variables.

But this is an extreme measure. As an absolute last resort you can launch the mac_server with your credentials in the environment variables, which should take care of authentication without requiring any interaction with the OS. However there is a more secure workaround.

In your SSH session, terminate the mac_server process and run the following command:

$ security authorizationdb read system.privilege.taskport > taskport.plist

Edit taskport.plist and change the authenticate-user option to false:

<key>authenticate-user</key>
<false/>

Then apply the changes:

$ sudo security authorizationdb write system.privilege.taskport < taskport.plist

This will completely disable the debugging authentication prompt (even across reboots), which should allow you to use the debug server over SSH without macOS bothering you about permissions.

Dealing With Slow Connections

When debugging over SSH you might experience some slowdowns. For example you might see this dialog appear for several seconds when starting the debugger:

During this operation IDA is fetching function names from the symbol tables for all dylibs that have been loaded in the target process. It is a critical task (after all we want our stack traces to look nice), but it is made complicated by the sheer volume of dylibs loaded in a typical macOS process due to the dyld_shared_cache. This results in several megabytes of raw symbol names that mac_server must transmit over the wire every time the debugger is launched.

We can fix this by using the same trick that IDA’s Remote iOS Debugger uses to speed up debugging - by extracting symbol files from the dyld cache and parsing them locally. Start by downloading the ios_deploy utility from our downloads page, and copy it to the remote mac:

$ scp ios_deploy user@remote:

Then SSH to the remote mac and run it:

$ ./ios_deploy symbols -c /var/db/dyld/dyld_shared_cache_x86_64h -d mac_symbols
Extracting symbols from /var/db/dyld/dyld_shared_cache_x86_64h => mac_symbols
Extracting symbol file: 1813/1813
mac_symbols: done
$ zip -r mac_symbols.zip mac_symbols

Copy mac_symbols.zip from the remote machine to your host machine and unzip it. Then open Debugger>Debugger options>Set specific options and set the Symbol path field:

Now try launching the debugger again, it should start up much faster.

Also keep the following in mind:

• Use /var/db/dyld/dyld_shared_cache_i386 if debugging 32-bit apps

• You must perform this operation after every macOS update. Updating the OS will update the dyld_shared_cache, which invalidates the extracted symbol files.

• The ios_deploy utility simply invokes dyld_shared_cache_extract_dylibs_progress from the dsc_extractor.bundle library in Xcode. If you don’t want to use ios_deploy there are likely other third-party tools that do something similar.

Debugging arm64 Applications on Apple Silicon

IDA 7.6 introduced the ARM Mac Debugger, which can debug any application that runs natively on Apple Silicon.

On Apple Silicon, the same rules apply (see Codesigning & Permissions above). The Local ARM Mac Debugger can only be used when run as root, so it is better to use the Remote ARM Mac Debugger with the debug server (mac_server_arm64), which can debug any arm64 app out of the box (see Using the Mac Debug Server).

We have included arm64 versions of the sample binaries used in the previous examples. We encourage you to go back and try them. They should work just as well on Apple Silicon.

Debugging arm64e System Applications

Similar to Intel Macs, IDA cannot debug system apps on Apple Silicon until System Integrity Protection is disabled.

But here macOS introduces another complication. All system apps shipped with macOS are built for arm64e - and thus have pointer authentication enabled. This is interesting because ptruath-enabled processes are treated much differently within the XNU kernel. All register values that typically contain pointers (PC, LR, SP, and FP) will be signed and authenticated by PAC.

Thus, if a debugger wants to modify the register state of an arm64e process, it must know how to properly sign the register values. Only arm64e applications are allowed to do this (canonically, at least).

You may have noticed that IDA 7.6 ships with two versions of the arm64 debug server:

mac_server_arm64e is built specifically for the arm64e architecture, and thus will be able to properly inspect other arm64e processes. We might want to try running this version right away, but by default macOS will refuse to run any third-party software built for arm64e:

According to Apple, this is because the arm64e ABI is not stable enough to be used generically. In order to run third-party arm64e binaries you must enable the following boot arg:

$ sudo nvram boot-args=-arm64e_preview_abi

After rebooting you can finally run mac_server_arm64e:

This allows you to debug any system application (e.g. /System/Applications/Calculator.app) without issue:

Also note that the arm64e ABI limitation means you cannot use the Local ARM Mac Debugger to debug system arm64e apps, since IDA itself is not built for arm64e. It is likely that Apple will break the arm64e ABI in the future and IDA might cease to work. We want to avoid this scenario entirely.

Using the Remote ARM Mac Debugger with mac_server_arm64e is a nice workaround. It guarantees ida.app will continue to work normally regardless of any breakages in the arm64e ABI, and we can easily ship new arm64e builds of the server to anybody who needs it.

Apple Silicon: TL;DR

To summarize:

• Use mac_server_arm64 if you’re debugging third-party arm64 apps that aren’t protected by SIP

• Use mac_server_arm64e if you’re feeling frisky and want to debug macOS system internals. You must disable SIP and enable nvram boot-args=-arm64e_preview_abi, then you can debug any app you want (arm64/arm64e apps, system/non-system apps, shouldn’t matter).

Support

If you have any questions about this writeup or encounter any issues with the debugger itself in your environment, don’t hesitate to contact us at support@hex-rays.com.

Our Mac support team has years of experience keeping the debugger functional through rapid changes in the Apple developer ecosystem. It is likely that we can resolve your issue quickly.

Here you can find a comprehensive set of step-by-step tutorials categorized by different debugging types and platforms.

General introduction into debugging with IDA

• Overview of

• Overview of

• IDA scriptable debugger: and

Local debugging tutorials

Windows local debugging:

Linux local debugging:

Remote debugging tutorials

PIN Tracer:

Other specialized debugging tutorials

Android/Dalvik debugging:

XNU debugging:

QEMU debugging:

Trace and replay debugger features:

Appcall mechanism:

Last updated on September 27, 2023 — v0.3

Preface

Starting with version 6.6, IDA Pro can debug Android applications written for the Dalvik Virtual Machine. This includes source level debugging too. This tutorial explains how to set up and start a Dalvik debugging session.

Installing Android Studio

First of all we have to install the Android SDK from the official site .

Environment Variables

IDA needs to know where the adb utility resides, and tries various methods to locate it automatically. Usually IDA finds the path to adb, but if it fails then we can define the ANDROID_SDK_HOME or the ANDROID_HOME environment variable to point to the directory where the Android SDK is installed to.

Android Device

Start the Android Emulator or connect to the Android device.

Information about preparing a physical device for development can be found at .

Check that the device can be correctly detected by adb:

Installing the App

IDA assumes that the debugged application is already installed on the Android emulator/device.

Please download:

and

from our site. We will use this application in the tutorial.

We will use adb to install the application:

Loading Application into IDA

IDA can handle both .apk app bundles, or just the contained .dex files storing the app’s bytecode. If we specify an .apk file, IDA can either extract one of the contained .dex files by loading it with the ZIP load option, or load all classes*.dex files when using the APK loader.

Dalvik Debugger Options

The main configuration of the dalvik debugger happens resides in "Debugger > Debugger Options > Set specific options":

Connection Settings

ADB executable

As mentioned above IDA tries to locate the adb utility. If IDA failed to find it then we can set the path to adb here.

Connection string

Specifies the argument to the adb connect command. It is either empty (to let adb figure out a meaningful target) or a <host>[:<port>] combination to connect to a remote device somewhere on the network.

Emulator/device serial number

Serial number of an emulator or a device. Passed to adb``'s -s option. This option is useful if there are multiple potential target devices running. For the official Android emulator, it is typically emulator-5554.

Start Application

Fill from AndroidManifest.xml

Press button and point IDA to either the APK or the AndroidManifest.xml file of the mobile app. IDA then automatically fetches the package name and application start activity, as well as the debuggable flag from the specified file.

Package Name

Package name containing the activity to be launched by the debugger.

Activity

Start activity to be launched by the debugger.

Alternative Start Command

Usually IDA builds the start command from the package and activity name and launches the APK from the command line as follows:

If that does not match your desired debugging setup, you can enter an alternative start command here. Note that you have to provide package and activity as part of the startup command.

APK Debuggable

The value of the debuggable flag, as extracted from the AndroidManifest.xml or the APK. APKs that do not have the debuggable flag set (most do not) cannot be started on unpatched phones. Hence, while this value is false, IDA will display a (silencable) warning when starting a debugging session. To produce a debuggable APK that has the flag set to true, please revert to third-party tooling.

Detect Local Variable Types

This controls the behavior of IDA’s type guessing engine. "Always" and "Never" are pretty self-explanatory: The options force-enable or force-disable type guessing. "Auto" means that type guessing is disabled for Android APIs < 28 and enabled on APIs >= 28. If you work with very old (i.e. API 23 and lower) Android devices and experience crashes during debugging, set this option to "Never". Note that when type guessing is disabled, IDA automatically assumes int for unknown variable types, which causes warnings on API 30 and above.

Local Variables with Type Guessing Deactivated

Local Variables with Type Guessing Activated

Other Options

Show object ID

If active, IDA shows the object ID assigned by the Java VM for composite (non-trivial) types in the local variables window.

Preset BPTs

If active, IDA sets breakpoints at the beginning of all (non-synthetic, non-empty) methods of the start activity class specified in the Activity field above.

Path to Sources

To use source-level debugging we have to set paths to the application source files. We can do it using the "Options > Sources path" menu item.

Our Dalvik debugger presumes that the application sources reside in the current (".") directory. If this is not the case, we can map current directory (".") to the directory where the source files are located.

Let us place the source files DisplayMessageActivity.java and MainActivity.java in the same directory as the MyFirstApp.apk package. This way we do not need any mapping.

Setting Breakpoints

Before launching the application it is reasonable to set a few breakpoints. We can rely on the decision made by IDA (see above the presetBPTs option) or set breakpoints ourselves. A good candidate is the onCreate method of the application’s main activity.

We can use the activity name and the method name onCreate to set a breakpoint:

Naturally, we can set any other breakpoints any time. For example, we can do it later, when we suspend the application.

Starting the Debugger

At last we can start the debugger. Check that the Dalvik debugger backend is selected. Usually it should be done automatically by IDA:

If the debugger backend is correct, we are ready to start a debugger session. There are two ways to do it:

• Launch a new copy of the application (Start process)

• Attach to a running process (Attach to process)

Launching the App

To start a new copy of the application just press <F9> or use the "Debugger > Start process" menu item. The Dalvik debugger will launch the application, wait until application is ready and open a debugger session to it.

We may wait for the execution to reach a breakpoint or press the “Cancel” button to suspend the application.

In our case let us wait until execution reach of onCreate method breakpoint.

Attaching to a Running App

Instead of launching a new process we could attach to a running process and debug it. For that we could have selected the "Debugger > Attach to process…​" menu item. IDA will display a list of active processes.

We just select the process we want to attach to.

Particularities of Dalvik Debugger

All traditional debug actions like Step into, Step over, Run until return and others can be used. If the application sources are accessible then IDA will automatically switch to the source-level debugging.

Below is the list of special things about our Dalvik debugger:

• In Dalvik there is no stack and there is no SP register. The only available register is IP.

• The method frame registers and slots (v0, v1, …​) are represented as local variables in IDA. We can see them in the "Debugger > Debugger Windows > Locals" window (see below)

• The stack trace is available from "Debugger > Debugger windows > Stack trace" (the hot key is <Ctrl-Alt-S>).

• When the application is running, it may execute some system code. If we break the execution by clicking on the “Cancel” button, quite often we may find ourselves outside of the application, in the system code. The value of the IP register is 0xFFFFFFFF in this case, and stack trace shows only system calls and a lot of 0xFFFFFFFF. It means that IDA could not locate the current execution position inside the application. We recommend to set more breakpoints inside the application, resume the execution and interact with application by clicking on its windows, selecting menu items, etc. The same thing can occur when we step out the application.

• Use “Run until return” command to return to the source-level debugging if you occasionally step into a method and the value of the IP register becomes 0xFFFFFFFF.

Locals Window

IDA considers the method frame registers, slots, and variables (v0, v1, …​) as local variables. To see their values we have to open the "Locals" window from the "Debugger > Debugger windows > Locals" menu item.

At the moment the debugger stopped the execution at the breakpoint which we set on onCreate method.

Perform “Step over” action (the hot key is <F8>) two times and open the "Locals" window, we will see something like the following:

If information about the frame is available (the symbol table is intact) or type guessing is enabled then IDA shows the method arguments, the method local variables with names and other non-named variables. Otherwise some variable values will not be displayed because IDA does not know their types.

Variables without type information are marked with "Bad type" in the "Locals" window. To see the variable value in this case please use the "Watch view" window and query them with an explicit type (see below).

Watch View Window

To open the "Watch view" window select the "Debugger > Debugger windows > Watch view" menu item. In this window we can add any variable to watch its value.

note that we have to specify type of variable if it is not known. Use C-style casts:

• (Object*)v0

• (String)v6

• (char*)v17

• (int)v7

We do not need to specify the real type of an object variable, the “(Object*)” cast is enough. IDA can derive the real object type itself.

Attention! On Android API versions 23 and below an incorrect type may cause the Dalvik VM to crash. There is not much we can do about it. Our recommendation is to never cast an integer variable to an object type, the Dalvik VM usually crashes if we do that. But the integer cast “(int)” is safe in practice.

Keeping the above in the mind, do not leave the cast entries in the "Watch view" window for a long time. Delete them before any executing instruction that may change the type of the watched variable.

Overall we recommend to debug on a device that runs at least Android API 24.

Troubleshooting

• Check the path to adb in the "Debugger specific options"

• Check the package and activity names

• Check that the emulator is working and was registered as an adb device. Try to restart the adb daemon.

• Check that the application was successfully installed on the emulator/device

• Check the output window of IDA for any errors or warnings

• Turn on more debug print in IDA with the -z50000 command line switch.

• Android APIs 24 and 25 are known to return wrong instruction sizes during single stepping. Try migrating to a different Android API if you have trouble with single steps.

• Android APIs 23 and below crash if type guessing is enabled. Remedy this by setting the Detect Local Variable Types option to Never or migrate to a newer Android API.

The IDA Win32 debugger allows remote debugging of Windows32 executables. This is especially suited to the safe analysis of unknown hostile code. Our remote debugger server must first be started on the target machine.

we select attach to remote win32

and of course specify our password and the port we'll use for the session.

the connection screen is indentical to the local one.

and we are now connected to a program running on the remote machine.

Check the tutorial about debuggind Windows apps with IDA Bochs:

Copyright 2020 Hex-Rays SA

Overview

This tutorial discusses optimal strategies for debugging native iOS applications with IDA Pro.

IDA Pro supports remote debugging on any iOS version since iOS 9 (including iPadOS). Debugging is generally device agnostic so it shouldn't matter which hardware you're using as long as it's running iOS. The debugger itself can be used on any desktop platform that IDA supports (Mac/Windows/Linux), although using the debugger on Mac makes more features available.

Note that IDA supports debugging on both jailbroken and non-jailbroken devices. Each environment provides its own unique challenges and advantages, and we will discuss both in detail in this writeup.

Getting Started

The quickest way to get started with iOS debugging is to use Xcode to install a sample app on your device, then switch to IDA to debug it.

In this example we'll be using an iPhone SE 2 with iOS 13.4 (non-jailbroken) while using IDA 7.5 SP1 on OSX 10.15 Catalina. Start by launching Xcode and use menu File>New>Project... to create a new project from one of the iOS templates, any of them will work:

After selecting a template, set the following project options:

Note the bundle identifier primer.idatest, it will be important later. For the Team option choose the team associated with your iOS Developer account, and click OK. Before building be sure to set the target device in the top left of the Xcode window:

Now launch the build in Xcode. If it succeeds then Xcode will install the app on your device automatically.

Preparing a Debugging Environment

Now that we have a test app installed on our device, let's prepare to debug it. First we must ensure that the iOS debugserver is installed on the device. Since our device is not jailbroken, this is not such a trivial task. By default iOS restricts all remote access to the device, and such operations are managed by special MacOS Frameworks.

Use the mount phase to install DeveloperDiskImage.dmg, which contains the debugserver:

The device itself is now ready for debugging. Now let's switch to IDA and start configuring the debugger. Load the idatest binary in IDA, Xcode likely put it somewhere in its DerivedData directory:

Then go to menu Debugger>Select debugger... and select Remote iOS Debugger:

When debugging a binary remotely, IDA must know the full path to the executable on the target device. This is another task that iOS makes surprisingly difficult. Details of the filesystem are not advertised, so we must use ios_deploy to retrieve the executable path. Use the path phase with the app's bundle ID:

Use this path for the fields in Debugger>Process options...

<img style="border:1px solid grey;" src="process_options1.png", width="600">

NOTE: the path contains a hex string representing the application's 16-byte UUID. This id is regenerated every time you reinstall the app, so you must update the path in IDA whenever the app is updated on the device.

Now go to Debugger>Debugger options>Set specific options... and ensure the following fields are set:

<img style="border:1px solid grey;" src="ios_options1.png", width="600">

Make special note of the Symbol path option. This directory contains symbol files extracted from your device. Both IDA and Xcode use these files to load symbol tables for system libraries during debugging (instead of reading the tables in process memory), which will dramatically speed up debugging.

Xcode likely already created this directory when it first connected to your device, but if not you can always use ios_deploy to create it yourself:

Also ensure that the Launch debugserver automatically option is checked. This is required for non-jailbroken devices since we have no way to launch the server manually. This option instructs IDA to establish a connection to the debugserver itself via the MacOS Frameworks, which will happen automatically at debugging start.

Lastly, Xcode might have launched the test application after installing it. Use the proclist phase to retreive the app's pid and terminate it with the kill phase:

Finally we are ready to launch the debugger. Go to main in IDA's disassembly view, use F2 to set a breakpoint, then F9 to launch the process, and wait for the process to hit our breakpoint:

You are free to single step, inspect registers, and read/write memory just like any other IDA debugger.

Source Level Debugging

You can also use IDA to debug the source code of your iOS application. Let's rebuild the idatest application with the DWARF with dSYM File build setting:

Since the app is reinstalled, the executable path will change. We'll need to update the remote path in IDA:

Be sure to enable Debugger>Use source-level debugging, then launch the process. At runtime IDA will be able to load the DWARF source information:

Note that the debugserver does not provide DWARF information to IDA - instead IDA looks for dSYM bundles in the vicinity of the idb on your local filesystem. Thus if you want IDA to load DWARF info for a given module, both the module binary and its matching dSYM must be in the same directory as the idb, or in the idb's parent directory.

For example, in the case of the idatest build:

IDA was able to find the idatest binary next to idatest.i64, as well as the dSYM bundle next to the parent app directory.

If IDA can't find DWARF info on your filesystem for whatever reason, try launching IDA with the command-line option -z440010, which will enable much more verbose logging related to source-level debugging:

Debugging DYLD

IDA can also be used to debug binaries that are not user applications. For example, dyld.

The ability to debug dyld is a nice advantage because it allows us to observe critical changes in the latest versions of iOS (especially regarding the shared cache) before a jailbreak is even available. We document this functionality here in the hopes it will be useful to others as well.

The target application will be a trivial helloworld program:

Compile and install this app on your device, then set the following fields in Debugger>Process options...

Under Debugger>Debugger options, enable Suspend on debugging start. This will instruct IDA to suspend the process at dyld's entry point, before it has begun binding symbols. Now launch the process with F9 - immediately the process will be suspended at __dyld_start:

Double-click on the helloworld module to bring up its symbol list and go to the _main function:

Note that function sub_1009CBF98 is the stub for puts:

The iOS debugger supports watchpoints. Now would be a good time to use one:

Resume the process and wait for dyld to trigger our watchpoint:

The instruction STR X21 [X19] triggered the watchpoint, and note the value in X21 (BB457A81BA95ADD8) which is the authenticated pointer to puts. Where did this value come from? We can see that X21 was previously set with MOV X21, X0 after a call to this function:

It seems like we're on the right track. Also note that IDA was able to extract a nice stack trace despite dyld's heavy use of PAC instructions to authenticate return addresses on the stack:

This leads us to the following logic in the dyld-733.6 source:

Here, fixupLoc (off_109CC00) and newValue (address of puts) are passed as the loc and target arguments for Arm64e::signPointer:

Thus, the pointer to puts is signed using its destination address in helloworld:__auth_got as salt for the signing operation. This is quite clever because the salt value is subject to ASLR and therefore cannot be guessed, but at this point the executable has already been loaded into memory – so it won’t change by the time the pointer is verified in the stub.

To see this in action, use F4 to run to the BRAA instruction in the stub and note the values of the operands:

The branch will use the operands to verify that the target address has not been modified after it was originally calculated by dyld. Since we haven't done anything malicious, one more single step should take us right to puts:

Just for fun, let's rewind the process back to the start of the stub:

Then overwrite the authenticated pointer to puts with a raw pointer to printf:

Now when we step through the stub, the BRAA instruction should detect that the authenticated pointer has been modified, and it will purposefully crash the application by setting PC to an invalid address:

Any attempt to resume execution will inevitably fail:

It seems we now have an understanding of secure symbol bindings in dyld. Fascinating!

Debugging the DYLD Shared Cache

This section discusses how to optimally debug system libraries in a dyld_shared_cache.

NOTE: full support for dyld_shared_cache debugging requires IDA 7.5 SP1

Debugging iOS system libraries is a challenge because the code is only available in the dyld cache. IDA allows you to load a library directly from the cache, but this has its own complications. A single module typically requires loading several other modules before the analysis becomes useful. Fortunately IDA is aware of these annoyances and allows you to debug such code with minimal effort.

To start, consider the following sample application that uses the CryptoTokenKit framework:

Assume this program has been compiled and installed on the device as ctk.app.

Instead of debugging the test application, let's try debugging the CryptoTokenKit framework itself - focusing specifically on the -[TKTokenWatcher init] method.

Initial Analysis

When opening the cache in IDA, choose the load option Apple DYLD cache for arm64e (single module) and select the CryptoTokenKit module:

Wait for IDA to finish the initial analysis of CryptoTokenKit. Immediately we might notice that the analysis suffers because of references to unloaded code. Most notably many Objective-C methods are missing a prototype, which is unusual:

However this is expected. Modern dyld caches store all Objective-C class names and method selectors inside the libobjc module. Objective-C analysis is practically useless without these strings, so we must load the libobjc module to access them. Since a vast majority of modules depend on libobjc in such a way, it is a good idea to automate this in a script.

For a quick fix, save the following idapython code as init.py:

Then reopen the cache with:

This will tell IDA to load libobjc immediately after the database is created, then perform the Objective-C analysis once all critical info is in the database. This should make the initial analysis acceptable in most cases. In the case of CryptoTokenKit, we see that the Objective-C prototypes are now correct:

Now let's go to the -[TKTokenWatcher init] method invoked by the ctk application:

If we right-click on the unmapped address 0x1B271C01C, IDA provides two options in the context menu:

In this case the better option is Load ProVideo:__auth_stubs, which loads only the stubs from the module and properly resolves the names:

This is a common pattern in the latest arm64e dyldcaches, and it is quite convenient for us. Loading a handful of __auth_stubs sections is enough to resolve most of the calls in CryptoTokenKit, which gives us some nice analysis for -[TKTokenWatcher init] and its helper method:

Debugger Configuration

Now that the static analysis is on par with a typical iOS binary, let's combine it with dynamic analysis. We can debug this database by setting the following options in Debugger>Process options:

Here we set the Input file field to the full path of the CryptoTokenKit module. This allows IDA to easily detect the dyldcache slide at runtime. When CryptoTokenKit is loaded into the process, IDA will compare its runtime load address to the imagebase in the current idb, then rebase the database accordingly.

By default the imagebase in the idb corresponds to the first module that was loaded:

Thus, it is easiest to set Input file to the module corresponding to the default imagebase.

Note however that we could also use this configuration:

Provided that we update the imagebase in the idb to the base of the libobjc module:

This will result in the same dyld slide and should work just as well, because the the imagebase and the Input file field both correspond to the same module. This is something to keep in mind when debugging dyldcache idbs that contain multiple libraries.

Now let's try launching the debugger. Set a breakpoint at -[TKTokenWatcher initWithClient:], use F9 to launch the process, then wait for our breakpoint to be hit:

IDA was able to map our database (including CryptoTokenKit, libobjc, and the satellite __auth_stubs sections) into process memory. We can single step, resume, inspect registers, and perform any other operation that is typical of an IDA debugging session.

Further Analysis

Note that after terminating the debugging session you can continue to load new modules from the cache. If a dyld slide has been applied to the database, new modules will be correctly loaded into the rebased address space. This did not work in previous versions of IDA.

For example, after a debugging session we might notice some more unresolved calls:

IDA is aware that the address space has shifted, and it will load the new code at the correct address:

You are free to load new modules and relaunch debugging sessions indefinitely.

\

Debugging System Applications

The previous examples used custom applications to demonstrate IDA's debugging capabilities. In this case IDA can utilize the debugserver included in Apple's iOS developer tools, but there are situations in which this server is not sufficient for our needs.

The debugserver will refuse to debug any application that we didn't build ourselves. To demonstrate this, try launching IDA with an empty database and use Debugger>Attach>Remote iOS Debugger to attach to one of the system daemons:

You will likely get this error message:

It is possible to install a custom version of the debugserver that can debug system processes, but this requires a jailbroken device. We document the necessary steps and IDA configuration here. The device used in this example is an iPhone 8 with iOS 13.2.2, jailbroken with checkra1n 0.10.1.

Patching the debugserver

First we must obtain a copy of the debugserver binary from the DeveloperDiskImage.dmg:

Now save the following xml as entitlements.plist:

This will grant the debugserver permission to debug any application, including system apps. Now we can copy the server to the device and run it:

Note that we specified 192.168.1.7 which is the IP of the host machine used in this example. Be sure to replace this with the IP of your host so that the server will accept incoming connections from IDA.

IDA Configuration

To enable debugging with the patched debugserver, set the following options in dbg_ios.cfg:

We're now ready to open a binary in IDA and debug it. Copy the itunesstored binary from your device, it is typically found here:

After loading the binary use Debugger>Select debugger and choose Remote iOS Debugger, then under Debugger>Process options set the following fields:

Since we set AUTOLAUNCH = NO, IDA now provides the Hostname and Port fields so we can specify how to connect to our patched debugserver instance.

Now use Debugger>Attach to process and choose itunesstored from the process list. Since we have modified the debugserver it should agree to debug the target process, allowing IDA to create a typically robust debugging environment:

Note that although we're not using the debugserver from DeveloperDiskImage.dmg, IDA still depends on other developer tools to query the process list. We discuss how to install the DeveloperDiskImage in the Getting Started section above, but for a quick workaround you can always just specify the PID manually:

Now that we've successfully attached to a system process, let's do something interesting with it. Consider the method -[PurchaseOperation initWithPurchase:]. This logic seems to be invoked when a transaction is performed in the AppStore. Set a breakpoint at this method, then open the AppStore on your device and try downloading an app (it can be any app, even a free one).

Immediately our breakpoint is hit, and we can start unwinding the logic that brought us here:

Stepping through this function, we see many Objective-C method call sites:

Instead of using F7 to step into the _objc_msgSend function, we can use shortcut Shift-O to take us directly to the Objective-C method that is being invoked:

It seems that we're well on our way to reverse-engineering transactions in the AppStore. The remaining work is left as an exercise for the reader :)

Conclusion

Hopefully by now we've shown that IDA's iOS Debugger is quite versatile. It can play by Apple's rules when debugging on a non-jailbroken device, and it can also be configured to use an enhanced debugserver when a jailbreak is available.

Also keep in mind that all previous examples in this writeup should work equally well with the patched debugserver. We encourage you to go back and try them.

\

Troubleshooting

Often times these packets contain messages or error codes that provide clues to the issue.

For more enhanced troubleshooting, you can also enable logging on the server side. Go to Debugger>Debugger options>Set specific options and set the Syslog flags field:

This will instruct the debugserver to log details about the debugging session to the iOS system log (all valid flags are documented under the SYSLOG_FLAGS option in dbg_ios.cfg).

Start collecting the iOS system log with:

Then launch the debugger. Now both the client (/tmp/ida.log) and the server (/tmp/sys.log) will log important events in the debugger session, which will often times reveal the issue.`

Notes

Debugging Linux Applications with IDA Pro, locally

Last updated on July 29, 2020 — v0.1

You may already know that IDA lets you debug an application from an already existing IDB, by selecting the debugger using the drop-down debugger list.

However, it is also possible to start IDA in a way that it will initially create an empty IDB, and then either:

• start a new process under its control

• attach to an existing process

Launch IDA with a fresh new process

To do so, you will have to launch IDA from the command line, like so:

IDA will then launch the /bin/ls program, and break at its entrypoint

Attaching IDA to an existing process

For this example, we’ll launch, from a shell, a /usr/bin/yes process, and attach to.

Now, we’ll launch IDA so it offers a selection of processes to (and use quick filtering (Ctrl+F) to quickly find our process):

IDA will then attach to the selected process, and leave it suspended at the place it was when it was attached to:

Introduction

The PIN tracer is a remote debugger plugin used to record execution traces. It allows to record traces on Linux and Windows (x86 and x86_64) from any of the supported IDA platforms (Windows, Linux and MacOSX). Support for MacOSX targets is not yet available.

PIN support for MacOSX

Recording traces on MacOSX target is not supported yet.

However, it’s possible to record traces from a Linux or Windows target using the MacOSX version of IDA.

Building the PIN tool

Before using the PIN tracer the PIN tool module (distributed only in source code form) must be built as the Intel PIN license disallows redistributing PIN tools in binary form.

First of all download PIN from , and unpack it on your hard drive.

the PIN tools are a little sensitive to spaces in paths. Therefore, we recommend unpacking in a no-space path. E.g., "C:\pin", but not "C:\Program Files (x86)\.

The building process of the PIN tool is different for Windows and Linux.

Building on Windows

1. Install Visual Studio. It is possible to build the PIN tool with the Express version of Visual Studio for C++.

2. Download the IDA pintool sources from: (*)

   pintool 6.9 and higher should be built with PIN version 3.0 and higher, for earlier versions of pintool you should use PIN build 65163.

3. Unpack the .zip file into /path/to/pin/source/tools/

4. Open /path/to/pin/source/tools/idapin/IDADBG.sln in Visual Studio, select the correct build configuration (either Win32 or x64) and build the solution.

Alternatively you can use GNU make:

1. Install GNU make as a part of cygwin or MinGW package

2. Unpack the .zip file into /path/to/pin/source/tools/

3. Prepare Visual Studio environment (e.g. %VCINSTALLDIR%\Auxiliary\Build\vcvars32.bat for 32-bit pintool or %VCINSTALLDIR%\Auxiliary\Build\vcvars64.bat for 64-bit one)

4. cd /path/to/pin/source/tools/idapin

5. make

Building on Linux

1. Install GCC 3.4 or later

3. Unpack the .zip file into /path/to/pin/source/tools/

4. Open a console, and do the following (only for versions of PIN prior to 3.0):

   1. cd /path/to/pin/ia32/runtime

   2. ln -s libelf.so.0.8.13 libelf.so

   3. cd /path/to/pin/intel64/runtime

   4. ln -s libelf.so.0.8.13 libelf.so

   5. cd /path/to/pin/source/tools/Utils

   6. ls testGccVersion 2>/dev/null || ln -s ../testGccVersion testGccVersion

5. cd /path/to/pin/source/tools/idapin

for building the x86 version, or

for the x64 version.

Pintool 6.9 and higher are compatible with versions 6.5-6.8 of IDA so currently you can use them.

Start process

Once the PIN tool module is built we can use it in IDA. Open a binary in IDA and wait for the initial analysis to finish. When it’s done select the PIN tracer module from the debuggers drop down list or via Debugger > Select debugger:

After selecting the PIN tracer module select the menu Debugger > Debugger options > Set specific options. The following new dialog will be displayed:

In this dialog at least the following options are mandatory:

1. PIN executable: This is the full path to the PIN binary (including the “pin.exe” or “pin” file name). In some versions “pin.sh” may exist – in this case you should use it.

2. Directory with idadbg: This is the directory where the idadbg.so or idadbg.dll PIN tool resides. Please note that only the directory must be specified.

Fill the form with the correct paths and press OK in this dialog and enable option Autolaunch PIN for localhost.

We can interact with the PIN tracer like with any other debugger module: add breakpoints and step into or step over functions by pressing F7 or F8 alternatively.

Now we put a breakpoint in the very first instruction of function main

and launch the debugger by pressing the F9 key or by clicking the Start button in the debugger toolbar.

Make several steps by pressing F8. We can see all the instructions that were executed changed their color:

Now let the application run and finish by pressing F9 again. After a while the process will terminate and IDA will display a dialog telling us that is reading the recorded trace. Once IDA reads the trace the debugger will stop and the instructions executed will be highlighted (like with the built-in tracing engine) as in the following picture:

We can see in the graph view mode the complete path the application took in some specific function by switching to the graph view, pressing space bar and then pressing “w” to zoom out:

Attach to an existing process

Instead of launching a new process we could attach to a running process and debug it. For that we could have selected the "Debugger > Attach to process…​" menu item. IDA will display a list of active processes.

We just select the process we want to attach to. IDA will then attach to the selected process, and leave it suspended at the place it was when it was attached to:

Remote debugging

In case of remote debugging you can run IDA and PIN backend on different platforms.

Starting the remote PIN backend

The first thing to do, is to start the PIN debugging backend on the target machine. Command line depends of bitness of the target application.

For example, a 64-bit application ls would be started for debugging by the following comand:

whereas a 32-bit one hello32 as follows:

there is a more complicated way to start an application regardless bitness:

Also you can attach to already running programs:

For example:

Connecting IDA to the backend

The next step is to select PIN tracer module in IDA via Debugger > Select debugger and switch IDA to remote PIN backend. For this you should disable option Autolaunch PIN for localhost in the PIN options dialod (Debugger > Debugger options > Set specific options):

and then tell IDA about the backend endpoint, through the menu action Debugger > Process options…​

Once IDA knows what host to contact (and on what port), debugging an application remotely behaves exactly the same way as if you were debugging it locally.

In order to remotely debug a 64 bit process running on Windows64, we start the remote debugging server on the target machine.

We start IDAG64 (the 32-bit hosted version of IDA that is fully 64 bit capable) and use the "attach to remote win64" command .

IDA displays a list of the processes running on the 64 bit machines, we choose, click...

and, here we are, welcome to the fancy world of 64 bit debugging! Yes, the registers are a bit wide... but we are looking into a fancy compression scheme that...

Introduction

This guide illustrates how to configure the Bochs debugger plugin under Linux/MacOS. Downloading and compiling Bochs Please download the Bochs source code tarball and extract it.

tar zxf bochs-2.5.1.tar.gz

Run the 'configure' script (it is possible to pass other switches) and make sure that the switches marked in bold are present:

./configure --enable-sb16 --enable-ne2000 --enable-all-optimizations \ --enable-cpu-level=6 --enable-x86-64 --enable-pci \ --enable-clgd54xx --enable-usb --enable-usb-ohci \ --enable-show-ips --with-all-libs \ --enable-debugger --disable-readline

Note: under MacOS Lion 10.7.3 use the following switches:

./configure --enable-cpu-level=6 --with-nogui --enable-debugger --enable-disasm --enable-x86-debugger --enable-x86-64 --disable-readline --enable-all-optimizations --enable-sb16 --enable-ne2000 --enable-pci --enable-acpi --enable-clgd54xx --enable-usb --enable-usb-ohci --enable-show-ips

For a complete installation guide please check: http://bochs.sourceforge.net/doc/docbook/user/compiling.html. Now run "make" and "make install". Then type "whereis bochs" to get something like:

lallous@ubuntu:~/dev/bochs-2.5.1$ whereis bochsbochs: /usr/local/bin/bochs /usr/local/lib/bochs

Configuring IDA and the Bochs debugger plugin

Opening a database and selecting the Bochs debugger

After installing Bochs, run IDA Pro and open a Windows PE file and select 'Debugger -> switch debugger' and select "Local Bochs Debugger": If a PE file was loaded, then the Bochs debugger plugin will operate in "PE mode":

In case the other two modes (IDB or Disk Image mode) are used then there is no need to specify any additional configurations options, otherwise please continue reading this guide. Before launching the debugger with F9, the Bochs debugger plugin needs to know where to find the MS Windows DLLs and which environment variables to use. Attempting to run the debugger without configuring it may result in errors like this:

Here is a basic list of DLLs that are needed by most programs: • advapi32.dll • comctl32.dll • comdlg32.dll • gdi32.dll • kernel32.dll • msvcrt.dll • mswsock.dll • ntdll.dll • ntoskrnl.exe • shell32.dll • shlwapi.dll • urlmon.dll • user32.dll • wininet.dll • ws2_32.dll • wsock32.dll Let us create a directory in $HOME/bochs_windir/ and place those DLLs there. Specifying the Windows DLL path and environment variables using the startup file The startup file is a script file found in idadir\plugins\bochs directory. If IDC was the currently active language then startup.idc is used, otherwise startup.ext (where ext is the extension used by the currently selected extlang). In this tutorial we will be working with IDC, so we will edit the startup.idc file. (Please note that changes to this file will affect all databases. For local changes (database specific configuration) take a copy of the startup script file and place it in the same directory as the database then modify it). It is possible to specify a path map for a complete directory, for example:

/// path /home/lallous/bochs_windir/=c:\windows\system32

This line means that /home/lallous/bochs_windir/* will be visible to the debugged program as c:\windows\system32* (for example /home/lallous/bochs_windir/ntdll.dll will be visible as c:\windows\system32\ntdll.dll)

If all DLLs referenced by the program are in the bochs_windir directory, then running the process again should work: (Bochs has already started and IDA switched to debugging mode.) There are two things that should be configured. Press “.” to switch to the output window (or use the Debugger / Modules list window to inspect the modules list):

1.	The path to bochsys.dll is still not properly mapped. In our case, we need to add the following line to the startup file: 
```
/// map /Users/elias/idasrc/current/bin/idaq.app/Contents/MacOS/plugins/bochs/bochsys.dll=c:\windows\system32\bochsys.dll
```
(As opposed to the path keyword that maps complete directories, the map keyword to map individual files).

To hide the presence of bochsys.dll, simply map it to another name: 
```
/// map /Users/elias/idasrc/current/bin/idaq.app/Contents/MacOS/plugins/bochs/bochsys.dll=c:\windows\system32\kvm.dll
```
6.	The executable's path: we also need to add a map for the executable itself or a path entry for the whole folder: 
```
/// path /Users/elias/idasrc/current/bin/=c:\malware
```

Now, after we run the program again we should get a more correct module list:

It is equally important to specify some environment variables. We will use the env keyword to define all the environment variables:

/// env PATH=c:\windows;c:\tools/// env USERPROFILE=C:\Users\Guest

Specifying the Windows DLL path and environment variables using environment variables An alternative way of configuring the DLLs path and environment variables is to use the IDABXPATHMAP and the IDABXENVMAP environment variables. To specify the path map, export the following environment variable:

$ export IDABXPATHMAP=/home/lallous/bochs_windir/=c:/windows/system32;\         /home/lallous/dev/idaadv/plugins/bochs/=c:/windows/system32;\         /home/lallous/temp=c:/malware

(Please note that the forward slash (/) will be replaced with a backslash automatically by the plugin) Similarly, specify the environment variables with the IDABXENVMAP environment variable:

$ export IDABXENVMAP="USERPROFILE=c:/Users/Guest++PATH=c:/windows;c:\tools"

(Please note that we used the ++ to separate between multiple variables)

In case you require to do specific changes (per database) to the startup file then please take a copy of it and place it in the same directory as the database. Refer to the help IDA Pro help file for more information.

Since version 4.8, IDA Pro supports remote debugging of x86/AMD64 Windows PE applications and Linux ELF applications over TCP/IP networks. Remote debugging is the process of debugging code running on one networked computer from another networked computer:

• The computer running the IDA Pro interface will be called the "debugger client".

• The computer running the application to debug will be called the "debugger server".

Remote debugging will be particularly useful in the following cases:

• To debug virus/trojans/malwares : in this way, the debugger client will be as isolated as possible from the compromised computer.

• To debug applications encountering a problem on one computer which is not duplicated on other computers.

• To debug distributed applications.

• To always debug from your main workstation, so you won't have to duplicate IDA configuration, documentation and various debugging related resources everywhere.

• In the future, to debug applications on more operating systems and architectures.

This small tutorial will present how to setup and use remote debugging in practice.

The remote IDA debugger server

In order to allow the IDA client to communicate with the debugger server over the network, we must first start a small server which will handle all low-level execution and debugger operations. The IDA distribution ships with a Windows debugger server (the win32_remote.exe file) and a Linux debugger server (the linux_server file). With these, we can:

• Locally debug x86/AMD64 Windows applications and DLLs from the IDA Windows graphical and text versions.

• Remotely debug x86 Linux applications and shared libraries from the IDA Windows graphical and text versions.

• Locally debug x86 Linux applications and shared libraries from the IDA Linux text version.

• Remotely debug x86/AMD64 Windows applications and DLLs from the IDA Linux text version.

So let's first copy the small Windows debugger server file to our debugger server.

This server accepts various command line arguments:

C:\> win32_remote -?
IDA Windows32 remote debugger server. Version 1.0. Copyright Datarescue 2004
Error: usage: ida_remote [switches]
 -p... port number
 -P... password
 -v verbose

Let's start it by specifying a password, to avoid unauthorized connections:

C:\>win32_remote -Pmypassword
IDA Windows32 remote debugger server. Version 1.0. Copyright Datarescue 2004
Listening to port #23946...

Note that the remote debugger server can only handle one debugger session at a time. If you need to debug several applications simultaneously on the same host, launch several servers on different network ports by using the -p switch.

Setting up the debugger client.

First, we copy the executable we want to debug from the debugger server (Windows or Linux) to the debugger client (Windows or Linux). We can then load this file into IDA, as usual. To setup remote debugging, we select the 'Process options...' menu item in the Debugger menu:

Specify the Application, Directory and Input file paths. Note that these file paths should be valid on the remote debugger server. Also do not forget to enter the host name or IP address of the debugger server: remote debugging will only be enabled if these settings are specified ! Finally, we enter the password we chose for the remote IDA debugger server.

Starting remote debugging.

Both debugger server and debugger client are now ready to start a remote debugging session. In fact, you can now use all debugger related commands as you would with the local Windows PE debugger or local Linux debugger! For example,we can run the process until EIP reaches the application entry point, by jumping to this entry point then pressing the F4 key:

If we now directly terminate the process (by pressing CTRL-F2) and look at win32_remote's output (on the debugger server), we indeed properly observe it accepted then closed our network connection:

C:\> win32_remote -Pmypassword

IDA Windows32 remote debugger server. Version 1.0. Copyright Datarescue 2004 Listening to port #23946...

Accepting incoming connection... Closing incoming connection...

Attaching to a running process.

Another interesting possibility is to attach to an already running process on the remote computer. If you click on the 'Attach to process...' command from the Debugger menu, IDA will display a listing of all remote running processes corresponding to the file in your disassembly database:

Double clicking on a process from the list will automatically suspend the process and attach to it, allowing you to debug it without starting it manually. This attach process works from Windows to Linux, from Windows to Windows, from Linux to Linux and from Linux to Windows.

Detaching from the debugged process.

Finally, if the debugger server is running Windows XP, Windows Server 2003 or Linux, you can also detach from a process you were currently debugging, simply by using the 'Detach from process' command in the Debugger menu:

IDA supports debugging of DLLs on Windows and shared libraries on Linux. On Windows, please note that IDA can also attach to Windows services running either locally or remotely. In particular, the 'Detach from process' command will be especially useful if you previously attached to a Windows service: it will allow you to stop the debugger without terminating a critical Windows service on the debugger server!

Since version 4.7, IDA offers a console Linux debugger and a console Linux disassembler (since version 5.1 IDA also offers a Mac OS X debugger and disassembler). The Linux version of IDA brings the power of combined disassembly and debugging to the Linux world. Like its Windows sibling, the IDA Linux Disassembler comes in two versions that differ only by the number of processors they support: click here for a list of processors supported by the Starter and Professional version of IDA.

The Linux version of IDA:

• is able to disassemble any file supported by the Windows version.

• supports all the features of the Windows console version, including interactivity, scripting and plugins.

• offer local debugging of Linux executables.

• can connect to Windows machines running our debugging server and debug Windows executables.

• remote debugging server that allows you to debug Linux programs from another Linux machine, or even a Windows one.

Below: the Linux Debugger working locally.

Below: the Windows Debugger about to debug a remote Linux binary.

A typical use of the remote linux debugger would be the safe analysis of an hostile Windows binary: the Linux debugger, for example, brings unprecedented flexibility and security to the virus analyst. A typical use of the remote Windows debugger would be Linux debugging in a comfortable, well known GUI.

The IDA debugger, disassembler and remote debuggers are not sold separately but are included in the normal IDA distribution.

Quick overview:

The Windbg debugger plugin is an IDA Pro debugger plugin that uses Microsoft's debugging engine (dbgeng) that is used by Windbg, Cdb or Kd.

To get started, you need to install the latest Debugging Tools from Microsoft website: https://msdn.microsoft.com/en-us/windows/hardware/hh852365

or from the Windows SDK / DDK package.

Please make sure you should install the x86 version of the debugging tools which is used by both IDA Pro and IDA Pro 64. The x64 version will NOT work.

After installing the debugging tools, make sure you select « Debugger / Switch Debugger » and select the WinDbg debugger.

Also make sure you specify the correct settings in the “Debugger specific options” dialog:

• User mode: Select this mode for user mode application debugging (default mode)

• Kernel mode: Select this mode to attach to a live kernel.

• Non Invasive debugging: Select this mode to attach to a process non-invasively

• Output flags: These flags tell the debugging engine which kind of output messages to display and which to omit

• Kernel mode debugging with reconnect and initial break: Select this option when debugging a kernel and when the connection string contains 'reconnect'. This option will assure that the debugger breaks as soon as possible after a reconnect.

  To make these settings permanent, please edit the IDA\cfg\dbg_windbg.cfg file.

  ** To specify the debugging tools folde**r you may add to the PATH environment variable the location of Windbg.exe or edit %IDA%\cfg\ida.cfg and change the value of the DBGTOOLS key.

  After the debugger is properly configured, edit the process options and leave the connection string value empty because we intend to debug a local user-mode application.

Now hit F9 to start debugging:

The Windbg plugin is very similar to IDA Pro's Win32 debugger plugin, nonetheless by using the former, one can benefit from the command line facilities and the extensions that ship with the debugging tools.

For example, one can type “!chain” to see the registered Windbg extensions:

“!gle” is another command to get the last error value of a given Win32 API call.

Another benefit of using the Windbg debugger plugin is the use of symbolic information.

Normally, if the debugging symbols path is not set, then the module window will only show the exported names. For example kernel32.dll displays 1359 names:

Let us configure a symbol source by adding this environment variable before running IDA:

set _NT_SYMBOL_PATH=srv*C:\Temp\pdb*http://msdl.microsoft.com/download/symbols

It is also possible to set the symbol path directly while debugging:

and then typing “.reload /f” to reload the symbols.

Now we try again and notice that more symbol names are retrieved from kernel32.dll:

Now we have 5818 symbols instead!

It is also possible to use the “x” command to quickly search for symbols:

(Looking for any symbol in any module that contains the word “continue”)

Debugging a remote process

We have seen how to debug a local user mode program, now let us see how to debug a remote process.

First let us assume that “pcA” is the target machine (where we will run the debugger server and the debugged program) and “pcB” is the machine where IDA Pro and the debugging tools are installed.

To start a remote process:

• On “pcA”, type:

• dbgsrv -t tcp:port=5000

(change the port number as needed)

• On “pcB”, setup IDA Pro and Windbg debugger plugin:

  • “Application/Input file”: these should contain a path to the debuggee residing in “pcA”

  • Connection string: tcp:port=5000,server=pcA

Now run the program and debug it remotely.

To attach to a remote process, use the same steps to setup “pcA” and use the same connection string when attaching to the process.

More about connection strings and different protocols (other than TCP/IP) can be found in “debugger.chm” in the debugging tools folder.

Debugging the kernel with VMWare

We will now demonstrate how to debug the kernel through a virtual machine.

In this example we will be using VMWare 6.5 and Windows XP SP3.

Configuring the virtual machine:

Run the VM and then edit “c:\boot.ini” file and add one more entry (see in bold):

[operating systems]

multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional" /noexecute=optin /fastdetect multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Local debug" /noexecute=optin /fastdetect /debug /debugport=com1 /baudrate=115200

For MS Windows Vista please see: http://msdn.microsoft.com/en-us/library/ms791527.aspxp

Actually the last line is just a copy of the first line but we added the “/debug” switch and some configuration values.

Now shutdown the virtual machine and edit its hardware settings and add a new serial port with option “use named pipes”:

Press “Finish” and start the VM. At the boot prompt, select “Local debug” from the boot menu:

Configuring Windbg debugger plugin:

Now run IDA Pro and select Debugger / Attach / Windbg

Then configure it to use “Kernel mode” debugging and use the following connection string:

com:port=\.\pipe\com_1,baud=115200,pipe

It is possible to use the 'reconnect' keyword in the connection string:

com:port=\.\pipe\com_1,baud=115200,pipe,reconnect

Also make sure the appropriate option is selected from the debugger specific options.

Please note that the connection string (in step 1) refers to the named pipe we set up in the previous steps.

Finally, press OK to attach and start debugging.

When IDA attaches successfully, it will display something like this:

If you do not see named labels then try checking your debugging symbols settings.

Note: In kernel mode IDA Pro will display one entry in the threads window for each processor.

For example a two processor configuration yields:

This screenshot shows how we are debugging the kernel and changing the disassembly listing (renaming stack variables, or using structure offsets):

At the end you can detach from the kernel and resume it or detach from the kernel and keep it suspended.

To detach and resume, simply select the “Debugger / Detach”, however to detach and keep the kernel suspended select “Debugger / Terminate Process”.

Debugging the kernel through kdsrv.exe

In some cases, when debugging a 64bit kernel using a 1394 cable then 64bit drivers are needed, thus dbgeng (32bits) will not work. To workaround this problem we need to run the kernel debugger server from the x64 debugging tools folder and connect to it:

• Go to “Debugging Tools (x64)” installation

• Run kdsrv.exe (change the port number/transport appropriately):

  • kdsrv -t tcp:port=6000

• Now run ida64 and specify the following connection string (change the transport value appropriately):

  • kdsrv:server=@{tcp:port=6000,server=127.0.0.1},trans=@{com:port=\.\pipe\com_3,baud=115200,pipe}

Since version 4.3, IDA offers a PE Windows debugger in addition to its Windows disassembler. The Windows debugger in IDA combines the power of static disassembly to dynamic debugging to allow its users to debug unknown binaries at a level close to source code. A Linux version of the debugger is also available, there is some more information about it here here

The Windows Debugger in IDA:

• is able to debug any file supported by the Windows DBG interface, including true 64 bits files.

• can benefit from all the features of the Windows Disassembler, including interactivity, scripting and plugins.

• offer local debugging of Windows executables.

• can connect to other Windows machines running our remote debugging server and debug Windows executables.

• can connect to our Linux remote debugging server and allows you to debug Linux executables from a familiar Windows environment.

Below: the Windows Debugger working locally.

Below: the Windows Debugger about to debug a remote Linux binary.

A typical use of the remote Windows debugger would be the analysis of an hostile Linux binary or an hostile Windows binary on a safe and clean machine. The IDA Windows debugger brings unprecedented flexibility and security to the virus analyst. Another typical use of the remote Windows debugger would be Linux debugging in a comfortable, well known GUI. Yet another possibility offered by our Windows debugger is 64 bit development. 64 bit development is still in its infancy and the IDA 64 bit debugger server allows you to debug 64 bit applications from within a stable Windows 32 environment.

Here are a few links to the IDA Windows Debugger on our site:

• multiple possible connections

• the debugger

• remote debugging

• tracing with the IDA Windows debugger.

• analysis of an obfuscated piece of hostile code

Current versions of VMWare Workstation include a GDB stub for remote debugging of the virtual machines running inside it. In version 5.4, IDA includes a debugger module which supports the remote GDB protocol. This document describes how to use it with VMWare. As an example, we'll debug a Linux kernel.

Debugging a Linux kernel

Let's assume that you already have a VM with Linux installed. Before starting the debugging, we will copy symbols for the kernel for easier navigation later. Copy either /proc/kallsyms or /boot/Sytem.map* file from the VM to host.

Now edit the VM's .vmx file to enable GDB debugger stub:

Add these lines to the file:

debugStub.listen.guest32 = "TRUE"

debugStub.hideBreakpoints= "TRUE"

monitor.debugOnStartGuest32 = "TRUE"

Save the file.

In VMWare, click "Power on this virtual machine" or click the green Play button on the toolbar.

A black screen is displayed since VMWare is waiting for a debugger to connect.

Start IDA.

If you get the welcome dialog, choose "Go".

Choose Debugger | Attach | Remote GDB debugger.

Enter "localhost" for hostname and 8832 for the port number.

Choose <attach to the process started on target> and click OK.

We land in the BIOS, but since we're not interested in debugging it, we can skip directly to the kernel. Inspect the kallsyms or System.map file you downloaded from the guest and search for the start_kernel symbol:

Press F2 or choose "Add breakpoint" from the context menu.

Press F9. You will see loading messages and then the execution will stop at the entrypoint.

Adding symbols

Symbols are very useful during debugging, and we can use the kallsyms or System.map file to add them to IDA. Go to File | Python command... and paste the following short script (don't forget to edit the file path):

ksyms = open(r"D:\kallsyms") # path to the kallsyms/map file for line in ksyms: if line[9]=='A': continue # skip absolute symbols addr = int(line[:8], 16) name = line[11:-1] if name[-1]==']': continue # skip module symbols idaapi.set_debug_name(addr, name) MakeNameEx(addr, name, SN_NOWARN) Message("%08X: %s\n"%(addr, name))

Click OK and wait a bit until it finishes. After that you should see the symbols in the disassembly and name list:

Happy debugging!

Copyright 2009 Hex-Rays SA

Debugging the Windows Kernel with VMWare and IDA WinDbg Plugin

We will now demonstrate how to debug the kernel through a virtual machine.

In this example we will be using VMware Workstation 15 Player and Windows 7.

It is highly recommended to read the article Windows driver debugging with WinDbg and VMWare

Configuring the virtual machine

Run the VM and use the bcedit command to configure the boot menu as stated in the article.

Edit the VM hardware settings and add a new serial port with option use named pipe:

Restart the VM to debug. At the boot prompt, select the menu item containing [debugger enabled] from the boot menu.

Configuring Windbg debugger plugin

The connection string com:port=\\.\pipe\com_2,baud=115200,pipe,reconnect for Windbg plugin should refer to the named pipe we set up in the previous steps.

Starting the debugger step by step

Start IDA Pro with an empty database:

> ida64 -t sample.i64

Select the Windbg debugger using "Debugger > Select debugger":

Then configure it to use “Kernel mode debugging” debugging in the “Debugger specific options” dialog:

After the debugger is properly configured, edit the process options and set the connection string:

Finally, start debugging using "Debugger > Attach to process":

IDA Pro may display a wait box "Refreshing module list" for some time. Then it will display something like this:

Starting the debugger using a command line option

The simplest way to start WinDbg Plugin is to run IDA Pro with the following option:

> ida64 -rwindbg{MODE=1}@com:port=\\.\pipe\com_2,baud=115200,pipe,reconnect+0 sample.i64

• {MODE=1} means "Kernel mode"

• +0 means the "<Kernel>" process

Debugging

In kernel mode IDA Pro will display one entry in the threads window for each processor.

For example a two processor configuration yields:

This screenshot shows how we are debugging the kernel and changing the disassembly listing (renaming stack variables, or using structure offsets):

At the end you can detach from the kernel and resume it or detach from the kernel and keep it suspended.

To detach and resume, simply select the “Debugger > Detach from process”, however to detach and keep the kernel suspended select “Debugger > Terminate Process”.

Debugging the kernel through kdsrv.exe

In some cases, when debugging a 64bit kernel using a 1394 cable then 64bit drivers are needed, thus dbgeng (32bits) will not work. To workaround this problem we need to run the kernel debugger server from the x64 debugging tools folder and connect to it:

• Go to “Debugging Tools (x64)” installation

• Run kdsrv.exe (change the port number/transport appropriately):

  • kdsrv -t tcp:port=6000

• Now run ida64 and specify the following connection string (change the transport value appropriately):

  • kdsrv:server=@{tcp:port=6000,server=127.0.0.1},trans=@{com:port=\\.\pipe\com_3,baud=115200,pipe}

Debugging facilities in IDA 4.9 and 4.9 SP

Since we enhanced the usability of IDA remote debugging options, many possibilities are now open. Explore the graph below to discover some of the possible connections.

• instant debugging, no need to wait for the analysis to be complete to start a debug session.

• easy connection to both local and remote processes.

• support for 64 bits systems and new connection possibilities.

• WindowCE remote debugging

click on the links below to explore some of the remote debugging features.

Introduction

IDA Pro 5.6 has a new feature: automatic running of the QEMU emulator. It can be used to debug small code snippets directly from the database. In this tutorial we will show how to dynamically run code that can be difficult to analyze statically.

Target

As an example we will use shellcode from the article in Phrack 66. It is self-modifying and because of alphanumeric limitation can be quite hard to undestand. So we will use the debugging feature to decode it.

The sample code is at the bottom of the article but here it is repeated:

80AR80AR80AR80AR80AR80AR80AR80AR80AR80AR80AR80AR80AR80AR80AR80AR80AR80AR 80AR80AR80AR80AR80AR80AR80AR80AR80AR00OB00OR00SU00SE9PSB9PSR0pMB80SBcACP daDPqAGYyPDReaOPeaFPeaFPeaFPeaFPeaFPeaFPd0FU803R9pCRPP7R0P5BcPFE6PCBePFE BP3BlP5RYPFUVP3RAP5RWPFUXpFUx0GRcaFPaP7RAP5BIPFE8p4B0PMRGA5X9pWRAAAO8P4B gaOP000QxFd0i8QCa129ATQC61BTQC0119OBQCA169OCQCa02800271execme22727

Copy this text to a new text file, remove all line breaks (i.e. make it a single long line) and save. Then load it into IDA.

Loading binary files into IDA

IDA displays the following dialog when it doesn't recognize the file format (as in this case):

Since we know that the code is for ARM processor, choose ARM in the "Processor type" dropdown and click Set. Then click OK. The following dialog appears:

When you analyze a real firmware dumped from address 0, these settings are good. However, since our shellcode is not address-dependent, we can choose any address. For example, enter 0x10000 in "ROM start address" and "Loading address" fields.

IDA doesn't know anything about this file so it didn't create any code. Press C to start disassembly.

Configuring QEMU

Before starting debug session, we need to set up automatic running of QEMU.

2. Edit cfg/gdb_arch.cfg and change "set QEMUPATH" line to point to the directory where you unpacked QEMU. Change "set QEMUFLAGS" if you're using an older version.

3. In IDA, go to Debug-Debugger options..., Set specific options.

4. Enable "Run a program before starting debugging".

5. Click "Choose a configuration". Choose Versatile or Integrator board. The command line and Initial SP fields will be filled in.

6. Memory map will be filled from the config file too. You can edit it by clicking the "Memory map" button, or from the Debugger-Manual memory regions menu item. See below for more details

Now on every start of debugging session QEMU will be started automatically.

Executing the code

By default, initial execution point is the entry point of the database. If you want to execute some other part of it, there are two ways:

1. Select the code range that you want to execute, or

2. Rename starting point ENTRY and ending point EXIT (convention similar to Bochs debugger)

In our case we do want to start at the entry point so we don't need to do anything. If you press F9 now, IDA will write the database contents to an ELF file (database.elfimg) and start QEMU, passing the ELF file name as the "kernel" parameter. QEMU will load it, and stop at the initial point.

Now you can step through the code and inspect what it does. Most of the instructions "just work", however, there is a syscall at 0x0010118:

ROM:00010118 SVCMI 0x414141

Since the QEMU configuration we use is "bare metal", without any operating system, this syscall won't be handled. So we need to skip it.

1. Navigate to 010118 and press F4 (Run to cursor). Notice that the code was changed (patched by preceding instructions):

2. Right-click next line (0001011C) and choose Set IP.

3. Press F7 three times. Once you're on BXPL R6 line, IDA will detect the mode switch and add a change point to Thumb code:

4. Go to 01012C and press U (Undefine).

5. Press Alt-G (Change Segment Register Value) and set value of T to 1. The erroneous CODE32 will disappear.

6. Go back to 00010128 and press C (Make code). Nice Thumb code will appear:

7. In Thumb code, there is another syscall at 00010152. If you trace or run until it, you can see that R7 becomes 0xB (sys_execve) and R0 points to 00010156.

Hint: if the code you're investigating has many syscalls and you don't want to handle them one by one, put a breakpoint at the address 0000000C (ARM's vector for syscalls). Return address will be in LR.

Saving results to database

If you want to keep the modified code or data for later analysis, you'll need to copy it to the database. For that:

1. Edit segment attributes (Alt-S) and make sure that segments with the data you need have the "Loader segment" attribute set.

2. Choose Debugger-Take memory snapshot and answer "Loader segments".

3. Now you can stop the debugging and inspect the new data. Note: this will update your database with the new data and discard the old. Repeated execution probably will not be correct.

Copyright 2019 Hex-Rays SA

Purpose

IDA 7.3 introduces the Remote XNU Debugger. It is designed to communicate with the GDB stub included with popular virtualization tools, namely VMWare Fusion (for OSX) and Corellium (for iOS). The debugger allows you to observe the Darwin kernel as it is running, while at the same time utilising the full power of IDA's analysis capabilities. It works equally well on Mac, Windows, and Linux.

This writeup is intended to quickly get you familiar with debugger, as well as offer some hints to make the experience as smooth as possible.

Debugging OSX with VMWare

To get started with debugging OSX, we will perform a simple experiment. This is the same experiment outlined in , but we will be performing the equivalent in IDA - which we hope you'll find is much simpler.

Begin with the following setup:

1. create an OSX virtual machine with . in this example the virtual machine is OSX 10.13.6, but the experiment should work with any recent OSX version.

2. open Terminal in the VM and enable some basic XNU debugging options:

3. shut down the VM and add the following line to the :

4. power on the virtual machine, open Terminal, and run this command:

   Let's use IDA to modify this version string.

Launch IDA, and when prompted with the window IDA: Quick start, choose Go to start with an empty database. Then go to menu Debugger>Attach>Remote XNU Debugger and set the following options:

Click OK, then select <attach to the process started on target>, and wait for IDA to attach. This step might take a few seconds (later we'll discuss how to speed things up). Once attached, the target is usually suspended in machine_idle:

IDA should have printed the message FFFFFF8000200000: process kernel has started, meaning it successfully detected the kernel image in memory. Now let's find the version string. Conveniently, the string appears in the kernel's symbol table, so we can simply use shortcut G and enter the name _version to jump right to it:

Use IDAPython to overwrite the bytes at this address:

Resume the process and allow the VM to run freely. Go back to Terminal in the VM and run the same command as before:

The output should look almost the same, except Darwin has been replaced with IDAPRO. So, we have modified kernel memory without breaking anything! You can continue to explore memory, set breakpoints, pause and resume the OS as you desire.

Using the KDK

Even if there is no KDK available for your OSX version, you can still utilise the KDK_PATH option in IDA to speed up debugging. For example, in the experiment above we could have done the following:

1. make your own KDK directory:
2. copy the kernelcache from your VM:
3. decompress the kernelcache:
4. set KDK_PATH in dbg_xnu.cfg:

Now whenever IDA needs to extract information from the kernel or kexts, it will parse the kernelcache file on disk instead of parsing the images in memory. This should be noticeably faster.

Debugging a Development Kernel

Our next goal is to use the KDK to create a rich database that can be used to debug XNU in greater detail. In this example we will debug the development kernel included in the Apple KDK. Let's open this file in IDA:

Wait for IDA to load the DWARF info and complete the autoanalysis. This may take a few minutes, but we only need to do it once.

While we wait, we can prepare the virtual machine to use the development kernel instead of the release kernel that is shipped with OSX (Note: System Integrity Protection must now be disabled in the VM). Open Terminal in the VM and run the following commands:

1. copy the development kernel from the KDK:
2. reconstruct the kernelcache:
3. reboot:
4. after rebooting, check that the development kernel was properly installed:

The VM is now ready for debugging.

IDA Configuration

Return to IDA and use Debugger>Select debugger to select Remote XNU Debugger. Then open Debugger>Process options and set the following fields:

Now go to Debugger>Debugger options>Set specific options and make sure the KDK path field is set:

You can ignore the other options for now, and press OK.

Assembly-Level Debugging + DWARF

IDA supports source-level debugging for the XNU Kernel. However for demonstration purposes we will focus on assembly-level debugging, while taking advantage of source-level DWARF information like local variables. This is a bit more stable, and is still quite useful.

Before attaching the debugger, open Options>Source paths... and un-check the checkbox:

Then click Apply. This will prevent IDA from complaining when it can't find a source file.

Finally, select Debugger>Attach to process>attach to the process started on target. After attaching, jump to function dofileread, and use F2 to set a breakpoint. Resume the debugger and and wait for the breakpoint to be hit (typically it will be hit right away, if not try simply running a terminal command in the guest). Once XNU hits our breakpoint, open Debugger>Debugger windows>Locals:

We can now perform detailed instruction-level debugging with the assistance of DWARF. You can continue to single step, set breakpoints, and inspect or modify local variables just like any other IDA debugger.

KEXT Debugging

IDA also supports debugging kext binaries. To demonstrate this, we will debug IONetworkingFamily, a submodule of IOKit that is typically shipped with the KDK. Begin by opening the binary in IDA:

Select Remote XNU Debugger from the debugger menu. Then in Debugger>Process options, set:

Note that we provide the bundle ID of the kext (com.apple.iokit.IONetworkingFamily) as the Input file field. This allows the debugger to easily identify the target kext at runtime.

Also note that loading all kexts in kernel memory can be a slow operation, which is why it is disabled by default. Open Debugger>Debugger options>Set specific options and ensure the KDK path field is set, then set the KEXT Debugging option to KDK only:

This tells the debugger to only load kexts that are present in the KDK. Since the KDK binaries are on the local filesystem, IDA can parse the kexts in a negligible amount of time - which is ideal since we're really only interested in IONetworkingFamily.

Now power on your VM and allow it to boot up. Once it is running idle, attach the debugger. Immediately IDA should detect the kernel and all relevant kexts in memory, including IONetworkingFamily:

Double-click to bring up the debug names for this module, and search for IONetworkInterface::if_ioctl:

Now set a breakpoint at this function and resume the OS. Typically the breakpoint will be hit right away, but if it isn't try performing an action that requires a network interface (for instance, performing a google search). Once execution breaks in the kext we can use the database to debug it in detail:

Debugging a Prelinked Kernelcache

For simplicity, all of the examples up until now have dealt with a subset of the kernel, but it is also possible to load a complete prelinked kernelcache in IDA and debug it. Naturally, we have some suggestions for this.

Extending the KDK

If you're interested in debugging the entire prelinked kernel, the biggest concern is speed. IDA must create a detailed and accurate depiction of kernel memory, which could contain hundreds of kext modules. If we're not careful, this can be slow.

Fortunately there is an easy solution. Try the following:

1. create a writable copy of Apple's KDK:
2. copy the kernelcache from your VM to the new KDK:
3. decompress the kernelcache:

Now IDA can use both the KDK and the kernelcache to extract debugging information for almost any kext at runtime. This should be fast.

Loading the Kernelcache

When loading a kernelcache, IDA now offers more load options:

In this example we want to load everything, so choose the kernel + all kexts option and wait for IDA to load all the subfiles and finish the autoanalysis. This will take a while but there's no way around it, it's a lot of code.

IMPORTANT NOTE: Try to avoid saving the IDA database file in the KDK directory. It is important to keep irrelevant files out of the KDK since they might slow down IDA's KDK parsing algorithm.

Copy kdk_utils.py to the plugins directory of your IDA installation. This plugin will create a new menu Edit>Other>KDK utils, with two new menu actions:

• Load KDK: This action will automatically detect all matching DWARF files in a given KDK, then apply the DWARF info to the subfiles in the database (including the kernel itself).

• Load DWARF for a prelinked KEXT: This action is useful if you have DWARF info for a prelinked kext that is not included in Apple's KDK. For a given DWARF file, the action will find a matching kext in the database and apply the DWARF info to this subfile.

Try opening Edit>Other>KDK utils>Load KDK and provide the KDK path:

Wait for IDA to scan the KDK for matching DWARF files and load them. This operation can also take a while, but it's worth it for all the extra structures, prototypes, and names that are added to the database. In the end we have a very detailed database that we are ready to use for debugging.

Now open Debugger>Process options and set the following options:

Then open Debugger>Debugger options>Set specific options and set the following fields:

Note that we set the KEXT Debugging option to all. This tells the debugger to detect every kext that has been loaded into memory and add it to the Modules list, including any non-prelinked kexts (there are likely only a handful of them, so it doesn't hurt).

Finally, power on the VM and attach to it with Debugger>Attach to process>attach to the process started on target. IDA should be able to quickly generate modules for the kernel and all loaded kexts:

You are now free to explore the entire running kernel! Try performing any of the previous demos in this writeup. They should work about the same, but now they are all possible with one single database.

Kernel ASLR + Rebasing

It is worth noting that rebasing has been heavily improved in IDA 7.3. Even large databases like the one we just created can now be rebased in just a few seconds. Previous IDA versions would take quite a bit longer. Thus, IDA should be able to quickly handle kernel ASLR, even when working with prelinked kernelcaches.

Debugging the OSX Kernel Entry Point

In this example we demonstrate how to gain control of the OS as early as possible. This task requires very specific steps, and we document them here. Before we begin, we must make an important note about a limitation in VMWare's GDB stub.

Physical Memory

Currently VMWare's 64-bit GDB stub does not allow us to debug the kernel entry point in physical memory. According to VMWare's support team, the correct approach is to use the 32-bit stub to debug the first few instructions of the kernel, then switch to a separate debugger connected to the 64-bit stub once the kernel switches to 64-bit addressing.

Since IDA's XNU debugger does not support 32-bit debugging, this approach is not really feasible (and it's not very practical anyway).

Workaround

Rather than add support for the 32-bit stub just to handle a few instructions, the official approach in IDA is to break at the first function executed in virtual memory (i386_init). This allows us to gain control of the OS while it is still in the early stages of initialization, which should be enough for most use cases.

Here's how you can do it:

1. Disable ALSR for the kernel. Open Terminal in the VM and run the following command:

   Then power off the VM.

2. Add this line to the .vmx file:

   This ensures that hardware breakpoints are enabled in the GDB stub. For most versions of VMWare, TRUE is the default value, but it's better to be safe.

3. Also add this line to the .vmx file:

   This will tell VMWare to suspend the OS before it boots.

4. Power on the VM. It will remain suspended until we attach the debugger.

5. Load a kernel binary in IDA, and set the following XNU debugger options:
6. Attach the debugger. The VM will be suspended in the firmware before the boot sequence has begun:
7. Now jump to the function _i386_init and set a hardware breakpoint at this location:

   We must use a hardware breakpoint because the kernel has not been loaded and the address is not yet valid. This is why steps 1 and 2 were important. It ensures the stub can set a breakpoint at a deterministic address, without trying to write to memory.

8. Resume the OS, and wait for our breakpoint to be hit:

   IDA should detect that execution has reached the kernel and load the kernel module on-the-fly. You can now continue to debug the kernel normally.

UEFI Debugging

It is possible to debug the EFI firmware of a VMWare Fusion guest. This gives us the unique opportunity to debug the OSX bootloader. Here's how it can be easily done in IDA:

First copy the bootloader executable from your VM:

Now shut down the VM and add this line to the .vmx file:

Load the boot.efi binary in IDA, open Debugger>Debugger options, check Suspend on library load/unload, and set Event condition to:

This will suspend the OS just before the bootloader entry point is invoked. Note: For some older versions of OSX, the bootloader will be named "boot.sys". You can check the name under the .debug section of the executable.

Now select Remote XNU Debugger from the Debugger menu, and set the following fields in Debugger>Process options:

We're now ready to start debugging the bootloader. Power on the VM (note that the VM is unresponsive since it is suspended), and attach to it with Debugger>Attach to process. After attaching IDA will try to detect the EFI_BOOT_SERVICES table. You should see the debugger print something like this to the console:

Now resume the process. You should see many UEFI drivers being loaded, until eventually boot.efi is loaded and IDA suspends the process:

At this point, the bootloader entry point function is about to be invoked. Jump to _ModuleEntryPoint in boot.efi and press F4. We can now step through boot.efi:

GetMemoryMap

To facilitate UEFI debugging, IDA provides an IDC helper function: xnu_get_efi_memory_map. This function will invoke EFI_BOOT_SERVICES.GetMemoryMap in the guest OS and return an array of EFI_MEMORY_DESCRIPTOR objects:

This function can be invoked at any point during firmware debugging.

UEFI Debugging + DWARF

If you build your own EFI apps or drivers on OSX, you can use IDA to debug the source code.

In this example we will debug a sample EFI application. On OSX the convention is to build EFI apps in the Mach-O format, then convert the file to PE .efi with the mtoc utility. In this example, assume we have an EFI build on our OSX virtual machine that contains the following files in the ~/TestApp directory:

• TestApp.efi - the EFI application that will be run

• TestApp.dll - the original Mach-O binary

• TestApp.dll.dSYM - DWARF info for the app

• TestApp.c - source code for the app

Here's how we can debug this application in IDA:

1. On your host machine, create a directory that will mirror the directory on the VM:
2. Copy the efi, macho, dSYM, and c files from your VM:
3. Open the TestApp.efi binary in IDA, and wait for IDA to analyze it.

   Note that you can improve the disassembly by loading the DWARF file from TestApp.dll.dSYM. You can do this with Edit>Plugins>Load DWARF file, or you can load it programatically from IDAPython:
4. Select Remote XNU Debugger from the debugger menu, and set the following fields in Debugger>Process options:
5. In Debugger>Debugger options, enable Suspend on library load/unload and set the Event condition field to:
6. In Debugger>Debugger options>Set specific options, set the following fields:

   Note that we must enable the Debug UEFI option, and set the UEFI symbols option so the debugger can find DWARF info for the EFI app at runtime.

7. If the usernames on the host and VM are different, we will need a source path mapping:
8. Reboot the VM and enter the EFI Shell

9. Attach the debugger. After attaching IDA will detect the firmware images that have already been loaded:
10. Resume the OS and launch TestApp from the EFI Shell prompt:

    At this point IDA will detect that the target app has been loaded, and suspend the process just before the entry point of TestApp.efi (because of step 5).

11. Now we can set a breakpoint somewhere in TestApp.efi and resume the OS. The debugger will be able to load source file and local variable information from TestApp.dll.dSYM:

    IMPORTANT NOTE: You must wait until TestApp has been loaded into memory before setting any breakpoints. If you add a breakpoint in the database before attaching the debugger, IDA might not set the breakpoint at the correct address. This is a limitation in IDA that we must work around for now.

Debugging iOS with Corellium

To get started with debugging iOS, we will perform a simple experiment to patch kernel memory. The device used in this example is a virtual iPhone XS with iOS 12.1.4, but it should work with any model or iOS version that Corellium supports. Begin by powering on your device and allow it to boot up. In the Corellium UI, look for the line labeled SSH under Advanced options:

Ensure you can connect to the device by running this command over ssh:

We will use IDA to patch this version string.

Now launch IDA, and when prompted with the window IDA: Quick start, choose Go to start with an empty database and open Debugger>Attach>Remote XNU Debugger. In the Corellium UI, find the hostname:port used by the kernel GDB stub. It should be specified in the line labeled kernel gdb:

And set the Hostname and Port fields in IDA's application setup window:

Now click on Debug options>Set specific options, and for the Configuration dropdown menu, be sure to select Corellium-ARM64:

You can ignore the other config options for now, and click OK.

Click OK again, and wait for IDA to establish a connection to Corellium's GDB stub (this may take a few seconds). Then select <attach to the process started on target> and wait for IDA to attach. This might take several seconds (we will address this later), but for now simply wait for IDA to perform the initial setup.

If IDA could detect the kernel, it should appear in the Modules list:

and the kernel version will be printed to the console:

Navigate to this address and use IDAPython to overwrite the string:

Resume the OS, and try running the same command as before:

If we could successfully write to kernel memory, IDAPRO should appear in the output.

Creating a KDK for iOS

Typically a Kernel Development Kit is not available for iOS devices, but we can still utilise the KDK_PATH option in IDA to achieve faster debugging. In the example above, the initial attach can be slow because IDA must parse the kernel image in memory (which can be especially slow if the kernel has a symbol table).

Here's how you can speed things up:

1. create the KDK directory:
2. copy the kernelcache from the virtual device:
4. set KDK_PATH in dbg_xnu.cfg:

Now whenever the debugger must extract information from the kernel, it will parse the local file on disk. This should be noticeably faster, especially if the device is hosted by Corellium's web service.

Debugging the iOS Kernel Entry Point

Corellium allows us to debug the first few instructions of kernel initialization. This can be very useful if we want to gain control of the OS as early as possible. In the Corellium UI, power on your device with the Start device paused option:

Now start IDA with an empty database and attach to the suspended VM:

From the XNU source, this is likely the _start symbol in osfmk/arm64/start.s, which simply branches to start_first_cpu. After stepping over this branch:

Press shortcut P to analyze start_first_cpu. This is where the kernel performs its initial setup (note that the value in X0 is a pointer to the boot_args structure). This function is interesting because it is responsible for switching the kernel to 64-bit virtual addressing. Typically the switch happens when this function sets X30 to a virtual address, then performs a RET:

Use F4 to run to this RET instruction. In this example X30 will now point to virtual address 0xFFFFFFF007B84474. After single stepping once more, we end up in arm_init in virtual memory:

After this single step, IDA detected that execution reached the kernel's virtual address space and automatically initialized the debugging environment. In this case a message will be printed to the console:

This signifies that IDA successfully detected the kernel base and created a new module in the Modules list. If the kernel has a symbol table, debug names will be available. Also note that PC now points inside the segment __TEXT_EXEC:__text instead of MEMORY, because the debugger parsed the kernel's load commands to generate proper debug segments.

Now that we know the address of arm_init, we can streamline this task:

1. power on the device with Start device paused

2. attach to the paused VM

3. set a hardware breakpoint at arm_init:
4. resume, and wait for the breakpoint to be hit

This gives us a quick way to break at the first instruction executed in virtual memory. You can continue debugging iOS as normal.

Known Issues and Limitations

Here is a list of known shortcomings in the XNU Debugger. Eventually we will address all of them, but it is unlikely we will resolve all of them by the next release. If any of the following topics are important to you, please let us know by sending an email to support@hex-rays.com. Issues with vocal support from our users are automatically prioritised.

iBoot debugging

Debugging the iOS firmware/bootloader is not yet supported. An On-Premise Corellium box is required for this functionality, so we will only implement it if there is significant demand.

32-bit XNU

The XNU Debugger does not support debugging 32-bit XNU. Since pure 32-bit OSes are quite outdated it is unlikely we will support them unless there is exceptional demand.

KDP

The XNU Debugger relies on the Remote GDB Protocol, and currently Apple's Kernel Debugging Protocol (KDP) is not supported. It is possible to add KDP support to IDA in the future.

Check the tutorial about tracing with IDA:

IDA Pro - Appcall user guide

Copyright 2023 Hex-Rays SA

Introduction

Appcall is a mechanism to call functions under a debugger session in the context of the debugged program using IDA's CLI (Command Line Interpreter) or from a script. Such a mechanism can be used for seamless blackbox function calling and use, fuzzing, process instrumentation, DLL injection, testing or extending applications.

Appcall mechanism highly depends on the type information of the called function. For that reason, it is necessary to have a correct function prototype before doing an Appcall, otherwise different or incorrect results may be returned.

In a nutshell, Appcall works by first hijacking the current thread's stack (please do switch threads explicitly if you want to Appcall in a different context), then pushing the arguments, and then temporarily adjusting the instruction pointer to the beginning of the called function. After the function returns (or an exception occurs), the original stack, instruction pointer, and other registers are restored, and the result is returned to the caller.

Please note that while most of the examples in this document are illustrated using a Windows user mode application, Appcall is not limited to Windows and can be used with any platform supported by IDA debuggers.

Quick start

Let's start explaining the basic concepts of Appcall using the IDC CLI. Let's imagine we have the following printf() in the disassembly somewhere:

It can be called by simply typing the following in the IDC CLI (press "." to jump to the CLI):

As you noticed, we invoked an Appcall by simply treating _printf as if it was a built-in IDC function. If the application had a console window, then you should see the message printed in it.

If you have a function with a mangled name or with characters that cannot be used as an identifier name in the IDC language, such as "_my_func@8", then you can use the LocByName function to get its address given its name, then using the address variable (which is callable) we issue the Appcall:

Or simply directly as:

Using AppCall with IDC

Apart from calling Appcall naturally as shown in the previous section, it is possible to call it explicitly using the dbg_appcall function:

The Appcall IDC function requires you to pass a function address, function type information (various forms are accepted) and the parameters (if any):

We've seen so far how to call a function if it already has type information, now suppose we have a function that does not:

Before calling this function with dbg_appcall we have two options:

1. Pass the prototype as a string

2. Or, parse the prototype separately and pass the returned type info object.

This is how we can do it using the first option:

As for the second option, we can use parse_decl() first, then proceed as usual:

Note that we used parse_decl() function to construct a typeinfo object that we can pass to dbg_appcall.

It is possible to permanently set the prototype of a function programmatically using apply_type():

In the following sections, we are going to cover different scenarios such as calling by reference, working with buffers and complex structures, etc.

Passing arguments by reference

To pass function arguments by reference, it suffices to use the & symbol as in the C language.

• For example to call this function:

We can use this code from IDC:

• To call a C function that takes a string buffer and modifies it:

We need to create a buffer and pass it by reference to the function:

__usercall calling convention

It is possible to Appcall functions with non standard calling conventions, such as routines written in assembler that expect parameters in various registers and so on. One way is to use the __usercall calling convention.

Consider this function:

And from IDC:

Variadic functions

In C:

And in IDC:

Calling functions that might cause exceptions

Exceptions may occur during an Appcall. To capture them, use the try/catch mechanism:

The exception object "e" will be populated with the following fields:

• description: description text generated by the debugger module while it was executing the Appcall

• file: The name of the file where the exception happened.

• func: The IDC function name where the exception happened.

• line: The line number in the script

• qerrno: The internal code of last error occurred

For example, one could get something like this:

In some cases, the exception object will contain more information.

Functions that accept or return structures

Appcall mechanism also works with functions that accept or return structure types. Consider this C code:

We can create a couple of records and link them up together:

Because we issued an Appcall, when listRecords is called, we expect to see the following output in the console:

We can then access the fields naturally (even the linked objects). We can verify that if we just dump the first record through the IDC CLI (or just by calling IDC's print function):

Notice how when rec1 is dumped, its next field is automatically followed and properly displayed. The same happens for rec2 and rec3.

We can also directly access the fields of the structure from IDC and have those changes reflected in the debugee's memory:

Notable observations:

• Objects are always passed by reference (no need to use the &)

• Objects are created on the stack

• Objects are untyped

• Missing object fields are automatically created by IDA and filled with zero

Calling an API that receives a structure and its size

Let us take another example where we call the GetVersionExA API function:

This API requires one of its input fields to be initialized to the size of the structure. Therefore, we need to initialize the structure correctly before passing it to the API to be further populated therein:

Now if we dump the ver object contents we observe something like this:

Working with opaque types

Opaque types like FILE, HWND, HANDLE, HINSTANCE, HKEY, etc. are not meant to be used as structures by themselves but like pointers.

Let us take for example the FILE structure that is used with fopen(); its underlying structure looks like this (implementation details might change):

And the fopen() function prototype is:

Let us see how we can get a "FILE *"" and use it as an opaque type and issue an fclose() call properly:

Nothing special about the fopen/fclose Appcalls except that we see the __at__ attribute showing up although it does not belong to the FILE structure definition. This is a special attribute that IDA inserts into all objects, and it contains the memory address from which IDA retrieved the object attribute values. We can use the __at__ to retrieve the C pointer of a given IDC object.

Previously, we omitted the __at__ field from displaying when we dumped objects output, but in reality this is what one expects to see as part of the objects attributes used in Appcalls. Let's create a user record again:

..and observe the output:

Please note that it is possible to pass as integer (which is a pointer) to a function that expects a pointer to a structure.

FindFirst/FindNext APIs example

In this example, we call the APIs directly without permanently setting their prototype first.

Using LoadLibrary/GetProcAddress

In this example, we are going to initialize the APIs by setting up their prototypes correctly so we can use them later conveniently.

Retrieving application's command line

Specifying Appcall options

Appcall can be configured with set_appcall_options() and passing one or more options:

• APPCALL_MANUAL: Only set up the appcall, do not run it (you should call cleanup_appcall() when finished). Please Refer to the "Manual Appcall" section for more information.

• APPCALL_DEBEV: If this bit is set, exceptions during appcall will generate IDC exceptions with full information about the exception. Please refer to the "Capturing exception debug events" section for more information.

It is possible to retrieve the Appcall options, change them and then restore them back. To retrieve the options use the get_appcall_options().

Please note that the Appcall options are saved in the database so if you set it once it will retain its value as you save and load the database.

Manual Appcall

So far, we've seen how to issue an Appcall and capture the result from the script, but what if we only want to setup the environment and manually step through a function?

This can be achieved with manual Appcall. The manual Appcall mechanism can be used to save the current execution context, execute another function in another context and then pop back the previous context and continue debugging from that point.

Let us directly illustrate manual Appcall with a real life scenario:

1. You are debugging your application

2. You discover a buggy function (foo()) that misbehaves when called with certain arguments: foo(0xdeadbeef)

3. Instead of waiting until the application calls foo() with the desired arguments that can cause foo() to misbehave, you can manually call foo() with the desired arguments and then trace the function from its beginning.

4. Finally, one calls cleanup_appcall() to restore the execution context

To illustrate, let us take the ref1 function (from the previous example above) and call it with an invalid pointer:

1. Set manual Appcall mode:
2. Call the function with an invalid pointer:

Directly after doing that, IDA will switch to the function and from that point on we can debug:

Now you are ready to single step that function with all its arguments properly set up for you. When you are done, you can return to the previous context by calling cleanup_appcall().

Initiating multiple manual Appcalls

It is possible to initiate multiple manual Appcalls. If manual Appcall is enabled, then issuing an Appcall from an another Appcall will push the current context and switch to the new Appcall context. cleanup_appcall() will pop the contexts one by one (LIFO style).

Such technique is useful if you happen to be tracing a function then you want to debug another function and then resume back from where you were!

Manual Appcalls are not designed to be called from a script (because they don't finish), nonetheless if you use them from a script:

We observe the following:

1. First Appcall will be initiated

2. The script will loop and display the values of i in IDA's output window

3. Another Appcall will be initiated

4. The script finishes. None of the two Appcalls actually took place

5. The execution context will be setup for tracing the last issued Appcall

6. After this Appcall is finished, we observe "Loop finished"

7. We issue cleanup_appcall and notice that the execution context is back to printf but this time it will print "Loop started"

8. Finally when we call again cleanup_appcall we resume our initial execution context

Capturing exception debug events

We previously illustrated that we can capture exceptions that occur during an Appcall, but that is not enough if we want to learn more about the nature of the exception from the operating system point of view.

It would be better if we could somehow get the last debug_event_t that occured inside the debugger module. This is possible if we use the APPCALL_DEBEV option. Let us repeat the previous example but with the APPCALL_DEBEV option enabled:

And in this case, if we dump the exception object's contents, we get these attributes:

Appcall related functions

There are some functions that can be used while working with Appcalls.

parse_decl/get_tinfo/sizeof

The get_tinfo() function is used to retrieve the typeinfo string associated with a given address.

The parse_decl() function is used to construct a typeinfo string from a type string. We already used it to construct a typeinfo string and passed it to dbg_appcall().

And finally, given a typeinfo string, one can use the sizeof() function to calculate the size of a type:

Accessing enum members as constants

In IDC, it is possible to access all the defined enumerations as if they were IDC constants:

Then one can type:

This syntax makes it even more convenient to use enumerations when calling APIs via Appcall.

Storing/Retrieving typed elements

It is possible to store/retrieve (aka serialize/deserialize) objects to/from the database (or the debugee's memory). To illustrate, let us consider the following memory contents:

And we know that this maps to a given type:

To retrieve (deserialize) the memory contents into a nice IDC object, we can use the object.retrieve() function:

Here is an example:

And now if we dump the contents of o:

and again we notice the __at__ which holds the address of the retrieved object.

To store (serialize) the object back into memory, we can use the object.store() function:

Here's an example continuing from the previous one:

And finally to verify, we go to the memory address:

Using Appcall with IDAPython

The Appcall concept remains the same between IDC and Python, nonetheless Appcall/Python has a different syntax (using references, unicode strings, etc.)

The Appcall mechanism is provided by ida_idd module (also via idaapi) through the Appcall variable. To issue an Appcall using Python:

One can take a reference to an Appcall:

• In case you have a function with a mangled name or with characters that cannot be used as an identifier name in the Python language, then use the following syntax:

• In case you want to redefine the prototype of a given function, then use the Appcall.proto(func_name or func_ea, prototype_string) syntax as such:

• To pass unicode strings you need to use the Appcall.unicode() function:

• To pass int64 values to a function you need to use the Appcall.int64() function:

Python Appcall code:

If the returned value is also an int64, then you can use the int64.value to unwrap and retrieve the value.

• To define a prototype and then later assign an address so you can issue an Appcall:

Things to note:

• We used the Appcall.Consts syntax to access enumerations (similar to what we did in IDC)

• If you replicate this specific example, a new memory page will be allocated. You need to refresh the debugger memory layout (with idaapi.refresh_debugger_memory()) to access it

Passing arguments by reference

• To pass function arguments by reference, one has to use the Appcall.byref():

• To call a C function that takes a string buffer and modifies it, we need to use the Appcall.buffer(initial_value, [size]) function to create a buffer:

• Another real life example is when we want to call the GetCurrentDirectory() API:

• To pass int64 values by reference:

We use the following Python code:

• To call a C function that takes an array of integers or an array of a given type:

First we need to use the Appcall.array() function to create an array type, then we use the array_object.pack() function to encode the Python values into a buffer:

Functions that accept or return structures

Like in IDC, we can create objects and pass them with at least two methods.

The first method involves using the Appcall.obj() function that takes an arbitrary number of keyword args that will be used to create an object with the arguments as attributes. The second method is by using a dictionary.

And finally, if you happen to have your own object instance then just pass your object. The IDAPython object to IDC object conversion routine will skip attributes starting and ending with "__".

FindFirst/FindNext example

Using GetProcAddress

Please note that we used the idaapi.inf_is_64bit() method to properly unwrap integer values that depends on the bitness of the binary.

Setting the Appcall options

In Python, the Appcall options can be set global or locally per Appcall.

• To set the global Appcall setting:

• To set the Appcall setting per Appcall:

Similarly, retrieving the Appcall options is done by either calling Appcall.get_appcall_options() or by reading the options attribute (for example: printf.options)

To cleanup after a manual Appcall use Appcall.cleanup_appcall().

Calling functions that can cause exceptions

An Appcall that generates an exception while executing in the current thread will throw a Python Exception object. This is inline with the IDC behavior we described above.

• Let us try when the Appcall options does not include the APPCALL_DEBEV flag:

This approach is useful if you want to know whether the Appcall passes or crashes.

Now if we want more details about the exception, then we use the APPCALL_DEBEV flag, which will cause an OSError exception to be raised and have its args[0] populated with the last debug_event_t:

If the Appcall caused a crash, then the debug_event variable will be populated with the last debug_event_t structure inside the OSError exception handler.

Appcall related functions in Python

Storing/Retrieving objects

Storing/Retrieving objects is also supported in Python:

1. Using the IDA SDK (through the idaapi Python module)

2. Using Appcall helper functions

In this example we show how to:

1. Unpack the DOS header at address 0x140000000 and verify the fields

2. Unpack a string and see if it is unpacked correctly

Let's start with the IDA SDK helper functions first:

Now to accomplish similar result using Appcall helper functions:

When it comes to storing, instead of using the Appcall's typedobj.retrieve(), we can use the typedobj.store() function:

Accessing enum members as constants

Like in IDC, to access the enums, one can use the Appcall.Consts object:

If the constant was not defined then an attribute error exception will be thrown. To prevent that, use the Appcall.valueof() method instead, which lets you provide a default value in case a constant was absent:

Since 2003 IDA offers a debugger that complements the static analysis nicely. In many cases, one just can't beat dynamic analysis. The IDA Debugger now supports 32-bit and 64-bit MS Windows executablesMS Windows, Linux, Mac OS X both locally and remotely. However, because the debugger API requires the mastery of our SDK and uses an event based model, it has proved quite difficult to use for some of our users.

• because the API uses an event based model makes it hard to program a linear sequence of actions in a natural way. The user is forced to install an event handler and to implement a finite state machine that implements the core logic of his plugin. While this may, in many ways, be a more powerful approach, this is probably too complex for more mundane tasks.

• because the API is only available at the plugin level, the simplest debugger actions requires writing a plugin which is a much bigger investment of time and efforts than writing a small IDC script.

IDA 5.2 will address both issues. The old event based model will remain available, but a simpler linear model will become available thanks to the function get_debugger_event(). This function pauses the execution of the plugin (or the script) until a new debugger event happens. The user can specify if she is interested only in the events that suspend the process or in all events. A timeout can also be confifured, after which the execution will continue if no event arose.

The new function allows us to drop the event based model (except in the cases when it is superior to linear logic) and write IDC scripts to control the debugger. For example, to launch the debugger, run to a specific location, print some data and single step twice, the following lines will suffice:

In IDA 5.1 this would have required a event handler and a small finite state automata, for a total more than 200 lines of code. Please note that, in the above example, the error handling code is omitted for clarity. In real life, you might want to check for unexpected conditions like an exception happening after StepInto().

To illustrate how easier it is to write scripts with the new approach, we rewrote the core functionality of the UUNP unpacker plugin. The original program requires about 600 lines of code and has a rather complex logic. The new script only requires 100 lines of code (almost half of them being comments and empty lines). More importantly, the script is easy to understand and modify for your needs.

This is a reimplementation of the uunp universal unpacker in IDC. It illustrates the use of the new debugger functions in IDA v5.2

Using Trace Replayer Debugger and Managing Traces in IDA

Copyright 2014 Hex-Rays SA

Introduction

Quick Overview

The trace replayer is an IDA pseudo debugger plugin that appeared first in IDA 6.3. This plugin can replay execution traces recorded with any debugger backend in IDA, such as local Win32 or Linux debuggers, WinDbg, remote GDB debugger, etc...

Following this tutorial

This tutorial was created using the Linux version of IDA and a Linux binary as target. However, it can be followed on any supported platform (MS Windows, Mac OS X and Linux) by setting up remote debugging. Please refer to the for more information regarding .

Supplied files

Among with the tutorial the following files are also provided at

Replaying and managing traces

Recording traces

Before using the trace replayer plugin we will need to record an execution trace of a program. We will use the following toy vulnerable program as an example:

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

int foo(char *arg, int size)

{

char *buf;

if ( strlen(arg) > size )

{

printf("Too big!\n");

return 1;

}

buf = malloc(size);

strcpy(buf, arg);

printf("Buffer is %s\n", buf);

free(buf);

return 0;

}

int main(int argc, char **argv)

{

if ( argc != 3 )

{

printf("Invalid number of arguments!\n");

return 2;

}

return foo(argv[1], atoi(argv[2]));

}

Please compile this sample program (in this example, we used GCC compiler for Linux) or use the supplied ELF binary, open the binary in IDA and wait until the initial analysis completes. When done, select a suitable debugger from the drop down list (“Local Linux debugger”, or “Remote Linux debugger” if you're following this tutorial from another platform):

We have two ways of telling IDA to record a trace:

1. Break on process entry point and manually enable tracing at this point.

2. Or put a trace breakpoint at the very first instruction of the program.

In the case we prefer the first approach we will need to click on the menu “Debugger → Debugger Options” and then mark the check box “Stop on process entry point” as shown bellow:

After checking this option press OK and run the program pressing F9. When the entry point is reached, we can select from the menu “Debugger → Tracing” one of the following three options:

1. Instruction tracing: All instructions executed will be recorded.

2. Function tracing: Only function calls and returns will be recorded.

3. Basic block tracing: Similar to instruction tracing but, instead of single stepping instruction by instruction, IDA will set temporary breakpoints in the end of every known basic block, as well as on function calls.

For this example we will select “Instruction tracing”. Check this option and let the program continue by pressing F9. The program will resume execution and finish quickly. Now, we have a recorded trace! To see it, select “Debugger → Tracing → Trace Window”. A new tab will open with a content similar to the following:

As previously stated, there are two ways to record traces: enabling it manually, or using an “Enable tracing” breakpoint. To set such a breakpoint we will go to the program's entry point (Ctrl+E) and put a breakpoint (F2) in the very first instruction. Then right click on the new breakpoint and select “Edit breakpoint”. In the dialog check the option “Enable tracing” and then select the desired “Tracing type” (for this example, we'll use “Instructions”):

Remove the “Stop on process entry point” option we set in the prior example and press F9 to run the program.

This way is more convenient than the first because the tracing is turned on automatically and does not need manual intervention.

Working with traces

Now we have a new recorded trace, no matter which method we used. What can we do with it? First, we can check which instructions were executed, as they are highlighted in the disassembly, like in the screenshot bellow:

(the highlight color can be changed in “Debugger → Tracing → Tracing Options”)

Highlighting makes it clear which instructions have been executed.

We can also check what functions have been executed (instead of instructions) by opening the “Trace Window” via “Debugger → Tracing → Trace Window”, right clicking on the list and then selecting “Show trace call graph”:

Now let's inspect the register values in order to understand why the check at 0x0848566 doesn't pass. Please select “Debugger → Switch debugger” and in the dialog box click on the “Trace replayer” radio button:

Click OK and press F4 in the first instruction of the “main” function.

The trace replayer will suspend execution at the “main” function and display the register values that were recorded when the program was executed:

We can single step by pressing F7, as usual. Let us keep pressing F7 until the “jz” instruction is reached:

The comparison “cmp [ebp+arg_0], 3” was not successful (ZF=0) so the check does not pass. We need to give to the program two arguments to pass this check and record a new trace.

Loading an overlay and viewing differences in flow

Before doing another run, let's save the first trace to a file. Select “Debugger → Tracing → Trace Window”, right click in the middle of the newly opened tab, and select “Save trace” from the popup menu:

Then save the file:

You will also be offered a chance to give the trace a description:

Now let's record a new trace but this time we will pass two command line arguments to the program. Select “Debugger → Process Options” and set “AAAA 4” as the arguments:

Close the dialog, revert to the “Local Linux debugger”, and press F9. A new trace will be recorded. If we check the “main” function we will see that different instructions have been executed this time:

Let's check which instructions are different between the first and the second run.

First, we will need to load the previous trace as “overlay”:

Select the trace we saved:

Note that we have now other options in the 'Overlay' submenu, now that there is an overlay present:

Now go back to the disassembly view and check how the disassembly code is highlighted in three different colors:

The code highlighted in yellow is the code executed in the current trace (the one listed in the “Trace Window”). The pink code was executed only in the overlay trace. And the code in purple is the code common to both traces. We can immediately see that there is some new code that have been executed, like the calls to atoi and foo.

Let's go to the “foo” function and see what happened here:

The code in yellow tells us that the check for the size at 0x800484FC passed and the calls to malloc, strcpy and printf were executed. Let's save this trace for later analysis and comparison with the future runs. As before, go to the trace window, right click on the list and select “Save trace”. Set the trace's description to 'Correct execution'.

It's time to record another trace with different arguments to see what happens. For this new execution, we will longer command line arguments (eight “A” characters instead of four). Let's change the arguments in “Debugger → Process Options”, switch back to the “Local Linux debugger”, and run it. We have a new trace. Let's compare it against the previously recorded one. As we did before, go to the “Trace Window”, right click on the list, select “Overlay”, then “Load overlay”, and select the trace with description “Correct execution”.

As we see, the code that alerts us the about a too big string was executed (it's highlighted in yellow). Let's save this recorded trace with the “String too big” description. Now we will record one more trace but this time we will use the number “-1” as the second command line argument.

Change the arguments in “Debugger → Process Options” as shown bellow:

Then switch back again, to the “Local Linux debugger” (or to “Remote Linux debugger” if needed) and run it by pressing F9. The process will crash somewhere in the call to strcpy:

Stop the debugger and save this trace (let's call it “Crash”). Then diff this trace against the “Correct execution” trace.

We will see the following in the disassembly view:

As we see, pretty much the same code as in the previous run was executed until the call to strcpy. It's time to replay this last execution and see what happened.

Diffing traces

When both a “main trace” and an “overlay trace” are present, the context menu item “Overlay → Subtract overlay” becomes available.

This allows one to subtract the list of events (e.g., instructions) that are present in the overlay, from the main trace:

Will give the following results:

As you can see, many events that were present in both the overlay & the main trace have been removed. Only those that were only present in the main trace remain.

Reverting the diff

The diffing operation is reversible:

This will restore the main trace as it were, before the contents of the overlay were removed from it.

Replaying traces

We know that the program is crashing somewhere in the call to strcpy but we don't know why the check at 0x080484FC passes since -1 is smaller than the size of the string (8 bytes). Let's put a breakpoint at the call to strlen at 0x080484F0, switch to the “Trace replayer” debugger, and "run" the program by pressing F9. Please note that we do not really run the program, we are merely replaying a previously recorded trace.

The debugger will stop at the strlen call:

In the trace replayer we can use all usual debugging commands like “run to cursor” (F4), “single step” (F7), or “step over” (F8). Let's press F8 to step over the strlen call and check the result:

It returns 8 as expected. Now move to the address 0x080484FC and press F4 or right click on this address, select “Set IP”, and press F7 (we need to inform the replayer plugin that we changed the current execution instruction in order to refresh all the register values). The difference between “Run to” (F4) and “Set IP” is that “Run to” will replay all events happened until that point but “Set IP” will directly move to the nearest trace event happened at this address (if it's in the recorded trace, of course).

Regardless of how we moved to this point IDA will display the following:

As we see, the jump was taken because CF was set to 1 in the previous instruction (“cmp edx, eax”). Let's step back to this instruction to see what values were compared. Select “Debugger → Step back” from the menu:

The flags are reset to 0 and we can see that EAX (0xFFFFFFFF) and EDX (8) are compared. Press F7 to step one instruction again and you will notice CF changes to 1. The instruction JBE performs an unsigned comparison between 8 and 0xFFFFFFFF and, as 8 <= 0xFFFFFFFF, the check passes. We just discovered the cause of the bug.

Let's continue analyzing it a bit more. Scroll down until the call to malloc at 0x08048517, right click, choose “Set IP”, and press F7 (or simply press F4). As we see, the argument given to malloc is 0xFFFFFFFF (4 GB).

Press F8 to step over the function call:

Obviously, malloc can not allocate so much memory and returns NULL. However, the program does not check for this possibility and tries to copy the contents of the given buffer to the address 0, resulting in a crash.

Summary

In this tutorial we showed you the basics of trace management and the trace replayer module in IDA. We hope you enjoy this new feature. Happy debugging!

Debugging a Windows executable locally and remotely

Last updated on September 01, 2020 - v0.2

This short tutorial introduces the main functionality of the IDA Debugger on Windows. IDA supports debugging of various binaries on various platforms, locally and remotely, but in this tutorial we will focus on debugging regular applications running on Windows.

Let’s see how the debugger can be used to locally debug a simple buggy C console program compiled under Windows.

Please use sample.exe.idb from samples.zip:

to follow this tutorial.

The buggy program

This program computes averages of a set of values (1, 2, 3, 4 and 5). Those values are stored in two arrays: one containing 8 bit values, the other containing 32-bit values.

Running this program gives us the following results:

Obviously, the computed average on the integer array is wrong. Let us use the IDA debugger to understand the origin of this error.

Loading the file

The debugger is completely integrated into IDA: to debug, we usually load the executable into IDA and create a database. We can disassemble the file interactively, and all the information which he will have added to the disassembly will be available during debugging. If the disassembled file is recognized as debuggable, the Debugger menu automatically appears in main window:

Since IDA has many debugger backends, we have to select the desired backend. We will use Local Windows debugger in our tutorial:

Instruction breakpoints

Once we located our int_average() function in the disassembly (it is at 0x40104A), we can add a breakpoint at its entry point, by selecting the Add breakpoint command in the popup menu, or by pressing the F2 key:

Program execution

Now we can start the execution. We can launch the debugger by pressing the F9 key or by clicking the Start button in the debugger toolbar. IDA displays a big warning message before really starting the debugger:

Indeed, running untrusted binaries on your computer may compromise it, so you should never run them. Since in our tutorial we are playing with a toy sample, it is okay, we can accept him. However, please consider using remote debugging for untrusted binaries.

Once we accept it, the program runs until it reaches our breakpoint:

Address evaluation

By analyzing the disassembled code, we can now locate the loop which computes the sum of the values, and stores the result in EAX. The [edx+ecx*4] operand clearly shows us that EDX points to the array and ECX is used as an index in it. Thus, this operand will successively point to each integer from the integers array:

Step by step and jump targets

Let us advance step by step in the loop, by clicking on the adequate button in the debugger toolbar or by pressing the F8 key. If necessary, IDA draws a green arrow to show us the target of a jump instruction:

The bug uncovered

Now, let’s have a look at value of [esp+count]. The ECX register (our index in the array) is compared to this register at each iteration: so, we can conclude that it is used as a counter in the loop. But, we also observe that it contains a rather strange number of elements: 14h (= 20). Remember that our original array contains only 5 elements! It seems we just found the source of our problem…​

Hardware breakpoints

To be sure, let us add a hardware breakpoint, just behind the last value of our integers array (in fact, on the first value of the chars array). If we reach this breakpoint during the loop, it will indeed prove that we read integers outside our array. For that jump to EDX, which points to the array, by clicking on a small arrow in the CPU register view:

IDA displays a sequence of bytes, so we need to create an array. Do the following:

• press Alt-D, D to create a doubleword

• press * and specify the size of 5 elements

Let us add a hardware breakpoint with a size of 4 bytes (the size of an integer) in Read/Write mode immediately after our array. Please note that the cursor is located after the array we created:

As foreseen, if we continue the execution, the hardware breakpoint detects a read access to the first byte of the chars array.

Please note that EIP points to the instruction following the one which caused the hardware breakpoint! It is in fact rather logical: to cause the hardware breakpoint, the preceding instruction has been fully executed, so EIP now points to the next one.

Caller’s mistake

By looking at the disassembly, we see that the value stored in [esp+count] comes from the count argument of our int_average() function. Let us try to understand why the caller gives us such a strange argument: if we go the call of int_average(), we easily locate the push 14h instruction, passing an erroneous count value to our function.

Now, by looking closer at the C source code, we understand our error: we used the sizeof() operator, which returns the number of bytes in the array, rather than returning the number of items in it! As, for the chars array, the number of bytes was equal to the number of items, we didn’t notice the error…​

Debugging a remote process

Our debuggers support debugging processes running on a remote computer. We just need to set up a remote debugging session and then we can debug the same way as in local debugging. Let us consider the following three simple steps.