hv command reference manual

hv credentials

In order to connect to the vault server, hv must at least have:

  • a username

  • a password

  • a hostname

For example:

$ hv -hhexvault.acme.com:65433 -uadmin -psecret users
LastActive Adm    Login        Email
---------- --- ------------ ------------
2022-06-27  *  admin
2022-06-22     alice       Alice <alice@acme.com>
Never          bob         Bob <bob@acme.com>
...

There are 3 ways to specify credentials (in decreasing order of priority):

All credentials, including usernames, are case-senstive, meaning that "Joe" and "joe" would be different users.

Command line

Passing credentials on the command line will always take precedence over environment variables and registry+keychain.

-uUSERNAME

specify username

-pPASSWORD

specify password

-hHOST

specify host (server:port) (if port is omitted, defaults to 65433)

-sSITENAME

specify site

--set

remember credentials. This option doesn’t require the credentials to be passed through the command line, credentials passed through environment variables will work as well

Environment variables

Credentials can also be passed through environment variables. They will take precedence over those possibly found in the registry+keychain.

VAULT_HOST

the server host name

VAULT_PORT

the server port

VAULT_USER

the username to connect to the server

VAULT_PASS

the user’s password

VAULT_SITE

the site to use (most commands need a site to operate)

Registry + keychain

Unless environment variables or command-line arguments are provided, hv will look for credentials in the registry (and the OS’s keychain for passwords.)

Credentials can be stored in the registry (and keychain) like so:

alice@alice_PC$ hv --set -ualice -palice -hvaultserver -salice_on_alicepc

The user, host (and optional site) will be persisted in the registry, while the password will be saved to the OS’s keychain.

For this operation to succeed, at least a user and host must be provided

In order to keep the various commands' syntax as clear as possible, we will assume that the user has stored credentials (in either the registry+keychain or environment variables) for the rest of this manual.

Best practices

We recommend persisting credentials using the registry+keychain method.

Once that is done, commands will become cleaner:

>./hv info

Hex-Rays Vault Server v1
Vault time: 2022-04-14 15:36:29, up since 2022-04-14 15:17:25
...

if you login to the server using hvui and save the login information, it will end up in the the registry+keychain method, and thus hv will then be able to use that information as well.

Path formats

Local paths refer to a file on the host file system.

Vault paths refer to a file mapped on the vault. They can start with // to refer to the root of the vault.

Some vault paths can optionally specify the revision of the path.

Special symbols were created to access specific revisions:

^

last revision available on the vault

=

current revision, that is synced on the site

*

all revisions

Special file revision symbols

subdir/

means all files in all subdirectories

subdir

means all files in all subdirectories (same as subdir/)

subdir/*

means all files in the directory

Directories and wildcards

Examples

Get the first revision of a file:

$ hv sync //malware/Ransomware.WannaCry/41aa.exe.i64#1
ok synced //malware/Ransomware.WannaCry/41aa.exe.i64#1 (838724 bytes)
ok sync completed

Sync to the last version of a file:

$ hv sync malware/Ransomware.WannaCry/41aa.exe.i64#^
ok synced //malware/Ransomware.WannaCry/41aa.exe.i64#3 (846916 bytes)
ok sync completed

Force sync to the current revision (we must specify -f to force a file transfer):

$ hv sync -f malware/Ransomware.WannaCry/41aa.exe.i64#=
ok synced //malware/Ransomware.WannaCry/41aa.exe.i64#2 (846916 bytes)
ok sync completed

Display md5 checksums of all revisions of a file:

$ hv md5 malware/Ransomware.WannaCry/41aa.exe.i64#*
ok 8F464140FA3DA4A20B03166F2E80325B //malware/Ransomware.WannaCry/41aa.exe.i64#1
ok E0F7B984151FEF497985F375C64FA5C7 //malware/Ransomware.WannaCry/41aa.exe.i64#2
ok 5C3B88306CF0D93DC35FFD67A710AE3B //malware/Ransomware.WannaCry/41aa.exe.i64#3

List Hex-Rays Vault server’s toplevel directory contents:

$ hv dir //
2022-06-02 10:29:30       140267 CL29/edit //malware/cppobj_virtcall.i64#9
2022-06-14 16:44:19      2173541 CL36/edit //iOS/dyld_ios16.i64#3

Plan to add a file to the vault:

$ hv add /path/to/local_rootdir/enable.png
ok added '//enabled.png'

Plan to add a directory:

$ hv add /path/to/local_rootdir/REsearch
ok added '//REsearch/vm2vm.dat'
ok added '//REsearch/vm2vm.exe'
ok added '//REsearch/vm2vm.i64'

Plan to delete a file:

$ hv del /path/to/local_rootdir/REsearch/*.dat
ok checked out '//REsearch/vm2vm.dat' for 'del' (worklist 1)

Show worklist to which files were added:

$ hv worklist show
WL 1 add  //REsearch/vm2vm.exe#0
WL 1 add  //REsearch/vm2vm.i64#0
WL 1 edit //cppobj_virtcall.i64#9
WL 1 add  //enabled.png#0

It is safe to interrupt a command using Ctrl-C. The file transfers in action will be gracefully terminated, so that no partially received files will be left on the disk. However, the requests that were delivered to the server will still be carried out up to the completion. For example, if the user asked to check out thousands of files for editing, this will be performed even if the user presses Ctrl-C after invoking the command.

If the command syntax specifies ellipsis (…​), it means that multiple path patterns can be specified. The path patterns can be specified using local paths or vault paths, which start with a double slash (//).

Commands

Sites

Commands in this section manipulate sites.

A user must be using a site in order for most commands to work correctly.

site add

site add [-u USER] SITENAME ROOTDIR [HOST]

Creates a new site.

The specified user will be the owner of the new site. If the user is not specified, the current user will own the site. Only the site owner can use a site.

Only admins can create sites for other users.

To use a site, it must be specified as described in the credentials section.

-u USER

The user (owner) of the new site, must be an existing username. Defaults to the current user. Admins can specify a different user.

SITENAME

The name of the site that will be created, it must be unique (no site can already exist with that name). It must not exceed 64 characters, and it must be composed of alphanumerics or underscore or dash. The first character cannot be a digit or a dash.

ROOTDIR

The absolute path to the directory that will hold the vault files.

HOST

The computer from which the site can be used. It can be specified as an empty string. In this case the server will let the site to be used by any computer. However, since it is a safety feature that prevents from inadvertently using a site from a wrong computer, we do not recommend to specify it as an empty string. When creating a site for the current user, the host defaults to the current computer.

Parameters

Examples:

Create a new site:

alice@alice_PC$ hv site add alicepc /home/alice/vault_site

Ensure that is exists:

alice@alice_PC$ hv sites
Site name User  Host     Last Used  Rootdir
--------- ----- -------- ---------- ------------
alicepc   alice alice_PC Never     /home/alice/vault_site

Remember the new site in the registry:

alice@alice_PC$ hv --set -salicepc
Information has been saved into the registry.

The new site is used in all future commands:

alice@alice_PC$ hv info |grep site
Client site: alicepc

site del

site del [-f] SITENAME

Deletes a site.

If -f was passed and the site has some pending worklists, they will be deleted.

This is not a reversible operation, so we recommend caution.

Only admins can delete sites that belong to other users.

-f

Force the deletion even if the site still has worklists.

SITENAME

Name of the site to delete.

Parameters

Example:

Delete the site, forcing deletion of the site’s worklists

alice@alice_PC$ hv site del alice_old_laptop
'alice_old_laptop' not empty

alice@alice_PC$ hv site del -f alice_old_laptop

site edit

site edit [-u USER] SITENAME ROOTDIR [HOST]

Edits an existing site’s details, such as the rootdir and the host it is bound to.

Admins can reassign a site to a new user or edit sites of other users.

-u USER

The new user (owner) of the site, can only be different than the previous owner if the current user is admin.

SITENAME

The name of the site that will be edited. It must exist and be owned by the current user, unless if the current user is admin.

ROOTDIR

The new absolute path to the directory that will hold the site files.

HOST

The new hostname that will be used for the site. It can be omitted if no changes are desired.

Parameters

Examples:

Change the root directory of a site:

alice@alice_PC$ hv site edit alicepc /home/alice/vault

Transfer ownership of site "local_on_shared_machine" to Bob:

alice@alice_PC$ hv site edit -u bob local_on_shared_machine /home/shared/projects re.acme.com

sites

sites [SITENAME]

Lists all sites.

Show a list of sites, and their associated information.

SITENAME

Name of the site to show.

Parameters

Example:

alice@alice_PC$ hv sites
Site name  User   Host       Last Used  Rootdir                Cur
---------- ------ ---------- ---------- ---------------------- ---
alicepc    alice  alice_PC   2022-06-22 /home/alice/vault_site  *
joe_laptop joe    ThinkPad14 2022-05-30 c:/work/vault
chrispc    chris  chris_PC   Never      W:/vault

Site filters

filt get

filt get [-s SITENAME]

Displays the filter table associated with the site.

Only admins can see filter tables of other users.

-s SITENAME

The sitename whose filter table should be displayed. If omitted, defaults to the current site.

Parameters

Examples:

Show the default (i.e., empty) filter table

alice@alice_PC$ hv filt get
# If the site filters were not set yet, the following info will be displayed:
alice@alice_PC$ hv filt get
# By default all vault files are visible.
# The admin can set up permissions to deny access
# to some files. The user too can set up filter
# patterns to make some files invisible. For that,
# each site has a user-controlled filter table.
#
# The below table controls vault file visibility.
# The table is scanned from the beginning to the end.
# Lines starting with '!' hide the matching files.
# Other lines make the matching files visible.
# If the first line starts with '!', all files are
# visible by default. Otherwise, all files are
# invisible by default.
# Example: !*.mov will hide all *.mov files.

Set site filters from the standard input:

alice@alice_PC$ hv filt set
/work/research/
<Ctrl-D>

Verify the new filters:

alice@alice_PC$ hv filt get
/work/research/
# The files outside of /work/research/ are not visible anymore

filt set

filt set [-s SITENAME] [@file]

Sets the filter table associated to the site, either interactively or from @file.

Information about the format of site filters can be retrieved by issuing thecommand.

Only admins can modify filter tables of other users.

-s SITENAME

The sitename whose filter table should be set. If omitted, defaults to current site.

@file

File containing the new table.

Parameters

Examples:

Make everything in the current site hidden, but .bak files

alice@alice_PC$ echo *.bak | hv filt set

Set site filters, from a file

alice@alice_PC$ cat @tablefile
*.idb
*.exe
alice@alice_PC$ hv filt set -s site1 @tablefile

File manipulation

add

add [-s] [-w WORKLIST_ID] PATH_PATTERN…​

Adds new file(s) to a worklist.

Issuing this command will not upload the file(s) to the server right away: the new file name(s) will be placed into a worklist, which then needs to be committed to the server. Once a worklist is committed, its files will be available to other users.

The specified file(s) are not required to exist, it is possible to add a file that does not exist yet.

The files must be inside the site’s rootdir.

The files will be filtered using hvignore rules.

-s

Silent mode; do not output any messages.

-w WORKLIST_ID

The id of the worklist that the file(s) will be added to. If omitted, defaults to worklist 1.

PATH_PATTERN…​

Local path to file(s) to add to the vault.

Parameters

Examples:

alice@alice_PC$ hv add new.idb
ok added '//new.idb'

add files to worklist 2

alice@alice_PC$ hv add -w 2 cuda_demo_suite/*
ok added '//cuda_demo_suite/bandwidthTest'
ok added '//cuda_demo_suite/busGrind'
ok added '//cuda_demo_suite/deviceQuery'
ok added '//cuda_demo_suite/nbody'
ok added '//cuda_demo_suite/nbody_data_files/nbody_galaxy_20K.bin'
ok added '//cuda_demo_suite/oceanFFT'
ok added '//cuda_demo_suite/oceanFFT_data_files/ocean.frag'
ok added '//cuda_demo_suite/oceanFFT_data_files/ocean.vert'
ok added '//cuda_demo_suite/oceanFFT_data_files/ref_slopeShading.bin'
ok added '//cuda_demo_suite/oceanFFT_data_files/ref_spatialDomain.bin'
ok added '//cuda_demo_suite/randomFog'
ok added '//cuda_demo_suite/randomFog_data_files/ref_randomFog.bin'
ok added '//cuda_demo_suite/vectorAdd'

copy

copy [-s] [-w WORKLIST_ID] SRC_PATH DST_PATH

Makes a copy of vault file(s).

This command creates a copy of the original file at the requested destination, and place the new file into a worklist. Once the worklist is committed, the new file will be visible to other users.

The source file will be downloaded from the server to the new file. If the source file was modified locally, those modifications won’t be part of the copy. This implies that if a file has just been added to the Hex-Rays Vault server but not committed yet, it can’t be copied because it does not exist on the server yet.

-s

Silent mode; do not output any messages.

-w WORKLIST_ID

The id of the worklist that the files will be added to. If omitted, defaults to worklist 1.

SRC_PATH

The source path.

DST_PATH

The destination path.

Parameters

Examples:

Copy newfile into the rust_samples subdirectory. The worklist #2 will hold the change.

alice@alice_PC$ hv copy -w 2 newfile rust_samples/newfile
ok copied '//newfile#1' to '//rust_samples/newfile'

Copy an entire subdirectory (note the trailing slash at the destination):

alice@alice_PC$ hv copy source_subdir/ destination/subdir/
ok copied '//source_subdir/aaa/sample.idb#1' to '//destination/subdir/aaa/sample.idb'
ok copied '//source_subdir/common.idb#1' to '//destination/subdir/common.idb'

Copy a subdirectory without recursion:

alice@alice_PC$ hv copy source_subdir/* destination/subdir/
ok copied '//source_subdir/common.idb#1' to '//destination/subdir/common.idb'

Copy a file that was just added but not yet committed, it will fail:

alice@alice_PC$ hv add test.text
ok added '//test.text'
alice@alice_PC$ hv copy test.text test.text.copy
no matching files for '//test.text'

move

move [-s] [-w WORKLIST_ID] SRC_PATH DST_PATH

Opens tracked file(s) for moving/renaming.

This is similar to performing a, followed by a: the new file will be checked out for copy while the original file will be checked out for deletion.

-s

Silent mode; do not output any messages.

-w WORKLIST_ID

The id of the worklist that the file(s) will be added to. If omitted, defaults to the worklist 1.

SRC_PATH

The source path.

DST_PATH

The destination path.

Parameters

Example:

alice@alice_PC$ hv move //VxWorks/CP05x/info.txt //VxWorks/CP05x/info.md
ok moved '//VxWorks/CP05x/info.txt#1' to '//VxWorks/CP05x/info.md'

alice@alice_PC$ hv wk show 1
WL 1 copy //VxWorks/CP05x/info.md#0
WL 1 del  //VxWorks/CP05x/info.txt#1

alice@alice_PC$ hv edit //VxWorks/CP05x/info.txt
file '//VxWorks/CP05x/info.txt' is already checked out

del

del [-s] [-w WORKLIST_ID] PATH_PATTERN…​

Opens tracked file(s) for deletion, adding them to a worklist.

Once the worklist is committed, the file(s) won’t be tracked anymore by the Hex-Rays Vault server, and will be removed from the local filesystem.

That this does not remove all revisions of the file on the server: that is the role of thecommand.

-s

Silent mode; do not output any messages.

-w WORKLIST_ID

The id of the worklist that the file(s) will be added to. If omitted, defaults to worklist 1.

PATH_PATTERN…​

Vault path of file(s) to delete.

Parameters

Example:

alice@alice_PC$ ls /path/to/site_rootdir/cat
/path/to/site_rootdir/cat

alice@alice_PC$ hv del -w2 cat
ok checked out '//cat' for 'del' (worklist 2)

alice@alice_PC$ ls /path/to/site_rootdir/cat
/path/to/site_rootdir/cat

alice@alice_PC$ hv commit 2 "Deleted 'cat'"
ok commit #39 completed

alice@alice_PC$ ls /path/to/site_rootdir/cat
ls: cannot access '/path/to/site/rootdir/cat': No such file or directory

edit

edit [-s] [-w WORKLIST_ID] PATH_PATTERN…​

Opens tracked file(s) for edit, adding them to a worklist.

This command is used to instruct the Hex-Rays Vault server that we will be working on files, so that it knows what revision of the file(s) that work will be based on and so later cmd.diff orcommands can work correctly.

-s

Silent mode; do not output any messages.

-w WORKLIST_ID

The id of the worklist that the file(s) will be added to. If omitted, defaults to worklist 1.

PATH_PATTERN…​

Vault path of file(s) to checkout for edit.

Parameters

Example:

alice@alice_PC$ hv edit cat.i64
ok checked out '//cat.i64' for 'edit' (worklist 1)

(...do some work...)

alice@alice_PC$ hv commit 1 "Analyzed 'main' function"
ok commit #12 completed

scan

scan [-a] [-e] [-d] [-s] [PATH_PATTERN…​]

Reconciles the contents of the current directory (or the one(s) provided) on the local filesystem, with those of the corresponding path(s) on the server.

This command will recursively look for:

  • new files (if -a is provided)

  • deleted files (if -d is provided)

  • modified files (if -e is provided)

If any is found will create a new worklist and, add those for addition/deletion/modification.

This command is particularly useful if the user didn’t have access to the server at a time it was necessary (e.g., to issue ancommand, while flying across the Atlantic.) Users can still get work done in such cases, and once they gain access to the server again, issue ato commit the changes.

The -e option causes thecommand to compute checksums of the local files, in order to compare them against those known to the server, in order to spot modifications.

If no options were given, defaults to -e -d.

The files found by the scan command will be filtered by hvignore.

-a

Checkout for add files that are present only on the client side.

-e

Checkout for edit files that are present on both the vault and the client side but differ.

-d

Checkout for delete files that are present only on the server side.

-s

Silent mode; do not output any messages.

PATH_PATTERN…​

Local path of file(s) to scan, if omitted defaults to current directory.

Parameters

Example:

alice@alice_PC$ hv scan -a -e -d //
added worklist 3
checked out '//afile' for 'del' (worklist 3)
checked out '//Win32.Emotet/29D6161522C7F7F21B35401907C702BDDB05ED47.bin.i64' for 'edit' (worklist 3)

Working with worklists

worklists

worklists [WORKLIST_ID] [USER]

Lists information about worklists.

Show a (possibly filtered) list of pending worklists, and their metadata:

  • the timestamp of when they were last changed

  • the number of files they contain

  • the owner

  • the site

  • their description

See also worklist show

WORKLIST_ID

Restrict to the provided worklist, defaults to showing all worklists.

USER

Restrict to user USER, defaults to the current user.

Parameters

Example:

alice@alice_PC$ hv worklists
WL 4 2022-06-27 17:24:51 2 files; $USER@$ALICE_SITE More work on L30DS2 firmware

Manipulating a worklist

The following worklist commands will also work with the shorter wk alias.

worklist add

worklist add DESCRIPTION

Creates a new worklist, with the provided description.

The worklist will initially be empty, and assigned a free ID.

Files can be associated to that new worklist when they are marked for addition, deletion, or edition.

DESCRIPTION

The description of the new worklist.

Parameters

Example:

alice@alice_PC$ hv worklist add "Working on the 'TMutexLocker' vtable"
added worklist 3
alice@alice_PC$ hv edit -w 3 //cppobj*
ok checked out '//cppobj_virtcall.i64' for 'edit' (worklist 3)
alice@alice_PC$ hv worklist add "vm2vm: WIP"
added worklist 4
alice@alice_PC$ hv edit -w 4 //REsearch/*
ok checked out '//REsearch/vm2vm.exe' for 'edit' (worklist 4)
ok checked out '//REsearch/vm2vm.i64' for 'edit' (worklist 4)

worklist show

worklist show [-s SITE] [-u USER] [WORKLIST_ID]

Lists worklist contents.

Show a list of files opened for editing, addition or deletion, and their associated worklist(s).

-s SITE

Restrict to site SITE. If omitted, defaults to the current site.

-u USER

Restrict to user USER. If omitted, defaults to the current user.

WORKLIST_ID

Restrict to the provided worklist, defaults to showing all worklists.

Parameters

Examples:

alice@alice_PC$ hv worklist show 3
WL 3 edit //cppobj_virtcall.i64#9
alice@alice_PC$ hv worklist show 4
WL 4 edit //REsearch/vm2vm.exe#1
WL 4 edit //REsearch/vm2vm.i64#1
alice@alice_PC$ hv worklist show
WL 4 edit //REsearch/vm2vm.exe#1
WL 4 edit //REsearch/vm2vm.i64#1
WL 3 edit //cppobj_virtcall.i64#9

Show the worklist contents of another user

alice@alice_PC$ hv worklist show -u ted
WL 4 edit //malware/unk_2022#1/6 SITE=ted_laptop
WL 2 copy //docs/onboarding.md#0 SITE=TEDPC
WL 1 del  //ida64.i64#6 SITE=TEDPC

worklist edit

worklist edit WORKLIST_ID DESCRIPTION

Edits a worklist description.

WORKLIST_ID

The worklist to modify.

DESCRIPTION

The new description for the worklist.

Parameters

Example:

change description of worklist 4

alice@alice_PC$ hv worklist edit 4 "vm2vm: resolved all offsets in 'main' function"

worklist del

worklist del WORKLIST_ID

Deletes a worklist.

This command will only succeed if the worklist is currently empty.

WORKLIST_ID

The worklist to delete.

Parameters

Example:

alice@alice_PC$ hv worklist del 3
worklist 3 is not empty
alice@alice_PC$ hv revert //cppobj*
ok reverted //cppobj_virtcall.i64
alice@alice_PC$ hv worklist del 3

Committing a worklist to the server

commit

commit [-f] [-s] WORKLIST_ID [DESCRIPTION]

Commits files to the vault (push).

This command uploads files from the local computer to the vault.

After a successful commit, the modifications made to the files contained in the worklist will be made available for other users.

A commit may fail if another user uploaded another revision of the changed files meanwhile. In this caseis necessary to merge the changes.

If the worklist does not yet have a proper description, the DESCRIPTION is mandatory.

-f

Force commit of unchanged files.

-s

Silent mode; do not output any messages.

WORKLIST_ID

The id of the worklist to commit to the vault.

DESCRIPTION

A description for the commit.

Parameters

Example:

alice@alice_PC$ hv commit 1
worklist 1 has empty description
alice@alice_PC$ hv commit 1 "more samples"
ok accepted //newfile#1 (5 bytes)
ok commit #2 completed

Syncing files, resolving & reverting

sync

sync [-f] [-p] [-s] [@COMMIT_ID] [PATH_PATTERN[=REVISION]…​]

Downloads the requested revisions of the files from the server, and stores them on the local filesystem.

If no paths are provided, all files from the server will be retrieved.

Requires that a site to be currently selected.

-f

Force sync. This will force a download of the files, even when the server thinks the client has the desired revision. This is a dangerous operation: any modification made to local files will be lost.

-p

The server will perform sync without really transferring files. This options is useful if the local files are already in sync but the server has stale info about them.

-s

Silent mode; do not output any messages.

@COMMIT_ID

Sync to state right after COMMIT_ID was committed, cannot be used with =REVISION.

PATH_PATTERN[=REVISION]…​

Vault path of file(s) to sync, if path is omitted, defaults to current directory, if no revision is specified, defaults to last revision available on vault (#^).

Parameters

Examples:

Sync all

alice@alice_PC$ hv sync

Sync only the specified subtree

alice@alice_PC$ hv sync -f //work/ds1_10

Sync a file to specific revision

alice@alice_PC$ hv sync //malware/Trojan.Shylock.Skype/D57D.i64#1
ok synced //malware/Trojan.Shylock.Skype/D57D.i64#1 (4374263 bytes)
ok sync completed

resolve

resolve METHOD PATH_PATTERN

Resolves conflicts in a file, using the specified strategy.

After the strategy is successfully applied and the local file has incorporated both the "local" and "remote" changes, it will be ready to be committed.

METHOD

One of "auto", "lmerge", "rmerge", "manual", "local" or "remote".