Trojan Insertion versus Layout Defenses for Modern ICs: Red-versus-Blue Teaming in a Competitive Community Effort

Context

This repository relates to our contest and benchmarking efforts for insertion of hardware Trojans (HTs) versus layout-level defenses. The contest itself was initially introduced as Advanced Security Closure of Physical Layouts to the community; see https://wp.nyu.edu/ispd23_contest/ for more details.

The latest TCHES publication with all details can be found at https://eprint.iacr.org/2024/1440.

Content

This repository includes the benchmarking framework, which covers

1. data management;
2. design and security evaluation, including insertion of HTs; and
3. parsing of results and scoring.

This repository does not contain (a) the benchmarks, the HTs, and the ASAP7 technology setup, nor (b) the best results from the contest, nor (c) the defense techniques, nor (d) the reference flow for physical design.

For (a), (b), and (c), see https://drive.google.com/drive/folders/10GJ5hX0BQupwqv1WMtitarsEuEE_Y-vV?usp=sharing; for (d), see https://github.com/Centre-for-Hardware-Security/asap7_reference_design. More details on which item/artifact to use how are described throughout this README.

Few non-essential notes on the branches in this repository -- feel free to skip ahead. The main branch of this repository contains a revised standalone version of the framework, which is tailored to be run locally and up-to-date. That branch is the one you want. Consider the branch gh_backend as FYI only. That branch contains the full framework, including the backend's web interface used during the contest itself, although again in some tailored version meant to be run locally. Not all latest edits are reflected in that branch.

Requirements

You need commercially licensed tools for physical design from Cadence, namely Innovus and Conformal/LEC.

Regarding Innovus, we have tested the framework (as well as the reference flow for physical design) for different versions, namely for 17.10, 18.10, 19.11, 20.11, 21.11, and 21.13. We found that results can vary across these versions. The version for the contest was 21.13-s100_1 -- if possible, it is highly recommended to use the very same or at least a similar version (21.x) for reproducibility of the published results.

For Conformal/LEC, the official version for the contest was 22.10-s300. We have not tested other versions, but we would expect that results generated by this tool are less sensitive, if at all, regarding different versions (as the related script and its underlying assessments are not complex at all).

To get the framework up and running, you need this repository as well as item (a) from Content; see also Folder Setup for more related details on handling of that item (a). The items (b), (c), and (d) are optional (i.e., not needed for operation of the framework as such), but it is highly recommended to look at these items the moment you want to pursue your on follow-up research for defenses. Note that (c) and (d) both come with their own dedicated README and instructions.

Alternative Access

If you do not have access to the commercial tools listed in Requirements, you can also get access to the framework used during the contest itself. This access will be in the same form as in the contest, i.e., you can upload your hardened layouts and receive back scores and HT-infested layouts. (However, you will not be given access to the actual backend itself.) See https://wp.nyu.edu/ispd23_contest/evaluation/#platform for more details and drop us an email at ispd23_contest@nyu.edu -- from a Google account -- so that we can provide you with access to your dedicated Google Drive as access point.

For that alternative option, you do not need to follow the Setup and Operation but the instructions given at https://wp.nyu.edu/ispd23_contest/evaluation/#platform.

Some non-essential remarks here. For the reference flow and the defense techniques provided via the external link given in Content, these are also implemented using Cadence Innovus. Thus, you would need to pull your own efforts if you want to transfer and extend those for open-source tools. However, there should be a good number of codes and tools online that support ASAP7, the open-source technology library of choice for this effort.

Setup and Operation

Folder Setup

Within the same local root folder where you've cloned this repository into, you need to unzip the archives benchmarks_HTs_ASAP7.zip and data_skeleton.zip obtained from https://drive.google.com/drive/folders/10GJ5hX0BQupwqv1WMtitarsEuEE_Y-vV?usp=sharing

Once done, the root folder structures should look like this (only partially shown here, namely till level 4):

	benchmarks
	└── _release
	    ├── README
	    ├── _AIC -> _final/
	    ├── _EXT -> _final/
	    ├── _ASAP7
	    │   ├── gds
	    │   ├── lef
	    │   ├── lib
	    │   ├── qrc
	    │   ├── README
	    │   └── techlef
	    └── _final
	        ├── aes
	        ├── camellia
	        ├── cast
	        ├── misty
	        ├── seed
	        └── sha256
	data
	├── AIC
	│   ├── CUEDA
	│   │   ├── aes
	│   │   ├── camellia
	│   │   ├── cast
	│   │   ├── misty
	│   │   ├── seed
	│   │   └── sha256
	│   ├── FDUEDA
	│   │   ├── aes
	│   │   ├── camellia
	│   │   ├── cast
	│   │   ├── misty
	│   │   ├── seed
	│   │   └── sha256
	│   └── NTHU-TCLAB
	│       ├── aes
	│       ├── camellia
	│       ├── cast
	│       ├── misty
	│       ├── seed
	│       └── sha256
	└── EXT
	    ├── CUEDA
	    │   ├── aes
	    │   ├── camellia
	    │   ├── cast
	    │   ├── misty
	    │   ├── seed
	    │   └── sha256
	    ├── FDUEDA
	    │   ├── aes
	    │   ├── camellia
	    │   ├── cast
	    │   ├── misty
	    │   ├── seed
	    │   └── sha256
	    └── NTHU-TCLAB
	        ├── aes
	        ├── camellia
	        ├── cast
	        ├── misty
	        ├── seed
	        └── sha256
	scripts
	├── eval
	│   ├── checks
	│   │   ├── check_pins.sh
	│   │   ├── check_rails_stylus.tcl
	│   │   ├── checks.tcl
	│   │   ├── check_stripes_area_stylus.tcl
	│   │   ├── check_stripes_coors_stylus.tcl
	│   │   ├── check_stripes_set2set_stylus.tcl
	│   │   ├── check_stripes_width_stylus.tcl
	│   │   └── lec.do
	│   ├── des
	│   │   ├── mmmc.tcl
	│   │   ├── PPA.AIC.tcl
	│   │   └── PPA.EXT.tcl
	│   ├── scores
	│   │   └── scores.sh
	│   └── sec
	│       ├── 1st_order
	│       └── ECO_TI
	└── gdrive
	    ├── ISPD23_daemon_procedures.sh
	    ├── ISPD23_daemon.settings
	    ├── ISPD23_daemon.sh
	    └── ISPD23_daemon_wrapper.sh

Some notes on the data skeleton.

1. The main folders within data/ are representative of the top-3 teams from the contest. You can just reuse any one or multiple of these folders, but do not create your own folders; they won't be operated on (unless you revise scripts/gdrive/ISPD23_daemon_procedures.sh for the lines with google_team_folders[]=).

2. The use of the 3rd-level subfolders (backup_work etc) is explained in Data In and Data Out further below.

3. While the data skeleton would also be automatically initialized by the daemon, using the one provided here is still helpful, especially for beginners of the framework, as having the skeleton ready helps avoiding hiccups during data arrangement; see also Operation further below on that.

Compilation

Compile the C++ code for pre-attack security evaluation considering exploitable regions:

cd scripts/eval/sec/1st_order/_cpp/
./compile.sh
cd -

Address compilation issues for your local C++ setup, if any.

Configuration

Configure the daemon through the following steps.

1. Required. File scripts/gdrive/ISPD23_daemon.settings. Revise line 32: local_root_folder="/data/nyu_projects/ISPD23" with your local path you've cloned the repository into.

2. Required. File scripts/gdrive/ISPD23_daemon.settings. Check all the lines that start with alias call_*. Revise for your local installation of the design tools, if needed.

3. Optional. File scripts/gdrive/ISPD23_daemon.settings. Revise line 13: round="AIC" for the desired operation. The available options are AIC (short for as in contest) or EXT (short for extended techniques). Please see the TCHES paper listed in Context for more details.

4. Optional. Files scripts/eval/sec/ECO_TI/TI_wrapper.*.sh. Revise lines 20, 21: max_current_runs_* according to the number and type of Innovus licenses available at your end. These settings dictate how many ECO runs for HT insertion are operated in parallel.

   Note that max_current_runs_aes mandates / only works with Innovus (invs) licenses, whereas max_current_runs_default works with both VDI (vdi) and Innovus (invs) licenses and automatically picks whatever licenses are free. Also note that these settings are "per design run", i.e., by default up-to 6 parallel ECO runs are allowed per evaluation job -- for the latter, the upper limit for parallel processing is configured in the file scripts/gdrive/ISPD23_daemon.settings, line 22: max_parallel_runs=8.

Data In

To put runs into the backend, arrange data through the following steps. See a full example toward the end of that subsection.

1. Choose the correct root folder for data input. First, depending on the mode you want to run, either consider data/AIC or data/EXT. Second, as indicated in Folder Setup, you can freely pick from any of the skeleton subfolder(s) in there, whereas you should not create your own (unless you revise the related code in the daemon). Third, pick the folder matching the benchmark you are working on. Finally, choose the downloads subfolder.

   For example, for the AIC mode on sha256, you could pick data/AIC/CUEDA/sha256/downloads as root folder.

2. Within that root folder data input, init new subfolders for each run. It is good practice to use unique IDs for these subfolders, e.g., run_001. (However, it is not mandatory -- in case subfolders with the same name are used again later on, the daemon will avoid conflicts for the data backup by extending the folder names with respective timestamps.)

   For example, for the AIC mode on sha256, you could generate data/AIC/CUEDA/sha256/downloads/run_001 and data/AIC/CUEDA/sha256/downloads/run_002.

3. For the new subfolder(s), put the design files you want to have evaluated. You must put both the DEF file and the Verilog netlist file. You must put only 1 pair of DEF and Verilog file in each subfolder -- use separate subfolders for multiple runs. The naming of the DEF and Verilog files does not matter.

To get started, you could run the benchmark design files as is, to obtain the baseline references. Continuing the above example, you would want to: cp benchmarks/_release/_final/sha256/design.{def,v} data/AIC/CUEDA/sha256/downloads/run_001/. Next, you could reproduce the full results documented in our TCHES paper listed in Context. For that, you need to obtain the best results/layouts from the external link given in Content.

Full Example: As indicated, you can arrange multiple runs at once. For example, to run all benchmarks through baseline evaluation, you could do:

for bench in $(ls benchmarks/_release/_final/); do
	mkdir data/AIC/CUEDA/$bench/downloads/run_001/
	cp benchmarks/_release/_final/$bench/design.{def,v} data/AIC/CUEDA/$bench/downloads/run_001/
done

Operation

Start the daemon, preferably from another independent bash session:

cd scripts/gdrive/
./ISPD23_daemon_wrapper.sh

Few important notes here.

1. Handle this as an actual daemon; keep it going as long as some runs are still working. Otherwise, if you interrupt it during some runs, you need to kill any lingering processes (innovus, lec, TI_init.*.sh, TI_wrapper.*.sh), and also cleanup the related work directories manually.

2. When seeking to push more runs into the download folders, it's better to first pause (Ctrl Z) or stop (Ctrl C) the daemon, then arrange the new data, and finally continue/restart the daemon. Either way (pausing or stopping the daemon), it's best to do so only once the daemon has finished all ongoing runs.

   FYI, that approach is important because, unlike the daemon version for the actual contest backend, this tailored version for local operation is more "fragile" in terms of data management, as we're bypassing the procedures handling data downloads and instead directly push files into the system. While this is not an issue as such, there can be easily race conditions: whenever the daemon checks for new downloads, it will right away arrange these files into the processing queue, and the moment your (manual) data arrangement is not done yet, any incomplete set of files would result in processing errors, and you'd have to redo all the concerned runs.

Data Out

Once the daemon has finished some run, you can access the results in the corresponding backup_work folder. Continuing the above example, you would find the results for the baseline runs in data/AIC/CUEDA/$bench/backup_work/run_001/.

Few notes for organization of the result files. For interpretation, please also refer to our TCHES paper listed in Context.

1. In case it exists, check reports/errors.rpt for any processing errors at your end (apart from the scoring errors for EXT that can be ignored; see point 3. below).

2. For AIC, all results are summarized in reports/scores.rpt. This file contains all the metrics (for both the baseline and the submission) and the scoring. More extended views -- but no further relevant data beyond that already included in scores.rpt -- are covered by the various files in reports/. See also the README files in benchmarks/_release/_final/$bench/README for detailed descriptions of the various files.

3. Similarly, for EXT, all results are summarized in scores.rpt.failed. Importantly, unlike the filename suggests, all metrics described in there are valid. Note that this file is normally declared as "failed" only because the scoring scheme proposed for AIC was not extended for EXT; the related computations fail. However, for EXT, we actually do not care about the scoring, but rather about the detailed insights provided by all the metrics. Again, please also refer to our TCHES paper for better understanding.

   In short, ignore the scoring in reports/scores.rpt.failed but focus on the valid raw metrics. Also ignore errors reported on scoring in reports/errors.rpt.

4. In the main folders, i.e., the parents of the reports/ folders, you can find further technical details. One further level up, the corresponding zip files contain all the work files, including all HT-infested layouts, and all log files.

   For beginners, it is recommended to focus only on the reports/scores.rpt files and possibly other report files, whereas all these technical details should only be of interest for advanced users which may also aim for their own follow-up research on layout defenses vs HT insertion.