Building Cryptech Software/Firmware/Bitstream From Source

Everything you need to build our software, firmware, and FPGA bitstreams from source yourself is publicly available, but the process is a bit complicated. Overall, there are two methods, one of which our developers use while writing this stuff, the other of which we use for the automated reproducible builds which go into our binary distributions. Both methods eventually boil down to "get the source code then run make", but the details differ.

What developers do

We check out copies of all the several dozen separate repositories and carefully arrange them in a tree structure which matches the official naming scheme. Yes, really. It's tedious, but we have a script to automate this. This works by parsing the .gitmodules file in the releng/alpha repository (see "reproducible builds", below).

Once you have this tree, you can hop around within it, building whichever bits are of interest to you. So if you want to rebuild just the HSM firmware (the C code that runs on the ARM), you would go to sw/stm32 and run make there.

What we do for reproducible builds

Reproducible builds use the same tree structure (as they must for the various Makefiles to work properly), but the entire tree is embedded in a git "superrepository" which also contains the release engineering goo necessary to make the whole thing work. Do git help submodule for an introduction to git's submodule mechanism.

With this model, one just checks out a copy of the superrepository, runs make in its top directory, and eventually the complete package pops out the other side.

git clone https://git.cryptech.is/releng/alpha.git
cd alpha
make

That's the good news. The bad news is that this process has higher demands on its build environment: it expects to find the a complete tool set, including the XiLinx synthesis tools, the several different cross compilers for the firmware, and the pbuilder system for building clean room packages for Ubuntu and Debian.

As a compromise, one can use this source tree as if it were the development source tree described above: just use the supermodule to pull down everything else, but then ignore the supermodule and build individual pieces as if you'd checked out all the repositories by hand.

Skip all this git mess and just download a tarball

There's another alternative, which is simpler than any of the above: just download the source tarball. Since the only build environments we support at the moment are Debian Jessie and Ubuntu Xenial, which also happen to be environments for which we build binary packages, you can just use APT:

apt-get source cryptech-alpha

Which will give you the same tree structure, but without all the git fun.

Build environment

Our software and firmware developers use the Debian and Ubuntu Linux distributions. Our current build box for binary packages runs Debian Jessie.

Our Verilog developers use various environments and have been known to use graphical tools, but synthesis of the bitstreams that go in our binary packages is done via the XiLinx command line tools on the same Debian Jessie machine as the software and firmware builds.

Which tools you need will of course depend on exactly what you're trying to do.

Most of the tools work on either 32-bit or 64-bit machines, but if you intend to run the full binary package build script, you'll need a 64-bit machine (or VM) because the tools won't build 64-bit binaries on a 32-bit machine.

Basic tool set (not all required for every purpose, but they're all supported Debian packages so it's usually easier just to install them all and not worry about it):

  apt-get install git pbuilder ubuntu-dev-tools rsync sudo
  apt-get install python-yaml python-serial python-crypto python-ecdsa
  apt-get install gcc-arm-none-eabi gdb-arm-none-eabi
  apt-get install gcc-avr binutils-avr avr-libc
  apt-get -t jessie-backports install debootstrap distro-info-data
  apt-get install reprepro ubuntu-archive-keyring

This is not an exhaustive list, because some of the other packages we use are pulled in by these as dependencies.

You will also need a copy of the XiLinx tools, which is tedious enough that it's described in a separate section, below.

Once you have all the tools installed, you'll need a copy of the source tree, as explained in the preceeding sections.

pbuilder requires a bit of setup (you can skip this if you're not trying to do the full binary package build):

for code in jessie xenial; do for arch in i386 amd64; do pbuilder-dist $code $arch create; done; done
ln -s jessie_result ~/pbuilder/jessie-amd64_result
ln -s xenial_result ~/pbuilder/xenial-amd64_result

Installing the XiLinx tools

XiLinx tools setup is a bit involved. You can skip this section if you don't intend to build FPGA bitstreams.

We use the command line versions of the XiLinx tools, but installing them requires a graphical environment, because the XiLinx installer and license manager are GUI tools. If you're running this on a server and don't already have a graphical environment installed, you can get away with something fairly minimal. For example, if you have a VNC viewer such as "Chicken of the VNC" on your laptop, you can get away with a fairly minimal X11 toolset:

apt-get install tightvncserver xterm icewm

If you're already running X11 on your laptop and are comfortable with extruding that to the build machine, eg, via ssh -Y, you can just use that (not recommended for long-haul use, eg, if the laptop is in Boston and the server is in Reykjavik).

You'll need to start by using a web browser to download the Xilinx ​ISE Design Suite.

XiLinx only supports specific versions of Red Hat and Suse Linux, but their tools do run on Debian and Ubuntu. A few caveats:

  • Debian and Ubuntu symlink /bin/sh to /bin/dash, which can't handle some of the syntax used in XiLinx's shell scripts, so you'll need to change that symlink to point to /bin/bash.

  • Although the XiLinx software can be installed as user or root, by default it wants to install into /opt/Xilinx, so you need to install as root if you want to do that.

  • The XiLinx tools are disk hogs, so if you're building a VM for this, you'll probably want to give it at least 30-40 GB of disk space.

Step-by-step installation:

  1. Unpack Xilinx_ISE_DS_Lin_14.7_1015_1.tar (or whatever version you have).
  2. In an X11 environment, cd to Xilinx_ISE_DS_Lin_14.7_1015_1, and run sudo ./xsetup
  3. Click through two screens of license agreements.
  4. Select ISE WebPACK.
  5. Unselect (or leave unselected) Install Cable Drivers.
  6. Go!

Well, not quite. You'll need to convince the ISE tools that you have a valid license to use the ISE tools. Go to http://www.xilinx.com/products/design-tools/ise-design-suite/ise-webpack.htm, click the Licensing Solutions link. On the page to which that takes you, expand the section Obtain a license for Free or Evaluation product. To download the ISE Webpack, you should have created an account, so now you can go to the Licensing Site and use that account to create a Certificate Based License.

You do not need to go through the HostID dance, just say Do It. You will then receive a certificate in email (not an X.509 certificate) which you will be able to use. Then start the ISE Webpack by issuing the command ise. Go to the Help menu and Manage Licenses. Use the resulting new License Manager window to install the .lic file. This process is complex and flakey.

Here's another description of installing ISE on Ubuntu.

The ise binary referred to above is in /opt/Xilinx/14.7/ISE_DS/ISE/bin/lin64/ise (or in .../lin/ise, but the pbuilder setup requires a 64-bit build machine).

When running this remotely under tightvncserver, setup looks something like this:

vncserver :0 -geometry 1280x768 -depth 16 -localhost
export DISPLAY=:0 XAUTHORITY=~/.Xauthority
icewm&

Then, either in the same shell as the above or in an xterm in the new display

cd Xilinx_ISE_DS_Lin_14.7_1015_1
sudo ./xsetup

cd
/opt/Xilinx/14.7/ISE_DS/ISE/bin/lin64/ise

It turns out you don't really need to run the whole ise tool to get to the license manager, you can just run

/opt/Xilinx/14.7/ISE_DS/common/bin/lin64/xlcm -manage

But you do have to source the appropriate settings file first, none of the XiLinx tools work properly without that:

. /opt/Xilinx/14.7/ISE_DS/settings64.sh