Building for PyLith development#

Tip

Unless you are an experienced PyLith developer, we strongly recommend using the PyLith development environment Docker image rather than using the installer to build all of the dependencies. The PyLith development Docker image contains all of the low-level dependencies.

There are a few modifications to the usual process when using the installer to build for PyLith development. There are several configure arguments relevant to using the installer for development:

--enable-developer: Enable installation of development tools (for example, generating documentation). PETSc is kept in its build directory to prevent conflicts when rebuilding.
--with-pylith-repo=URL: Clone the PyLith source code from this URL. The URL corresponds to your fork of the PyLith repository. If you are accessing GitHub via SSH, the URL usually has the form git@github.com:YOUR_GITHUB_USERNAME/pylith.git. If you are accessing GitHub via HTTPS, the URL usually has the form https://github.com/YOUR_GITHUB_USERNAME/pylith.git. The default URL is the geodynamics/pylith repository, geodynamics/pylith.git.
--with-pylith-git=BRANCH: Build branch BRANCH of PyLith. If you are just starting development and have not created any feature branches, then use the main branch.
--with-spatialdata-repo=URL: Clone the spatialdata source code from this URL. The default is the geodynamics/spatialdata repository, geodynamics/spatialdata.git. You do not need to change this unless you want to contribute to development of the spatialdata library.

Other arguments commonly used when configuring for PyLith development include:

--with-debugging: Use debugging (generate debugging symbols and use low-level optimization) when building spatialdata, PETSc, and PyLith.
--enable-swig: Build the SWIG library for generating the Python/C++ interface.
--enable-pcre: Build the PCRE library for use by SWIG.

Tip

We always have debugging turned on when we are developing. We build the code in directories separate from the source code. This makes it easy to build an optimized version in a different directory. We use environment variables to set the paths to the desired build.

Setting up the environment#

Prerequisites:

C/C++-14 compiler
git
make
fork of PyLith repository

We use the following directory structure:

pylith-developer/
├── pylith_installer-4.0.0-0 # source code for the installer
├── build-debug # top-level directory for building with debugging
└── pylith-debug # directory where PyLith and other CIG code is installed by installer

Important

The installer will create a virtual Python environment for PyLith in pylith-debug.

You will need to activate the virtual Python environment for PyLith in every shell where you rebuild or use PyLith.

Running configure will generate a bash script setup.sh in the build-debug directory. You should activate the virtual environment using this bash script, because we include any additional environment variables that need to be set in the setup.sh script.

Listing 4 Activate the environment using setup.sh#

source $PYLITH_DIR/build-debug/setup.sh

Note

We do not install PETSc to $PYLITH_DIR/pylith-debug. Instead we run leave its build in place. This prevents conflicts between the installed location and the local source files when rebuilding.

Listing 5 Setting up the installer for PyLith development.#

# Set some variables to hold information about our installer setup.
# The PYLITH_REPO should be https://github.com/YOUR_GITHUB_USERNAME/pylith.git if using a personal
# access token or git@github.com:YOUR_GITHUB_USERNAME/pylith.git if using SSH.
PYLITH_DIR=$HOME/pylith-developer
PYLITH_REPO=git@github.com:YOUR_GITHUB_USERNAME/pylith.git
PYLITH_BRANCH=main

# Create a top-level directory for PyLith.
mkdir -p $PYLITH_DIR
cd $PYLITH_DIR

# Place the installer source code tarball in $PYLITH_DIR and then unpack the tarball.
tar -xf pylith_installer-4.0.0-0.tar.gz

Linux#

This configuration is for the Ubuntu 22.04 Linux distribution. The setup is similar for other Linux distributions. Some older distributions may not provide Python 3 packages for the PyLith dependencies.

Install all of the system packages listed in the user installation for Ubuntu 22.04. You will also need to set the PYTHON_VERSION, HDF5_INCDIR, and HDF5_LIBDIR environment variables as in the user installation. Then, install the additional packages recommended for development:

apt-get install -y --no-install-recommends \
      python3-autopep8 \
      uncrustify \
      gnupg2 \
      lcov \
      gdb \
      valgrind

We use autopep8 and uncrustify to format the source code, gdb and valgrind for debugging, and lcov to measure code coverage of tests.

Configure the installer.

mkdir $PYLITH_DIR/build-debug
cd $PYLITH_DIR/build-debug
$PYLITH_DIR/pylith_installer-4.0.0-0/configure \
    --enable-developer \
    --with-debugging \
    --with-pylith-git=$PYLITH_BRANCH \
    --with-pylith-repo=$PYLITH_REPO \
    --with-fetch=curl \
    --with-make-threads=$(nproc) \
    --prefix=$PYLITH_DIR/pylith-debug \
    --with-hdf5-incdir=${HDF5_INCDIR} \
    --with-hdf5-libdir=${HDF5_LIBDIR} \
    --disable-cmake \
    --disable-sqlite \
    --disable-numpy \
    --disable-hdf5 \
    --enable-catch2 \
    --enable-h5py \
    --enable-netcdf \
    --enable-netcdfpy

Check the configure output to make sure it ran without errors and the configuration matches what you want. Setup your environment, and then build the remaining dependencies and then Pythia, SpatialData, PETSc, and PyLith.

cd $PYLITH_DIR/build-debug
source setup.sh
make

macOS#

Important

You will need to install XCode or XCode command line tools before configuring the installer.

This configuration is for Monterey (12.6.X). We use the Apple clang/clang++ compiler. macOS does not provide a standard installation of Python 3, so we use the installer to build Python 3. XCode does not contain the autotools suite (automake, autoconf, libtool), so we also use the installer to build autotools.

Tip

Instead of having the installer build Python 3, a good alternative is to install the macOS Python binary provided by python.org. In this case you do not need to install OpenSSL, libffi, or Python.

Important

The installer will create a virtual environment for PyLith. You will need to activate this virtual environment for PyLith in every shell (terminal) where you want to build or use PyLith. You may want to add source $PYLITH_DIR/pylith-debug/bin/activate to your .bashrc or local setup.sh file.

Configure the installer.

mkdir $PYLITH_DIR/build-debug
cd $PYLITH_DIR/build-debug
${HOME}/src/pylith/pylith_installer-4.0.0-0/configure  \
    --enable-developer \
    --with-debugging \
    --with-pylith-git=$PYLITH_BRANCH \
    --with-pylith-repo=$PYLITH_REPO \
    --with-fetch=curl \
    --prefix=$PYLITH_DIR/pylith-debug \
    --enable-force-install \
    --with-make-threads=8 \
    --with-fortran=no \
    --enable-autotools \
    --enable-mpi=mpich \
    --enable-openssl \
    --enable-libffi \
    --enable-curl \
    --enable-sqlite \
    --enable-python \
    --enable-swig \
    --enable-pcre \
    --enable-catch2 \
    --enable-tiff \
    --enable-proj \
    --enable-hdf5 \
    --enable-cmake \
    CC=clang CXX=clang++

cd $PYLITH_DIR/build-debug
source setup.sh
make