Installation

Before we start, make sure you are on a linux machine, OSX machine or a linux virtual machine. OSX users must install the developer version, (pip packages are not compiled for OSX for now). On Windows 10 an Ubuntu Linux subsystem can installed via WSL, which should offer better performance than a virtual machine.

Spam works on Python 3.5+, but if you need python 2.7 support the git install still works correctly.

The install process is in two steps: installing system dependencies, then installing spam.

If you “just” want to use spam, please follow Installing system dependencies corresponding to your system, then Set up a Python virtual environment for spam, then follow the Install spam with pip (for users). This means using pip to install spam, you’ll have occasional updates and will not be able to contribute code.

If you want to contribute to spam, or a cutting-edge version (which is less stable), please follow Installing system dependencies, then Set up a Python virtual environment for spam, and then Install spam with git (for developers).

Before we start, commands for you to execute

$ look -like this





Installing system dependencies

This step is required for both user and developer install. The installation of these required packages (for spam compilation and for some of our Python dependencies). The details of this step is very system-dependent, please see below for your system:

Note: we use R and rpy2 only for the generation of random fields. It’s considered on optional dependency, however test_excursions.py will fail without it.

System dependencies for Debian/Ubuntu Linux

If you’re on Debian/Ubuntu:

$ sudo apt update

$ sudo apt upgrade

$ sudo apt install git python3-dev python3-virtualenv python3-tk gcc g++ libeigen3-dev r-base libicu-dev libgmp-dev libmpfr-dev mpi-default-bin mpi-default-dev libcgal-dev gmsh libfreetype6-dev libxml2-dev libxslt-dev

Please see note for version 20.04 of Ubuntu *. Now you’re ready to go to: Set up a Python virtual environment for spam



System dependencies for Scientific Linux 6.9

Warning: This is probably out of date.

If you’re on Scientific Linux 6.9 (and you don’t want to update to a reasonably modern distribution, e.g., it’s installed on a cluster):

$ sudo yum install epel-release

$ sudo sh -c 'wget -qO- http://people.redhat.com/bkabrda/scl_python27.repo >> /etc/yum.repos.d/scl.repo'

$ sudo yum install git swig python27 python27-python-libs python27-runtime python27-python-devel python-pip python-virtualenv gcc eigen3-devel R gmp-devel mpfr-devel openmpi openmpi-devel mpich-devel mpich-autoload

Now you’re ready to go to: Set up a Python virtual environment for spam



System dependencies to install for other Linux distributions

For other linux installations our dependencies are currently:

Python components:

  • python3 and python3 development files

  • python3 virtual env (highly recommended to not install spam right into the system)

  • python3 tk libraries for matplotlib

Compilation dependencies:

  • gcc and/or g++

  • eigen3 development files

  • CGAL development files

External programs we call:

  • R

  • gmsh

  • mpi (optional)

Dependencies of the pip packages we use:

  • libfreetype6-dev (for matplotlib)

  • libxml2-dev

  • libxslt-dev

  • libicu-dev

  • libgmp-dev

  • libmpfr-dev

Once these are install you’re ready to go to: Set up a Python virtual environment for spam



System dependencies for OSX 10.14

  • Install xcode

  • Install the following packages from source e.g., to /usr/local, or with homebrew (e.g., brew install cmake). Homwbrew is recommended for simplicity.

    • cmake

    • eigen3

    • openMPI

    • gmsh 4.4.1

    • gmp

    • mpfr

    • cgal

  • install Python 3.7.4 package from python.org for OSX

  • (optional, see above) install R 3.6.1 package from R for OSX. The rpy2 Python module requires llvm to compile correctly (since there is no openMP support in Apple’s clang).

    • Install llvm with homebrew (e.g., brew install llvm) - or the instructions from llvm.org for installing llvm from source (with the clang component), and install openmp from llvm.org as well.

    • llvm may require that you install OSX SDK headers using the following command (which in turn require the Xcode command line tools):

    $ xcode-select --install
$ open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

  • add the following to ~/.profile:

# general /usr/local variables  , also needed by brew
export PATH=$PATH:/usr/local/bin
export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/lib
export DYLD_FALLBACK_LIBRARY_PATH=$DYLD_FALLBACK_LIBRARY_PATH:/usr/local/lib

###########
# for LLVM:
export LLVM_PREFIX=/usr/local/opt/llvm  # or your llvm install location if not using brew
export PATH=$PATH:$LLVM_PREFIX/bin
export LIBRARY_PATH=$LIBRARY_PATH:$LLVM_PREFIX/lib
export DYLD_FALLBACK_LIBRARY_PATH=$DYLD_FALLBACK_LIBRARY_PATH:$LLVM_PREFIX/lib
export CC=$LLVM_PREFIX/bin/clang
export CXX=$LLVM_PREFIX/bin/clang++
#the following line may not be needed on all installations
export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolcha\
in/usr/include/c++/v1
#if LLVM came with brew you may need this too:
export KMP_DUPLICATE_LIB_OK=TRUE
#############

Thanks to Adrian Sheppard, Steve Hall, and Ryan Hurley for these OSX install notes.

Now you’re ready to go to: Set up a Python virtual environment for spam



System dependencies for Windows 10

If you’re on Windows 10, please install Ubuntu 18.04 WSL (Windows Subsystem for Linux), and follow System dependencies for Debian/Ubuntu Linux.

In order for graphs and images to appear with matplotlib you need to first install XMing and have it running. In your Ubuntu subsystem you must then run the following in the terminal:

echo "export DISPLAY=:0;" >> ~/.bashrc

Now you’re ready to go to: Set up a Python virtual environment for spam






Set up a Python virtual environment for spam

If the dependencies above installed correctly, now you can go ahead and create a virtual environment for spam. If you’re using Anaconda, please see this note: Installing into Anaconda Environment.

You can create the virtual environment any writable folder on your computer. The virtual environment is created, in the folder you want, using:

$ virtualenv -p python3 spam

This will create a virtual environment called spam in the folder spam. You can then activate the virtual environment:

$ source spam/bin/activate
(spam) $

You should see (spam) written at the beginning of your command line. You will need to activate this virtual environment in the terminal each time you want to use spam – check for the (spam) at the command prompt!

You should then upgrade your virtual environment to the latest version of pip and setuptools:

(spam) $ pip install -U pip setuptools

Congratulations, now that your virtual environment is set up, you can either:




Install spam with pip (for users)

Execute the following line…

(spam) $ pip install spam

spam should now be installed! Check whether:

(spam) $ spam-ldic --help

prints out the help for the function, if it does, you’re done!




Install spam with git (for developers)

1. Clone spam

It can be cloned anywhere on the computer. For example in a folder ~/bin. Before cloning go to the directory:

$ cd ~/bin/

Then clone the repository:

$ git clone https://gricad-gitlab.univ-grenoble-alpes.fr/ttk/spam.git

Now cd into the spam directory that has been downloaded:

$ cd spam

2. Set up virtual environment

Activate the previously-created virtual environment:

$ source path/to/virtualenv/spam/bin/activate
(spam) $

Install pip requirements:

(spam) $ pip install -r requirements.txt

and (optionally) the development tools to calculate coverage, rpy2, and build documentation:

(spam) $ pip install -r requirements-dev.txt

3. Install spam into the virtualenv

(spam) $ python setup.py install

In the future, when you do

(spam) $ git pull

to download the latest version of the code, you’ll have to repeat setup install above.






Installing Jupyter for use with spam (optional)

You need to install Jupyter (with pip or you package manager).

If you work in a virtualenv, you have to add ipykernel in it :

(spam) $ pip install ipykernel

and declare your virtualenv in Jupyter :

(spam) $ python -m ipykernel install --user --name=spam

Then you should be able to select this kernel from Jupyter and download the Jupyter Notebook files from the examples.






Installing into Anaconda Environment

If you’re using an Anaconda Python environment, and you want to install spam from pip please try the following:

conda create -n spam python=3.7
conda activate spam
module load mpi
pip install spam
spam-ldic --help

Thanks to Jan-Willem Buurlage for the tip.






Problem with tifffile ?

You may find that tifffile complains that it needs a c file to compile _tifffile. It is not a spam issue but a pip/tifffile one. It happens when you build tifffile for the first time in a virtualenv without numpy. After that, the packages without C extension is cached, and each time you pip install, you will use the cached version…

Quick solution, install tifffile with no-cache-dir option :

(spam) $ pip --no-cache-dir install tifffile

Longer and cleaner solution :

  • get and rm the cached package:

(spam) $ pip uninstall tifffile
(spam) $ pip --verbose install tifffile|grep cached
...
Using cached wheel link: file:///path/to/file
(spam) $ rm /path/to/file

  • uninstall and reinstall in an env with numpy :

(spam) $ pip install numpy
(spam) $ pip --verbose install tifffile |grep tiffile.so
...
adding 'tifffile/_tifffile.so'
...
*

There is a temporary problem in generating builds consistent with PyPi with this version. The easiest fix is to uncomment the ==N.M.O.P values from requirements.txt for the following: numpy, tifffile, SimpleITK, mpi4py, pybind11, opencv-python, lxml