Build Environment Variables

See also: in the source tree BROKEN src/product/scripts/browserEnvironment.txt

The kent/src build makes use of environment variables to configure include and lib paths used when compiling. The most important are MYSQLLIBS and MYSQLINC -- see step 3 of kent/src/product/README.building.source. This page focus on several optional environment variables, with defaults provided by kent/src/inc/common.mk.

External library configuration (USE_*)

Several browser features require the installation of additional external software packages. They are disabled unless their corresponding environment variables are set to 1. If you are compiling on hgwdev, then you can use any or none of these as you wish (but you may get an annoying track handler warning if you don't set USE_BAM to 1, and then visit hg18 in hgTracks). To set a variable USE_XXX to 1, if you have a ~/.cshrc or ~/.tcshrc variable, add this:

# Enable UCSC XXX features
setenv USE_XXX 1

and if you have a ~/.bashrc, add this:

# Enable UCSC XXX features
export USE_XXX=1

After editing, source the file that you edited (e.g. "source ~/.cshrc"). Then, do a clean build of your tree, to make sure that both libraries and CGIs/utils are compiled with consistent settings.

cd kent/src
make clean
make

USE_SAMTABIX

As of our software release v275 (end of October 2012), it is no longer necessary to separately download, patch and configure samtools and tabix as described in the following section (Obsoleted by USE_SAMTABIX). Instead, you can simply download and build UCSC's samtabix package as follow:

git clone http://genome-source.soe-.ucsc.edu/samtabix.git samtabix
cd samtabix
make

and then set two environment variables: set USE_SAMTABIX to 1 and set SAMTABIXDIR to your local samtabix build directory. (If you have manually installed libsamtabix.a and *.h files in separate directories, then set SAMTABIXINC to the directory that contains the .h files, and SAMTABIXLIB to <libDirectory>/libsamtabix.a.)

Obsoleted by USE_SAMTABIX

The following environment variables are still supported for the time being, but if you're using them, we recommend installing the samtabix package and switching to USE_SAMTABIX above.

USE_BAM (+KNETFILE_HOOKS)

BAM is the compressed binary form of the SAM/BAM file format for short read alignments. The Genome Browser can display BAM data tracks when this is enabled. The samtools C library must be installed first. Then, in addition to setting USE_BAM to 1, you must define another variable or two depending on how you have installed samtools locally. If you have done the default build, which places libbam.a in the top-level samtools directory along with the .c and .h files, then set the environment variable SAMDIR to that directory. If you have manually installed libbam.a and .h files in separate directories, then set SAMINC to the directory that contains the .h files, and SAMLIB to <libDirectory>/libbam.a .

UCSC has an extension to samtools that allows its network module (knetfile) to be configured as a wrapper on a different module, so we can configure it to use UCSC's network code. This speeds up BAM display by caching parts of the data file that have already been accessed, and supports https and http with authentication. If you obtain the extension from UCSC KNETFILE_HOOKS (or if it is someday incorporated into the main distribution of samtools as we hope) and recompile samtools with it, then you can set KNETFILE_HOOKS to 1 to make the UCSC code utilize the extension.

USE_TABIX (+KNETFILE_HOOKS)

Tabix is a compression and indexing utility that works on any tab-separated file that is sorted by genomic position (sequence name, start offset). As of Mar. 16, 2011, when USE_TABIX is set to 1, the kent/src tree has library support for reading tabix-compressed files, and development of VCF+tabix track display is underway. The tabix C library, which is distributed from the same site as samtools, must be installed first. Then, in addition to USE_TABIX=1, one or more variables pointing to the local installation must be defined. If you have done the default build, which places libtabix.a in the top-level tabix directory along with the .c and .h files, then set the environment variable TABIXDIR to that directory. If you have manually installed libtabix.a and .h files in separate directories, then set TABIXINC to the directory that contains the .h files, and TABIXLIB to <libDirectory>/libtabix.a .

UCSC has an extension to tabix that allows its network module (knetfile) to be configured as a wrapper on some other module, so we can configure it to use UCSC's network code. This speeds up fetching of remote data by aching parts of the data file that have already been accessed, and supports https and http with authentication. If you obtain the extension from UCSC KNETFILE_HOOKS (or if it is someday incorporated into the main distribution of tabix as we hope) and recompile samtools with it, then you can set KNETFILE_HOOKS to 1 to make the UCSC code utilize the extension.

Note: both samtools and tabix libraries include bgzf_* functions for compressing and decompressing data. The functions are almost identical, but not completely. Therefore, linking with both libraries can fail unless the function names are uniquified. Included in UCSC's KNETFILE_HOOKS patch for tabix is a change of bgzf_* function names to ti_bgzf_*, which prevents the clash. If KNETFILE_HOOKS is set to 1, both samtools and tabix must be patched using their respective KNETFILE_HOOKS patch files.

USE_PNG

This is no longer an option as of the middle of 2010. You must set USE_PNG=1 in your shell environment to build the source tree and have the browser function. You may need to install the libpng-devel package if it is not already. To check your installed packages on linux:

rpm -qa | grep -i png

libpng-devel often installs its include files in places such as:

/usr/include/libpng10

In which case you may need to define:

PNGINCL="-I/usr/include/libpng10"

in your build environment to direct the compiler to that directory to find png.h

PNG (Portable Networks Graphics) is a widely used image file format, supported by all major browsers, that generally produces smaller image file sizes than the Genome Browser's default format, GIF. The libpng library must be installed before setting USE_PNG to 1; if on a linux system, use your favorite package manager (rpm, yum, apt-get etc.). A caveat: since kent/src/inc/common.h includes setjmp.h, libpng's pngconf.h must be edited to remove the syntax error that is intentionally generated when setjmp.h has already been included.

USE_SSL

SSL (Secure Socket Layer) provides encrypted transfer of data. In the Genome Browser, URLs that begin with "https://" or "ftps://" are supported only when this is enabled. The libssl library (development version w/headers) must be installed first; if on a linux system, use your favorite package manager (rpm, yum, apt-get etc.).

Compilation environment variables

COPT

Default: -O -g (optimize but keep debugger symbols). When debugging gets gnarly due to optimizations, recompile with COPT=-ggdb (or if not using gcc/gdb, just -g).

HG_WARN

By default, we use -Wall and add in a few others on specific platforms. When OSTYPE does not include "darwin" or "solaris", -Werror is added. This is appended to common.mk's CFLAGS.

OS

If OS is Windows_NT (as it is in the Cygwin environment), then necessary file suffixes are tweaked for a.out and executable names.

STRIP

Default: STRIP=strip is applied to executables after compilation, stripping their symbol tables and making them nice and small but not debugger-friendly. To bypass strip, set STRIP=true and make will run 'true exec' instead of 'strip exec'.

Installation environment variables

BINDIR

Installation directory for utility programs compiled from C.

CGI_BIN

Installation directory for CGI programs (hgTracks, hgc etc)

DOCUMENTROOT

/usr/local/apache/htdocs by default; if you use a different web server or htdocs root for Genome Browser files, change this. CGI programs expect to read some files from this directory.

SCRIPTS

Installation directory for utility programs written in an interpreted scripting language (bash, tcsh, perl etc.).

cygwin compile issues

MySQL libraries

There seems to be a problem of finding a compatible libmysql system to work on Windows. The source tree would build and link but end up with a SIGSEGV in any call into the MySQL functions.

common.mk

This problem may get fixed in the code, but as of July 2010 the following changes were needed in src/inc/common.mk to get the libraries to compile on cygwin with gcc-4.3.4: remove the -Werror at line 112 defining HG_WARN

HG_WARN = -Wall -Wformat -Wimplicit -Wreturn-type

and to use the openssl libraries, add a line 27 following the -lssl definition to include -lcrypto:

L+=-lssl
L+=-lcrypto

samtools build

Attempting to build the samtools library on cygwin required a couple of lines changed in the Makefile, as found in this note samtools on cygwin

add -D_WIN32 to the DFLAGS definition and add an LFLAGS:
LFLAGS= -lws2_32 -lgcc -lcygwin -lbz2 -lz -lstdc++

and optionally change the linker command for the samtools executable build:

$(CC) $(CFLAGS) -o $@ $(AOBJS) -Xlinker --enable-auto-import -lm libbam.a -lm $(LIBPATH) $(LIBCURSES) -lz -L. -lbam $(LFLAGS)

You do not need this executable to build the kent libraries since they only use libbam.a

Set SAMDIR SAMINC and SAMLIB in your environment. If you build libbam.a in .../samtools-0-1.8/ the settings are:

SAMDIR=/path/to/samtools-0-1.8
SAMINC=${SAMDIR}
SAMLIB=${SAMDIR}/libbam.a