Browser Mirrors

This page contains information for users interested in mirroring the UCSC Genome Browser on their own servers. See also http://genome.ucsc.edu/mirror.html

Partial Mirrors

A complete mirror of all assemblies requires a large amount of disk space (currently on the order of a terabyte). However, it is not too difficult to set things up so that only a portion of assemblies are mirrored. The following scripts and auxiliary files are used for this purpose at Cornell (http://genome-mirror.bscb.cornell.edu).

• Media:doDownloads.sh - Script for downloading selected files from hgdownload.cse.ucsc.edu
• Media:doUpdateDb.sh - Script for updating local mysql database with downloaded files.
• Media:databases - File identifying which databases to mirror.
• Media:gbdb.exclude - File identifying directories to be excluded when rsyncing /gbdb.

These programs are run nightly via cron, using the following crontab entry:

0 0 * * * /usr/data/mirror-download/doAll.sh

where doAll.sh is a simple wrapper for doDownloads.sh and doUpdateDb.sh, viz.

#!/bin/bash -e

# do downloads and updates # for use with cron

echo "#####################################" /usr/data/mirror-download/doDownloads.sh

echo "#####################################" /usr/data/mirror-download/doUpdateDb.sh

echo "#####################################" echo "Successfully updated mirror."