Curated Hubs

QA on hgwdev

From a user's perspective, the curated hubs should behave like all other assemblies. The data files for curated hubs live in /gbdb like all other assemblies:

• /gbdb/$db/

You make any HTML edits in /trackDb and continue with a make to update hgwdev and hgwbeta. Performing a make will populate one of the following directories:

• /gbdb/$db/hubs/alpha
• /gbdb/$db/hubs/beta
• /gbdb/$db/hubs/public

These directories have the HTML files for the track description pages, hub.txt, and trackDb.txt files.

To keep the file list organized, you should create a list of the general /gbdb files and another file list for the alpha/beta/public directories.

Script to Update Existing Curated Hubs

There is now a script that will run the 'make' for the curated assembly in trackDb, then print the list of /gbdb files that need to be pushed to have the track appear on hgwbeta/RR/euro/asia.

$ makeHub.sh db [alpha/beta/public]

The script will run a cd ~/kent/src/hg/makeDb/trackDb and run the make specified. The script will then print a list of /gbdb files that must be pushed to update the curated hub.

Example output:

$ makeHub.sh hs1 public

Updating trackDb and friends for hs1

find . -type l -exec rm {} \;
git pull
Already up-to-date.
make public_all
make[1]: Entering directory `/cluster/home/jairo/kent/src/hg/makeDb/trackDb'
ssh  qateam@hgw0.soe.ucsc.edu find /gbdb/hs1  -type f -size +1c  > hs1.gbdbList.txt
HGDB_CONF=/cluster/home/jairo/.hg.conf.beta ./loadTracks -addVersion  -strict -gbdbList=hs1.gbdbList.txt -release=public trackDb_public hgFindSpec_public hs1
hgTrackDb -release=public -addVersion -strict -gbdbList=hs1.gbdbList.txt human hs1 trackDb_public ../../lib/trackDb.sql .
Loaded 615 track descriptions total
Loaded database hs1
tdbQuery -check -release=public -strict 'select count(*) from hs1' -root=/cluster/home/jairo/kent/src/hg/makeDb/trackDb
0
hgFindSpec -release=public -strict human hs1 hgFindSpec_public ../../lib/hgFindSpec.sql .
Loaded 31 search specs total
Loaded database hs1
rm -f hs1.gbdbList.txt
HGDB_CONF=/cluster/home/jairo/.hg.conf.beta ./checkMetaDb public metaDb_public hs1
HGDB_CONF=/cluster/home/jairo/.hg.conf.beta HGDB_TRACKDB=trackDb_public ./buildTrix trackDb_public metaDb_public cv/public/cv.ra qateam@hgwbeta.soe.ucsc.edu /data/trix hs1
# now build the relatedTracks table for each db
HGDB_CONF=/cluster/home/jairo/.hg.conf.beta ./buildRelatedTracks relatedTrack_public hs1
ssh  qateam@hgwbeta.soe.ucsc.edu rm -rf /dev/shm/trackDbCache/hs1.trackDb_public
HGDB_CONF=/cluster/home/jairo/.hg.conf.beta HGDB_TRACKDB=trackDb_public ./maybeBuildHub  public trackDb_public hs1
make[1]: Leaving directory `/cluster/home/jairo/kent/src/hg/makeDb/trackDb'
/hive/users/jairo/tracks/hs1/crispr

--------------------------------------------------------------------

/gbdb/hs1/hubs/public/T2T_Encode.html
/gbdb/hs1/hubs/public/assembly.html
/gbdb/hs1/hubs/public/cactus.html
/gbdb/hs1/hubs/public/catLiftOffGenesV1.html
/gbdb/hs1/hubs/public/censat.html
/gbdb/hs1/hubs/public/clinVar20220313.html
/gbdb/hs1/hubs/public/cpgIslandExt.html
/gbdb/hs1/hubs/public/cpgIslands.html
/gbdb/hs1/hubs/public/crisprHs1.html
/gbdb/hs1/hubs/public/dbSNP155.html
/gbdb/hs1/hubs/public/gc5Base.html
/gbdb/hs1/hubs/public/gwasSNPs2022-03-08.html
/gbdb/hs1/hubs/public/hgLiftOver.html
/gbdb/hs1/hubs/public/hgUnique.html
/gbdb/hs1/hubs/public/hub.txt
/gbdb/hs1/hubs/public/hubAndGenome.txt
/gbdb/hs1/hubs/public/mappability.html
/gbdb/hs1/hubs/public/microsatellites.html
/gbdb/hs1/hubs/public/proseq.html
/gbdb/hs1/hubs/public/rdnaModel.html
/gbdb/hs1/hubs/public/refSeqComposite.html
/gbdb/hs1/hubs/public/rnaseq.html
/gbdb/hs1/hubs/public/sedefSegDups.html
/gbdb/hs1/hubs/public/sgdpCopyNumber.html
/gbdb/hs1/hubs/public/sgdpCopyNumber_subset.html
/gbdb/hs1/hubs/public/simpleRepeat.html
/gbdb/hs1/hubs/public/t2tRepeatMasker.html
/gbdb/hs1/hubs/public/trackDb.txt
/gbdb/hs1/hubs/public/windowMasker.html
/gbdb/hs1/hubs/public/xenoRefGene.html

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
When releasing the assembly to the RR, you will FIRST have to push
the /gbdb files above from hgwdev --> hgwbeta --> RR/euro/asia.

Note: Please push /gbdb data files (i.e. bigBeds) BEFORE running this script.

Staging on hgwbeta

When staging the assembly to hgwbeta, you will FIRST have to push:

• /gbdb files
• Create an empty database for the assembly

Then do a make beta DBS=$db in trackDb to update the /gbdb/$db/hubs/beta directory. Using the gbdbPush script, push the entire contents of that directory to hgwbeta.

Staging on hgwbeta-public

Similar to the staging process on hgwbeta, the /gbdb files will have to the RR since /gbdb is shared between hgwbeta-public and the RR.

• Push /gbdb files to the RR
• Do a make public DBS=$db
• Use the gbdbPush script to push the /gbdb/$db/hubs/public directory from hgwdev to hgwbeta

Releasing to the RR

Once everything looks good on hgwbeta-public:

• Push the empty database to the RR (tdbPush script works just fine)
• Ask cluster-admin to push the /gbdb/$db/hubs/public from hgwbeta to the RR.

After the assembly is on the RR, finish the QA like any other assembly (edit newsarch.html, credits.html, FAQreleases.html, etc.)