ThreeStateTrackDb

This is a proposal for modifying the trackDb release process developed by Brian, Brooke, and Markd. It is intended as a straight-forward modification to the existing process to address some of the current problems; not as a major,comprehensive change.

This address the following issues:

• Release tags currently do not prevent modifications to tracks already on the RR from leaking to the RR while in QA. This is caused by the release tag not accurately modeling the process. This is addressed by adding a third state: "public".
• Managing releases of updated versions of ENCODE composite tracks is complex, requiring editing a potentially large number of release tags in trackDb.ra stanza files. Also, the release tag doesn't currently work on the composite track stanza.

trackDb release process

The three state trackDb release process is:

• alpha state: make alpha run on hgwdev loads all tracks with release alpha into trackDb regardless of the table existing.
• beta state: make beta run on hgwbeta loads all tracks with release beta into trackDb only if the table exists.
• public state: make public run on hgwbeta loads all tracks with release public into trackDb_public only of the table exists.

Having no release tag is equivalent to having all 3 release tags present.

QA can look at trackDb_public on hgwbeta-public.cse.ucsc.edu as a final check before pushing it to the RR. Then trackDb_public will be pushed from mysqlbeta to mysqlrr, where it will be renamed trackDb.

Tracks that don't already exist on the RR function as they do now:

• Developer creates table(s) and adds a corresponding stanza to trackDb.ra, with no release tags
• QA pushes table to mysqlbeta, does a make beta, and QAs track
• When track is ready, QA does a make public, then pushes table and trackDb_public to mysqlrr (and trackDb_public is renamed trackDb along the way)

If a developer subsequently wants to make changes to the track, it would work like this:

• the old stanza:

track someTrack
shortLabel Mediocre RNAs
visibility hide

• becomes two stanzas:

track someTrack
release alpha
shortLabel Great RNAs
visibility pack

track someTrack
release beta,public
shortLabel Mediocre RNAs
visibility hide

• nothing leaks out to the RR before it is ready. QA looks at the changes on hgwbeta by changing the first stanza to 'release alpha,beta' and the second stanza to release public

• when it is deemed worthy, the trackDb.ra entry can be collapsed back to one stanza, with no release tags (although leaving release alpha,beta,public in there would have the exact same effect):

track someTrack
shortLabel Great RNAs
visibility pack

Managing large composite tracks

To address this issue, each large composite track will be moved to its own file, named in the form: trackDb.compositeName.version.ra, and include all of its contained track stanzas. To minimize the amount of editing required, the include directive will be modified to have a release attribute. Since includes are processed line-per-line, not as part of a stanza, an attribute is an easier approach than adding a release tag.

For example, if a developer created the file encGencodeV1.trackDb.ra, the following line could be added to trackDb.ra (or trackDb.wgEncode.ra, or whatever):

include encGencodeV1.trackDb.ra alpha

When QA gets it, this will become:

include encGencodeV1.trackDb.ra alpha,beta

And when it is released:

include encGencodeV1.trackDb.ra alpha,beta,public

When changes need to be made to an already-released composite track, the composite trackDb file is copied, the version number in the name is incremented, and it is added to cvs. So two copies of the entire file will exist, and trackDb.ra will look like:

 include encGencodeV2.trackDb.ra alpha
 include encGencodeV1.trackDb.ra beta,public

The changes won't leak to the RR before QA approves them and promotes the version 2 file to alpha,beta,public. Once an older version of a composite track is no longer on any systems, the corresponding .ra file can be cvs removed.

Note that the version numbers here are just sequential for a given composite track, and they do not necessarily correspond to the ENCODE "V2" track releases.