CGI Testing

Also see: version notes and CGI_testing_responsibilities.

As of v255 CGIs, we are focusing testing on hgTables and hgTracks. Other CGIs no longer receive systematic testing every CGI release; only bug fixes, new features and changes need to be tested. There should be redmine tickets associated with changes, but also check the version notes for changes related to your CGIs.

Testing on hgwbeta needs to be done in IE on Windows machine.

While testing, open the error console on your web browser and watch for errors. In FireFox, go to Tools -> Error Console. You should explicitly choose the "Errors" tab if you are using FF, because the warnings tab shows almost exclusively false positives (e.g. CSS "Errors" that we and almost all other sites generate)." Errors in IE are flagged by a little error icon in the lower left hand corner of the web browser window (usually a message like "Done, with errors on page"). If you click on that icon, you see the error. This page tells you how to look at the console under IE (useful if there are multiple errors on the page). Larry says that many non-obvious bugs leave behind evidence in the web browser error log, so it can be very useful.

A good track for uncovering bugs is CSHL Long RNA Seq. It's huge!

hgTracks

TrackCheck

The TrackCheck robot checks the first 4 links at the default position for every track in every assembly (except for UCSC Genes). If zoomcount is set to something higher than 1, it also zooms out and checks the first 4 links there, and repeats this until zoomcount is reached. TrackCheck is 1 of 4 test robots that are run automatically on hgwbeta as part of the build process (the hgTracks tester runs it on hgw0).

• Check the TrackCheck output (located in hgwdev:/cluster/bin/build/scripts/logs/) for the current build version for any errors found hgwbeta (it runs automatically on hgwbeta as part of the build process, currently with zoomCount=3).
• If there are errors found on hgwbeta, look at the error logs on hgwbeta for clues about what is causing it, since the error reported by TrackCheck is often just something like "error getting zoom button," with no other information. The error logs are located at hgwbeta:/usr/local/apache/logs/. The file called "error_log" is the latest one, and those with a number appended (error_log.1, etc.) are older. Look for errors that say they are coming from hgwdev's IP address (128.114.50.189) around the time the robots were running.
• Even if there are no errors reported on hgwbeta, it is useful to scan the error logs for unreported errors triggered by TrackCheck. (This whole process is admittedly lame. A better robot would just report the errors directly.)
• On CGI push day, run TrackCheck on hgw0. It is not required that the hgTracks tester watch the error logs on hgw0 while TrackCheck is running, but it can be helpful. To get to the apache logs, do:

$ ssh qateam@hgw0
$ cd logs
$ less -S error_log  (Once you are viewing the log, typing SHIFT-F will allow you to follow incoming errors.  To exit the log, type CTRL-C and then Q)

• To follow the error_log from hgw0 without the CGI_TIME updates use the following instead:

$ tail -f error_log | grep -v CGI_TIME

• Until October 2012, trackCheck was run on hgw0 on release day with zoomCount=1 for all assemblies. Because running trackCheck with zoomCount=1 is not very informative and because running trackCheck with zoomCount=3 for all assemblies takes too long, as of October 2012, trackCheck will be run differently:
  • For most releases, trackCheck will be run on hgw0 on release day with zoomCount=3 for hg19, mm9 and mm10 only.
  • For all releases ending in 0 or 5 (e.g., v280, v285, etc.), trackCheck will be run on hgw0 with zoomCount=3 for all assemblies. Because of the amount of time this will take, the CGIs will need to be pushed to hgw0 on Friday afternoon so trackCheck can run over the weekend. This will need to be coordinated with the push shepherd and the admins.
  • A full trackCheck will still be run on beta as a normal part of the release cycle.

main page testing

• Manually check anything that has been an issue during the past release cycle.
• Set position to a region of about 20bp and turn on all of the tracks in hg18 to dense (using configure page). You may need to do this in several chunks (as of the v244 CGIs, if you get too many tracks on at once, you will see an error that says "Image is over 32,000 pixels high (32013 pix) at the following track which is now hidden:"). Make sure all tracks load okay. -- addendum: chr22:39,957,028-39,957,030 on hg19 seems to load all tracks in one fell swoop.
• Check that drag/reorder of tracks works
• Check that reverse works
• Check that resize works
• Check the various options on the configure page
• Try a few of the right click functions

track search

• Try a search that will bring up the 3 different kinds of tracks, which are are shaded with different colors:
  • tan: super-track (has a folder/wrench icon)
  • light orange: composite (has a folder/wrench icon)
  • light yellow: an actual data track (not a container)

(note that ENCODE Regulation track on hg18 is a special kind of super-track)

• Make sure you can configure tracks via the folder/wrench icons as well as by adjusting the "view" drop-downs (after hitting submit on a configure page, you should be returned to the track search page)
• Try the different sort types ("Alphabetically" sorts on the longLabel)
• Check that the track description pops up when you click a blue track name
• Check that metadata appears when you click the down arrow "(v)" after a track name
  • Check that the links within the metadata work
• Try a couple of advanced searches
  • Make sure searching with "ENCODE terms" is working

Information on how visibilities work can be found here

hgTables

Notes from Jim about what hgTablesTest robot does: the hgTables robot goes through the first two tables in the first four tracks in each group, and brings up the schema and summary page on a 5 megabase window, and then does the various outputs, checking that there are the right number of items. It does not test filters or intersections.

The output of the hgTablesTest robot is in /cluster/bin/build/scripts/logs/.

1. check that all of the ways to limit output work: position, "define regions," "paste identifiers," filter, filter on related tables
2. check all output formats; for "sequence," make sure the different sequence types work
3. check output for track types that are not checked by hgTablesTest, such as: microarray names (available on GNF Atlas 2 track), MAF (available on multiz tracks), . . . please add more!
4. do an intersection with some different options and output formats selected
5. do a subtrack merge (select a table from a composite track to get the option)
6. create a custom track from the TB
7. import a track hub
8. do a correlation
9. try sending output to Galaxy and GREAT
10. make sure summary/stats button works
11. try different combinations of the above

hgc

This CGI displays the detail page for individual items in a track. For wiggles, it displays the details about the range of data displayed by the track in the current browser window.

Testing:

• details are displayed
• links on the details page work
• composite tracks have specific things like metadata link (see no. 2 in hgApi).

hgTrackUi

This CGI displays the track controls (trackDb .ra file settings determine the settings) and description page (determined by the trackDb .html file) of each track. A couple ways to get to hgTrackUi are clicking on the gray mini-button on the left-hand side of the browser image or by clicking on the title of the track's pull-down menu on the hgTracks page.

Here are some tracks that are popular or have complex hgTrackUis:

• ENCODE
  • test the correspondence between the matrix and the list of tracks when you turn on or off different tracks, including the graying out of boxes in the matrix
  • test different views for multiple tracks while testing correspondence between matrix and list of tracks
  • test one track that has 3 dimensional matrix beta RNA-seq, RNA-extract dimension
  • try sorting list of tracks by different columns
  • test links at the top of the page to downloads, subtracks and description
  • test that the buttons allowing users to go to super-tracks from child-tracks at the top of the page work
  • test hgApi as it pertains to hgTrackUi (see no. 1 in hgApi)
  • test filterBy settings (use Gencode)
• Conservation (a "full" track like the one found on hg19)
  • test all aspects and buttons of the multiz track, including codon frames and the mini-wiggles visible when both the subtrack and parent track are on "full"
  • use this as an opportunity to test all aspects of a wiggle track
  • try filtering by score
• SNPs
  • test that adding different gene prediction tracks do change the details page for individual SNPs
  • test one of the Filtering Options (e.g. Minimum Average Heterozygosity, Maximum Weight, etc.)
  • test the filtering by attribute
  • test color specification, including changing the default color for a specification
• mRNAs
  • test filtering by one and more than one search term, as well as different filtering options
  • test that the coloring by codons/alignment function as expected

Other non-standard trackUis include Chain/Nets, TransMap, Human Proteins, Restriction Enzymes. A fun one to test is HapMap LD Phased on hg18 since it has a radically different hgTrackUi.

Information on how visibilities on composite tracks work can be found here

top

hgGene

Pick a UCSC Gene / Known Gene on one of the organisms with a knownGene track (Human, Mouse, Rat, D. melanogaster, C. elegans, or S. cerevisiae). For example h19.SIRT-beta hg19.SIRT1-hgw0 Then check:

• Under "Sequence and Links to Tools and Databases"
  • the internal links (in green boxes)
  • the external links (in blue boxes). Make an effort to see that these link to gene-specific pages. There should be a "User Annotations" link (to the wiki track) should be available for the later human and mouse assemblies.
• Check a representative link from each section.
• Make sure you can change the expression ratio to blue/yellow.
• In the "Orthologous Genes in Other Species" section, make sure that if there is a link in any particular column, all entries of that column have links (except for Zebrafish -- it doesn't have an hgGene page to link back to).
• Make sure the documentation links at the bottom of the page work.
• Check LS-SNP and Chimera links Example.SOD1-hgw0
• Check that can collapse and expand sections
• Check in Gene Alleles, all buttons, table sort, variant link
• Check BACE1 Beta-uc010rxh.2 hgw0-uc010rxh.2 to ensure entire page loads. (Unique variant issue, RM 11894)

hgNear

Notes from Jim about what the hgNear robot does: The hgNear robot goes turns on each column and makes sure that it doesn't crash, and has the appropriate number of items. It tests lightly the gene filters, just working on the filters on the gene column itself. It also tests each sort, making sure it doesn't crash and still you get the right number of items.

1. To test on this click on "Gene Sorter" from the blue navigation bar.
2. Type a gene into the search text box
3. Pick a gene that has a long list of results.
4. Test sort by.
5. Test configure (make sure new columns show up and test the order feature)
   • Expression colors (change them to see if they are displaying)
6. Click the column headers to make sure that the descriptions show up
7. Click on a couple of entires to make sure they display in the browser.
8. Click on "sequence" from the sorter
9. Click on "filter" and try filtering from a specific column. Don't for get to put a "*" next to the search term or it most likely won't return any results.

hgCustom

1. Test all three methods of entering a Custom Track (CT): pasted/typed in text, uploaded by file, URL (see Custom Track Examples or Brian's links). dev beta hgw0
2. Test the detail pages for custom tracks uploaded by clicking on the browser image for tracks. (Ex. bedDetail displays info -Brian's Rachel set displays BC###)
3. Test editing, deleting, adding, updating, HTML docs, etc.
4. Test adding multiple tracks at once (multiple tracks in one file, multiple URLs, and pasting in multiple tracks)
5. Test custom track output in relation to the Table Browser.

• See also examples page here: Custom Track Examples Hiram's page: http://genome-test.cse.ucsc.edu/~hiram/ctDb/hg19.chr21.ct.txt, with BrianL's additions: http://hgwdev.cse.ucsc.edu/~brianlee/customTracks/examples.txt

hgLogin

Note that genome.ucsc.edu is hard coded in to several hgLogin links, so you cannot test many components on hgw0 only, you need to wait till it hits all RR machines. Testing on beta works as expected.

You access the hgLogin CGI from the hgSession CGI (http://hgwbeta.soe.ucsc.edu/cgi-bin/hgSession)

1. Create a new account (click "Create Account" from hgSession):
   1. make sure it catches you if:
      • email addresses don't match
      • passwords entered do not match
   2. make sure you get confirmation email
   3. click confirmation email link and log into account
   4. sign out and then back in to accout
2. Change account password (it should not log you out during this process):
   • Log out and back in with new password
3. Test retrieving a "lost" user name (must start signed out):
   1. At login screen click "Can't Access Your Account?" and follow prompts:
      • Testing an unregistered email address: Enter an email address that you know does NOT have an account, but you have access to and can check (e.g. a person email acct)
        • You should not get an email to an address that does not have an account (see redmine)
      • Testing a registered email address: Enter the email address you used to make the new account:
        • You should get an email with the correct username(s) associated with the account)
          • Note that more than one user name can be associated with an account, and if so, the email should list all associated user names
4. Test retrieving a "lost" password:
   1. At login screen click "Can't Access Your Account?" and follow prompts
      • You should receive an email with a new password
      • Log in with the new (temporary) password
        • You will now be asked to change the temp password provided in the email
        • After changing password, you should STILL be logged in (this is currently not the case and is a bug)

hgVisiGene

1. test the search box by entering a gene name
2. check the zoom buttons
3. check that the "Gene" link opens the correct gene details page, and that that "visiGene" link in that gene details page retrieves the correct images in visiGene
4. pick an image and check all of the links for that page
5. for images composed of several smaller images, check that the pane descriptions are displaying correctly (not all panes will have an associated pane description)

(Note: the images from Mahoney are a subset of the MGI/Jax images. The Mahoney images should list two sources and should show two sets of acknowledgements.)

hgPal

There are 3 ways to get to the hgPal CGI:

1. check a "Protein FASTA" click-through from a UCSC Gene details page
2. check a "CDS FASTA" click-through from a RefSeq Genes details page
3. using the Table Browser, choose "CDS FASTA" as the output format (this should work for any genePred track)

Check that different settings give the expected results.

hgBlat

1. perform both a nucleotide and a protein search with default settings
2. make sure colors listed in description section are right
3. zoom in on an alignment and test "View details of parts of alignment within browser window"
4. try different sorts and output types
5. make sure all the buttons work, including uploading a file
6. try uploading a file with too many bases or too many queries and verify error message

top

hgPcr

1. test some perfectly matching primers, including a pair on the negative strand
2. find a pair of primers that shouldn't match UCSC Genes and test that they don't
3. vary settings and input primers for several assemblies and see if results make sense
4. for human and mouse browsers, test the "UCSC Genes" target. Check that UI functions as expected
5. check that the UI functions as expected for regular primers as well

hgLiftOver, hgConvert

1. Choose an assembly and go to hgTracks
2. Under the "View" drop down menu, click the "In Other Genomes (Convert)" link
3. Compare the output to the same conversion using liftOver:
4. LiftOver is at Home -> Utilities -> Batch Coordinate Conversion (liftOver)
5. Test both position and BED format
6. Test a variety of settings
7. Try converting multiple positions at once
8. Try uploading a file

pbGateway, pbGlobal, pbTracks

1. general testing; click around
2. enter a protein symbol
3. review results page
4. click into pbGlobal and check display

hgSession

1. check that a new session can be saved
2. check that old sessions are still there
3. delete a session
4. check browser & email button; click the title of session & make sure you can save changes to the description
5. try loading a session via a file and via a URL
6. logout and try to load a session that can be shared and one that can't be shared

hgGenome

1. find it by going Home -> Genome Graphs
2. upload a dataset from a file
3. upload a data set from a URL
4. import data from a track
5. change some configurations
6. check that "browse regions" & "sort genes" work
7. check the correlate button

Some good genome graph data for testing: http://hgwdev.cse.ucsc.edu/~rhead/genomeGraphsWithColumns

cartDump, cartReset

1. check cart
2. reset cart
3. check cart again

See also: cart test protocol

hgApi

This CGI is responsible for the metadata "down arrow" links (formerly "..." links) on the hgTrackUi page and the "metadata" links on the hgc details pages of composite tracks with metadata.

1. click on a down arrow link and check that it opens up - no need to check the content. You only need to test one down arrow link - if hgApi is broken, it will break all down arrow links.
2. This CGI can also be tested by clicking on a "metadata" link in hgc details and check that it opens up - no need to check the content.

Probably doing 1 or 2 is sufficient, but it is a good habit to have the testers of hgTrackUi and hgc check 1 and 2 respectively, that way we are double covered in case one person forgets.

hgEncodeApi

This CGI is currently used internally by these pages:

• Experiment Matrix on human & mouse
• ChIP-seq Experiement Matrix on human & mouse
• Experiment Summary on human & mouse

These pages are part of the ENCODE portal. This CGI obtains the experiments from the hgFixed.encodeExp table on the respective machine (gets pushed every Friday from hgwdev -> mysqlbeta -> mysqlrr) and obtains the dataTypes, cellTypes, and antibodies from the controlled vocabulary. Testing:

1. Make sure the matrix loads on the Experiment Matrix pages: human (beta | hgw0) & mouse (beta | hgw0).
2. Make sure the matrix loads on the ChIP-seq Experiment Matrix pages: human (beta | hgw0) & mouse (beta | hgw0).
3. Make sure the summary tables load on the Experiment Summary pages: human (beta | hgw0) & mouse (beta | hgw0).
4. Make sure that clicking on rows in the Experiment Summary and the boxes in the Experiment Matrix and ChIP-seq Experiment Matrix pages take you to results in track/file search (depending on radio button selection).
5. hgEncodeVocab description should be pulled up when clicking on the cell type matrix headings (y-axis). For the Assays, they aren't clickable but there should be a tool-tip with the full title of the assay when you hover over.
6. Leaving and returning from the Experiment Matrix pages (human and mouse) it should remember if the "Overview" was closed or open.

hgEncodeVocab

1. Go to an ENCODE track, most matrix headers should be links. Click on them, should take you to a page with a row of info about the term.
2. Go to a couple of these pages and check that the sections are displaying (these pages use hgEncodeVocab to display the info on them):
   • Registered Variables: beta, hgw0
   • Antibodies: beta, hgw0
   • Human Cell Types: beta, hgw0
   • Mouse Cell Types: beta, hgw0
3. Check a few links in the metadata from hgc details (click "metadata" link to view other links) or hgTrackUi (click arrow in subTrack list below matrix to open metadata for each subTrack with vocab terms)

hgFileUi

1. Check that the Downloads links (below matrix) on hgTrackUi page of a track that is known to have fileSortOrder goes to hgFileUi, e.g. Broad Histone hg19 (beta | hgw0)
2. Manipulate a URL to directly access hgFileUi (beta | hgw0)
3. Make sure download button prompts a download (and doesn't take you to an error page)
4. Check the File Filter multi-select drop-downs, look for (interdependence) greyed out options
5. File accounting
6. Check the sorting of columns
7. Check the "Track Settings" link takes you back to the track's hgTrackUi page
8. Check the "File Search" link takes you to File Search

top

More info about this CGI:

Only tracks with a fileSortOrder line in their trackDb blocks will have links to the hgFileUi page, however, all composites with downloads and metadata should have a functioning hgFileUi page. If a track has a fileSortOrder line, then the "Downloads" links in the top-most light blue bar and under the subtrack list will link to the appropriate hgFileUi page. If not, then the same Download links will go to the index.html on hgdownload (or hgdownload-test), but you can still get to the hgFileUi page for a track of this kind by manipulating the URL: <machine>.ucsc.edu/cgi-bin/hgFileUi?g=<trackname> (for example: genome.ucsc.edu/cgi-bin/hgFileUi?g=wgEncodeCaltechRnaSeq). The first track with the fileSortOrder and therefore links that go to hgFileUi is ENCODE Broad Histone on hg19

hgFileSearch

(aka File Search)

About:

• Each ENCODE assembly has File Search, e.g. hg19 File Search.
• From File Search, users can search all the ENCODE download files in an assembly (across all composite tracks).
• Navigate to this CGI via the Search links on the ENCODE portal, then click one of the File Search buttons.

Testing:

1. Try searching on the different fields and drop-downs and different combinations
2. Try sorting the columns
3. Check the folder icon links back to hgFileUi for that track
4. Check the clear and cancel buttons work
5. Try clicking "Download" button to see if you can download a file
6. Encode terms: plus and minus buttons, drop down check-box lists, term specific help links

hgHubConnect

Started with v254 CGIs, which coincides with Jim's suggestion to not systematically test all CGIs.

• Load a hub with a URL (http://genome.ucsc.edu/goldenPath/help/examples/hubDirectory/hub.txt ) dev beta hgw0

( Or use public DNA Methylation hub: http://smithlab.usc.edu/trackdata/methylation/hub.txt for multiple assemblies,mm9,hg18,hg19+) dev beta hgw0

• Check to be sure it displays properly for assembly and click items and make changes to track settings.
• Go the table browser, test getting some output from the hub.
• From the table browser, try changing the genome away from the hub's assembly too, and test getting some output.
• After getting output from a hub in the Table Browser,then change species, and get output from something else, then a hub output from another species (say mm9 from the above methylation hub, or clade:DNA methylation, checkout arabidopsis).

Selenium_HubTest1 script available for automated FireFox Selenium PlugIn testing, defunct with v301 release.

hgVai

hgVai will generate a set of data to work with if the Artificial Variants dataset is chosen, but it will also work on a pgSNP or VCF custom track, a VCF track that is part of a data hub, or on a list of rs IDs. This data hub has a VCF file in it: http://hgwdev.cse.ucsc.edu/~rhead/hubs/hub.txt.

Once some sample data is loaded via a custom track and/or a track hub (it's good to test both), navigate to the VAI from the Tools menu or by clicking the "go to variant annotation integrator" button in hgCustom. Then:

• Try selecting different gene sets for annotations. You should get different effect predictions in places where the gene tracks differ. Note that the effect predictions are not optional; they will appear in the output no matter what.

• Try adding optional annotations: dbNSFP scores, the rsID from dbSNP, conserved elements, conservation scores. Note that the optional annotations (at least for now) depend only on the positions of the variants in the user data; they are not affected by the gene set selected.

• Try filtering the results in different ways.

• Make sure the different output formats work.

• Make sure the "maximum number of variants to be processed" drop-down (near the top of the page) is working.

• Make sure you can change "region to annotate" to a position and output is limited to that postion.

• Since this page uses javascript, it's important to test any changes on Windows/IE.

hgUserSuggestion

This is new. For URLs supplied below, don't forget to adjust the server and the "txtInput" field (the simple captcha code) as required for the page you're submitting from.

• Make a legitimate suggestion and see if it actually goes through.
• Try mismatched email addresses
• Try adding multiple email addresses
• Try a URL like the following http://hgwdev.cse.ucsc.edu/cgi-bin/hgUserSuggestion?suggestName=a&suggestEmail=jcasper@soe.ucsc.edu&suggestCfmEmail=jcasper@soe.ucsc.edu&suggestCategory=CRAZY&suggestSummary=test&suggestDetails=testing&txtInput=73147&do.suggestSendMail=1 to see if you can submit to a random category.
• Try a URL like http://hgwdev.cse.ucsc.edu/cgi-bin/hgUserSuggestion?suggestName=a&suggestEmail=jcasper@soe.ucsc.edu&suggestCfmEmail=jcasper@soe.ucsc.edu&suggestCategory=Tracks&suggestSummary=test&suggestDetails=testing&txtInput=73147&suggestWebsite=http://www.junk.com&do.suggestSendMail=1 to make sure the hidden field filter works.

genome-euro

As of v286, genome-euro is now active. Once the CGI testing process is complete and the CGIs get pushed out to the RR, genome-euro should receive some basic testing to ensure that nothing obvious is broken.

You can shell into genome-euro by using the command: ssh qateam@genome-euro

First, go to /usr/local/apache/cgi-bin to make sure the new CGIs have made it to genome-euro (ls -l to check dates). Minimally, the following should be performed:

• Monitor the error log on genome-euro for ~20 minutes (or for the duration of the testing, whichever is longer) to make sure no unexpected errors appear
  • The error log on genome-euro is located at: /var/log/httpd
  • Use the command: tail -f /usr/local/apache/logs/error_log | grep -v CGI_TIME
  • To stop monitoring, hit CTRL-C
  • If you want to also open a second log to watch all use you can use less and follow the log the with SHIFT-F
  • less -S /usr/local/apache/logs/error_log ...then type SHIFT-F
• View the default tracks in hgTracks for hg19, mm9 and mm10
• Click into a UCSC Genes details page
• Click into a RefSeq Genes details page
• Load an hg19 custom track
• Perform an hg19 blat search
• Perform in-silico PCR on hg19
• Perform a liftOver from hg19 to mm9
• Perform a conversion from hg19 to mm9

top