Monitoring Tasks: Difference between revisions
From Genecats
Jump to navigationJump to search
(check for companies) |
(added new column so we can switch folks around) |
||
| Line 4: | Line 4: | ||
<th>task</th> | <th>task</th> | ||
<th>frequency</th> | <th>frequency</th> | ||
<th>who</th> | <th>who (now)</th> | ||
<th>who (July '13)</th> | |||
</tr> | </tr> | ||
<tr> | <tr> | ||
<td colspan=" | <td colspan="4" style="background-color:gray"><b>from qateam cron</b></td> | ||
</tr> | </tr> | ||
| Line 14: | Line 15: | ||
<td>ask Galt/Jim about blat updates</td> | <td>ask Galt/Jim about blat updates</td> | ||
<td>3x per year</td> | <td>3x per year</td> | ||
<td></td> | |||
<td>Pauline, Steve, Jonathan</td> | <td>Pauline, Steve, Jonathan</td> | ||
</tr> | </tr> | ||
| Line 20: | Line 22: | ||
<td>Respond to items on the broken links report: Check to make sure that broken links reported are actually broken. If so, fix the link if it is clear what the new link should be and push to the RR. If not, contact the appropriate person at the web site in question to find out what the new link should be.</td> | <td>Respond to items on the broken links report: Check to make sure that broken links reported are actually broken. If so, fix the link if it is clear what the new link should be and push to the RR. If not, contact the appropriate person at the web site in question to find out what the new link should be.</td> | ||
<td>daily</td> | <td>daily</td> | ||
<td></td> | |||
<td>Steve</td> | <td>Steve</td> | ||
</tr> | </tr> | ||
| Line 26: | Line 29: | ||
<td>Respond to the trackDbGlobal report: Make public and request a push of trackDb and friends for any assemblies that show up as changed in the trackDbGlobal report.</td> | <td>Respond to the trackDbGlobal report: Make public and request a push of trackDb and friends for any assemblies that show up as changed in the trackDbGlobal report.</td> | ||
<td>weekly</td> | <td>weekly</td> | ||
<td></td> | |||
<td>Steve</td> | <td>Steve</td> | ||
</tr> | </tr> | ||
| Line 32: | Line 36: | ||
<td>receive output from hgTracksRandom cron; alert admins to problems</td> | <td>receive output from hgTracksRandom cron; alert admins to problems</td> | ||
<td>as needed</td> | <td>as needed</td> | ||
<td></td> | |||
<td>many: kuhn, rhead, pauline, katrina, brianlee, braney, luvina, gary, ann, steve, jcasper</td> | <td>many: kuhn, rhead, pauline, katrina, brianlee, braney, luvina, gary, ann, steve, jcasper</td> | ||
</tr> | </tr> | ||
| Line 38: | Line 43: | ||
<td>receive output from gbLoaded cron job and alert braney if anything seems very outdated</td> | <td>receive output from gbLoaded cron job and alert braney if anything seems very outdated</td> | ||
<td>daily, problems every 3-6 months</td> | <td>daily, problems every 3-6 months</td> | ||
<td></td> | |||
<td>Brooke, Steve, Jonathan</td> | <td>Brooke, Steve, Jonathan</td> | ||
</tr> | </tr> | ||
| Line 44: | Line 50: | ||
<td>receive output from showProcesslist.csh</td> | <td>receive output from showProcesslist.csh</td> | ||
<td>daily, no action</td> | <td>daily, no action</td> | ||
<td></td> | |||
<td>Brooke, Steve, Jonathan</td> | <td>Brooke, Steve, Jonathan</td> | ||
</tr> | </tr> | ||
| Line 50: | Line 57: | ||
<td>get output from checkTableStatus.csh on Mondays (this is a report on whether the genbank table dumps are up-to-date...if we could get rid of our dependence on those, we could get rid of this cron job)</td> | <td>get output from checkTableStatus.csh on Mondays (this is a report on whether the genbank table dumps are up-to-date...if we could get rid of our dependence on those, we could get rid of this cron job)</td> | ||
<td>weekly</td> | <td>weekly</td> | ||
<td></td> | |||
<td>Brooke, Steve, Jonathan</td> | <td>Brooke, Steve, Jonathan</td> | ||
</tr> | </tr> | ||
| Line 56: | Line 64: | ||
<td>get an email from backupCentral.csh each month (there's a redmine ticket on whether we should change this one: http://redmine.soe.ucsc.edu/issues/7834)</td> | <td>get an email from backupCentral.csh each month (there's a redmine ticket on whether we should change this one: http://redmine.soe.ucsc.edu/issues/7834)</td> | ||
<td>monthly</td> | <td>monthly</td> | ||
<td></td> | |||
<td>Brooke, Steve, Jonathan</td> | <td>Brooke, Steve, Jonathan</td> | ||
</tr> | </tr> | ||
| Line 62: | Line 71: | ||
<td>receive an email on additions and subtractions to files in /gbdb on hgnfs1</td> | <td>receive an email on additions and subtractions to files in /gbdb on hgnfs1</td> | ||
<td>daily</td> | <td>daily</td> | ||
<td></td> | |||
<td>Brooke, Steve, Jonathan</td> | <td>Brooke, Steve, Jonathan</td> | ||
</tr> | </tr> | ||
| Line 68: | Line 78: | ||
<td>receive an email from cron if qateam DailySandboxUpdate fails</td> | <td>receive an email from cron if qateam DailySandboxUpdate fails</td> | ||
<td>daily</td> | <td>daily</td> | ||
<td></td> | |||
<td>Brooke, Steve, Jonathan</td> | <td>Brooke, Steve, Jonathan</td> | ||
</tr> | </tr> | ||
| Line 74: | Line 85: | ||
<td>receive output from checkMetaAday.csh; alert (rhead, steve, jcasper) if anything is out of sync (checks that tables are in sync between beta and RR for database of the day)</td> | <td>receive output from checkMetaAday.csh; alert (rhead, steve, jcasper) if anything is out of sync (checks that tables are in sync between beta and RR for database of the day)</td> | ||
<td>daily</td> | <td>daily</td> | ||
<td></td> | |||
<td>Luvina</td> | <td>Luvina</td> | ||
</tr> | </tr> | ||
| Line 80: | Line 92: | ||
<td>receive output from syncAday.csh; alert (rhead, steve, jcasper) if anything is out of sync (compares hgcentralbeta to hgcentral for db of the day)</td> | <td>receive output from syncAday.csh; alert (rhead, steve, jcasper) if anything is out of sync (compares hgcentralbeta to hgcentral for db of the day)</td> | ||
<td>daily</td> | <td>daily</td> | ||
<td></td> | |||
<td>Luvina</td> | <td>Luvina</td> | ||
</tr> | </tr> | ||
| Line 86: | Line 99: | ||
<td>get output from checkGenomeMysql</td> | <td>get output from checkGenomeMysql</td> | ||
<td>daily</td> | <td>daily</td> | ||
<td></td> | |||
<td>Brooke, Steve, Jonathan</td> | <td>Brooke, Steve, Jonathan</td> | ||
</tr> | </tr> | ||
<tr> | <tr> | ||
<td colspan=" | <td colspan="4" style="background-color:gray"><b>from build cron</b></td> | ||
</tr> | </tr> | ||
| Line 96: | Line 110: | ||
<td>receive cron output for tableDescriptions build errors</td> | <td>receive cron output for tableDescriptions build errors</td> | ||
<td>rare</td> | <td>rare</td> | ||
<td></td> | |||
<td>Brooke, Pauline</td> | <td>Brooke, Pauline</td> | ||
</tr> | </tr> | ||
<tr> | <tr> | ||
<td colspan=" | <td colspan="4" style="background-color:gray"><b>from galt's cron</b></td> | ||
</tr> | </tr> | ||
| Line 106: | Line 121: | ||
<td>run ReleaseLog script (this requires a copy of the kent repository)</td> | <td>run ReleaseLog script (this requires a copy of the kent repository)</td> | ||
<td>daily</td> | <td>daily</td> | ||
<td>Galt</td> | |||
<td>Galt</td> | <td>Galt</td> | ||
</tr> | </tr> | ||
| Line 112: | Line 128: | ||
<td>back up the qapushq</td> | <td>back up the qapushq</td> | ||
<td>every 30 minutes during the work day/week</td> | <td>every 30 minutes during the work day/week</td> | ||
<td>Galt</td> | |||
<td>Galt</td> | <td>Galt</td> | ||
</tr> | </tr> | ||
<tr> | <tr> | ||
<td colspan=" | <td colspan="4" style="background-color:gray"><b>from braney's cron</b></td> | ||
</tr> | </tr> | ||
| Line 122: | Line 139: | ||
<td>braney has a cron that runs /cluster/bin/x86_64/hubPublicCheck and complains when the hubPublic table doesn't have the same strings as the actual hub.txt file. This should be claimed by a QAer and transferred to the qateam cron.</td> | <td>braney has a cron that runs /cluster/bin/x86_64/hubPublicCheck and complains when the hubPublic table doesn't have the same strings as the actual hub.txt file. This should be claimed by a QAer and transferred to the qateam cron.</td> | ||
<td>runs daily, complains rarely?</td> | <td>runs daily, complains rarely?</td> | ||
<td></td> | |||
<td>braney</td> | <td>braney</td> | ||
</tr> | </tr> | ||
<tr> | <tr> | ||
<td colspan=" | <td colspan="4" style="background-color:gray"><b>from ann's cron</b></td> | ||
</tr> | </tr> | ||
| Line 132: | Line 150: | ||
<td>Ann has a cron that sends a reminder to check the list of folks who have mirror sites. This list is created by a script of Hiram's and located here: http://genome-test.cse.ucsc.edu/admin/tools/useCount.html. The information comes from the phoneHome program in the CGIs. Sometimes companies set up mirrors, which is not OK.</td> | <td>Ann has a cron that sends a reminder to check the list of folks who have mirror sites. This list is created by a script of Hiram's and located here: http://genome-test.cse.ucsc.edu/admin/tools/useCount.html. The information comes from the phoneHome program in the CGIs. Sometimes companies set up mirrors, which is not OK.</td> | ||
<td>monthly</td> | <td>monthly</td> | ||
<td> | <td></td> | ||
<td>Ann</td> | |||
</tr> | </tr> | ||
<tr> | <tr> | ||
<td colspan=" | <td colspan="4" style="background-color:gray"><b>non-cron jobs</b></td> | ||
</tr> | </tr> | ||
| Line 142: | Line 161: | ||
<td>Check old forum (http://redmine.soe.ucsc.edu/forum/index.php) for MLQs that may have slipped through the cracks: Check to make sure that the last response for every ticket came from one of us. If not, look it up in redmine to ensure that A) there is indeed a redmine ticket for it, and B) it has been responded to. If not, notify the appropriate MLM with as much information about the MLQ as possible.</td> | <td>Check old forum (http://redmine.soe.ucsc.edu/forum/index.php) for MLQs that may have slipped through the cracks: Check to make sure that the last response for every ticket came from one of us. If not, look it up in redmine to ensure that A) there is indeed a redmine ticket for it, and B) it has been responded to. If not, notify the appropriate MLM with as much information about the MLQ as possible.</td> | ||
<td>weekly</td> | <td>weekly</td> | ||
<td></td> | |||
<td>Steve</td> | <td>Steve</td> | ||
</tr> | </tr> | ||
| Line 148: | Line 168: | ||
<td>update sla.html whenever we have RR downtime</td> | <td>update sla.html whenever we have RR downtime</td> | ||
<td>hopefully, not much!</td> | <td>hopefully, not much!</td> | ||
<td>Pauline, Jonathan, Steve</td> | |||
<td>Brooke, Pauline</td> | <td>Brooke, Pauline</td> | ||
</tr> | </tr> | ||
| Line 154: | Line 175: | ||
<td>update the two crontabs in genecats/qa/crontabs/*.crontab (and log in as qateam and restart the qateam crontab with updates)</td> | <td>update the two crontabs in genecats/qa/crontabs/*.crontab (and log in as qateam and restart the qateam crontab with updates)</td> | ||
<td>as needed</td> | <td>as needed</td> | ||
<td>Pauline, Jonathan</td> | |||
<td>Brooke, Pauline</td> | <td>Brooke, Pauline</td> | ||
</tr> | </tr> | ||
<tr> | <tr> | ||
<td colspan=" | <td colspan="4" style="background-color:gray"><b>ENCODE</b></td> | ||
</tr> | </tr> | ||
| Line 164: | Line 186: | ||
<td>Check genome-preview with Selenium test, ensures various big ENCODE tracks are accessible, and that machine is still running</td> | <td>Check genome-preview with Selenium test, ensures various big ENCODE tracks are accessible, and that machine is still running</td> | ||
<td>weekly</td> | <td>weekly</td> | ||
<td>Brian</td> | |||
<td>Brian</td> | <td>Brian</td> | ||
</tr> | </tr> | ||
| Line 170: | Line 193: | ||
<td>Receive validation checks on ENCODE cv.ra metadata, but won't change unless cv.ra changes</td> | <td>Receive validation checks on ENCODE cv.ra metadata, but won't change unless cv.ra changes</td> | ||
<td>rare/never</td> | <td>rare/never</td> | ||
<td>Brian</td> | |||
<td>Brian</td> | <td>Brian</td> | ||
</tr> | </tr> | ||
| Line 176: | Line 200: | ||
<td>Update broken links from cron on hgEncodeVocab/cv.ra checkLinks</td> | <td>Update broken links from cron on hgEncodeVocab/cv.ra checkLinks</td> | ||
<td>rare, fixes needed about once a month</td> | <td>rare, fixes needed about once a month</td> | ||
<td>Brian</td> | |||
<td>Brian</td> | <td>Brian</td> | ||
</tr> | </tr> | ||
Revision as of 15:15, 26 July 2013
| task | frequency | who (now) | who (July '13) |
|---|---|---|---|
| from qateam cron | |||
| ask Galt/Jim about blat updates | 3x per year | Pauline, Steve, Jonathan | |
| Respond to items on the broken links report: Check to make sure that broken links reported are actually broken. If so, fix the link if it is clear what the new link should be and push to the RR. If not, contact the appropriate person at the web site in question to find out what the new link should be. | daily | Steve | |
| Respond to the trackDbGlobal report: Make public and request a push of trackDb and friends for any assemblies that show up as changed in the trackDbGlobal report. | weekly | Steve | |
| receive output from hgTracksRandom cron; alert admins to problems | as needed | many: kuhn, rhead, pauline, katrina, brianlee, braney, luvina, gary, ann, steve, jcasper | |
| receive output from gbLoaded cron job and alert braney if anything seems very outdated | daily, problems every 3-6 months | Brooke, Steve, Jonathan | |
| receive output from showProcesslist.csh | daily, no action | Brooke, Steve, Jonathan | |
| get output from checkTableStatus.csh on Mondays (this is a report on whether the genbank table dumps are up-to-date...if we could get rid of our dependence on those, we could get rid of this cron job) | weekly | Brooke, Steve, Jonathan | |
| get an email from backupCentral.csh each month (there's a redmine ticket on whether we should change this one: http://redmine.soe.ucsc.edu/issues/7834) | monthly | Brooke, Steve, Jonathan | |
| receive an email on additions and subtractions to files in /gbdb on hgnfs1 | daily | Brooke, Steve, Jonathan | |
| receive an email from cron if qateam DailySandboxUpdate fails | daily | Brooke, Steve, Jonathan | |
| receive output from checkMetaAday.csh; alert (rhead, steve, jcasper) if anything is out of sync (checks that tables are in sync between beta and RR for database of the day) | daily | Luvina | |
| receive output from syncAday.csh; alert (rhead, steve, jcasper) if anything is out of sync (compares hgcentralbeta to hgcentral for db of the day) | daily | Luvina | |
| get output from checkGenomeMysql | daily | Brooke, Steve, Jonathan | |
| from build cron | |||
| receive cron output for tableDescriptions build errors | rare | Brooke, Pauline | |
| from galt's cron | |||
| run ReleaseLog script (this requires a copy of the kent repository) | daily | Galt | Galt |
| back up the qapushq | every 30 minutes during the work day/week | Galt | Galt |
| from braney's cron | |||
| braney has a cron that runs /cluster/bin/x86_64/hubPublicCheck and complains when the hubPublic table doesn't have the same strings as the actual hub.txt file. This should be claimed by a QAer and transferred to the qateam cron. | runs daily, complains rarely? | braney | |
| from ann's cron | |||
| Ann has a cron that sends a reminder to check the list of folks who have mirror sites. This list is created by a script of Hiram's and located here: http://genome-test.cse.ucsc.edu/admin/tools/useCount.html. The information comes from the phoneHome program in the CGIs. Sometimes companies set up mirrors, which is not OK. | monthly | Ann | |
| non-cron jobs | |||
| Check old forum (http://redmine.soe.ucsc.edu/forum/index.php) for MLQs that may have slipped through the cracks: Check to make sure that the last response for every ticket came from one of us. If not, look it up in redmine to ensure that A) there is indeed a redmine ticket for it, and B) it has been responded to. If not, notify the appropriate MLM with as much information about the MLQ as possible. | weekly | Steve | |
| update sla.html whenever we have RR downtime | hopefully, not much! | Pauline, Jonathan, Steve | Brooke, Pauline |
| update the two crontabs in genecats/qa/crontabs/*.crontab (and log in as qateam and restart the qateam crontab with updates) | as needed | Pauline, Jonathan | Brooke, Pauline |
| ENCODE | |||
| Check genome-preview with Selenium test, ensures various big ENCODE tracks are accessible, and that machine is still running | weekly | Brian | Brian |
| Receive validation checks on ENCODE cv.ra metadata, but won't change unless cv.ra changes | rare/never | Brian | Brian |
| Update broken links from cron on hgEncodeVocab/cv.ra checkLinks | rare, fixes needed about once a month | Brian | Brian |
