Cell Browser data flow and architecture: Difference between revisions
| Line 19: | Line 19: | ||
2. The output files from <code>cbBuild</code> are placed inside <code>/usr/local/apache/htdocs-cells</code>. Note that the original configuration files and expression matrices inside the dataset directory are converted into either JSON or binary files (BIN). These files are used by the Cell Browser to ....... (help) build a cell browser. | 2. The output files from <code>cbBuild</code> are placed inside <code>/usr/local/apache/htdocs-cells</code>. Note that the original configuration files and expression matrices inside the dataset directory are converted into either JSON or binary files (BIN). These files are used by the Cell Browser to ....... (help) build a cell browser. | ||
3. Once the dataset is on cells-test, the next destination is [https://cells-beta.gi.ucsc.edu/ cells-beta]. You will push the directory and files from <code>htdocs-cells</code> onto <code>/usr/local/apache/htdocs-cells-beta</code> using the command: | |||
<pre> | |||
cbPush dir-name | |||
</pre> | |||
Note that cbPush requires you to input a directory name. You can paste in multiple names at the end. | |||
A good alias to have in your .bashrc that pushes the current directory you are in onto beta: | |||
<pre> | |||
alias cbPushDir='cbPush "${PWD##*/}"' | |||
</pre> | |||
4. Once your dataset is on beta, you are almost there! Once the dataset is checked over for potential bugs, you will use the command: | |||
<pre> | |||
sudo cellsPush | |||
</pre> | |||
You will be prompted to type in a password, use your <code>hgwdev</code> password. Once you do that, the datasets will be built onto the hgw0, hgw1, and hgw2 machines! Voila! | |||
== '''System Architecture Map''' == | |||
[[File:cb_sysarchmap.png|1045px]] | [[File:cb_sysarchmap.png|1045px]] | ||
Revision as of 23:18, 25 August 2022
How does data flow between the different machines?
How does building a cell browser work?
What files are copied over? Which ones are transformed into another format?
1. Data is first deposited in a dataset directory inside /hive/data/inside/cells on hgwdev and then gets built onto cells-test using the command:
# For datasets with no additional subsets cbBuild -o alpha # For dataset collections you will use the recursive option "-r" cbBuild -r -o alpha
2. The output files from cbBuild are placed inside /usr/local/apache/htdocs-cells. Note that the original configuration files and expression matrices inside the dataset directory are converted into either JSON or binary files (BIN). These files are used by the Cell Browser to ....... (help) build a cell browser.
3. Once the dataset is on cells-test, the next destination is cells-beta. You will push the directory and files from htdocs-cells onto /usr/local/apache/htdocs-cells-beta using the command:
cbPush dir-name
Note that cbPush requires you to input a directory name. You can paste in multiple names at the end.
A good alias to have in your .bashrc that pushes the current directory you are in onto beta:
alias cbPushDir='cbPush "${PWD##*/}"'
4. Once your dataset is on beta, you are almost there! Once the dataset is checked over for potential bugs, you will use the command:
sudo cellsPush
You will be prompted to type in a password, use your hgwdev password. Once you do that, the datasets will be built onto the hgw0, hgw1, and hgw2 machines! Voila!
System Architecture Map
