materials/week2.html

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  <meta http-equiv="Content-Style-Type" content="text/css" />
  <meta name="generator" content="pandoc" />
  <title></title>
  <style type="text/css">code{white-space: pre;}</style>
  <link rel="stylesheet" href="pandoc.css" type="text/css" />
</head>
<body>
<h1 id="week-2-advanced-hpc-usage">Week 2: Advanced (?) HPC usage</h1>
<p>Some background reading:</p>
<ul>
<li><a href="https://hpc.oit.uci.edu/">HPC at UCI</a></li>
<li><a href="https://hpc.oit.uci.edu/HPC_Overview.html">Hardware overview</a></li>
<li><a href="https://hpc.oit.uci.edu/~krthornt/BioClusterGE.pdf">An old tutorial that I wrote</a></li>
<li><a href="https://github.com/ThorntonLab/biocluster">Newer tutorial that I wrote</a></li>
</ul>
<p><a href="HPC.html">My notes for week2 lecture</a></p>
<h1 id="lab-exercise.">Lab exercise.</h1>
<p>We are going to concoct a “bioinformatics workflow” that has the following features:</p>
<ol style="list-style-type: decimal">
<li>It uses dependent jobs.</li>
<li>Each step checks that the output from the previous step exists.</li>
<li>Ideally, we do everything using only bash commands. (You could do a lot of it with custom Python scripts, but that adds a lot of extra work.)</li>
</ol>
<p>The workflow will do the following:</p>
<ol style="list-style-type: decimal">
<li>Job1: create a list of all files in your user’s home directory.</li>
<li>Job2: get the line count for each file in the list created in Job1.</li>
<li>Job3: Make a list of every file with fewer than 100 lines, using the output from Job2.</li>
<li>Job4: (or 3b.) Make a list of every file smaller than 5Kb in size.</li>
</ol>
<p>The goal here is that you get a robust set of scripts that check for errors, etc. Although this example is really trivial, you will also learn some new shell commands that are really useful.</p>
</body>
</html>