Appendix A — Linux Cheatsheet for Viromics

A scannable quick reference for the commands used most often across this book. Copy, adapt, and keep it open in a second terminal. Commands are grouped by task so you can jump straight to what you need.

The one habit that saves the most time

Preview before you overwrite or delete. Run a ls, head, or a --dry-run first, and only then commit to the real command. Most lost-data disasters come from a hasty rm or a redirect (>) onto the wrong file.

A.1 Navigation

pwd                          # where am I
ls -lh                       # list with human-readable sizes
ls -lhtr                     # sort by time, newest last (good for logs)
cd ~/viromics_course         # jump to the project root
cd -                         # jump back to the previous directory
tree -L 2                    # show the folder tree, two levels deep
du -sh *                     # size of everything in the current folder

A.2 Files and folders

mkdir -p raw_reads trimmed_reads reports   # -p makes parents, no error if present
cp -r results results_backup               # -r copies folders
mv old_name new_name                       # rename or move
rm file                                    # delete a file (no undo)
rm -r folder                               # delete a folder and its contents
ln -s /big/disk/db databases               # symlink a large DB into the project
find . -name "*.fastq.gz" -size +1G        # find large FASTQ files

A.3 Viewing and inspecting

head -n 20 file.tsv          # first 20 lines
tail -n 20 file.tsv          # last 20 lines
tail -f logs/run.log         # follow a log live (Ctrl-C to stop)
less -S table.tsv            # page a wide table without wrapping (q to quit)
wc -l file.tsv               # count lines
zcat reads.fastq.gz | head   # peek inside a gzip file without unzipping
column -t -s $'\t' file.tsv  # pretty-print a TSV

A.4 Text processing (grep, awk, sed, cut, sort, uniq)

grep -c "^>" contigs.fa                     # count sequences in a FASTA
grep -v "^#" table.tsv                      # drop comment/header lines
grep -i "virsorter" logs/*.log              # case-insensitive search across logs

cut -f1,3 table.tsv                         # keep columns 1 and 3
cut -f1 table.tsv | tail -n +2              # column 1, minus the header

sort -k2,2 -t$'\t' table.tsv                # sort by column 2 (tab-separated)
sort -k5,5nr table.tsv                      # sort by column 5, numeric, descending
sort file.txt | uniq -c | sort -nr          # count and rank unique values

awk -F'\t' 'NR>1 && $4=="High-quality"' checkv.tsv   # filter rows by a column value
awk -F'\t' '$2>=10000' lengths.tsv                   # keep contigs >= 10 kb
awk -F'\t' '{sum+=$2} END{print sum}' lengths.tsv    # sum a column

sed 's/old/new/g' file.txt                  # replace text (prints to screen)
sed -i 's/old/new/g' file.txt               # replace in place (edits the file)
sed 's/ .*//' contigs.fa                    # keep only the first word of FASTA headers
tr ',' '\t' < file.csv > file.tsv           # convert commas to tabs
paste a.txt b.txt                           # merge two files column-wise

A.5 FASTA and FASTQ with seqkit

seqkit stats *.fastq.gz                                 # N, length stats, N50
seqkit seq -m 1000 contigs.fa > contigs_min1kb.fa       # keep sequences >= 1 kb
seqkit grep -f ids.txt contigs.fa > selected.fa         # extract by ID list
seqkit grep -v -f drop.txt contigs.fa > kept.fa         # exclude an ID list
seqkit fx2tab -n -l contigs.fa > lengths.tsv            # name + length table
seqkit rmdup -s contigs.fa > dedup.fa                   # remove exact duplicates
seqkit subseq -r 1:5000 genome.fa                       # slice a region
seqkit sort -l -r contigs.fa > by_length.fa             # sort by length, longest first
seqkit replace -p ".*" -r "ctg_{nr}" contigs.fa         # renumber headers cleanly

A.6 Conda and mamba

conda env list                                          # list environments
mamba create -n viromics-core -c conda-forge -c bioconda fastqc fastp multiqc
conda activate viromics-core
mamba install -c conda-forge -c bioconda seqkit         # add a tool to the active env
conda deactivate
conda env export --no-builds > env.yml                  # record an environment
mamba env create -f env.yml                             # rebuild it elsewhere
conda config --set channel_priority strict              # avoids many solver conflicts

Use mamba, and one job per environment

mamba resolves dependencies far faster than the classic conda solver. When two heavy tools conflict, do not fight the solver — give each its own environment (virsorter2, genomad, checkv, …) and switch between them.

A.7 Bash scripting

cat > run.sh << 'EOF'
#!/usr/bin/env bash
set -euo pipefail                 # stop on errors, unset vars, and failed pipes
THREADS=12
for r1 in trimmed_reads/*_R1.fastq.gz; do
  sample=$(basename "$r1" _R1.fastq.gz)
  echo ">> processing $sample"
  # your command here, using "$sample" and "$THREADS"
done
EOF

chmod +x run.sh
bash run.sh 2>&1 | tee logs/run.log     # run and save all output to a log

Handy building blocks:

VAR="value"; echo "$VAR"                # set and use a variable
export VIROME_DB=~/databases            # export so child processes see it
sample=$(basename "$path" .fastq.gz)    # strip a directory and suffix
for f in *.fa; do echo "$f"; done       # loop over files
cmd1 && cmd2                            # run cmd2 only if cmd1 succeeds
cmd1 || echo "cmd1 failed"              # fallback on failure

A.8 Long-running jobs with tmux and screen

Assemblies, database downloads, and prediction runs can take hours. Detach them so a dropped SSH connection does not kill the job.

tmux new -s viromics        # start a named session
#   ... launch your long command inside it ...
# Detach: press Ctrl-b then d       (the job keeps running)
tmux ls                     # list sessions
tmux attach -t viromics     # reattach later

# screen equivalents:
screen -S viromics          # start
# Detach: Ctrl-a then d
screen -ls                  # list
screen -r viromics          # reattach

# Or run detached and log to a file:
nohup bash run_phase4_main_pipeline.sh > logs/phase4.log 2>&1 &

A.9 Monitoring resources

htop                        # live CPU/RAM per process (q to quit; F9 to kill)
top                         # always-available fallback for htop
df -h                       # free disk space per filesystem
du -sh ~/viromics_course/*  # which project folders are eating disk
du -sh databases/*          # database sizes
free -h                     # free and used RAM
nproc                       # how many CPU threads are available
ls -lh out/*.bam            # check output file sizes as a job progresses

Watch disk during assembly

Assembly and mapping create large temporary files. Run df -h before you start a big job, and again if it fails with a write error — a full disk is the most common silent cause of a crashed assembly.

# Linux Cheatsheet for Viromics A scannable quick reference for the commands used most often across this book. Copy, adapt, and keep it open in a second terminal. Commands are grouped by task so you can jump straight to what you need. ::: {.callout-tip} ## The one habit that saves the most time Preview before you overwrite or delete. Run a `ls`, `head`, or a `--dry-run` first, and only then commit to the real command. Most lost-data disasters come from a hasty `rm` or a redirect (`>`) onto the wrong file. ::: ## Navigation ```bash pwd # where am I ls -lh # list with human-readable sizes ls -lhtr # sort by time, newest last (good for logs) cd ~/viromics_course # jump to the project root cd - # jump back to the previous directory tree -L 2 # show the folder tree, two levels deep du -sh * # size of everything in the current folder ``` ## Files and folders ```bash mkdir -p raw_reads trimmed_reads reports # -p makes parents, no error if present cp -r results results_backup # -r copies folders mv old_name new_name # rename or move rm file # delete a file (no undo) rm -r folder # delete a folder and its contents ln -s /big/disk/db databases # symlink a large DB into the project find . -name "*.fastq.gz" -size +1G # find large FASTQ files ``` ## Viewing and inspecting ```bash head -n 20 file.tsv # first 20 lines tail -n 20 file.tsv # last 20 lines tail -f logs/run.log # follow a log live (Ctrl-C to stop) less -S table.tsv # page a wide table without wrapping (q to quit) wc -l file.tsv # count lines zcat reads.fastq.gz | head # peek inside a gzip file without unzipping column -t -s $'\t' file.tsv # pretty-print a TSV ``` ## Text processing (grep, awk, sed, cut, sort, uniq) ```bash grep -c "^>" contigs.fa # count sequences in a FASTA grep -v "^#" table.tsv # drop comment/header lines grep -i "virsorter" logs/*.log # case-insensitive search across logs cut -f1,3 table.tsv # keep columns 1 and 3 cut -f1 table.tsv | tail -n +2 # column 1, minus the header sort -k2,2 -t$'\t' table.tsv # sort by column 2 (tab-separated) sort -k5,5nr table.tsv # sort by column 5, numeric, descending sort file.txt | uniq -c | sort -nr # count and rank unique values awk -F'\t' 'NR>1 && $4=="High-quality"' checkv.tsv # filter rows by a column value awk -F'\t' '$2>=10000' lengths.tsv # keep contigs >= 10 kb awk -F'\t' '{sum+=$2} END{print sum}' lengths.tsv # sum a column sed 's/old/new/g' file.txt # replace text (prints to screen) sed -i 's/old/new/g' file.txt # replace in place (edits the file) sed 's/ .*//' contigs.fa # keep only the first word of FASTA headers tr ',' '\t' < file.csv > file.tsv # convert commas to tabs paste a.txt b.txt # merge two files column-wise ``` ## FASTA and FASTQ with seqkit ```bash seqkit stats *.fastq.gz # N, length stats, N50 seqkit seq -m 1000 contigs.fa > contigs_min1kb.fa # keep sequences >= 1 kb seqkit grep -f ids.txt contigs.fa > selected.fa # extract by ID list seqkit grep -v -f drop.txt contigs.fa > kept.fa # exclude an ID list seqkit fx2tab -n -l contigs.fa > lengths.tsv # name + length table seqkit rmdup -s contigs.fa > dedup.fa # remove exact duplicates seqkit subseq -r 1:5000 genome.fa # slice a region seqkit sort -l -r contigs.fa > by_length.fa # sort by length, longest first seqkit replace -p ".*" -r "ctg_{nr}" contigs.fa # renumber headers cleanly ``` ## Conda and mamba ```bash conda env list # list environments mamba create -n viromics-core -c conda-forge -c bioconda fastqc fastp multiqc conda activate viromics-core mamba install -c conda-forge -c bioconda seqkit # add a tool to the active env conda deactivate conda env export --no-builds > env.yml # record an environment mamba env create -f env.yml # rebuild it elsewhere conda config --set channel_priority strict # avoids many solver conflicts ``` ::: {.callout-tip} ## Use `mamba`, and one job per environment `mamba` resolves dependencies far faster than the classic conda solver. When two heavy tools conflict, do not fight the solver — give each its own environment (`virsorter2`, `genomad`, `checkv`, …) and switch between them. ::: ## Bash scripting ```bash cat > run.sh << 'EOF' #!/usr/bin/env bash set -euo pipefail # stop on errors, unset vars, and failed pipes THREADS=12 for r1 in trimmed_reads/*_R1.fastq.gz; do sample=$(basename "$r1" _R1.fastq.gz) echo ">> processing $sample" # your command here, using "$sample" and "$THREADS" done EOF chmod +x run.sh bash run.sh 2>&1 | tee logs/run.log # run and save all output to a log ``` Handy building blocks: ```bash VAR="value"; echo "$VAR" # set and use a variable export VIROME_DB=~/databases # export so child processes see it sample=$(basename "$path" .fastq.gz) # strip a directory and suffix for f in *.fa; do echo "$f"; done # loop over files cmd1 && cmd2 # run cmd2 only if cmd1 succeeds cmd1 || echo "cmd1 failed" # fallback on failure ``` ## Long-running jobs with tmux and screen Assemblies, database downloads, and prediction runs can take hours. Detach them so a dropped SSH connection does not kill the job. ```bash tmux new -s viromics # start a named session # ... launch your long command inside it ... # Detach: press Ctrl-b then d (the job keeps running) tmux ls # list sessions tmux attach -t viromics # reattach later # screen equivalents: screen -S viromics # start # Detach: Ctrl-a then d screen -ls # list screen -r viromics # reattach # Or run detached and log to a file: nohup bash run_phase4_main_pipeline.sh > logs/phase4.log 2>&1 & ``` ## Monitoring resources ```bash htop # live CPU/RAM per process (q to quit; F9 to kill) top # always-available fallback for htop df -h # free disk space per filesystem du -sh ~/viromics_course/* # which project folders are eating disk du -sh databases/* # database sizes free -h # free and used RAM nproc # how many CPU threads are available ls -lh out/*.bam # check output file sizes as a job progresses ``` ::: {.callout-warning} ## Watch disk during assembly Assembly and mapping create large temporary files. Run `df -h` before you start a big job, and again if it fails with a write error — a full disk is the most common silent cause of a crashed assembly. :::