Linux: how to print header line of tab-delimeted file in column with numbers

Let's say there is a header of certain atab-delimited file that has a lot of parameters. There are many examples: annotation files, results of certain algorithms, etc...

Sometimes it's interesting to show only select certain parameters. This can be performed easily using cut command. For example, this is how to report only chromosome name and start-end positions of gene exon from GTF file:

cut -f 1,4,5 Homo_sapiens.GRCh38.78.gtf | less

To perform such task using cut command it's important to report exact indexes of required columns. But, what if there are too many columns (more than 30 for example)? How to detect exact indexes?

Here's an example of a command to detect index numbers of a first line of a file:

head -n 1 fusions.detailed.txt | tr "\t" "\n" | cat -n

This example creates a list of all parameters of a detailed report from InFusion tool. Result of this example can be found here (numbers and words in bold font).

Stupid penguin strikes back: Ubuntu does not boot after installing proprietary ATI video driver

Well, in my opinion Linux will never be a user-friendly operating system suitable for home usage. There are a lot issues but there is one particular problem I hate at most: drivers for new hardware. You can only be safe when installing Linux on at least 1 year old computer. But if you have a brand new laptop, get ready.

There will be blood.

Ok, maybe I'm a bit angry, but I just lost 2 hours by fixing some stupid stuff.

Recently I got a new laptop. There was a bunch of problems after installing Ubuntu 12.04 along with Windows 8 (btw, the installation itself was a bit of a quest): overheating, reduced battery time, system suspension failures. What else? Ah, yes, after installing native ATI video drivers the system did not boot. There were no error message, it just hanged.

Here's how I fixed it. Maybe it will be of help to other "hackers" out there.

***

The goal of the rescue operation is to remove the new driver and restore the default video-driver.

First, load in the recovery mode.
Select item "root" console. Btw, you can try loading in fail safe mode, but in my case it did not work.
Remount the file system in read/write mode:

mount -o remount,rw /

Then remove the video-driver:

apt-get purge fglrx*

Remove X11 configuration file:

mv /etc/X11/xorg.conf /etc/X11/xorg.conf.broken

Reboot and pray that it worked ;)

Help was found here and here.

Dismissed!

Output subset of columns from file

Nice tip to remember: Unix command line tool cut allows to output not only selected columns from file, but also a subset of columns.

Example:

kokonech@ultor:~/playgrnd/scythe$ cut -f 1,2,3,4,5,6 scythe_output/fusions.txt
#ref1 break_pos1 strand1 ref2 break_pos2 strand2
chr20 49446917 + chr17 58761341 +
kokonech@ultor:~/playgrnd/scythe$ cut -f 1-6 scythe_output/fusions.txt
#ref1 break_pos1 strand1 ref2 break_pos2 strand2
chr20 49446917 + chr17 58761341 +


Counting size of transcript from GTF file

Here I use grep to get records associated with query transcript and awk to perform the calculations:

cat test.gtf | grep "transcript_id \"ENSMUST00000105216\"" | egrep -v "CDS|start_codon|stop_codon" | awk '{l += $5 -$4 + 1}END{print l}'

Note that egrep is used to filter out all records containing CDS, start_codon or stop_codon. egrep is applied because it supports regular expression "logical or" statement.

Source of power

Came across a cool website with a lot of info on Unix tools such as bash, awk, sed:
http://www.grymoire.com/Unix/

Finding a Debian package with with a missing include file

Have you ever had this problem: you build some application or library from source and have this stupid missing include file, which indicates that you don't have a dependency. Now you wonder which package you should install.

Luckily in Debian one can do the following:

apt-file search readproc.h


Source is here.

See ya folks!

Ssh witout printing password

Always forget this stuff...

Here is the link

Sum up a column in a tab delimited file

Suppose we have a tab-delimited file:

kokonech@ultor:~/playgrnd/densityAnalysis/density_repeats.64.LTRs.bed$ head 24h-i-input.bam.bed
1 83886031 83886750 ERVL-E-int 980 -1 N 187 LTRs 1 0
1 83886031 83886750 ERVL-E-int 980 -1 N 187 LTRs 2 0
1 83886031 83886750 ERVL-E-int 980 -1 N 187 LTRs 3 0
1 83886031 83886750 ERVL-E-int 980 -1 N 187 LTRs 4 0
1 83886031 83886750 ERVL-E-int 980 -1 N 187 LTRs 5 0
1 83886031 83886750 ERVL-E-int 980 -1 N 187 LTRs 6 0
1 83886031 83886750 ERVL-E-int 980 -1 N 187 LTRs 7 0
1 83886031 83886750 ERVL-E-int 980 -1 N 187 LTRs 8 0
1 83886031 83886750 ERVL-E-int 980 -1 N 187 LTRs 9 0
1 83886031 83886750 ERVL-E-int 980 -1 N 187 LTRs 10 0


We want to sum up a column, let's say the 11th.
A piece of cake using Awk:

awk '{a+=$11}END{printf "%i\n",a}' 24h-i-input.bam.bed


Small tricks for tab-delimeted files

Working on my current project includes a lot of manipulation with tab-delimited files (SAM,GFF, etc) Here are some nice tricks to remember.

Output only second field of a delimited file:
cat samout | cut -f 2

The same with awk:
awk '{print $2}'

Not related to tab-delimited files but still very useful :)

Compare two files based on their content:
cat file1 file2 | sort | uniq -u

Linux BASH tricks: recovering items

Very nice commands to know about

Run the last-command name: !!
Recover last entered item: Alt + "."
Search over history: Ctrl + R