Showing posts with label benchmark. Show all posts
Showing posts with label benchmark. Show all posts

Thursday, 27 September 2012

using gzipped geno files for snptest

seems like using gzipped input files doesn't affect the analysis timings of snptest
will do this with binary geno (bgen) files and update

$ grep 'User' 6template-snptest.sh.log.gz.1 6template-snptest.sh.log
6template-snptest.sh.log.gz.1: User time (seconds): 669.87
6template-snptest.sh.log: User time (seconds): 662.10

Wednesday, 1 August 2012

SSD / HDD benchmarking on Linux

Found this gem of a wiki on archlinux the distro for speed.
https://wiki.archlinux.org/index.php/SSD_Benchmarking

There are several HDD benchmarking utils/ways
some like dd (see exerpt below) are avail on all linux systems by default and good for quick and dirty benchmarking
like I recently found that running on a HDD RAID that was twice as fast might have saved me half the time on the samtools mpileup command!


Using dd

Note: This method requires the command to be executed from a mounted partition on the device of interest!
First, enter a directory on the SSD with at least 1.1 GB of free space (and one that obviously gives your user wrx permissions) and write a test file to measure write speeds and to give the device something to read:
$ cd /path/to/SSD
$ dd if=/dev/zero of=tempfile bs=1M count=1024 conv=fdatasync,notrunc
1024+0 records in
1024+0 records out
w bytes (x GB) copied, y s, z MB/s
Next, clear the buffer-cache to accurately measure read speeds directly from the device:
# echo 3 > /proc/sys/vm/drop_caches
$ dd if=tempfile of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
w bytes (x GB) copied, y s, z MB/s
Now that the last file is in the buffer, repeat the command to see the speed of the buffer-cache:
$ dd if=tempfile of=/dev/null bs=1M count=1024
1024+0 records in
1024+0 records out
w bytes (x GB) copied, y s, z GB/s

Friday, 9 April 2010

bowtie build time statistics / benchmark

Recently Built colorspace index for hg19 (from ftp://ftp.sanger.ac.uk/pub/1000genomes/tk2/main_project_reference/)

AMD Phenom II X4 955 chip
8 GB RAM

SATA2 HDD

Wrote 822714402 bytes to primary EBWT file: hg19.rev.1.ebwt
Wrote 358098108 bytes to secondary EBWT file: hg19.rev.2.ebwt
Re-opening _in1 and _in2 as input streams
Returning from Ebwt constructor
Headers:
    len: 2864784823
    bwtLen: 2864784824
    sz: 716196206
    bwtSz: 716196206
    lineRate: 6
    linesPerSide: 1
    offRate: 5
    offMask: 0xffffffe0
    isaRate: -1
    isaMask: 0xffffffff
    ftabChars: 10
    eftabLen: 20
    eftabSz: 80
    ftabLen: 1048577
    ftabSz: 4194308
    offsLen: 89524526
    offsSz: 358098104
    isaLen: 0
    isaSz: 0
    lineSz: 64
    sideSz: 64
    sideBwtSz: 56
    sideBwtLen: 224
    numSidePairs: 6394609
    numSides: 12789218
    numLines: 12789218
    ebwtTotLen: 818509952
    ebwtTotSz: 818509952
Total time for backward call to driver() for mirror index: 01:24:19

Gosh and I thought it will take hours!

Datanami, Woe be me