blast ftp

NCBI BLAST FTP Site

Tao Tao, Ph.D.
User Service
NCBI, NLM, NIH

TOC

1. Introduction content
2. File list and content content
3. Techinical Support

NCBI BLAST ftp site (ftp.ncbi.nlm.nih.gov) provides software packages for standalone blast, client server blast, and wwwblast packages for various platforms. It also provides commonly used blast databases in preformatted as well as FASTA format. Documents on the blast executables and other related subjects are also available from this site.

This file describes the the subdirectories and files found on this ftp site. It also prvoides the basic information on file content and on how the files should be used.

2. File list and content

This section list and describes the files found on the BLAST FTP site. File content for each directory/subdirectory is described in a separate table.

2.1 ftp.ncbi.nlm.nih.gov/blast/ general directory content

The blast ftp directory contains several subdirectories each for a specific set of files.

Table 2.1 ftp.ncbi.nlm.nih.gov/blast/ general directory content
File/Dir Name	Content
blastftp.html	README on FTP site (this file)
db	databases in preformatted or FASTA form
demo	demonstration programs and documents from blast developers
documents	documents for programs in standalone blast, netblast, and wwwblast programs
executables	archives for binary distribution of blast programs
matrices	protein and nucleotide score matrices, only a subset are supported by blast
temp	temporary directory for miscellaneous files

2.2 /blast/db/ directory content

Databases larger than two gigabytes (2 GB) are formatted in multiple volumes, which are named using the "database.##.tar.gz" convention. All relevant volumes are required. An alias file is provided so that the database can be called using the alias name without the extension (.nal or .pal). For example, to call est database, simply use "–d est" option in the commandline (without the quotes).

Certain databases are subsets of a larger parental database. For those databases, mask files, rather than actual databases, are provided. The mask file needs the parent database to function properly. The parent databases should be generated on the same day as the mask file. For example, to use swissprot preformatted database, swissprot.tar.gz, one will need to get the nr.tar.gz with the same date stamp.

To use the preformatted blast database file, first inflate the file using gzip (unix, linux), WinZip (window), or StuffIt Expander (Mac), then extract the component files out from the resulting tar file using tar (unix, linux), WinZip (Window), or StuffIt Expander (Mac). The resulting files are ready for BLAST.

Table 2.2 /blast/db/ directory content
File/Dir Name	Content
FASTA	subdirectory with databases in FASTA format
blastdb.html	content list of the blast databases
est.*.tar.gz	volumes of the est database, all are needed to reconstitute complete est database
est_human.tar.gz	human est database, a mask file requires all volumes of est to work
est_mouse.tar.gz	mouse est database, a mask file requires all volumes of est to work
est_others.tar.gz	non-human, non-mouse est database, a mask file reqires both volumes of est
gss.*.tar.gz	genomic survery sequence database, all are needed to reconstitute complete gss database
htgs.*.tar.gz	volumes of the htgs database, all are needed to reconstitute complete htgs database
human_genomic.tar.gz	human chromosome database containing concatenated contigs with adjusted gaps represented by N's
nr.*.tar.gz	volumes of non-redundant protein database, all are needed to reconstitute complete nr
nt.*.tar.gz	volumes of the nucleotide "nr" database, all are needed to reconstitute complete nt database
other_genomic.*.tar.gz	volumes of chromosome database for organisms other than human, all are needed to reconstitute complete other_genomic database
pataa.tar.gz	patent protein database
patnt.tar.gz	patent nucleotide database
pdbaa.tar.gz	protein sequence database from pdb entries, a mask file requires all nr.*.tar.gz to function
pdbnt.tar.gz	nucleotide sequence database from pdb entries. They are not the coding sequences for the corresponding protein structure entries!
sts.tar.gz	sequence tag site database
swissprot.tar.gz	swissprot sequence database, last major release. It requires all nr.*.tar.gz to work properly
taxdb.tar.gz	taxonomy id database for use with the above file to retrieve taxonomic information for specific entries
wgs.*.tar.gz	volumes of wgs assembly database, all volumes are needed to reconsititute wgs database

2.2.1 /blast/db/FASTA/ subdirectory content

The FASTA database files are now stored in this subdirectory, it does contain some additional databases that are not available via the NCBI BLAST pages. Due to file size issues, the full est database is not provided. One needs to get the three subsets and concatenate them together to get the complete est database.

These databases will need to be formatted using formatdb program found in the standalone blast executable package. The recommended commandlines to use are:

formatdb –i input_db –p F –o T for nucleotide
formatdb –i input_db –p T –o T for protein

For additional information on formatdb, please see the formatdb.html at:

/blast/documents/formatdb.html

2.2.1 blast/db/FASTA/ subdirectory content
File/Dir Name	Content
alu.a.gz	proteins translated from alu.n
alu.n.gz	alu repeat sequences
drosoph.aa.gz	Drosophila protein from genome annotation
drosoph.nt.gz	Drosophila genome
ecoli.aa.gz	E.coli K-12 proteins from genome annotation
ecoli.nt.gz	E.coli K-12 genomic contigs
est_human.gz	human subset of the est database
est_mouse.gz	mouse subset of the est database
est_others.gz	subset of est other than human or mouse entries
gss.gz	Genomic Survey Sequences (mostly BAC ends)
htgs.gz	High Throughput Genomic Sequences
human_genomic.gz	Human chromosomes formed by concatenating genomic contig assemblies (NT_######) and adjusting the gaps with N’s
igSeqNt.gz	Immunoglobulin nucleotide sequences
igSeqProt.gz	Immunoglobulin protein sequences
mito.aa.gz	protein from the annotated mitochondrial genomes
mito.nt.gz	mitochondrial genomes
month.aa.gz	protein sequences released or updated in the past 30 days
month.est_human.gz	human subset of EST released/updated in the past 30 days
month.est_mouse.gz	mosue subset of EST released/updated in the past 30 days
month.est_others.gz	non-human non-mouse EST, released or updated in the past 30 days
month.gss.gz	gss entries released/updated in the past 30 days
month.htgs.gz	htgs entries released/updated in the past 30 days
month.nt.gz	subset of nt released/updated in the past 30 days
nr.gz	non-redundant protein sequence database
nt.gz	nucleotide database from GenBank excluding the batch division htgs, est, gss,sts, pat divisions, and wgs entries. Not non-redundant
other_genomic.gz	Chromosome entries other than human
pataa.gz	Patent protein sequence database
patnt.gz	Patent nucleotide sequence database
pdbaa.gz	protein sequences for pdb entries
pdbnt.gz	nucleotide entries for pdb entries. They are NOT the coding sequence forthe corresponding protein entries
sts.gz	Sequence Tag Sites database
swissprot.gz	swissprot database, last major release
vector.gz	vector sequences from synthetic (syn) division of GenBank
wgs.gz	Whole Genome Shotgun sequence assembly
yeast.aa.gz	protein translations from yeast genome annotation
yeast.nt.gz	yeast genomic sequence

2.3 File content for /blast/demo/ directory

This directory contains some technical presentations from the BLAST developers along with some demo tools or documentation relevant to BLAST.

Table 2.3 File content for /blast/demo/ directory
File/Dir Name	Content
README.blast_demo	readme for blast_demo package
README.first	readme for this directory
README.parse_blast_xml	readme for parse_blast_xml package
benchmark	Package with sample database and query for gauging the performance of BLAST on different platforms
blast_demo.tar.gz	blast_demo package on blast db, blast object, and reformating blast alignment from blastobj file
blast_programming.ppt	PowerPoint presentation on BLAST programing
ieee_blast.final.ppt	PowerPoint presentation (IEEE conference)
ieee_talk.pdf	PDF file for presentation (IEEE conference)
ieee_talk.pdf	Above IEEE presentation in PDF format
old	Containing an older version of blast_demo.tar.gz
parse_blast_xml.tar.gz	demo package on parsing xml styled blast output
splitd.ppt	PowerPoint presentation on NCBI BLAST server’s splitd implementation
test_suite.tar.gz	test package

2.4 File content for /blast/documents/ directory

This directory contains copies of the documentation on different BLAST programs distributed from this ftp site under the /blast/executables/ directory. blast.txt also contains detailed release history.

2.4 File content for /blast/documents/ directory
File/Dir Name	Content
bl2seq.html	readme for bl2seq (standalone "Align Two Sequences")
blast-sc2004.pdf	POSTER describing NCBI splitd system
blast.html	General readme for blast setup
blastall.html	readme for blastall and blastpgp
blastclust.html	readme for blastclust
blastdb.html	readme for blast databases
blastftp.html	readme for blast ftp site (this document)
blastpgp.html	readme for blastpgp, standalone PSI-BLAST
developer	subdirectory with additional documentation: blast_seqalign.txt: describing seqalign function readdb.txt: describing readdb function scoring.pdf: describing scoring systems used by BLAST urlapi.txt: a short introduction on BLAST URL API
fastacmd.html	readme for fastacmd, a sequence retrieval tool
filter.html	readme on filter strings and their functions
formatdb.html	readme for formatdb, which converts fasta sequences into blast database
formatrpsdb.html	readme for formatrpsdb, which converts scoremat into rpsblast database
history.html	history of changes/bug fixes introduced to blast package
impala.html	readme for impala
index.html	list of this group of files/subdirs
megablast.html	readme for megablast
netblast.html	readme for netblast (blastcl3)
rpsblast.html	readme for rpsblast
seedtop.html	readme for seedtop, a standalone pattern match tool
web_blast.pl	Example perl script for doing URLAPI blast
xml	subdirectory with .dtd and .mod field description files for blast xml output NCBI_BlastOutput.dtd: dtd file for blast xml output NCBI_BlastOutput.mod: mod file for blast xml output NCBI_Entity.mod: mod file for NCBI xml file README.blxml: readme on blast xml output

2.5 File content for /blast/executables/ directory

This directory contains several subdirectories each for a specific subsets of executable BLAST programs.

2.5.1 File content for /blast/executables/LATEST/ subdirectory

This directory contains the latest official release of precompiled BLAST executable programs.

The binaries can be divided into three groups. blast initialed files are equivalent. They contain the standalone command line blast binary programs for different platforms. Users need this package to set up BLAST locally. It also provides the tools necessary to prepare custom databases and retrieve sequences from these prepared databases.

The netblast initialed archives the blastcl3 program which functions by formulating BLAST search locally first before forwarding the search to NCBI blast server for process. The search results returned by NCBI BLAST server is saved to an user-specified file on local computer disk. Users do not need to maintain local databases, nor can they search custom databases locally.

The wwwblast initialed archive contains the web pages with embedded blast search forms similar to that of NCBI. They can process the BLAST request submitted through web and search against local set of databases and return the result to a browser window. wwwblast is now in sync with the NCBI toolkit and the two above two packages. Installation requires existing web server (apache) setup.

Table 2.5.1 File content for /blast/db/executables/LATEST/ directory
File/Dir Name	Content
blast-x.y.z-axp64-tru64.tar.gz	command line blast for COMPAQ/HP alpha machine (OSF 5.1 and above)
blast-x.y.z-ia32-freebsd.tar.gz	command line blast for PC running freeBSD
blast-x.y.z-ia32-linux.tar.gz	command line blast for PC running Linux
blast-x.y.z-ia32-solaris.tar.gz	command line blast for PC runnig solaris
blast-x.y.z-ia32-win32.exe	command line blast for PC running windows
blast-x.y.z-mips64-irix.tar.gz	command line blast for 64-bits SGI machine
blast-x.y.z-ppc32-macosx.tar.gz	command line blast for MacOSX
blast-x.y.z-sparc64-solaris.tar.gz	command line blast for Sparc running Solaris
blast-x.y.z-x64-linux.tar.gz	command line blast for 64-bits Linux
ncbi.tar.gz	NCBI toolkit
ncbiz.exe	NCBI toolkit for PC
netblast-x.y.z-axp64-tru64.tar.gz	netblast for COMPAQ/HP alpha machine (OSF 5.1 and above)
netblast-x.y.z-ia32-freebsd.tar.gz	netblast for PC running freeBSD
netblast-x.y.z-ia32-linux.tar.gz	netblast for PC running Linux
netblast-x.y.z-ia32-solaris.tar.gz	netblast for PC runnig solaris
netblast-x.y.z-ia32-win32.exe	netblast for PC running windows
netblast-x.y.z-mips64-irix.tar.gz	netblast for 64-bits SGI machine
netblast-x.y.z-ppc32-macosx.tar.gz	netblast for MacOSX
netblast-x.y.z-sparc64-solaris.tar.gz	netblast for Sparc running Solaris
netblast-x.y.z-x64-linux.tar.gz	netblast for 64-bits Linux
wwwblast-x.y.z-axp64-tru64.tar.gz	web server blast for COMPAQ/HP alpha machine (OSF 5.1 and above)
wwwblast-x.y.z-ia32-freebsd.tar.gz	web server blast for PC running freeBSD
wwwblast-x.y.z-ia32-linux.tar.gz	web server blast for PC running Linux
wwwblast-x.y.z-ia32-solaris.tar.gz	web server blast for PC runnig solaris
wwwblast-x.y.z-mips64-irix.tar.gz	web server blast for 64-bits SGI machine
wwwblast-x.y.z-ppc32-macosx.tar.gz	web server blast for MacOSX
wwwblast-x.y.z-sparc64-solaris.tar.gz	web server blast for Sparc running Solaris
wwwblast-x.y.z-x64-linux.tar.gz	web server blast for 64-bits Linux
NOTE: Currently, we do not have access to IBM AIX server to make the binary for that platform. Also, we do not make wwwblast package of window.

2.5.2 /blast/executables/release/ subdirectory content

This directory contains past major releases of BLAST, as far back as version 2.0.10. Each release is in its own subdirectory.

2.5.2 File content for /blast/executables/release/ subdirectory
File/Dir Name	Content
2.0.10	Release version 2.0.10
2.0.11	Release version 2.0.11
2.0.12	Release version 2.0.12
2.0.13	Release version 2.0.13
2.0.14	Release version 2.0.14
2.0.7	Release version 2.0.7
2.0.8	Release version 2.0.8
2.0.9	Release version 2.0.9
2.1.2	Release version 2.1.2
2.1.3	Release version 2.1.3
2.2.10	Release version 2.2.10
2.2.11	Release version 2.2.11
2.2.12	Release version 2.2.12
2.2.13	Release version 2.2.13
2.2.3	Release version 2.2.3
2.2.4	Release version 2.2.4
2.2.5	Release version 2.2.5
2.2.6	Release version 2.2.6
2.2.7	Release version 2.2.7
2.2.8	Release version 2.2.8
2.2.9	Release version 2.2.9

2.5.3 /blast/executables/snapshot/ subdirectory content

This directory contains compilation of blast packages with bug fixes in between major versioned releases. The subdirectories are named with date. There was not snapshot or patch since the end of 2004, which as for release 2.2.10. Available old patches will not be listed here.

2.6 /blast/matrices directory content

This directory contains the scoring matrices, which are files that can be used by BLAST alignment assessment. The file are text files with special format that can be viewed directly by a browser.

For valid statistical analysis, blastn uses only identity matrix and blastp only supports a limited subset of the BLOSUM and PAM matrices: BLOSUM45, BLOSUM62, BLOSUM80, plus PAM30 and PAM70.

2.7 /blast/temp/ directory content

For temporary file storage and miscellaneous files or tools. Current empty.

3. Techinical Support

Additional questions/comments on this ftp site should be directed to NCBI blast-help group at:

blast-help@ncbi.nlm.nih.gov

Other questions on general NCBI resources should be directed to:

info@ncbi.nlm.nih.gov