The Switch

Old Dispatcher

New Dispatcher

Firewalling

Work Arounds

Additional Info

NCBI Dispatcher Parameters

There are two pieces of code in C toolkit that allow an application to contact NCBI dispatchers, the new and the old one. The old dispatcher is being phased out and is no longer supported at NCBI. All new services that are added get configured with the new dispatcher. The code to access the new dispatcher had been placed in the toolkit a while ago, and should be ready for use in all applications compiled and linked against the toolkit released since beginning of 2002 [recent repository check showed that actually the cut-off date can perhaps be half a year earlier, in early summer 2001]. Older applications have to be upgraded by re-compiling/re-linking in order to take advantage of the new code.

In the beginning of August 2002, the switch was made in the toolkit's code that now directs all network requests to go through the new dispatcher. So all applications linked against the toolkit at a later date should be using the new dispatcher automatically.

Below is the description of parameters and differences between the two dispatchers, and instructions on how to set an existing application to use the new dispatcher's code without rebuilding.

Although the dispatcher should work behind the scenes absolutely transparently for the user, some applications may fail if for example there are on-site configurational discrepancies between the two dispatchers. In fact, every attempt was made to keep the configurations consistent at the time of the switch, but still there is a little chance that something could have been missed.

The switch

A dispatcher to use can be controlled by the environment variable SRV_CONN_MODE as follows:

SRV_CONN_MODE=SERVICE	chooses the new dispatcher [default for all programs built since mid August, 2002]
SRV_CONN_MODE=WWW	chooses the old dispatcher, without firewalling [this was the default prior to the switchover in mid August, 2002]
SRV_CONN_MODE=FIREWALL	chooses the old dispatcher, with firewalling

Parameters of the OLD dispatcher

SRV_ENGINE_HOST SRV_ENGINE_PORT	host and port to locate dispatcher. Default = www.ncbi.nlm.nih.gov:80 (should normally not be changed)
SRV_ENGINE_URL	path part of URI to locate dispatcher. Default = /Service/nph-dispd.cgi (should normally not be changed, either)
SRV_CONN_TIMEOUT	time (as a floating point value, seconds) to wait for network activity, at most. Default = 30. Special value INFINITE is also recognized to set the timeout value to "wait forever". Note: If you are in the United States, the default timeout of 30 seconds should suffice. From foreign countries with poor Internet connections to the U.S., you may want to increase this value.
SRV_CONN_TRY	how many connection attempts to perform if unsuccessful, default = 3
SRV_HTTP_PROXY_HOST SRV_HTTP_PROXY_PORT	host and port of transparent HTTP proxy (if in use at the client's end), default = none
SRV_PROXY_HOST	CERN-like (non-transparent) proxy host (if in use at the client's end), default = none
SRV_DEBUG_PRINTOUT	switch to print debugging/trace output, default = NO Activate printing with YES, 1, or TRUE.
NI_SERVICE_NAME_<name>	reassignment of the service name, e.g.: NI_SERVICE_NAME_ID1 ID1Prod used for temporary name override, if a developer instructed you to do so (usually, neither required nor advised). Note: Service name must be all capitalized for this setting to work, e.g. "Entrez" becomes ENTREZ.

Atlernate service name can also be specified via the registry file by defining they key SERVICE_NAME with desired value, which should be located in the section having the service name in all capitals. The following registry fragment changes the name of the Entrez service:

[ENTREZ]
SERVICE_NAME=EntrezTest

Parameters of the NEW dispatcher

The following additional parameters are also recognized (and are the preferred way to tune up the new dispatcher). These parameters can be specified in the environment as printed in the table, or they can be read from the registry file's section [CONN], but then without the prefix CONN. For example, the environment setting CONN_FIREWALL=TRUE is equivalent to the following registry file fragment:

[CONN]
FIREWALL=TRUE

There are also per-service settings, taking the form <NAME>_CONN_<PARAMETER> in the environment, or just like printed below if found in the section [<NAME>] in the registry file. For example, the following settings of FIREWALL in the environment and the registry file are equivalent, and affect Bounce service only (leaving all other services with default or their individual settings, if any):

environment:

BOUNCE_CONN_FIREWALL=TRUE

registry file:

[BOUNCE]
CONN_FIREWALL=TRUE

The order of significance (from the most to the least) of specifications is as follows:

1. Per-service environment variable, e.g. BOUNCE_CONN_FILREWALL=TRUE;
2. Per-service registry key, e.g. CONN_FIREWALL=TRUE in the section [BOUNCE];
3. Common environment setting, e.g. CONN_FIREWALL=TRUE;
4. Common registry key from section [CONN], e.g. FIREWALL=TRUE;
5. Default setting.

CONN_HOST CONN_PORT	see SRV_ENGINE_HOST and SRV_ENGINE_PORT above.
CONN_PATH	see SRV_ENGINE_URL, default = /Service/dispd.cgi
CONN_ARGS	arguments for service/dispatcher, going after the '?' in CGI's parameters, default = none
CONN_REQ_METHOD	HTTP request method to use (default = POST, may also be one of ANY and GET).
CONN_TIMEOUT	see SRV_CONN_TIMEOUT.
CONN_MAX_TRY	see SRV_CONN_TRY.
CONN_HTTP_PROXY_HOST CONN_HTTP_PROXY_PORT	see SRV_HTTP_PROXY_HOST and SRV_HTTP_PROXY_PORT.
CONN_PROXY_HOST	see SRV_PROXY_HOST.
CONN_DEBUG_PRINTOUT	see SRV_DEBUG_PRINTOUT. Additional value DATA is provided to print the network data as being transferred at the transport [TCP] layer.
CONN_STATELESS	TRUE or FALSE, default = FALSE. A proper value has to be set depending on the nature of the client (Web-browsers are CONN_STATELESS=TRUE). For dispatching by default, this variable is not to be touched!
CONN_FIREWALL	TRUE or FALSE, default = FALSE. A proper value has to be chosen depending on whether the client is behind a firewall or not. Use TRUE if so! This parameter of the new dispatcher is analogous to switching the old dispatcher into firewall mode by specifying FIREWALL in SRV_CONN_MODE.
<name>_CONN_SERVICE_NAME	see description for NI_SERVICE_NAME_... above.

When intermixed with older (SRV_...) parameters, the order of consideration is the following: the new parameters (CONN_...) first as described in the table above, then SRV_... parameters, thus all SRV_... parameters taking precedence over all CONN_... parameters. The only exception is the service name override ({..._CONN|NI}_SERVICE_NAME_...), which works the different way:

• All other parameters loaded, first CONN_... using the original service name as specified in the application (e.g. Entrez) from the registry/environment, followed by SRV_... found in the registry/environment.
• SERVICE_NAME registry key in the service's section followed by NI_SERVICE_NAME_... environment variable (if exist) are then consulted for an alternate name.
• ..._CONN_SERVICE_NAME environment variable (or service's registry section with SERVICE_NAME key) in then consulted, using a new name from the substitution above (if occurred). This process is recursive but with no loop detection, i.e. A_CONN_SERVICE_NAME=B and B_CONN_SERVICE_NAME=A would create an infinite loop, which breaks the application. The following registry/environment combination changes the name of the Entrez service into Entrez2:

  environment:

  NI_SERVICE_NAME_ENTREZ=EntrezTest
  

  registry:

  [ENTREZ]
  SERVICE_NAME=Entrez1 ; Note that this key is overriden in the env
  
  [ENTREZ_TEST]
  SERVICE_NAME=Entrez2
  

Note that if a client connects to new NCBI dispatcher, then the dispatcher might be able to switch the client into pseudo-firewall mode, should the dispatcher notice that this might be needed for the client to further succeed with the service requested. This however is not a substitute for CONN_FIREWALL parameter, and an application, which connects from behind a firewall, must still use CONN_FIREWALL=TRUE in order to function flawlessly.

Also note that C toolkit can have debugging printouts disabled in the core, thus {SRV|CONN}_DEBUG_PRINTOUT will have no or very little effect. In order to override the core settings, environment setting DEBUG_PRINTOUT=TRUE could be used. Unlike {SRV|CONN}_DEBUG_PRINTOUT the last key has no effect outside the environment (and thus specifying DEBUG_PRINTOUT in the registry file will not affect in-core logging settings). This override option was recently added to the toolkit, and applications built before mid August 2002 lack this feature.

For applications linked against C++ toolkit, environment settings DIAG_POST_LEVEL=0 and DIAG_TRACE=1 would enable the debugging printouts regardless of the in-core settings. You may also use these keys in the [DEBUG] section of the registry file (but not in the [NET_SERV] or [CONN] sections).

Taking care of firewall connections

If you are behind a firewall, it must be configured correctly to access NCBI services. Your network administrators may have done this already. If not, please have them read the section below.

The following section is intended for network administrators:

Using NCBI services from behind a security firewall requires opening ports in your firewall. The ports to open are:

Firewall Port        IP Address
--------------------------------
 5860..5870         130.14.29.112
 5860..5870         165.112.7.12

If your firewall is not transparent, the firewall port number should be mapped to the same port number on the external host.

While not every port from the table above might be currently in use, we stongly recommend you to have them all configured on your firewall.

TROUBLESHOOTING: You can test whether these ports are accessible from your host by just running, for example (see the "Ports to open" list above):

  telnet 130.14.29.112 5861

| > telnet 130.14.29.112 5861
| Trying 130.14.29.112...
| Connected to 130.14.29.112.
| Escape character is '^]'.
| test
| NCBI Firewall Daemon:  Invalid ticket. Connection closed.
| Connection closed by foreign host.

There is also an auxiliary UNIX shell script fwd_check.sh to check all of the above addresses.

To check the ports as they are seen from within NCBI you can use this test page, which shows you whether the ports are open and the firewall daemon is listening on them. The page does not however provide you with the information whether these ports are accessible from your site. You still have to use fwd_check.sh to check that.

Working around connectivity problems

Executable Psrventr.REAL, which implements old Entrez service on Solaris sparc, was confirmed flaky, and had to be removed from configurations of both new and old dispatchers on the sparcs.

Although this should not affect the users of the new dispatcher (as the service had been moved to and is currently provided from a number of Intel machines), the users of the old dispatcher (which was not ported to Linux/Intel platform, and overall is considered for phasing out as inflexible and outdated) may be affected.

The workaround is to use the code in the toolkit that has been around long ago but was not made active until mid August 2002 - we then switched the toolkit to use the new dispatcher code by default.

For that, the user has to define SRV_CONN_MODE=SERVICE either in the environment, or in the configuration (.ini) file, section [NET_SERV], and can do so without an absolute necessity of downloading the new toolkit code from NCBI and rebuilding. Only very-very old applications will not be able to use the switch, and have to be upgraded by linking against freshened libraries.

If an application runs from behind a firewall, then CONN_FIREWALL=TRUE should also be set in the environment (in addition to SRV_CONN_MODE=SERVICE). Please note that CONN_FIREWALL must remain set even if you switch into the newer toolkit release, which does not require setting SRV_CONN_MODE for the use of the new dispatcher.

The overall rule in the case when your application fails to connect to an NCBI service is to undertake the following sequence of steps:

1. Try to force your application to use the new dispatcher code by setting SRV_CONN_MODE to SERVICE in the environment/registry;
2. If this does not help, or if you are connecting from behind a firewall, try to set CONN_FIREWALL to TRUE in addition to the above setting, and run your application again;
3. If you are still unable to connect, set SRV_DEBUG_PRINTOUT to TRUE, and collect the output. (You may also need to set environment variable DEBUG_PRINTOUT to TRUE in addition, if the in-core settings prevent debugging output from being released onto the terminal. Please note that unlike other {CONN|SRV}... keys described in this document, DEBUG_PRINTOUT is honored only if set in the environment and has no effect if put in the registry file.) Beware that debugging output may be copious.
   • If with all the above settings you can find references to either Service/nph-dispd.cgi or Service/nph-ncbid.cgi in the collected output, it means that your application was linked against very old version of the toolkit and has to be rebuilt;
   • If you cannot find references to either of these CGIs, then you can consider contacting NCBI Help Desk to resolve your problem.

Small reference on how to set the environment in various shells (to know the shell in UNIX/Linux, execute "echo $SHELL"):

• C-like shells: csh or tcsh

      setenv SRV_DEBUG_PRINTOUT TRUE
  

• Born-like shells: sh, ash, ksh, zsh, or bash

      SRV_DEBUG_PRINTOUT=TRUE; export SRV_DEBUG_PRINOUT
  

  In ksh, zsh, and bash the two commands above can be combined:

      export SRV_DEBUG_PRINTOUT=TRUE
  

• DOS shell (MS-Windows/DOS): command.com or cmd.exe

      set SRV_DEBUG_PRINTOUT=TRUE
  

Additional Information

Note: Very old NCBI clients used different application configuration settings and ports than listed above. These clients are no longer supported by NCBI.

More technical and brief note on dispatcher selection and parameters is in network/nsclilib/readme located in the C toolkit source tree.

Description of the new NCBI dispatcher can be found here.

Discussion of firewall settings can be found here and also provided in directory doc of the C toolkit.

If you have a problem, which you are unable to resolve on your own, or you have a question, please contact info@ncbi.nlm.nih.gov for further information.