<!-- doc/src/sgml/config.sgml -->

<sect1 id="runtime-config-failover">
 <title>Failover and Failback</title>

 <para>
  <emphasis>Failover</emphasis> means automatically detaching
  <productname>PostgreSQL</productname>backend node which is not
  accessible by <productname>Pgpool-II</productname>.  This happens
  automatically regardless the configuration parameter settings and is
  so called <emphasis>automatic failover</emphasis>
  process. <productname>Pgpool-II</productname> confirms the
  inaccessibility of <productname>PostgreSQL</productname> backend node
  by using following methods:

  <itemizedlist>
   <listitem>
    <para>
     Regular health check process
     (see <xref linkend="runtime-config-health-check"> for more
      details). The heath check process tries to connect
      from <productname>Pgpool-II</productname>
      to <productname>PostgreSQL</productname> node to confirm
      its healthiness. If it fails to connect, it is possible
      that there's something wrong with network connection
      between <productname>Pgpool-II</productname>
      and <productname>PostgreSQL</productname>,
      and/or <productname>PostgreSQL</productname> does not work
      properly. <productname>Pgpool-II</productname> does not
      distinguish each case and just decides that the
      particular <productname>PostgreSQL</productname> node is
      not available if health check fails.
    </para>
   </listitem>

   <listitem>
    <para>
     An error occurs while connecting
     to <productname>PostgreSQL</productname>, or network level
     errors occur while communicating with it.
     <productname>Pgpool-II</productname> will just disconnect
     the session to client
     if <xref linkend="guc-failover-on-backend-error"> is off in
      that case However.
    </para>
   </listitem>

   <listitem>
    <para>
     In the case When clients already connect
     to <productname>Pgpool-II</productname>
     and <productname>PostgreSQL</productname> is shutdown
     (please note that if no client connects
     to <productname>Pgpool-II</productname> at all, shutting
     down of <productname>PostgreSQL</productname> does not
     trigger a failover).
    </para>
    <para>
     In addition, in the streaming replication mode, shutting down
     of <productname>PostgreSQL</productname> does not trigger a
     failover, if the node is a standby node and is not the load
     balance node even
     if <xref linkend="guc-failover-on-backend-shutdown"> is on.
    </para>
   </listitem>
  </itemizedlist>

 </para>

 <para>
  If <xref linkend="guc-failover-command"> is configured and a
  failover happens, <xref linkend="guc-failover-command"> gets
  executed. <xref linkend="guc-failover-command"> should be provided
  by user. From 4.1 an example script for failover command is provided
  as <filename>failover.sh.sample</filename> which can be a good
  starting point for you.
 </para>
 <para>
  The major role of failover command is choosing new primary server
  from existing standby servers and promoting it for example. Another
  example would be let the administrator know that a failover happens
  by sending a mail.
 </para>

 <para>
  While a failover could happen when a failure occurs, it is
  possible to trigger it by hand. This is called a <emphasis>switch
   over</emphasis>. For instance, switching over
  a <productname>PostgreSQL</productname> to take its backup would
  be possible. Note that switching over just sets the status to
  down and never bringing <productname>PostgreSQL</productname>
  down. A switch over can be triggered by
  using <xref linkend="PCP-DETACH-NODE"> command.
 </para>

 <para>
  A <productname>PostgreSQL</productname> node detached by failover or
  switch over will never return to the previous state (attached state)
  automatically in the default setting. Restarting
  <productname>Pgpool-II</productname> with -D option or running <xref
  linkend="PCP-ATTACH-NODE"> makes it to the attached state again. It
  is recommended to confirm the replication_state of <xref
  linkend="SQL-SHOW-POOL-NODES"> is "streaming" before doing that. The
  state indicates that the standby server is properly connected to the
  primary server through streaming replication and both databases are
  in sync.
 </para>
 <para>
  From 4.1 a new parameter <xref linkend="guc-auto-failback"> can be
  used to do above automatically. See <xref
  linkend="guc-auto-failback"> for more details.
 </para>

 <sect2 id="runtime-config-failover-settings">

  <title>Failover and Failback Settings</title>

  <variablelist>

   <varlistentry id="guc-failover-command" xreflabel="failover_command">
    <term><varname>failover_command</varname> (<type>string</type>)
     <indexterm>
      <primary><varname>failover_command</varname> configuration parameter</primary>
     </indexterm>
    </term>
    <listitem>
     <para>
      Specifies a user command to run when a <productname>PostgreSQL</> backend node gets detached.
      <productname>Pgpool-II</productname> replaces the following special characters
      with the backend specific information.
     </para>

     <table id="failover-command-table">
      <title>failover command options</title>
      <tgroup cols="2">
       <thead>
	<row>
	 <entry>Special character</entry>
	 <entry>Description</entry>
	</row>
       </thead>

       <tbody>
	<row>
	 <entry>%d</entry>
	 <entry>DB node ID of the detached node</entry>
	</row>
	<row>
	 <entry>%h</entry>
	 <entry>Hostname of the detached node</entry>
	</row>
	<row>
	 <entry>%p</entry>
	 <entry>Port number of the detached node</entry>
	</row>
	<row>
	 <entry>%D</entry>
	 <entry>Database cluster directory of the detached node</entry>
	</row>
	<row>
	 <entry>%m</entry>
	 <entry>New main node ID</entry>
	</row>
	<row>
	 <entry>%H</entry>
	 <entry>Hostname of the new main node</entry>
	</row>
	<row>
	 <entry>%M</entry>
	 <entry>Old main node ID</entry>
	</row>
	<row>
	 <entry>%P</entry>
	 <entry>Old primary node ID</entry>
	</row>
	<row>
	 <entry>%r</entry>
	 <entry>Port number of the new main node</entry>
	</row>
	<row>
	 <entry>%R</entry>
	 <entry>Database cluster directory of the new main node</entry>
	</row>
	<row>
	 <entry>%N</entry>
	 <entry>Hostname of the old primary node (<productname>Pgpool-II</productname> 4.1 or after)</entry>
	</row>
	<row>
	 <entry>%S</entry>
	 <entry>Port number of the old primary node (<productname>Pgpool-II</productname> 4.1 or after)</entry>
	</row>
	<row>
	 <entry>%%</entry>
	 <entry>'%' character</entry>
	</row>

       </tbody>
      </tgroup>
     </table>

     <note>
      <para>
       The "main node" refers to a node which has the
       "youngest (or the smallest) node id" among live the
       database nodes. In <link linkend="running-mode">streaming
	replication mode</link>, this may be different from
       primary node. In <xref linkend="failover-command-table">,
	%m is the new main node chosen
	by <productname>Pgpool-II</productname>. It is the node
	being assigned the youngest (smallest) node id which is
	alive. For example if you have 3 nodes, namely node 0, 1,
	2. Suppose node 1 the primary and all of them are healthy
	(no down node). If node 1 fails, failover_command is
	called with %m = 0. And, if all standby nodes are down and primary node
    failover happens, failover_command is called with %m = -1 and %H,%R,$r = "".
      </para>
     </note>

     <note>
      <para>
       When a failover is performed,
       basically <productname>Pgpool-II</productname> kills all
       its child processes, which will in turn terminate all the
       active sessions to
       <productname>Pgpool-II</productname>. After that <productname>Pgpool-II</productname>
       invokes the <command>failover_command</command> and after the command completion
       <productname>Pgpool-II</productname> starts new child processes
       which makes it ready again to accept client connections.
      </para>
      <para>
       However from <productname>Pgpool-II</productname> 3.6, in the
       steaming replication mode, client sessions will not be
       disconnected any more when a failover occurs if the session
       does not use the failed standby server. Please note that if a
       query is sent while failover is processing, the session will be
       disconnected. If the primary server goes down, still all
       sessions will be disconnected. Health check timeout case will
       also cause the full session disconnection. Other health check
       error, including retry over case does not trigger full session
       disconnection.
      </para>
     </note>

     <note>
      <para>
       You can run <command>psql</command> (or whatever command)
       against backend to retrieve some information in the
       script, but you cannot run <command>psql</command> against
       <productname>Pgpool-II</productname> itself, since the
       script is called from <productname>Pgpool-II</productname>
       and it needs to run
       while <productname>Pgpool-II</productname> is working on
       failover.
      </para>
     </note>

     <para>
      A complete failover_command example can be found
      in <xref linkend="example-cluster">.
     </para>

     <para>
      This parameter can be changed by reloading the <productname>Pgpool-II</> configurations.
     </para>
    </listitem>
   </varlistentry>

   <varlistentry id="guc-failback-command" xreflabel="failback_command">
    <term><varname>failback_command</varname> (<type>string</type>)
     <indexterm>
      <primary><varname>failback_command</varname> configuration parameter</primary>
     </indexterm>
    </term>
    <listitem>
     <para>
      Specifies a user command to run when a <productname>PostgreSQL</> backend node gets attached to
      <productname>Pgpool-II</productname>. <productname>Pgpool-II</productname>
      replaces the following special characters with the backend specific information.
      before executing the command.
     </para>

     <table id="failback-command-table">
      <title>failback command options</title>
      <tgroup cols="2">
       <thead>
	<row>
	 <entry>Special character</entry>
	 <entry>Description</entry>
	</row>
       </thead>

       <tbody>
	<row>
	 <entry>%d</entry>
	 <entry>DB node ID of the attached node</entry>
	</row>
	<row>
	 <entry>%h</entry>
	 <entry>Hostname of the attached node</entry>
	</row>
	<row>
	 <entry>%p</entry>
	 <entry>Port number of the attached node</entry>
	</row>
	<row>
	 <entry>%D</entry>
	 <entry>Database cluster directory of the attached node</entry>
	</row>
	<row>
	 <entry>%m</entry>
	 <entry>New main node ID</entry>
	</row>
	<row>
	 <entry>%H</entry>
	 <entry>Hostname of the new main node</entry>
	</row>
	<row>
	 <entry>%M</entry>
	 <entry>Old main node ID</entry>
	</row>
	<row>
	 <entry>%P</entry>
	 <entry>Old primary node ID</entry>
	</row>
	<row>
	 <entry>%r</entry>
	 <entry>Port number of the new main node</entry>
	</row>
	<row>
	 <entry>%R</entry>
	 <entry>Database cluster directory of the new main node</entry>
	</row>
	<row>
	 <entry>%N</entry>
	 <entry>Hostname of the old primary node (<productname>Pgpool-II</productname> 4.1 or after)</entry>
	</row>
	<row>
	 <entry>%S</entry>
	 <entry>Port number of the old primary node (<productname>Pgpool-II</productname> 4.1 or after)</entry>
	</row>
	<row>
	 <entry>%%</entry>
	 <entry>'%' character</entry>
	</row>

       </tbody>
      </tgroup>
     </table>

     <note>
      <para>
       You can run <command>psql</command> (or whatever command)
       against backend to retrieve some information in the
       script, but you cannot run <command>psql</command> against
       <productname>Pgpool-II</productname> itself, since the
       script is called from <productname>Pgpool-II</productname>
       and it needs to run
       while <productname>Pgpool-II</productname> is working on
       failback.
      </para>
     </note>

     <para>
      This parameter can be changed by reloading the <productname>Pgpool-II</> configurations.
     </para>
    </listitem>
   </varlistentry>

   <varlistentry id="guc-follow-primary-command" xreflabel="follow_primary_command">
    <term><varname>follow_primary_command</varname> (<type>string</type>)
     <indexterm>
      <primary><varname>follow_primary_command</varname> configuration parameter</primary>
     </indexterm>
    </term>
    <listitem>

     <para>
      Specifies a user command to run after failover on the
      primary node failover. In case of standby node failover, the
      command will not be executed. This command also runs if a
      node promote request is issued by
      <xref linkend="PCP-PROMOTE-NODE"> command. This works only
       in streaming replication mode.
     </para>

     <para>
      Since the command is executed within a child process forked
      off by <productname>Pgpool-II</productname> after failover
      is completed, execution of follow primary command does not
      block the service
      of <productname>Pgpool-II</productname>. Here is a pseud
      code to illustrate how the command is executed:
      <programlisting>
for each backend node
{
    if (the node is not the new primary)
        set down node status to shared memory status
        memorize that follow primary command is needed to execute
}
if (we need to executed follow primary command)
{
    fork a child process
    (within the child process)

    for each backend node
    {
        if (the node status in shared memory is down)
            execute follow primary command
    }
}
      </programlisting>
     </para>

     <para>
      <productname>Pgpool-II</productname> replaces the following special characters
      with the backend specific information before executing the command.
     </para>

     <table id="follow-primary-command-table">
      <title>follow primary command options</title>
      <tgroup cols="2">
       <thead>
	<row>
	 <entry>Special character</entry>
	 <entry>Description</entry>
	</row>
       </thead>

       <tbody>
	<row>
	 <entry>%d</entry>
	 <entry>DB node ID of the detached node</entry>
	</row>
	<row>
	 <entry>%h</entry>
	 <entry>Hostname of the detached node</entry>
	</row>
	<row>
	 <entry>%p</entry>
	 <entry>Port number of the detached node</entry>
	</row>
	<row>
	 <entry>%D</entry>
	 <entry>Database cluster directory of the detached node</entry>
	</row>
	<row>
	 <entry>%m</entry>
	 <entry>New primary node ID</entry>
	</row>
	<row>
	 <entry>%H</entry>
	 <entry>Hostname of the new primary node</entry>
	</row>
	<row>
	 <entry>%M</entry>
	 <entry>Old main node ID</entry>
	</row>
	<row>
	 <entry>%P</entry>
	 <entry>Old primary node ID</entry>
	</row>
	<row>
	 <entry>%r</entry>
	 <entry>Port number of the new primary node</entry>
	</row>
	<row>
	 <entry>%R</entry>
	 <entry>Database cluster directory of the new primary node</entry>
	</row>
	<row>
	 <entry>%N</entry>
	 <entry>Hostname of the old primary node (<productname>Pgpool-II</productname> 4.1 or after)</entry>
	</row>
	<row>
	 <entry>%S</entry>
	 <entry>Port number of the old primary node (<productname>Pgpool-II</productname> 4.1 or after)</entry>
	</row>
	<row>
	 <entry>%%</entry>
	 <entry>'%' character</entry>
	</row>

       </tbody>
      </tgroup>
     </table>

     <note>
      <para>
       If <varname>follow_primary_command</varname> is not empty, then after failover
       on the primary node gets completed in Native Replication mode with streaming replication,
       <productname>Pgpool-II</productname> degenerates all nodes except the new primary
       and starts new child processes to be ready again to accept connections from the clients.
       After this, <productname>Pgpool-II</productname> executes the command configured
       in the <varname>follow_primary_command</varname> for each degenerated backend nodes.
      </para>
     </note>
     <para>
      Typically <varname>follow_primary_command</varname> command
      is used to recover the standby from the new primary by calling
      the pcp_recovery_node command.  In
      the <varname>follow_primary_command</varname>, it is
      recommended to check whether
      target <productname>PostgreSQL</productname> node is running
      or not using pg_ctl since already stopped node usually has a
      reason to be stopped: for example, it's broken by hardware
      problems or administrator is maintaining the node.  If the
      node is stopped, skip the node. If the node is running, stop
      the node first and recovery it.
      Although it is possible to recover by directly
      executing <command>pg_basebackup</command> or other commands
      instead of <xref linkend="PCP-RECOVERY-NODE">, you need to
      execute <xref linkend="PCP-ATTACH-NODE"> to
      notice <productname>Pgpool-II</productname> that the standby
      server is ready to use. Otherwise the standby server remains in
      down status. (Since <xref linkend="PCP-RECOVERY-NODE">
      internally executes <xref linkend="PCP-ATTACH-NODE">, you don't
      need to execute <xref linkend="PCP-ATTACH-NODE"> after
      recovering.
      A complete <varname>follow_primary_command</varname> example can
      be found in <xref linkend="example-cluster">.
     </para>
     <para>
      This parameter can be changed by reloading the <productname>Pgpool-II</> configurations.
     </para>
    </listitem>
   </varlistentry>

   <varlistentry id="guc-failover-on-backend-shutdown" xreflabel="failover_on_backend_shutdown">
    <term><varname>failover_on_backend_shutdown</varname> (<type>boolean</type>)
     <indexterm>
      <primary><varname>failover_on_backend_shutdown</varname> configuration parameter</primary>
     </indexterm>
    </term>
    <listitem>
     <para>
      When set to on, <productname>Pgpool-II</productname> detects a
      backend shutdown event by checking particular error code
      <literal>57P01</literal> and <literal>57P02</literal> on a
      session established between the client and backend. If it
      detects those error codes, a failover is triggered on that node.
      When this is set to off, <productname>Pgpool-II</productname>
      only report an error and disconnect the session in case of such
      errors.  The default is off.
     </para>
     <para>
      If no client is connected,
      <productname>Pgpool-II</productname> will not detect the event
      even if the parameter is on.
     </para>
     <para>
      In addition, in the streaming replication mode, shutting down
      of <productname>PostgreSQL</productname> does not trigger a
      failover, if the node is a standby node and is not the load
      balance node even
      if <xref linkend="guc-failover-on-backend-shutdown"> is on.  In
      order to find which node is the load balance node, you can
      use <xref linkend="SQL-SHOW-POOL-NODES">
      or <xref linkend="PCP-PROC-INFO">.
     </para>

     <para>
      Please note that <literal>57P01</literal> is not only sent at a
      shutdown event, but also sent when the backend currently
      connected is killed by <function>pg_terminate_backend</function>
      or SIGTERM signal. This is annoying because it results in
      unwanted failover. To avoid this, turn off this parameter.
     </para>

     <note>
      <para>
       It is recommended to turn on the backend health checking
       (see <xref linkend="runtime-config-health-check">)
	when <varname>failover_on_backend_shutdown</varname> is set to off.
      </para>
     </note>

     <para>
      This parameter is available
      in <productname>Pgpool-II</productname> 4.3 or later.  This
      parameter can be changed by reloading
      the <productname>Pgpool-II</productname> configurations.
     </para>
    </listitem>
   </varlistentry>

   <varlistentry id="guc-failover-on-backend-error" xreflabel="failover_on_backend_error">
    <term><varname>failover_on_backend_error</varname> (<type>boolean</type>)
     <indexterm>
      <primary><varname>failover_on_backend_error</varname> configuration parameter</primary>
     </indexterm>
    </term>
    <listitem>
     <para>
      When set to on, <productname>Pgpool-II</productname> considers the reading/writing
      errors on the PostgreSQL backend connection as the backend node failure and trigger the
      failover on that node after disconnecting the current session.
      When this is set to off, <productname>Pgpool-II</productname> only report an error
      and disconnect the session in case of such errors.
     </para>
     <note>
      <para>
       It is recommended to turn on the backend health checking
       (see <xref linkend="runtime-config-health-check">)
	when <varname>failover_on_backend_error</varname> is set to off.
      </para>
     </note>
     <para>
      This parameter can be changed by reloading the <productname>Pgpool-II</> configurations.
     </para>
     <note>
      <para>
       Prior to <productname>Pgpool-II</productname> <emphasis>V4.0</emphasis>,
       this configuration parameter name was <varname>fail</varname><emphasis>_</emphasis><varname>over_on_backend_error</varname>.
      </para>
     </note>
    </listitem>
   </varlistentry>

   <varlistentry id="guc-search-primary-node-timeout" xreflabel="search_primary_node_timeout">
    <term><varname>search_primary_node_timeout</varname> (<type>integer</type>)
     <indexterm>
      <primary><varname>search_primary_node_timeout</varname> configuration parameter</primary>
     </indexterm>
    </term>
    <listitem>
     <para>
      Specifies the maximum amount of time in seconds to search for the
      primary node when a failover scenario occurs.
      <productname>Pgpool-II</productname> will give up looking for the primary
      node if it is not found with-in this configured time.
      Default is 300 and Setting this parameter to 0 means keep trying forever.
     </para>
     <para>
      This parameter is only applicable in the streaming replication mode.
     </para>
     <para>
      This parameter can be changed by reloading the <productname>Pgpool-II</> configurations.
     </para>
    </listitem>
   </varlistentry>

   <varlistentry id="guc-detach-false-primary" xreflabel="detach_false_primary">
    <term><varname>detach_false_primary</varname> (<type>boolean</type>)
     <indexterm>
      <primary><varname>detach_false_primary</varname> configuration parameter</primary>
     </indexterm>
    </term>
    <listitem>
     <para>
      If set to on, detach false primary node. The default is
      off. This parameter is only valid in streaming replication
      mode and for <productname>PostgreSQL</productname> 9.6 or
      after since this feature
      uses <function>pg_stat_wal_receiver</function>.
      If <productname>PostgreSQL</productname> 9.5.x or older
      version is used, no error is raised, just the feature is
      ignored.
     </para>
     <para>
      If there's no primary node, no checking will be performed.
     </para>
     <para>
      If there's no standby node, and there's only one primary
      node, no checking will be performed.
     </para>
     <para>
      If there's no standby node, and there's multiple primary
      nodes, leave the primary node which has the youngest node
      id and detach rest of primary nodes.
     </para>
     <para>
      If there are one or more primaries and one or more standbys,
      check the connectivity between primary and standby nodes by
      using <function>pg_stat_wal_receiver</function>
      if <productname>PostgreSQL</productname> 9.6 or after. In
      this case if a primary node connects to all standby nodes,
      the primary is regarded as "true" primary. Other primaries
      are regarded as "false" primary and the false primaries will
      be detached if <varname>detach_false_primary</varname> is
      true.  If no "true" primary is found, nothing will happen.
     </para>
     <para>
      When <productname>Pgpool-II</productname> starts, the
      checking of false primaries are performed only once in
      the <productname>Pgpool-II</productname> main
      process. If <xref linkend="guc-sr-check-period"> is greater
       than 0, the false primaries checking will be performed at
       the same timing of streaming replication delay checking.
     </para>

     <note>
      <para>
       <xref linkend="guc-sr-check-user"> must
	be <productname>PostgreSQL</productname> super user or
	in "pg_monitor" group to use this feature. To
	make <xref linkend="guc-sr-check-user"> in pg_monitor
	 group, execute following SQL command
	 by <productname>PostgreSQL</productname> super user
	 (replace "sr_check_user" with the setting
	 of <xref linkend="guc-sr-check-user">):
	  <programlisting>
	   GRANT pg_monitor TO sr_check_user;
	  </programlisting>
	  For <productname>PostgreSQL</productname> 9.6, there's
	  no pg_monitor group
	  and <xref linkend="guc-sr-check-user"> must
	   be <productname>PostgreSQL</productname> super user.
      </para>
     </note>

     <note>
      <para>
       if watchdog is enabled, detaching false primary is only done by
       leader watchdog node. Even
       if <xref linkend="guc-failover-require-consensus"> is on,
       detaching the false primary being performed is solely by the
       judgment of the leader watchdog. No majority consensus is
       required.
      </para>
     </note>

     <para>
      This parameter is only applicable in the streaming replication mode.
     </para>
     <para>
      This parameter can be changed by reloading
      the <productname>Pgpool-II</productname> configurations.
     </para>

     <para>
      <figure>
       <title>Detecting false primaries</title>
       <mediaobject>
	<imageobject>
	 <imagedata fileref="detach_false_primary.gif">
	</imageobject>
       </mediaobject>
      </figure>
     </para>

    </listitem>
   </varlistentry>

   <varlistentry id="guc-auto-failback" xreflabel="auto_failback">
    <term><varname>auto_failback</varname> (<type>boolean</type>)
     <indexterm>
      <primary><varname>auto_failback</varname> configuration parameter</primary>
     </indexterm>
    </term>
    <listitem>
     <para>
      When set to on, standby node be automatically failback, if the node status
      is down but streaming replication works normally. This is useful when
      standby node is degenerated by pgpool because of the temporary network failure.
     </para>

     <para>
      To use this feature, streaming replication check (see <xref linkend="runtime-streaming-replication-check"> for more details)
       must be enabled, and <productname>PostgreSQL</productname> 9.1 or later
       is required as backend nodes. This feature uses <function>pg_stat_replication</function>
       on primary node. The automatic failback is performed on standby node only.
       Note that failback_command will be executed as well if failback_command is not empty.
       If you plan to detach standby node for maintenance, set this parameter to off beforehand.
       Otherwise it's possible that standby node is reattached against your intention.
     </para>
     <para>
      The default is off. This parameter can be changed by reloading the <productname>Pgpool-II</> configurations.
     </para>

     <note>
      <para>
       <xref linkend="guc-auto-failback"> may not work, when replication slot is used.
       There is possibility that the streaming replication is stopped, because
       <xref linkend="guc-failover-command"> is executed and replication slot is deleted by
       the command.
      </para>
     </note>
    </listitem>
   </varlistentry>

   <varlistentry id="guc-auto-failback-interval" xreflabel="auto_failback_interval">
    <term><varname>auto_failback_interval</varname> (<type>integer</type>)
     <indexterm>
      <primary><varname>auto_failback_interval</varname> configuration parameter</primary>
     </indexterm>
    </term>
    <listitem>
     <para>
      Specifies the minimum amount of time in seconds for execution interval of auto failback.
      Next auto failback won't execute until that specified time have passed
      after previous auto failback. When <productname>Pgpool-II</productname> frequently detect
      backend down because of network error for example, you may avoid repeating
      failover and failback by setting this parameter to large enough value.
      The default is 60. Setting this parameter to 0 means that auto failback don't wait.
     </para>
    </listitem>
   </varlistentry>

  </variablelist>
 </sect2>

 <sect2 id="runtime-config-failover-in-the-raw-mode">

  <title>Failover in the raw Mode</title>

  <para>
   Failover can be performed in raw mode if multiple backend servers are defined.
   <productname>Pgpool-II</> usually accesses the backend specified by
   <literal>backend_hostname0</> during normal operation. If the
   <literal>backend_hostname0</> fails for some reason,
   <productname>Pgpool-II</> tries to access the backend specified by
   <literal>backend_hostname1</>. If that fails, <productname>Pgpool-II</>
   tries the <literal>backend_hostname2, 3</> and so on.
  </para>

 </sect2>

</sect1>
