Linux specific commands to run when troubleshooting core files for NetBackup support cases

The following steps should be taken when the failure of a NetBackup process or daemon has generated a core file on a Linux system.  This will help ensure the relevant information is collected for when the core file needs to be analyzed as part of a support escalation.  For a list of the basic supporting information that should also be gathered whenever a core file is generated see the document linked in the Related Articles.

Enabling core file generation:

First, run this command to see if the system is configured to generate core files (by default, most Linux systems will not generate core files):
# ulimit -c

If the command returns a zero, then core files have been disabled on the system.

To enable core files run the following command:
# ulimit -c unlimited

This will allow core files to be generated with an unlimited size.  If disk space is a concern, replace "unlimited" with the maximum number of blocks that is desired for a core file - but please be aware that restricting the maximum size of a core file may prevent identification of the problem that caused the process to abort.

NOTE: These changes will only apply until the next reboot.

To enable core file generation on Red Hat Linux on future reboots...

Enable this globally by editing the /etc/sysconfig/init file and add this line:
DAEMON_COREFILE_LIMIT='unlimited'

Edit the file /etc/profile, to change the line that reads:
ulimit -S -c 0 > /dev/null 2>&1
to:
ulimit -S -c unlimited > /dev/null 2>&1


To enable core file generation on SuSE Linux on future reboots...

Check that the HARDCORELIMIT and SOFTCORELIMIT values in /etc/sysconfig/ulimit are set to "unlimited":

HARDCORELIMIT="unlimited"
SOFTCORELIMIT="unlimited"

...or alternatively edit the file /etc/initscript, and add the "ulimit -c unlimited" command close to the end of the file, before the script executes the program:

# initscript   Executed by init(8) for every program it
#              wants to spawn like this:
#
#              /bin/sh /etc/initscript <id> <level> <action> <process>
#

# Set umask to safe level, and enable core dumps.
umask 022
PATH=/bin:/sbin:/usr/bin:/usr/sbin
export PATH

ulimit -c unlimited

# Execute the program.
eval exec "$4"

NOTE: Changes to "ulimit" settings do not affect daemon processes which are already running, so you need to restart any daemons for which you want to enable core dumps. Alternatively, you can reboot the system to have the new settings take effect for all daemons.

By default, core dumps are not generated by programs that are running setuid, to prevent sensitive information being leaked
To temporarily enable core dumps for setuid programs (not persistent over reboots) by running the following commands:

• For Red Hat Enterprise Linux 6:
  DAEMON_COREFILE_LIMIT=unlimited
  export DAEMON_COREFILE_LIMIT
  ulimit -c unlimited
  echo 2 > /proc/sys/fs/suid_dumpable

• For Red Hat Enterprise Linux 5: "suidsafe" (recommended) - protect privileged information by having the core dump be owned by and only readable for root:
  echo 2 > /proc/sys/fs/suid_dumpable

• For Red Hat Enterprise Linux 5: "debug" (may cause privileged information to be leaked):
  echo 1 > /proc/sys/fs/suid_dumpable

• For Red Hat Enterprise Linux 4:
  echo 2 > /proc/sys/kernel/suid_dumpable

• For Red Hat Enterprise Linux 3:
  echo 1 > /proc/sys/kernel/core_setuid_ok

• Changes to these /proc values take effect immediately, so they will also apply to daemons that are already running.

To enable core dumps for setuid programs (persistent over reboots):

• For Red Hat Enterprise Linux 6:
  Un-comment the following line in /etc/security/limits.conf:
  #* soft core 0
  ...and change the value from zero to unlimited:
  * soft core unlimited

  Add the following line to /etc/sysconfig/init:
  DAEMON_COREFILE_LIMIT=’unlimited’

  Add the following lines to /etc/sysctl.conf:
  kernel.core_pattern = /tmp/core-%e-%s-%u-%g-%p-%t
fs.suid_dumpable = 2

  Reload the settings in /etc/sysctl.conf :
  sysctl -p

• For Red Hat 3, 4 and 5, edit /etc/sysctl.conf to add the following:
  fs.suid_dumpable = 2        # RHEL 5 only
  kernel.suid_dumpable = 2    # RHEL 4 only
  kernel.core_setuid_ok = 1   # RHEL 3 only
  kernel.core_pattern = /tmp/core.%e.%t.%p

  Reload the settings in /etc/sysctl.conf :
  sysctl -p