gpmt gp_log_collector
This tool collects Greenplum and system log files, along with the relevant configuration parameters, and generates a file which can be provided to VMware Customer Support for diagnosis of errors or system failures.
Usage
gpmt gp_log_collector [-failed-segs | -c <ID1,ID2,...>| -hostfile <file> | -h <host1,host2,...>]
[ -start <YYYY-MM-DD> ] [ -end <YYYY-MM-DD> ]
[ -dir <path> ] [ -segdir <path> ] [ -a ] [-skip-master] [-with-gpbackup] [-with-gptext] [-with-gptext-only] [-with-gpcc] [-with-gpss] [-gpss_logdir <gpss_log_directory>]
[-with-pxf] [-with-pxf-only] [-with-gpupgrade] [-with-gpdr-primary] [-with-gpdr-recovery]
Options
-failed-segs
The tool scans gp_configuration_history to identify when a segment fails over to their mirrors or simply fails without explanation. The relevant content ID logs will be collected.
-free-space
Free space threshold which will exit log collection if reached. Default value is 10%.
-c
Comma separated list of content IDs to collect logs from.
-hostfile
Hostfile with a list of hostnames to collect logs from.
-h
Comma separated list of hostnames to collect logs from.
-start
Start date for logs to collect (defaults to current date).
-end
End date for logs to collect (defaults to current date).
-dir
Working directory (defaults to current directory).
-segdir
Segment temporary directory (defaults to /tmp).
-a
Answer Yes to all prompts.
-skip-master
When running gp_log_collector, the generated tarball can be very large. Use this option to skip Greenplum Master log collection when only Greenplum Segment logs are required.
-with-gpbackup
Beginning with Greenplum 6.22, this option enables you to collect logs related to backup and restore.
With this option, gpmt collects these log files from $GPADMIN_HOME/gpAdminLogs:
- gpbackup_.log
- gpbackup_helper_.log
- gpbackup_ plugin .log
- gprestore_.log
These are collected from the provided --backup-dir or default backup directory:
- gpbackup__config.yaml
- gpbackup__metadata.sql
- gpbackup__report
- gpbackup__toc.yaml
- gprestore___report
- gpbackup___report
Also, the pg_log file is collected from the master and segment hosts.
-with-gptext
Collect all GPText logs along with Greenplum logs.
-with-gptext-only
Collect only GPText logs.
-with-gpcc
Collect log files related to Greenplum Command Center. Log files are collected from the following locations:
- $GPCC_HOME/logs/*
- $GPCC_HOME/conf/app.conf
- $HOME/gpmetrics/* (Greenplum Command Center 6.8.0 and later on Greenplum Database 6.x or Greenplum Command Center 4.16.0 and later on Greenplum Database 5.x)
- $MASTER_DATA_DIRECTORY/gpmetrics (Greenplum Command Center 6.7.0 and earlier on Greenplum Database 6.x or Greenplum Command Center 4.15.0 and earlier on Greenplum Database 5.x)
- The output of the gppkg -q --all command
-with-gpss : Collect log files related to Greenplum Streaming Server. If you do not specify a directory with the -gpsslogdir option, gpmt collects logs from the gpAdminLogs directory. Log files are of the format gpss_
-with-pxf
Collect all PXF logs along with Greenplum logs.
-with-pxf-only
Collect only PXF logs.
-with-gpupgrade
Collect all gpupgrade logs along with Greenplum logs.
-with-gpdr-primary
Collect all logs relevant for a Greenplum Disaster Recovery primary cluster, along with Greenplum logs.
-with-gpdr-recovery
Collect all logs relevant for a Greenplum Disaster Recovery recovery cluster, along with Greenplum logs.
Note: : Hostnames provided through -hostfile or -h must match the hostname column in gp_segment_configuration.
The tool also collects the following information:
| Source | Files and outputs |
|---|---|
| Database parameters | version uptime pg_resqueue pg_resgroup_config pg_database gp_segment_configuration gp_configuration_history Initialization timestamp |
| Segment server parameters | uname -a sysctl -a psaux netstat -rn netstat -i lsof ifconfig free df -h top sar |
| System files from all hosts | /etc/redhat-release /etc/sysctl.conf /etc/sysconfig/network /etc/security/limits.conf /var/log/dmesg |
| Database-related files from all hosts | $SEG_DIR/pg_hba.conf $SEG_DIR/pg_log/ $SEG_DIRE/postgresql.conf ~/gpAdminLogs |
| GPText files | Installation configuration file: $GPTXTHOME/lib/python/gptextlib/consts.py gptext-state -D \<gptext data dir>/solr*/solr.in \<gptext data dir>/solr*/log4j.properties \<gptext data dir>/zoo*/logs/* commands/bash/-c_echo $PATH commands/bash/-c_ps -ef | grep solr commands/bash/-c_ps -ef | grep zookeeper |
| PXF files | pxf cluster status pxf status PXF version Logs/ CONF/ Run/ |
| gpupgrade files | ~/gpAdminLogs on all hosts $HOME/gpupgrade on master host $HOME/.gpupgrade on all hosts Source cluster's pg_log files located in $MASTER_DATA_DIRECTORY/pg_log on master host Target cluster's pg_log files located in $(gpupgrade config show --target-datadir)/pg_log on master host Target cluster's master data directory |
| Greenplum Disaster Recovery files | GPDB pg_logs gpdr_YYYYMMDD.log files in gpAdminLogs GPDR config files in $GPDR_HOME/configs/ GPDR pgbackrest log files in $GPDR_HOME/logs/ GPDR state files in $GPDR_HOME/state/ |
Note: : Some commands might not be able to be run if user does not have the correct permissions.
Examples
Collect Greenplum master and segment logs listed in a hostfile from today:
Collect logs for any segments marked down from 21-03-2016 until today:
Collect logs from host sdw2.gpdb.local between 2016-03-21 and 2016-03-23:
Collect only GPText logs for all segments, without any Greenplum logs: