11 Jul 2011 by Simon Greaves
Working with ESX(i) log files is important when troubleshooting issues within the virtual environment. You can view and search log files in ESX(i) and in vCenter Server using a few different methods.
Using the vSphere client
The direct console user interface (DCUI)
A web browser
A syslog or vMA appliance
An SSH connection to the host
PowerCLI using the Get-Log command
When using SSH, use the following commands to view and search the log files.
Use more to page through the log files one page at a time
Use tail to view the end of the log files
Use grep to search
Use pipe | to link commands together
Use pipe | to grep to search through files
Use cat to concatenate & use grep to search
Use find | print | grep <filename> to search for a file
Example
Cat hostd.log | grep <search variable> | more
vCenter log files are in a vpxd-xx.log format where xx is a numerical value that increases when each log file is 5MB in size.
The log file numbers rotate when the vpxd service is started or when the log reaches 5MB in size.
The log files are located in the c:\programdata\VMware\VMware virtual center\logs
Other log files include
Vpxd-alert-x.log
Vpxd-profile-x.log
/var/log/VMkernel - VMkernel messages
/var/log/vmkwarning
/var/log/vmksummary
/var/log/vmware/hostd.log
/var/log/messages - service console
/var/log/vmware/vpx/vpxa.log - vSphere client agent
/var/log/aam/VMware__xxx.log - HA
/var/log/vmkiscsid.log - iSCSI
/var/log/boot-logs/sysboot.log - boot log
/var/log/messages - combination of the VMkernel and vmkwarning
/var/log/vmware/hostd.log - host management service
/var/log/vmware/vpx/vpxa.log - vSphere client agent
/var/log/sysboot.log - boot log
/var/log/vmware/aam/vmware__xxx.log - HA
Within ESX(i), rotation for most log files is controlled by /etc/logrotate.conf
To view available options run man logrotate
With both ESX and ESX(i) hostd.log rotation is controlled with /etc/vmware/hostd/config.xml
vpxa.log rotation is controlled with /etc/opt/VMware/vpxa/vpxa.cfg
Should you wish to edit the rotation control files you can use Nano in ESX or vi in ESX and ESXi.
I will focus on vi as I am more familiar with it and with the arrival of vSphere 5 there will no longer be ESX, and as such no native support for Nano.
a - append
i - insert
O/o - open new line - O is line above, o is line below
r - replace
: - search or save options
/ - search
wq - write and quit
x - delete individual characters
dd - delete line
$ - go to the end of the line
ESC - break out of current mode
Log bundles can be accessed through the VMware folder on the start menu, by clicking generate vCenter server log bundle. This runs the vc-support windows scripting file located at c:\program filesVMware\virtual infrastructure\virtual centrescript\svc-support.wsf and cscript.
You can also download it through the vSphere client and by connecting to the ESX(I) server using scp with Veeam FastSCP or WinSCP. To do this you have to enable tech support mode first.
An alternative way of generating log bundles is through the vm-support command run through an SSH connection to the COS or through the vMA. Running vm-support will generate a tar compressed file.
[root@esxhost]# /usr/bin/vm-support
With ESXi it is possible to place log files on shared storage. To set this open the vSphere client connection to the host, click configuration>advanced settings>syslog select local and enter the path to the shared storage. Enter the log file location as [datastorename]/logfiles/hostname.log.
vilogd is a service that performs log collections.
You can manage it with the vilogger commands. vilogger is used to enable and disable or configure the log collections with these commands.
To use vilogger, first ensure that vi-fastpass is enabled using vifp list server to list out the current vi-fastpass enabled servers, if no servers are listed use vifp addserver <servername> and vifptarget -s <servername> to add again.
vilogger enable
vilogger list
vilogger update policy
Control the vilogd service with etc/init.d/vmware-vilogd <start|stop|restart>
vilogger has several parameters available, an example of which are
--numrotation number of files to collect
--maxfilesize specified in MB
--collectionperiod how often to poll, specified in seconds
Example
vilogger enable --server <servername> --numrotation 20 --maxfilesize 10 --collectionperiod 10
This command will collect the following logs from the ESXi host
hostd.log
messages.log
vpxa.log
To scroll through the log files one page at a time use the more command.
Example
more hostd.log
You can configure the vMA as a syslog receiver to collect log files from the ESX and ESXi server. Run the commands listed below to configure.
sudo service rsyslog stop
sudo nano /etc/sysconfig/rsyslog
This will open nano so you can edit the following information
change SYSLOGD_OPTIONS="-m 0" to SYSLOGD_OPTIONS="-r -m 0"
Save and exit the file
sudo service rsyslog start
sudo iptables -I INPUT -i eth0 -p udp --dport 514 -j ACCEPT
sudo nano /etc/rc.local
Edit the file to add the iptables line below to the end of the rc.local file
iptables -I INPUT -i eth0 -p udp --dport 514 -j ACCEPT
To configure ESX to use vMA as a syslog server add the IP address of the vMA to the /etc/syslog.conf file.
vi /etc/syslog.conf
Add the following lines to the bottom of the file
*.* @<IP_Address_Of_vMA>
Open the firewall with
/usr/sbin/esxcfg-firewall -o 514,udp,out,syslog
Finally restart the syslog service with
sbin/services syslog restart
Use the vSphere client by going to configuration>advanced>syslog settings and enter into the syslog.Remote.Hostname section the name of the vMA.
Alternatively assuming vi-fastpass is enabled run
vifptarget -s [<ESXihost>]
vicfg-syslog -s [<vMA>]
vifptarget -c
Tagged with: vSphere Command Line
Comments are closed for this post.