EMC Isilon Customer Troubleshooting Guide: Troubleshoot ... · Swap the front panels Is the front...
Transcript of EMC Isilon Customer Troubleshooting Guide: Troubleshoot ... · Swap the front panels Is the front...
1 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an
Isilon node
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________
___________________________
Abstract
This troubleshooting guide helps you to troubleshoot issues with the front panel of an Isilon node.
January 15, 2018
EMC ISILON CUSTOMER TROUBLESHOOTING GUIDE
TROUBLESHOOT ISSUES WITH THE FRONT PANEL OF AN ISILON NODE
OneFS 7.1 – 8.1
2 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an
Isilon node
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________
___________________________
Contents and overview
Before you begin
Page 3
Appendix A
If you need further assistance
Start troubleshooting
Page 4
Service lights
Page 5
Swap the front panels
Page 6
Restart the isi_lcd_d process
Page 9
Note Follow all of these steps, in order, until you reach a resolution.
1. Follow these
steps.
2. Perform
troubleshooting
steps in order.
3. Appendixes
Appendix B
How to use this flowchart
Restart the isi_lcd_d daemon
Page 10
Restart the isi_lcd_d process
Page 11
3 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an
Isilon node
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________
___________________________
Configure screen logging through SSH
We recommend that you configure screen logging to log all session input and output during your troubleshooting session.
This log file can be shared with EMC Isilon Technical Support, if you require assistance at any point during troubleshooting.
Note: The screen session capability does not work in OneFS 7.1.0.6 and 7.1.1.2. If you are running either of these versions,
configure logging by using the local SSH client logging feature.
1. Open an SSH connection to the cluster and log in by using the root account .
Note: If the cluster is in compliance mode, use the compadmin account to log in. All compadmin commands must be
preceded by the sudo prefix.
2. Change the directory to /ifs/data/Isilon_Support by running the following command:
cd /ifs/data/Isilon_Support
3. Run the following command to capture all input and output from the session:
screen -L
This will create a file named screenlog.0 that will be appended to during your session.
4. Perform troubleshooting.
Before you begin
CAUTION!If the node, subnet, or pool that you are working on goes down during the course of
troubleshooting and you do not have any other way to connect to the cluster , you could
experience data unavailability.
Therefore, make sure that you have more than one way to connect to the cluster before
you start this troubleshooting process. The best method is to have a serial cable
available. This way, if you are unable to connect through the network, you will still be
able to connect to the cluster physically.
For specific requirements and instructions for making a physical connection to the
cluster, see article 304071 on the EMC Online Support site.
Before you begin troubleshooting, confirm that you can connect through either another
subnet or pool, or that you have physical access to the cluster.
4 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an
Isilon node
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________
___________________________
Start troubleshooting
Remove the front panel from the
node, wait 30 seconds, and then
reseat the panel.
Start
IntroductionStart troubleshooting here. If you need
help to understand the flowchart
conventions that are used in this guide,
see Appendix B: How to use this
flowchart.
If you have not done so already, log in to
the cluster and configure screen logging
through SSH, as described on page 3.
Note For instructions on how to
remove a front panel, refer to the
Front Panel Replacement Guide
for your node type.
Is the front panel fully lit
and responsive after you
reseated it?
End troubleshooting
Go to Page 5
No
Yes
5 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an
Isilon node
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________
___________________________
Service lights
Is the Isilon logo on the
front of the unit flashing and is
the Unit ID light on the back of
the node illuminated?
Run the following command on the
node with flashing lights for your
version of OneFS:
OneFS 8.0 – 8.1.0
isi servicelight disabled
OneFS 7.1 – 7.2
isi servicelight off
Yes
Go to Page 6
No
Page
5
You could have arrived here from:
Page 4 - Start troubleshooting
Is the front panel
fully lit and
responsive now?
End troubleshooting
Note the page number that you
are currently on.
Upload log files and contact Isilon Technical
Support, as instructed in Appendix A.
No
Yes
6 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an
Isilon node
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________
___________________________
Swap the front panels
Is the front panel
functioning, except for
the decorative
lighting?
Remove a fully functioning front panel
from another node and install the
unresponsive front panel in its place.
No Yes
Page
6
You could have arrived here from:
Page 5 - Service lights
Is the
unresponsive front
panel lit and
responsive on the
new node?
No
Yes
Contact Support for a replacement panel.
Note the page number that you
are currently on.
Upload log files and contact Isilon Technical
Support, as instructed in Appendix A.
Go to Page 7
Note Decorative lighting is the LED strip below the
Isilon logo.
This indicates that the panel is
bad. Contact Isilon Technical
Support, as instructed in
Appendix A, to order a
replacement panel and continue
troubleshooting.
7 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an
Isilon node
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________
___________________________
Swap the front panels (2)
Install the known good front panel
from the responsive node on the
problematic node.
Page
7
You could have arrived here from:
Page 6 - Swap the panels
Is the known good front
panel lit and responsive on
the problematic node?
No
Yes
Go to Page 8
End troubleshooting
8 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an
Isilon node
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________
___________________________
Swap the front panels (3)
Page
8
You could have arrived here from:
Page 7 - Swap the panels (2)
Is the problem related to
the LCD control panel or
the alert LED?
No
Yes
Is the problem related to
one of the other LEDs?
Note the page number that you
are currently on.
Upload log files and contact Isilon Technical
Support, as instructed in Appendix A.
No
Go to Page 9Yes
Go to Page 10
9 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an
Isilon node
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________
___________________________
Restart the isi_lcd_d daemon
Page
9
You could have arrived here from:
Page 8 - Swap the panels (3)
Open an SSH connection on the node
with the unresponsive front panel and
log in by using the root account.
Run the following command:
ps aux | grep isi_lcd_d
Does the output
look similar to the example
output shown on this
page?
Example ps aux | grep isi_lcd_d output
ps aux | grep isi_lcd_d
10141 2.0 0.1 28428 22352 ?? S 6:25PM 0:00.23 isi_lcd_d: manager (isi_lcd_d)
10140 0.0 0.1 22908 16656 ?? S 6:25PM 0:00.02 isi_lcd_d: server (isi_lcd_d)
10146 0.0 0.1 22908 16536 ?? S 6:25PM 0:00.00 isi_lcd_d: buttons (isi_lcd_d)
Go to Page 11
Go to Page 10
Yes
No
Restart the isi_lcd_d daemon by
running the following command:
isi_lcd_d start
Make note of the column values in bold
text. These are the process IDs (PIDs)
and you will need them for the next steps.
10 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an
Isilon node
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________
___________________________
Restart the isi_lcd_d process
Page
10 You could have arrived here from:
Page 9 - Restart the daemon
Did the PIDs change from
when you ran the command
on the previous page?
Yes
No
Reboot the node:
reboot
Restart the isi_lcd_d process by
running the following command:
killall isi_lcd_d
Run the following command:
ps aux | grep isi_lcd_d
Compare the current PIDs with the PIDs from the output gathered on page 10.______
Return to Page 4
This indicates that the isi_lcd_d
process was successfully restarted.
Are the lights on
the front panel
fully functional?
No
End troubleshooting Yes
This indicates that the
isi_lcd_d process
might be hung.
11 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an
Isilon node
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________
___________________________
Restart the isi_lcd_d process (2)
Page
11
You could have arrived here from:
Page 10 - Restart the daemon
Run the following command:
ps aux | grep isi_lcd_d
Does the output
look similar to the
example output shown
on this page?
Example ps aux | grep isi_lcd_d output
ps aux | grep isi_lcd_d
10141 2.0 0.1 28428 22352 ?? S 6:25PM 0:00.23 isi_lcd_d: manager (isi_lcd_d)
10140 0.0 0.1 22908 16656 ?? S 6:25PM 0:00.02 isi_lcd_d: server (isi_lcd_d)
10146 0.0 0.1 22908 16536 ?? S 6:25PM 0:00.00 isi_lcd_d: buttons (isi_lcd_d)
Reboot the node:
reboot
No
Is the front panel
fully lit and
responsive now?Yes
End
troubleshootingYes
No
Return to Page 4
12 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an
Isilon node
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________
___________________________
Contact EMC Isilon Technical Support
If you need to contact Isilon Technical Support during troubleshooting, reference the page or step that you need help with.
This information and the log file will help Isilon Technical Support staff resolve your case more quickly.
Appendix A: If you need further assistance
Upload node log files and the screen log file to EMC Isilon Technical Support
1. When troubleshooting is complete, type exit to end your screen session.
2. Gather and upload the node log set and include the SSH screen log file by using the command appropriate for your
method of uploading files. If you are not sure which method to use, use FTP.
ESRS:
isi_gather_info --esrs --local-only -f /ifs/data/Isilon_Support/screenlog.0
FTP:
isi_gather_info --ftp --local-only -f /ifs/data/Isilon_Support/screenlog.0
HTTP:
isi_gather_info --http --local-only -f /ifs/data/Isilon_Support/screenlog.0
SMTP:
isi_gather_info --email --local-only -f /ifs/data/Isilon_Support/screenlog.0
SupportIQ:
Copy and paste the following command.
Note: When you copy and paste the command into the command-line interface, it will appear on multiple lines (exactly
as it appears on the page), but when you press Enter, the command will run as it should.
isi_gather_info --local-only -f /ifs/data/Isilon_Support/screenlog.0 --noupload \
--symlink /var/crash/SupportIQ/upload/ftp
3. If you receive a message that the upload was unsuccessful , refer to article 16759 on the EMC Online Support site for
directions on how to upload files over FTP.___________
13 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an
Isilon node
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________
___________________________
Decision diamondYes No
Process stepProcess step with command:
command xyz
Go to Page #
Page
# Note Provides context and additional
information. Sometimes a note is linked
to a process step with a colored dot.
CAUTION!Caution boxes warn that
a particular step needs
to be performed with
great care, to prevent
serious consequences.
End point Document ShapeCalls out supporting documentation
for a process step. When possible,
these shapes contain links to the
reference document.
Sometimes linked to a process step
with a colored dot.
Optional process step
Directional arrows indicate
the path through the
process flow.
IntroductionDescribes what the section helps you to
accomplish.
You could have arrived here from:
Page # - Page title
Appendix B: How to use this flowchart
14 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot issues with the front panel of an
Isilon node
For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.
We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________
___________________________
Copyright © 2018 Dell Inc. or its subsidiaries. All rights reserved.
Dell believes the information in this publication is accurate as of its publication date. The
information is subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS-IS. DELL MAKES NO
REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE
INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
USE, COPYING, AND DISTRIBUTION OF ANY DELL SOFTWARE DESCRIBED IN THIS
PUBLICATION REQUIRES AN APPLICABLE SOFTWARE LICENSE.
Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other
trademarks may be the property of their respective owners.
EMC Corporation
Hopkinton, Massachusetts 01748-9103
1-508-435-1000 in North America 1-866-464-7381
www.EMC.com