Troubleshoot Windows File System Permissions for your Isilon Cluster

1 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot Windows File System Permissions

for your Isilon Cluster

For links to all Isilon customer troubleshooting guides, visit the Customer Troubleshooting - Isilon Info Hub.

We appreciate your help in improving this document. Submit your feedback at http://bit.ly/isi-docfeedback._________________

___________________________

Abstract

This guide will help you to troubleshoot problems with gaining access to the Isilon cluster.

December 5, 2017

EMC ISILON CUSTOMER TROUBLESHOOTING GUIDE

TROUBLESHOOT WINDOWS FILE SYSTEM PERMISSIONS FOR YOUR ISILON CLUSTER

OneFS 7.2.0 - 8.0.0

http://bit.ly/isi-docfeedback


https://community.emc.com/docs/DOC-49017





___________________________

Contents and overview

Page 3 Before you begin

Appendix A If you need further assistance

Note Follow all of these steps, in order, until you reach a resolution.

1. Follow these

steps.

2. Perform

troubleshooting

steps in order.

3. Appendixes

Appendix B How to use this flowchart

Page 4 Start troubleshooting

Page 5 Authentication provider status

Page 6 Protocol

Page 7 SMB protocol

Page 9 Multiprotocol

Page 12 Missing permissions

Page 13 Mismatched permissions

Page 15 Matching permissions

Page 19 NFS protocol

0 NFS - Map lookup UID

Page 21 NFS - Resolve user's UID

Page 22 NFSv4 - Domain names

Appendix C Example isi smb shares view --share=<share> --zone=<zone>

output

Appendix D Example isi auth mapping token --zone<zone>

--user="<domain>\<user>" output

Appendix E Example isi_run -z <zoneID> "ls -led/lend <basefolder>" output

Appendix F Examples of permissions

Appendix G Commands to create or modify permissions








___________________________

Configure screen logging through SSH

We recommend that you configure screen logging to log all session input and output during your troubleshooting session.

This log file can be shared with Isilon Technical Support, if you require assistance at any point during troubleshooting.

Note: The screen session capability does not work in OneFS 7.1.0.6 and 7.1.1.2. If you are running either of these versions,

you can configure logging by using your local SSH client's logging feature.

1. Open an SSH connection to the cluster and log in by using the root account .

Note: If the cluster is in compliance mode, use the compadmin account to log in. All compadmin commands must be

preceded by the sudo prefix.

2. Change the directory to /ifs/data/Isilon_Support by running the following command:

cd /ifs/data/Isilon_Support

3. Run the following command to capture all input and output from the session:

screen -L

This will create a file named screenlog.0 that will be appended to during your session.

4. Perform troubleshooting.

Before you begin

CAUTION!If the node, subnet, or pool that you are working on goes down during the course of

troubleshooting and you do not have any other way to connect to the cluster, you could

experience data unavailability.

Therefore, make sure that you have more than one way to connect to the cluster before

you start this troubleshooting process. The best method is to have a serial console

connection available. This way, if you are unable to connect through the network, you

will still be able to connect to the cluster physically.

For specific requirements and instructions for making a physical connection to the

cluster, see article 304071 on the EMC Online Support site.

Before you begin troubleshooting, confirm that you can connect through either another

subnet or pool, or that you have physical access to the cluster.




https://support.emc.com/kb/304071





___________________________

Start troubleshooting

IntroductionStart troubleshooting here. For an overview

of the conventions used in this flowchart, see

Appendix B: How to use this flowchart.

Check the status of all authentication providers by

running the following command:

isi auth status

See example output at the bottom of this page.

Start

If you have not done so already, log in to

the cluster and configure screen logging

through SSH, as described on page 3.

Are any authentication

providers reporting

as offline?

Go to Page 6Go to Page 5

Yes No

Example isi auth status outputCluster-1# isi auth status

ID Active Server Status

-------------------------------------------------

lsa-local-provider:System - active

lsa-local-provider:ZONE2 - active

lsa-file-provider:System - active

lsa-ldap-provider:LDAPTest - online

lsa-nis-provider:NIStest - offline

lsa-ads-provider:ADtest - online

-------------------------------------------------

Total: 5








___________________________

Authentication provider status

Which authentication providers

are reporting as offline?

Page

5

You could have arrived here from:

Page 4 - Start troubleshooting

LDAPActive Directory NIS

Go to: EMC Isilon Customer

Troubleshooting Guide:

Troubleshoot Windows Active

Directory Authentication



Troubleshoot Your LDAP

Authentication Provider



Troubleshoot Problems with

your NIS Authentication

Provider




http://www.emc.com/collateral/TechnicalDocument/docu63151.pdf







___________________________

Protocol

Page

6


Page 4 - Start troubleshooting

Which protocol

is in use?

Note the page number that you

are currently on.

Upload log files and contact Isilon Technical

Support, as instructed in Appendix A.

Other

Go to Page 19

NFSSMB

Go to Page 7








___________________________

SMB protocol

Page

7


Page 6 - Protocol

Page 20 - NFS - Map lookup UID

Page 21 - NFS - Resolve user's UID

Check the SMB share permissions by running the following command, where

<share> is the share name, and <zone> is the zone name:

isi smb shares view --share="<share>" --zone="<zone>"

See Appendix C for example output.

Is the user or

group in question

listed with read

permissions?

Grant the user or group

read permissions.

See Appendix G for

commands.

No

Does the user

require write

permissions?

Yes

Grant the user write

permissions. See

Appendix G for

commands.

Yes

Go to Page 8

No

Gather the user's token by running the following command, where:

<zone> is the name of the zone.

<domain> is the name of the domain.

<user> is the name of the user.

isi auth mapping token --zone=<zone> --user="<domain>\<user>"

See Appendix D for example output.

__________

__________

_______________

___________________________

______________________________








___________________________

SMB protocol (2)

Page

8


Page 7 - SMB protocol

Find the zone ID by running the following command, where <zone>

is the name of the zone:

isi zone zones view <zone>

See the example output at the bottom of this page.

Example isi zone zones view <zone> outputCluster-1# isi zone zones view System

Name: System

Path: /ifs

Cache Size: 9.54M

Map Untrusted:

Auth Providers: -

NetBIOS Name:

All Auth Providers: Yes

User Mapping Rules: -

Home Directory Umask: 0077

Skeleton Directory: /usr/share/skel

Audit Success: create, delete, rename, set_security, close

Audit Failure: create, delete, rename, set_security, close

HDFS Authentication: all

HDFS Keytab: /etc/hdfs.keytab

HDFS Root Directory: /ifs

WebHDFS Enabled: Yes

Syslog Forwarding Enabled: No

Syslog Audit Events: create, delete, rename, set_security

Zone ID: 1

Go to Page 9








___________________________

Multiprotocol

Does the output show any

User Mapping Rules?

See the example output at the

bottom of this page.

Page

9


Page 8 - SMB protocol (2)


are currently on.



Yes

Go to Page 10

No

Example isi zone zones view <zone> outputCluster-1# isi zone zones view System

Name: System

Path: /ifs

Cache Size: 9.54M

Map Untrusted:

Auth Providers: -

NetBIOS Name:

All Auth Providers: Yes

User Mapping Rules: -

Home Directory Umask: 0077

Skeleton Directory: /usr/share/skel

Audit Success: create, delete, rename, set_security, close

Audit Failure: create, delete, rename, set_security, close

HDFS Authentication: all

HDFS Keytab: /etc/hdfs.keytab

HDFS Root Directory: /ifs

WebHDFS Enabled: Yes

Syslog Forwarding Enabled: No

Syslog Audit Events: create, delete, rename, set_security

Zone ID: 1




10 - EMC Isilon Customer Troubleshooting Guide: Troubleshoot Windows File System

Permissions for your Isilon Cluster



___________________________

Multiprotocol (2)

Page

10


Page 9 - Multiprotocol

Compare the user token to the on-disk permissions of the file and all

parent files up to /ifs. Start at the problematic file and run the following

commands one time for each file or folder in the tree, starting with the

base folder, where <zoneID> is the zone ID, and <basefolder> is

the base folder for the share or export:

isi_run -z <zoneID> "ls -led <basefolder>"

isi_run -z <zoneID> "ls -lend <basefolder>"

See Appendix E for example output for both commands.

Note The ls -led command lists names

and the ls -lend command lists the

stored UID/GID/SID identities. When

comparing the ls -led and

ls -lend output to the user token,

ls -led can help you to identify the

names, and ls -lend can help you

to verify that the stored identities

numerical representations (GID or

SID) are correct. Comparing names to

numerical identities ensures that you

are dealing with the correct users and

groups.

Did you get the error

Unable to read

security descriptor?


are currently on.



Yes

Go to Page 11No

__________








___________________________

Multiprotocol (3)

Page

11


Page 10 - Multiprotocol (2)

Compare the output of the isi auth mapping token --zone=<zone> --user="<domain>\<user>"

command (Appendix D) to the output of the isi_run -z <zoneID> "ls -led/lend <basefolder>"

commands (Appendix E). The ls -led and ls -lend output should match the same group in the user

token. Specifically, compare the user name, SID and GID returned.

See Appendix F for example output and explanation of mismatched permissions.

Are the expected

permissions missing,

mismatched, or matching?

Missing Mismatched Matching

Go to Page 12 Go to Page 13 Go to Page 15

_________

_________

_________








___________________________

Missing permissions

Page

12

Missing



Add the missing access control lists (ACLs) by using your preferred method (for

example, Windows Explorer) and retest the connection.

For more information, refer to the Microsoft article: File and Folder Permissions.

Has the original issue

been resolved?


are currently on.



No

End troubleshootingYes




https://msdn.microsoft.com/en-us/library/bb727008.aspx





___________________________

Mismatched permissions

Page

13

Mismatched



Are you using

SID history?

Go to:EMC Isilon Customer


Troubleshoot Identity MappingNo

Make sure that the SID on the file matches

the primary SID that the user token shows.

Yes

Do they

match?


are currently on.



Yes

No

Go to Page 14









___________________________

Mismatched permissions (2)

We do not support SID history.

Adjust the data permissions to add the user or

group's primary SIDs and retest the connection.

Page

14


Page 13 - Mismatched permissions

Has the original

issue been

resolved?

End troubleshooting


are currently on.



No

Yes








___________________________

Matching permissions

Page

15

Matching



Is the group in

question a domain

local group?

Is the cluster joined

directly to the domain

where the domain local

group resides?

Yes

Go to Page 17No

Go to Page 16Yes

No

Note For more information, see

the Microsoft article:

Group scope.

Domain local groups work only in the domain where they were

created. Reevaluate your permissions and access model to

include domain local groups, or ensure the cluster is joined to the

domain where the domain local group was created.

End troubleshooting




https://technet.microsoft.com/en-us/library/cc755692(v=ws.10).aspx





___________________________

Multiprotocol (4)

Page

16


Page 15 - Matching permissions

Are you connecting

through the access zone

in which the domain

local group resides?

Go to Page 17Yes

Please try to connect through that

access zone and retest the connection.

No

Can you

connect now?Go to Page 17No

Has your original

issue been resolved?

Yes

End troubleshooting

Yes

Go to Page 17No








___________________________

Multiprotocol (5)

Page

17


Page 15 - Matching permissions


Do the user token and

the ls -led or ls -lend

output match?

See Appendix E for

example output.



Troubleshoot Windows Active

Directory Authentication

Yes

Identify which group or user should

be listed on the file permissions.

No

Add the missing ACLs by using your preferred

method (for example, Windows Explorer) and retest

the connection.

For more information, refer to the Microsoft article:

File and Folder Permissions.

Go to Page 18

__________________________

_____________________

__________





https://msdn.microsoft.com/en-us/library/bb727008.aspx





___________________________

Multiprotocol (6)

Note For more information, see:

Identities, Access Tokens, and the

Isilon OneFS User Mapping Service.

Page

18



Recheck file or folder permissions

to verify that the outputs from the

user token, the ls -led and

ls -lend match.

Do they

match?



Troubleshoot Identity MappingNo

Retest the user's access

to the file or folder.

Yes

Has the original

issue been

resolved?


are currently on.



No

End troubleshootingYes




https://support.emc.com/docu50075






___________________________

NFS protocol

Page

19


Page 6 - Protocol

Is the export mounted

on the client?


are currently on.



No

Are you using NFSv3

or NFSv4?

Yes

Go to Page 22

Go to Page 20

NFSv4

NFSv3








___________________________

NFS - Map lookup UID

Page

20


Page 19 - NFS protocol

Page 22 - NFSv4 - Domain names

Verify that Map Lookup UID setting is enabled by running the

following command, where <export> is the ID of the export:

isi nfs exports view <export> | egrep -i "lookup"

See the box at the bottom of this page for example output.

Example isi nfs exports view <export> | egrep -i "lookup" output

Cluster-1# isi nfs exports view 1 | egrep -i "lookup"

Map Lookup UID: Yes

According to the output, is

Map Lookup UID setting

enabled?

Go to Page 21

Return to Page 7

Yes No

From the client machine, collect the user's

UID, primary GID, and supplemental GIDs.

Typically, this is done by running the id

command. Your distribution of Linux, UNIX, or

FreeBSD may or may not have this command.

___________________

____________________________

Perform another lookup of

the user token.








___________________________

NFS - Resolve user's UID


Page 20 - NFS - Map lookup UID

Page

21

Try to resolve the user's UID by running the following command,

where <uid> is the user's UID, and <zone> is the zone name:

isi auth mapping token --uid=<uid> --zone=<zone>

Note In versions of OneFS prior to 7.2,

NFSv3 works only in the System

zone. If you are trying to access a

zone other than the System zone,

consider adjusting your workflow or

upgrading to OneFS 7.2 to gain that

feature.

Did the user's

UID resolve?

Does a user with this UID

exist in one of the

authentication providers?

No

Return to Page 7Yes


are currently on.



YesEnd troubleshooting

NoThis is expected behavior. If the user

does not exist in the authentication

provider, access will be denied.








___________________________

NFSv4 - Domain names

Page

22


Page 19 - NFS protocol

On the server, make sure that the NFSv4 domain

name is in the correct case.

On the client, make sure that the domain name is

defined correctly.

Example paths:

Linux: /etc/dmap.conf

Solaris: /etc/default/nfs

Verify that the NFSv4 domain name and client

domain name match. These domain names are

case sensitive.

Do the NFSv4 domain

name and client domain

names match?

Return to Page 20

Return to Page 20

No

Yes

Adjust the names so that they match exactly.








___________________________

Contact Isilon Technical Support

If you need to contact Isilon Technical Support during troubleshooting, reference the page or step that you need help with.

This information and the log file will help Isilon Technical Support staff resolve your case more quickly.

Appendix A: If you need further assistance

Upload node log files and the screen log file to Isilon Technical Support

1. When troubleshooting is complete, in the command-line interface, type exit to end your screen session.

2. Gather and upload the node log set and include the SSH screen log file by using the command appropriate for your

method of uploading files. If you are not sure which method to use, use FTP.

ESRS:

isi_gather_info --esrs --local-only -f /ifs/data/Isilon_Support/screenlog.0

FTP:

isi_gather_info --ftp --local-only -f /ifs/data/Isilon_Support/screenlog.0

HTTP:

isi_gather_info --http --local-only -f /ifs/data/Isilon_Support/screenlog.0

SMTP:

isi_gather_info --email --local-only -f /ifs/data/Isilon_Support/screenlog.0

SupportIQ:

Copy and paste the following command.

Note: When you copy and paste the command into the command-line interface, it will appear on multiple lines (exactly

as it appears on the page), but when you press Enter, the command will run as it should.

isi_gather_info --local-only -f /ifs/data/Isilon_Support/screenlog.0 --noupload \

--symlink /var/crash/SupportIQ/upload/ftp

3. If you receive a message that the upload was unsuccessful , refer to article 304567 on the EMC Online Support site for

directions on how to upload files over FTP.

___________




http://www.emc.com/contact/contact-us.htm

https://support.emc.com/kb/304567





___________________________

Decision diamondYes No

Process stepProcess step with command:

command xyz

Go to Page #

Page

# Note Provides context and additional

information. Sometimes a note is linked

to a process step with a colored dot.

CAUTION!Caution boxes warn that

a particular step needs

to be performed with

great care, to prevent

serious consequences.

End point Document ShapeCalls out supporting documentation

for a process step. When possible,

these shapes contain links to the

reference document.

Sometimes linked to a process step

with a colored dot.

Optional process step

Directional arrows indicate

the path through the

process flow.

IntroductionDescribes what the section helps you to

accomplish.


Page 4 - Start Troubleshooting

Appendix B: How to use this flowchart








___________________________

Appendix C: Example output

Example isi smb shares view --share=<share> --zone=<zone> output



Example isi smb shares view --share=<share> --zone=<zone> output

Cluster-1# isi smb shares view --share=testshare --zone=system

Share Name: testshare

Path: /ifs/home

Description:

Client-side Caching Policy: manual

Automatically expand user names or domain names: False

Automatically create home directories for users: False

Browsable: True

Permissions:

Account Account Type Run as Root Permission Type Permission

----------------------------------------------------------------

Everyone wellknown False allow read

TestUser wellknown False allow write

----------------------------------------------------------------

Total: 1

Access Based Enumeration: No

Access Based Enumeration Root Only: No

Allow Delete Readonly: No

Allow Execute Always: No

Change Notify: norecurse

Create Permissions: default acl

Directory Create Mask: 0700

Directory Create Mode: 0000

File Create Mask: 0700

File Create Mode: 0100

Hide Dot Files: No

Host ACL: -

Impersonate Guest: never

Impersonate User:

Mangle Byte Start: 0XED00

Mangle Map: 0x01-0x1F:-1, 0x22:-1, [snip]

Ntfs ACL Support: Yes

Oplocks: Yes

Strict Flush: Yes

Strict Locking: No








___________________________

Appendix D: Example output

Example isi auth mapping token --zone=<zone> --user="<domain>\<user>" output




Example isi auth mapping token --zone=<zone> --user="<domain>\<user>" outputCluster-1# isi auth mapping token --zone=System --user="TEST\testuser1"

User

Name: TEST\testuser1

UID: 3501

SID: S-1-5-21-377814043-3192668432-1337460308-1886

On Disk: 3501

ZID: 1

Zone: System

Privileges: -

Primary Group

Name: TEST\domain users

GID: 1000000

SID: S-1-5-21-377814043-319232-133708-513

On Disk: S-1-5-21-377814043-319232-133708-513

Supplemental Identities

Name: TEST\ad_group-1

GID: 1000001

SID: S-1-5-21-377814043-319232-1337460308-1887


GID: 1000002

SID: S-1-5-21-377814043-319232-1337460308-1888


GID: 1000003

SID: S-1-5-21-377814043-319232-1337460308-1889

Name: Users

GID: 1545

SID: S-1-5-32-545

Name: Authenticated Users

UID: -

GID: -

SID: S-1-5-11

Name: NIS_Group-2

GID: 3002

SID: S-1-22-2-3002

Name: NIS_Group-1

GID: 3001

SID: S-1-22-2-3001

Name: NIS_Group-3

GID: 3003

SID: S-1-22-2-3003

________________________________________








___________________________

Example isi_run -z <zoneID> "ls -led <basefolder>" output

Cluster-1# isi_run -z 1 "ls -led /ifs"

drwxrwxrwx 5 root wheel 65 Apr 21 12:01 /ifs

OWNER: user:root

GROUP: group:wheel

SYNTHETIC ACL

0: user:root allow

dir_gen_read,dir_gen_write,dir_gen_execute,std_write_dac,delete_child

1: group:wheel allow dir_gen_read,dir_gen_write,dir_gen_execute,delete_child

2: everyone allow dir_gen_read,dir_gen_write,dir_gen_execute,delete_child

Example isi_run -z <zoneID> "ls -lend <basefolder>" output

Cluster-1# isi_run -z 1 "ls -lend /ifs"

drwxrwxrwx 5 0 0 65 Apr 21 12:01 /ifs

OWNER: user:0

GROUP: group:0

SYNTHETIC ACL

0: user:0 allow dir_gen_read,dir_gen_write,dir_gen_execute,std_write_dac,delete_child

1: group:0 allow dir_gen_read,dir_gen_write,dir_gen_execute,delete_child

2: SID:S-1-1-0 allow dir_gen_read,dir_gen_write,dir_gen_execute,delete_child

Appendix E: Example output

Example isi_run -z <zoneID> "ls -led/lend <basefolder>" output





_____________________

_____________________

_____________________








___________________________

Appendix F: Examples of permissions

Example of a permission that should have matched but shows the wrong identity. In this example the user is: TEST\testuser1, the SID is S-1-5-21-4087762976-3323327-7495-1118, and the UID is 1001.

Cluster-1# ls -led multi

-rw-r--r-- 1 TEST\testuser1 wheel 0 Sep 4 15:41 multi

OWNER: user:TEST\testuser1

GROUP: group:wheel

SYNTHETIC ACL

0: user:TEST\testuser1 allow file_gen_read,file_gen_write,std_write_dac

1: group:wheel allow file_gen_read

2: everyone allow file_gen_read

TEST\testuser1 exists in AD and LDAP and this is the expected output:

Cluster-1# ls -lend multi

-rw-r--r-- 1 1001 0 0 Sep 4 15:41 multi

OWNER: user:1001

GROUP: group:0

SYNTHETIC ACL

0: user:1001 allow file_gen_read,file_gen_write,std_write_dac

1: group:0 allow file_gen_read

2: SID:S-1-1-0 allow file_gen_read

If the identities were not correctly matched, the output might look like this:


-rw-r--r-- 1 1001 0 0 Sep 4 15:41 multi

OWNER: SID:S-1-5-21-4087762976-3323327-7495-1118

GROUP: group:0

SYNTHETIC ACL

0: SID:S-1-5-21-4087762976-3323327-7495-1118 allow file_gen_read,file_gen_write,std_write_dac



Continued on next page.



________








___________________________

Appendix F: Examples of permissions (2)

Example of permissions that match but that give the wrong permissionsAn extra ACE has been added to the example on the previous page, giving users in TEST\testgroup1 read access. If the

expectation was that users in TEST\testgroup1 should be able to write or modify, then this is the wrong permission:

Cluster-1# ls -led multi

-rw-r--r-- + 1 TEST\testuser1 wheel 0 Sep 4 15:41 multi

OWNER: user:TEST\testuser1

GROUP: group:wheel

0: group:TEST\testgroup1 allow file_gen_read

1: user:TEST\testuser1 allow file_gen_read,std_write_dac

2: group:wheel allow file_gen_read

3: everyone allow file_gen_read


-rw-r--r-- + 1 1001 0 0 Sep 4 15:41 multi

OWNER: user:1001

GROUP: group:0


1: user:1001 allow file_gen_read,std_write_dac




Page 28 - Appendix F: Examples of permissions




30 - EMC Isilon Customer Troubleshooting Guide: [title]



___________________________

Appendix G: Commands to create or modify permissions



Commands to create or modify permissions for a user or a group

Create for a user:

isi smb shares permission create --share=<share> --user=<user> --permission-type=allow --permission-type=<read/change/full>

Create for a group:

isi smb shares permission create --share=<share> --group=<group> --permission-type=allow --permission-type=<read/change/full>

Modify for a user:

isi smb shares permission modify --share=<share> --user=<user> --permission-type=allow --permission-type=<read/change/full>

Modify for a group:

isi smb shares permission modify --share=<share> --group=<group> --permission-type=allow --permission-type=<read/change/full>








___________________________

Copyright © 2017 Dell Inc. or its subsidiaries. All rights reserved.

Dell believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS-IS. DELL MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. USE, COPYING, AND DISTRIBUTION OF ANY DELL SOFTWARE DESCRIBED IN THIS PUBLICATION REQUIRES AN APPLICABLE SOFTWARE LICENSE.

Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be the property of their respective owners.

EMC CorporationHopkinton, Massachusetts 01748-91031-508-435-1000 in North America 1-866-464-7381www.EMC.com




https://www.emc.com

Troubleshoot Windows File System Permissions for your Isilon Cluster

Documents

Transcript of Troubleshoot Windows File System Permissions for your Isilon Cluster