Transparent fileservices for Windows, Unix and Mac

Leveraging ProLiant Storage Servers and Enterprise Virtual Array together

with Windows Storage Server, ExtremeZ-IP and Cluster Extension EVA

Monday, 10-Nov-08 Heinz-Hermann Adam (adamh@uni-muenster.de)

Agenda

•  Who we are and what we do •  Initial Situation •  Goal •  Components and Challenges •  Implemented Solution •  Migration Process •  Status of Operation

Who we are

•  WWU Münster is one of the three major universities in Germany –  ~ 40,000 students –  ~ 5,000 scientists and

staff –  Over 100 fields of

study

•  The Natural Sciences Department is ~¼ of the university –  Major user and

provider of compute resources

–  IT is a Volunteer driven operation

•  Not much dedicated staff

What we do

•  Provide and maintain resources for students, scientists and staff in Biology, Chemistry and Physics –  ~ 4,000 Computers –  ~ 12,000 Users

•  Compute resources –  Scientific Computing

•  SMP and Clusters •  Development

Environment

–  Desktop Applications •  Windows •  Linux •  Mac OS

–  File and Print Sharing

Initial Situation 2005/2006

•  Replacement of IT Infrastructure in Operation since 1998: Overdue

•  Isolated Data-Silos of Direct Attached Storage –  OpenVMS –  Windows –  Tru64 UNIX –  Linux

•  Changed Focus Platforms

Goal

•  Consolidation –  Versatile Storage System

•  Storage Capacity •  Data Protection •  Reliability, Availabilty, Fault-Tolerance

–  Highly Available Fileservice •  Transparent to client operating systems

–  Unified Computersystem •  Scientific Computing ( HPC) •  Infrastructure Services (Active Directory etc.)

–  Manpower •  Data Pools

•  OpenVMS 7.3-2 Cluster running Advanced Server 7.3A ECO-4 (Pathworks)

•  Transparent Filesystems –  OpenVMS –  Windows

•  Several Windows based Fileservers

A first step (2005) – a proof-of-principle

Prior to 2005 Beginning in 2005 •  ProLiant Storage Server

Cluster attached to an EVA 3000 storage array

•  Transparent Filesystems –  Windows –  Linux

•  Single Windows Storage Server based NAS-Cluster

Architectural move in 2005

before 2005 Conception

Second step (2006) – maturing the solution

•  Two „independent“ sites

•  More storage –  Mirroring of essential

file systems

•  Larger NAS-System –  Performance –  Availability

Components of the Solution

•  Microsoft Active Directory •  Windows Server 2003 R2/

Microsoft Services for Unix •  Windows Storage Server 2003 R2 Cluster •  Continous Access & Cluster Extension EVA •  Linux and Samba 3 •  Grouplogic ExtremeZ-IP

Active Directory

•  X.509 based Directory Service with an extensible Schema –  Can hold information not only for Windows, but also

for e.g. Unix/Linux users, groups and computers

•  Windows Server 2003 R2 or Microsoft Services for Unix Schema extension necessary –  Forest-wide operation

•  Leverages industry standard LDAP and Kerberos protocol

User management for non-Windows platforms

•  Linux/Unix –  Pluggable Authentication Module

•  Uses Kerberos

–  Name Service Switch •  Uses LDAP

•  Macintosh –  Open Directory Framework

•  Uses LDAP and Kerberos

–  Unix-based

Windows Server 2003 R2/Microsoft Services for Unix

•  Schema and Userinterface Extension on Domain Controllers

•  Server for NFS on Fileservers (NAS) –  Exports Windows Directories as „Network File

System“

Schema extension

Users •  msSFU30NisDomain

–  No need for NIS on Windows

•  msSFU30UidNumber •  msSFU30LoginShell •  msSFU30HomeDirectory •  msSFUGidNumber

–  Primary Group

Groups •  msSFU30NisDomain

–  No need for NIS on Windows

•  msSFU30GidNumber •  msSFU30PosixMember

–  Beware the storage limitation for an Active Directory attribute/object

Windows Storage Server Cluster

•  Microsoft Cluster Service •  Consists of Cluster

Groups (= „virtual Servers“) –  Default Cluster Group

•  Contains Quorum ressource –  Additional Groups for

production Resources •  One per node in the cluster •  Disks, Shares, VSS Tasks

–  Loadbalancing –  Faulttolerance

Windows Storage Server Cluster

•  No real (active-active) cluster –  Failover cluster

•  No load balancing –  Static load distribution

between nodes, based on cluster group configuration

–  One cluster group per cluster member

Continous Access & Clusterextension EVA

•  Stretched cluster –  Two SAN connected locations

•  Continous Access –  Synchronous writes to mirrored Vdisks on

both EVAs •  If connection between EVAs is broken,

changes are logged •  After re-establishing connection, changes

are commited to remote EVA

•  Quorum –  Odd number of nodes in the cluster and

at a minimum a third location •  Majority node set cluster

•  Clusterextension –  Failover between EVAs at different sites –  Automatic, no operator intervention

required

Clusterextension EVA

•  Resource in MSCS –  One per cluster group –  Talks to EVA Storage

Management Appliance (one per EVA required)

–  Cluster node only talks to EVA local to its site

–  SMA changes Vdisk presentation etc. automatically upon Offline and Online Operation of the CLX resource specific to a certain Cluster node

Multi-Protocol Challenges – Part I

•  Access for Unix Servers –  NFS on ACL secured VLANs

•  Access for Unix Clients –  NFS no option for Clients (No File

Security) –  CIFS (native Windows

Implementation) •  No support for special files, e.g.

sockets •  Limitation to allowed characters in a

file name, e.g. „:“ •  Filesystem behaviour prevents

some „features“, e.g. start of a KDE session

–  CIFS (SaMBa/Linux Implementation)

•  Linux Server mounts file systems via NFS and re-shares them via Samba 3

Server for NFS on Fileservers

•  File Name Handling –  Allows otherwise impossible file names

•  Unix: .DCOPserver_myhost_:0 •  Windows: .DCOPserver_myhost_20 •  C:\SFU\common\__Translate__NFS_File_Names__.txt •  0x00 0x3a : 0x00 0xb2 ; replace client : with 2 on server

–  NFS created files beginning with a „.“ are hidden files on Windows as well (via the DOS hidden flag)

•  For multi-protocoll access, e.g. sharing a directory simultaneously to Windows and NFS clients –  Microsoft Knowledge Base Article 321049

•  HKLM\Software\Microsoft\Server forNFS\Current Version\Mapping –  KeepInheritance = 1

•  Otherwise NFS created files and folders do not inherit NTFS ACLs from parent directories, rendering the inaccesible from Windows

–  E.g. for Backup applications

Multi-Protocol Challenges

•  Samba in Active Directory –  Security = ADS

•  Import Windows Shares via NFS –  Windowscluster:/home /homes nfs auto 0 0

•  Export Windows Share via Samba –  [homes]

•  Browseable = no •  Writeable = yes

–  Unix extensions = yes –  Mangled names = no

Multi-Protocol Challenges – Part II

•  Access for Macintosh Clients –  Compatibility Issues with CIFS Client on Mac OS X (file system

semantics) –  Microsoft Services for Macintosh

•  Provide Apple Filing Protocol access to Windows files and directories

•  Not cluster-aware –  Manual Procedure (generic script cluster resource) takes more than two hours

to bring AFP shares online •  Do not scale well

–  Limited to 2.9 million files or 1.6 million directories combined on all AFP volumes shared

–  Only achievable with SFM having the systems paged pool on its own •  Ancient software, introduced with NT 3.x

–  No longer maintained –  Discontinued in Windows Server 2008

GroupLogic ExtremeZ-IP

•  Native Apple Filing Protocol 3.1 Implementation on Windows –  TCP/IP, no need for AppleTalk –  Microsoft Cluster Service aware –  Transparent to failover within the cluster –  Kerberos support

•  Does everything Microsoft Services for Macintosh should do –  And more (e.g. TimeMachine support) –  Dfs support comming soon

Status of Installation

Moving the data from VMS to Windows

•  4 user disks as a VMS searchlist –  Disk$user_f, disk$user_k, disk$user_r, disk$user_z

•  Analyzing current usage and size –  5,000 – 6,000 users –  100 MB diskquota –  Overcommitting

•  Planning (2005) for –  7,000+ users (currently ~12,000) –  650 MB diskquota (currently 2-10 GB) –  Overcommitting

•  Microsoft Dfs helps a lot, if you have it in place beforehand

Moving data from VMS to Windows

•  Data transfer Advanced Server Storage Server –  Robocopy

•  Copying ISAM/indexed files (e.g. mail.mail) may crash Pathworks •  Exclude from copying, they are not useful under Windows , Linux or

Mac anyhow

–  Multi-stage copying •  Full copy

–  Test all services with production data –  Have some guinea pigs

•  Incremental copy –  Update changes from production system, after successfull test –  Switch users to the new system

–  Adjust Distributed Filesystem and User accounts

Our Way to Data Pools

•  Versatile Storage System –  1 GB units

•  All Servers connected to the SAN

•  NAS-Cluster for Filesharing •  Partitionable SMP Shared

Memory System –  Itanium2

–  2-24 CPU •  Bladesystem

–  X86-64 –  VMware Virtual Infrastructure

Q&A – Questions? Please!