32-bit Virtual Memory Constraints in Windows: an Update
description
Transcript of 32-bit Virtual Memory Constraints in Windows: an Update
![Page 1: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/1.jpg)
Demand Technology Software, Inc.
32-bit Virtual Memory Constraints in
Windows: an UpdateMark FriedmanDemand Technology
1020 Eighth Avenue South, Suite 6, Naples, FL 34102 USA
phone: (239) 261-8945 fax: (239) 261-5456
e-mail: [email protected]://www.demandtech.com
![Page 2: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/2.jpg)
2Demand Technology, Inc. 32-bit VM Constraints
Outline Topics
Virtual Memory concepts
Virtual Memory constraints in 32-bit Windows
/3 GB option Physical Address
Extension (PAE) Address Windowing
Extensions (AWE) 64-bit relief
Vir t ualaddress
space
V ir t ualaddress
space
P hysical M em ory
Page Tables
![Page 3: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/3.jpg)
Intel x860100010011 0000011010 000001101100
IA -32 Virtual address translat ion
Page tableSelector
Page tableentry
byte offset
Control Register 3
Page Directory
Private
System
10000
7F F E F F F F
A 4000000
C1000000
E 0000000
F F F FF F F F
PTEs/ cache extension
System data cache
Paged system pool
(offset of per process Page Table)
Page Tables Physical pages
Paging file
(invalid)
80000000
C0000000
![Page 4: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/4.jpg)
4Demand Technology, Inc. 32-bit VM Constraints
Intel x86 in PAE mode
(ptr) Page Directory Index Page Table Index Byte Offset
Virtual Address Translation (PAE)
PDE
PageDirectory
PageDirectoryPointers
PDE
PDE
PDE
PDE
64-bit PTE
PageTable
RAM(Physical Memory)
CR3
31 29 20 011
![Page 5: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/5.jpg)
5Demand Technology, Inc. 32-bit VM Constraints
Virtual Memory Manager
4 GB virtual per process address space Lower 2 GB - Private
Page 0 reserved Code pages Heap
Upper 2 GB - System System code Shared dlls System cache
System
User
0
x'1 0000'16
No Access
x'8000 0000'16
x'ffff 0000'16
System Code
Device Driver Code
Nonpaged Pool
Paged Pool
PTEs
File Cache
![Page 6: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/6.jpg)
6Demand Technology, Inc. 32-bit VM Constraints
Virtual Memory Manager
All address spaces share the virtual addresses from the same upper 2 GB of the System area
All virtual addresses in the lower 2 GB User area are unique
Shared memory in the System range can be used for IPC
User
UserUser
System
User
0
x'1 0000'16
x'8000 0000'16
x'ffff 0000'16
System Code
Device Driver Code
Nonpaged Pool
Paged Pool
PTEs
File Cache
![Page 7: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/7.jpg)
7Demand Technology, Inc. 32-bit VM Constraints
Private
System
10000
7F F E F F F F
A 4000000
C1000000
E 0000000
F F F F F F F F
PTEs/ cache extension
System data cache
Paged system pool
80000000
C0000000
Virtual Memory map
Paged Memory mapped
files Shared DLLs
File Server IIS (html, jpg, gif)
Page tables; kernel thread stacks
nonPaged: I/O buffers used by
device drivers TCP Session data
![Page 8: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/8.jpg)
8Demand Technology, Inc. 32-bit VM Constraints
Virtual Memory Manager
Commit Limit A hard limit on the number of
Virtual Memory pages that the system will allocate
Sizeof (RAM) + paging file(s) Memory allocations start to fail
as % Committed Bytes in Use 100%
For smaller systems (< 2 GB RAM), Committed Bytes tends to become exhausted prior to running out of VM
![Page 9: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/9.jpg)
9Demand Technology, Inc. 32-bit VM Constraints
Monitoring Committed Bytes
![Page 10: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/10.jpg)
10Demand Technology, Inc. 32-bit VM Constraints
Monitoring Process\(*)Virtual Bytes
![Page 11: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/11.jpg)
11Demand Technology, Inc. 32-bit VM Constraints
Virtual Memory Manager
Commit Limit But page files are
extendible!
Should you or shouldn’t you?
![Page 12: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/12.jpg)
12Demand Technology, Inc. 32-bit VM Constraints
Paging file extension
![Page 13: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/13.jpg)
13Demand Technology, Inc. 32-bit VM Constraints
System memory pools
No hard limits on the sizes of the system pools
One reserved area of virtual memory can fill up faster than the others;
Area Max Size
file cache 960 MB
PTEs 660 MB
Paged pool 512 MB
nonPaged Pools
256 MB
![Page 14: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/14.jpg)
14Demand Technology, Inc. 32-bit VM Constraints
Monitoring the System Pools
![Page 15: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/15.jpg)
15Demand Technology, Inc. 32-bit VM Constraints
Private
System
10000
7F F E F F F F
A 4000000
C1000000
E 0000000
F F F F F F F F
PTEs/ cache extension
System data cache
Paged system pool
80000000
C0000000
Virtual Memory map
Virtual memory map is dynamic, subject to adjustment by the OS
One reserved area of virtual memory can fill up faster than the others
When any one area fills, VMM routines can empty the file cache to acquire more vm for the paged/nonpaged pools
Paged Pool
nonPaged PoolPTEs
Cache
![Page 16: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/16.jpg)
16Demand Technology, Inc. 32-bit VM Constraints
Default Paged and Nonpaged pool size can be overridden:
![Page 17: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/17.jpg)
17Demand Technology, Inc. 32-bit VM Constraints
Default Paged and Nonpaged pool sizes can be overridden:
Dynamic adjustment of system virtual memory when one pool is exhausted.
Workloads most vulnerable: Terminal server with an excess of processes &
threads Servers booted with the /3 GB option
NonPagedPoolSize, NonPagedPoolSize, and SystemPages can be set explicitly. Max pool size is only available from the debugger!
![Page 18: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/18.jpg)
Virtual Memory constraints in Terminal Server
1
10
100
1000
10000
100000
1000000
0 50 100 150 200 250
Terminal Server Sessions (x86)
Free
Sys
tem
PTE
s
0
10
20
30
40
50
60
% P
roce
ssor
Bus
yFree System PTEs
% Processor Time
![Page 19: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/19.jpg)
19Demand Technology, Inc. 32-bit VM Constraints
Extended Virtual Addressing
Boot.ini /3 GB switch Virtual storage constraint relief for some
applications Squeezes the System VM into 1 GB
Physical Address Extension (PAE) Supports 37-bit real addresses on Xeon processors
Address Windowing Extensions (AWE) Permits processes to address real memory > 4 GB
![Page 20: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/20.jpg)
20Demand Technology, Inc. 32-bit VM Constraints
Virtual Memory Manager
Boot.ini /3 GB switch Virtual storage
constraint relief for some server applications
SQL Server Exchange 2000 Etc.
/userva=SizeInMB subparameter, where SizeinMB can be any value between 2048 and 3072
System
User
0
x'1 0000'16No Access
x'c000 0000'16
x'ffff 0000'16
System Code
Device Driver Code
Nonpaged Pool
Paged Pool
PTEs
File Cache
/userva
![Page 21: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/21.jpg)
21Demand Technology, Inc. 32-bit VM Constraints
Exchange 2003 memory tuning Exchange Server 2003
recommendations/3 GB /Userva = 3030SystemPages = -1HeapDeCommitFreeBlockThreshold = 0x00040000msExchESEParamCacheSizeMax = 311296 (1.2 GB)
See KB #815372
![Page 22: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/22.jpg)
22Demand Technology, Inc. 32-bit VM Constraints
Virtual Memory constraints
Try the /3 GB switch 1 GB system area is not enough virtual memory for some
applications Possible shortage of Free System Page Table Entries Possible shortage of Nonpaged Pool
– Where Session data from TCP connections is stored
Due to fragmentation, it may not be possible to failover a 2 GB+ private address space (e.g., SQL Server, MS Exchange database – store.exe) using Microsoft Cluster Server (MCS)
During address space recovery on the standby node, the entire virtual memory allocation is acquired at one time
![Page 23: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/23.jpg)
23Demand Technology, Inc. 32-bit VM Constraints
Physical Address Extension
PAE is supported on most recent Intel servers Physical addresses are 37-bits Up to 128 GB of RAM can be installed
New format Page Table Entries (PTEs) are 64-bits
Process address spaces are still limited to 4 GB. How can they exploit 64 GB of RAM? Expand sideways, using multiple address spaces Large Memory Enabled (LME) device drivers Address Windowing Extensions (AWE)
Boot.ini /pae switch Limitation: system addresses can be no higher than 16 GB
![Page 24: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/24.jpg)
24Demand Technology, Inc. 32-bit VM Constraints
Intel x86 in PAE mode
(ptr) Page Directory Index Page Table Index Byte Offset
Virtual Address Translation (PAE)
PDE
PageDirectory
PageDirectoryPointers
PDE
PDE
PDE
PDE
64-bit PTE
PageTable
RAM(Physical Memory)
CR3
31 29 20 011
![Page 25: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/25.jpg)
25Demand Technology, Inc. 32-bit VM Constraints
Physical Address Extension
Expand sideways, using multiple address spaces e.g., MS SQL
Server
SQL Server instance #1
0
3 GB
4 GB(System Range)
SQL Server instance #2
0
3 GB
4 GB(System Range)
SQL Server instance #3
0
3 GB
4 GB(System Range)
System Range
0
12 GB
RAM VM
![Page 26: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/26.jpg)
26Demand Technology, Inc. 32-bit VM Constraints
IIS 6.0 (expand sideways)
User
Kernel
HTTP
TCP
IP
Network Interface
HTTP Kernel Mode Driver
(http.sys)
HTTP Response Cache
(Physical Memory)
LSSAS
WindowsAuthentication
SSL
Inetinfo
IIS Administration
FTP
Metabase
SMTP
NNTP
SVCHOST
W3SVC
W3wpW3wp
Aspnet_filter.dll
Mscoree.dll
Aspnet_isapi.dll
<code-behind>.dll
Default.aspx
Application Pool
W3wpW3wp
Aspnet_filter.dll
Mscoree.dll
Aspnet_isapi.dll
<code-behind>.dll
Default.aspx
Application Pool
![Page 27: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/27.jpg)
27Demand Technology, Inc. 32-bit VM Constraints
Configuring IIS 6.0 Application Pools
![Page 28: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/28.jpg)
28Demand Technology, Inc. 32-bit VM Constraints
Address Windowing Extensions (AWE)
API that allows processes to address real memory locations outside their 4 GB virtual addressing range Create and manage memory Overlays Used in conjunction with PAE
AllocateUserPhysicalPages
MapUserPhysicalPages
Virtual Alloc
![Page 29: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/29.jpg)
29Demand Technology, Inc. 32-bit VM Constraints
AWESystem
User
0
x'1 0000'16No Access
x'c000 0000'16
x'ffff 0000'16
System Code
Device Driver Code
Nonpaged Pool
Paged Pool
PTEs
File Cache
x'8000 0000'16
AWE Region (Nonpaged memory)
Physical Address Range(Reserved)
Physical Address Range(Reserved)
Physical Address Range(Reserved)
MapUserPhysicalPage
Physical Address Range(Reserved)
Note: Frequent unmapping and remapping of Physical Memory blocks is expensive!
![Page 30: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/30.jpg)
30Demand Technology, Inc. 32-bit VM Constraints
AWE support
MS SQL Server 2000: /3 GB, PAE, & AWE Multiple process instances awe enabled = 1 AWE activity counters in the Buffer Manager object
Oracle: /3 GB, PAE, & AWE AWE_WINDOW_MEMORY Also Set max server Oraclememory
SAS: /3 GB, PAE Work library can be placed in PAE memory
![Page 31: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/31.jpg)
31Demand Technology, Inc. 32-bit VM Constraints
System Virtual Memory tuning
HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management settings
NonpagedPoolSize Defaults based on the size of RAM, up to 256 MB.
Can be set explicitly.-1 extends NonpagedPoolSize to its maximum
PagedPoolSize Defaults based on the size of RAM, Up to 650 MB.
Can be set explicitly; -1 extends PagedPoolSize to its maximum
SystemPages Defaults based on the size of RAM. Can be set explicitly; -1 extends SystemPages to its maximum
LargeSystemCache 960 MB Zero value allocates minimum sized file cache
![Page 32: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/32.jpg)
32Demand Technology, Inc. 32-bit VM Constraints
Default Paged and Nonpaged pool size can be overridden:
![Page 33: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/33.jpg)
33Demand Technology, Inc. 32-bit VM Constraints
Default Paged and Nonpaged pool size can be overridden (Win 2003):
System Pages = x’ffff ffff’16 or -1 Maximizes the number of PTEs that can be built Potentially useful with /PAE or Terminal Services
PagedPoolSize = x’ffff ffff’16 or -1 Allows the OS maximum flexibility to determine the size of
the Paged Pool Or PagedPoolSize and NonPagedPoolSize can be set
explicitly
LargeSystemCache controls the maximum virtual address range of the file cache 512 MB – 1 GB
![Page 34: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/34.jpg)
34Demand Technology, Inc. 32-bit VM Constraints
!vm Debugger command runs the Page Frame Number (PFN) database
Available Pages: 4920 ( 19680 Kb)ResAvail Pages: 358 ( 1432 Kb)Locked IO Pages: 251 ( 1004 Kb)Free System PTEs: 204387 ( 817548 Kb)Free NP PTEs: 28645 ( 114580 Kb)Free Special NP: 0 ( 0 Kb)Modified Pages: 596 ( 2384 Kb)Modified PF Pages: 660 ( 2640 Kb)NonPagedPool Usage: 2750 ( 11000 Kb)NonPagedPool Max: 33768 ( 135072 Kb)PagedPool 0 Usage: 3544 ( 14176 Kb)PagedPool 1 Usage: 1359 ( 5436 Kb)PagedPool 2 Usage: 1340 ( 5360 Kb)PagedPool Usage: 6243 ( 24972 Kb)PagedPool Maximum: 138240 ( 552960 Kb)Shared Commit: 6842 ( 27368 Kb)Special Pool: 0 ( 0 Kb)Shared Process: 3688 ( 14752 Kb)PagedPool Commit: 6398 ( 25592 Kb)Driver Commit: 1630 ( 6520 Kb)Committed pages: 211846 ( 847384 Kb)Commit limit: 320257 ( 1281028 Kb)
![Page 35: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/35.jpg)
35Demand Technology, Inc. 32-bit VM Constraints
!poolused Debugger command accountsfor all system pool allocations(plus pooltags.txt documentation)
lkd> !poolused 2 Sorting by NonPaged Pool Consumed
Pool Used: NonPaged Paged Tag Allocs Used Allocs Used LSwi 1 2576384 0 0 NV 287 1379120 14 55272 File 2983 504920 0 0 MmCm 16 435248 0 0 LSwr 128 406528 0 0 Devi 267 377472 0 0 Thre 452 296512 0 0 PcNw 12 278880 0 0 Irp 669 222304 0 0
![Page 36: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/36.jpg)
36Demand Technology, Inc. 32-bit VM Constraints
Poolmon utility (from DDK)
![Page 37: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/37.jpg)
37Demand Technology, Inc. 32-bit VM Constraints
64-bit Virtual Memory Relief
Component 64-bit 32-bit
Process Virtual Address Space 16 TB 4 GB
User Address Space 8 TB 2 GB
System Paged Pool 128 GB 512 MB*
System Non-paged Pool 128 GB 256 MB*
System cache 1 TB 1 GB
System PTEs 128 GB 660 MB*
Hyperspace 8 GB 4 MB
Paging File size 512 TB 4 GB
![Page 38: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/38.jpg)
38Demand Technology, Inc. 32-bit VM Constraints
64-bit Relief
Windows supports two flavors of 64-bit hardware IA-64 (Itanium) AMD64 and Intel x64 (aka EMT)
Runs 32-bit applications with almost no performance penalty
32-bit applications can allocate their entire 4 GB virtual memory range
![Page 39: 32-bit Virtual Memory Constraints in Windows: an Update](https://reader033.fdocuments.net/reader033/viewer/2022051517/56815a9c550346895dc82085/html5/thumbnails/39.jpg)
39Demand Technology, Inc. 32-bit VM Constraints
Questions
?