Microsoft Windows Internals Microsoft Windows Server 2003, Windows XP, and Windows 2000

by Solomon, David; Russinovich, Mark

• ISBN13: 9780735619173
• ISBN10: 0735619174
• Edition: 4th
• Format: Paperback
• Copyright: 2004-12-08
• Publisher: Microsoft Press

Summary

The classic, in-depth developer's guide to the Windows kernel now covers Windows .NET Server 2003, Windows XP, and Windows 2000. Written by noted Windows internals experts David Solomon and Mark Russinovich in collaboration with the Microsoft Windows .NET Server product development team, this book packs the latest concepts and terms, kernel and source code specifics, undocumented interfaces, component and tool descriptions, and architectural perspectives that reveal the inner workings of the operating system. Special callouts highlight information that is specific to a particular version of Windows, and an advanced troubleshooting section helps you more easily decipher-and exploit-system operations and performance.

Author Biography

Mark Russinovich is a Technical Fellow in the Windows Azure group at Microsoft. He is coauthor of Windows Sysinternals Administrator’s Reference, co-creator of the Sysinternals tools available from Microsoft TechNet, and coauthor of the Windows Internals book series. David A. Solomon is coauthor of the Windows Internals book series and has taught his Windows internals class to thousands of developers and IT professionals worldwide, including Microsoft staff. He is a regular speaker at Microsoft conferences, including TechNet and PDC.

Table of Contents

Historical Perspective	xix
Foreword	xxiii
Acknowledgments	xxv
Introduction	xxvii
Concepts and Tools	1	(34)
Windows Operating System Versions	1	(2)
Foundation Concepts and Terms	3	(22)
Windows API	3	(2)
Services, Functions, and Routines	5	(1)
Processes, Threads, and Jobs	6	(8)
Virtual Memory	14	(2)
Kernel Mode vs. User Mode	16	(5)
Terminal Services and Multiple Sessions	21	(1)
Objects and Handles	22	(1)
Security	23	(1)
Registry	24	(1)
Unicode	25	(1)
Digging into Windows Internals	25	(9)
Performance Tool	27	(1)
Windows Support Tools	27	(1)
Windows Resource Kits	27	(1)
Kernel Debugging	28	(5)
Platform Software Development Kit (SDK)	33	(1)
Device Driver Kit (DDK)	34	(1)
Sysinternals Tools	34	(1)
Conclusion	34	(1)
System Architecture	35	(50)
Requirements and Design Goals	35	(1)
Operating System Model	36	(1)
Architecture Overview	37	(14)
Portability	40	(1)
Symmetric Multiprocessing	41	(5)
Scalability	46	(1)
Differences Between Client and Server Versions	47	(2)
Checked Build	49	(2)
Key System Components	51	(33)
Environment Subsystems and Subsystem DLLs	53	(10)
Ntdll.dll	63	(1)
Executive	63	(2)
Kernel	65	(2)
Hardware Abstraction Layer	67	(2)
Device Drivers	69	(6)
System Processes	75	(9)
Conclusion	84	(1)
System Mechanisms	85	(98)
Trap Dispatching	85	(39)
Interrupt Dispatching	87	(22)
Exception Dispatching	109	(10)
System Service Dispatching	119	(5)
Object Manager	124	(25)
Executive Objects	126	(2)
Object Structure	128	(21)
Synchronization	149	(17)
High-IRQL Synchronization	151	(4)
Low-IRQL Synchronization	155	(11)
System Worker Threads	166	(2)
Windows Global Flags	168	(3)
Local Procedure Calls (LPCs)	171	(4)
Kernel Event Tracing	175	(3)
Wow64	178	(4)
Wow64 Process Address Space Layout	179	(1)
System Calls	179	(1)
Exception Dispatching	179	(1)
User Callbacks	179	(1)
File System Redirection	180	(1)
Registry Redirection and Reflection	180	(1)
I/O Control Requests	181	(1)
16-bit Installer Applications	182	(1)
Printing	182	(1)
Restrictions	182	(1)
Conclusion	182	(1)
Management Mechanisms	183	(68)
The Registry	183	(28)
Viewing and Changing the Registry	183	(1)
Registry Usage	184	(1)
Registry Data Types	185	(1)
Registry Logical Structure	186	(6)
Troubleshooting Registry Problems	192	(5)
Registry Internals	197	(14)
Services	211	(26)
Service Applications	212	(5)
Service Accounts	217	(6)
The Service Control Manager	223	(2)
Service Startup	225	(4)
Startup Errors	229	(1)
Accepting the Boot and Last Known Good	230	(1)
Service Failures	231	(1)
Service Shutdown	232	(1)
Shared Service Processes	233	(3)
Service Control Programs	236	(1)
Windows Management Instrumentation	237	(12)
WMI Architecture	237	(2)
Providers	239	(1)
The Common Information Model and the Managed Object Format Language	240	(3)
The WMI Namespace	243	(1)
Class Association	244	(3)
WMI Implementation	247	(1)
WMI Security	248	(1)
Conclusion	249	(2)
Startup and Shutdown	251	(38)
Boot Process	251	(23)
x86 and x64 Preboot	251	(4)
The x86/x64 Boot Sector and Ntldr	255	(9)
The IA64 Boot Process	264	(2)
Initializing the Kernel and Executive Subsystems	266	(3)
Smss, Csrss, and Winlogon	269	(4)
Images that Start Automatically	273	(1)
Troubleshooting Boot and Startup Problems	274	(12)
Last Known Good	274	(1)
Safe Mode	274	(5)
Recovery Console	279	(2)
Solving Common Boot Problems	281	(5)
Shutdown	286	(2)
Conclusion	288	(1)
Processes, Threads, and Jobs	289	(86)
Process Internals	289	(11)
Data Structures	289	(8)
Kernel Variables	297	(1)
Performance Counters	297	(1)
Relevant Functions	298	(2)
Flow of CreateProcess	300	(13)
Stage 1: Opening the Image to Be Executed	302	(2)
Stage 2: Creating the Windows Executive Process Object	304	(4)
Stage 3: Creating the Initial Thread and Its Stack and Context	308	(1)
Stage 4: Notifying the Windows Subsystem about the New Process	309	(1)
Stage 5: Starting Execution of the Initial Thread	310	(1)
Stage 6: Performing Process Initialization in the Context of the New Process	310	(3)
Thread Internals	313	(10)
Data Structures	313	(7)
Kernel Variables	320	(1)
Performance Counters	321	(1)
Relevant Functions	322	(1)
Birth of a Thread	322	(1)
Examining Thread Activity	323	(2)
Thread Scheduling	325	(43)
Overview of Windows Scheduling	326	(1)
Priority Levels	327	(3)
Windows Scheduling APIs	330	(1)
Relevant Tools	331	(2)
Real-Time Priorities	333	(1)
Thread States	334	(4)
Dispatcher Database	338	(2)
Quantum	340	(5)
Scheduling Scenarios	345	(2)
Context Switching	347	(1)
Idle Thread	348	(1)
Priority Boosts	348	(9)
Multiprocessor Systems	357	(9)
Multiprocessor Thread-Scheduling Algorithms	366	(2)
Job Objects	368	(5)
Conclusion	373	(2)
Memory Management	375	(110)
Introduction to the Memory Manager	375	(7)
Memory Manager Components	376	(1)
Internal Synchronization	377	(1)
Configuring the Memory Manager	378	(1)
Examining Memory Usage	378	(4)
Services the Memory Manager Provides	382	(19)
Large and Small Pages	382	(2)
Reserving and Committing Pages	384	(1)
Locking Memory	385	(1)
Allocation Granularity	385	(1)
Shared Memory and Mapped Files	386	(2)
Protecting Memory	388	(2)
No Execute Page Protection	390	(2)
Copy-on-Write	392	(2)
Heap Manager	394	(5)
Address Windowing Extensions	399	(2)
System Memory Pools	401	(12)
Configuring Pool Sizes	401	(3)
Monitoring Pool Usage	404	(4)
Look-Aside Lists	408	(1)
Driver Verifier	409	(4)
Virtual Address Space Layouts	413	(12)
x86 User Address Space Layouts	415	(2)
x86 System Address Space Layout	417	(1)
x86 Session Space	418	(3)
System Page Table Entries	421	(1)
64-Bit Address Space Layouts	422	(3)
Address Translation	425	(14)
x86 Virtual Address Translation	425	(9)
Translation Look-Aside Buffer	434	(1)
Physical Address Extension (PAE)	435	(2)
IA-64 Virtual Address Translation	437	(1)
x64 Virtual Address Translation	438	(1)
Page Fault Handling	439	(9)
Invalid PTEs	440	(1)
Prototype PTEs	441	(2)
In-Paging I/O	443	(1)
Collided Page Faults	444	(1)
Page Files	444	(4)
Virtual Address Descriptors	448	(2)
Section Objects	450	(7)
Working Sets	457	(1)
Demand Paging	458	(1)
Logical Prefetcher	458	(11)
Placement Policy	462	(1)
Working Set Management	463	(3)
Balance Set Manager and Swapper	466	(1)
System Working Set	467	(2)
Page Frame Number Database	469	(14)
Page List Dynamics	472	(3)
Modified Page Writer	475	(1)
PFN Data Structures	476	(3)
Low and High Memory Notification	479	(4)
Conclusion	483	(2)
Security	485	(52)
Security System Components	488	(4)
Protecting Objects	492	(24)
Access Checks	493	(13)
Security Descriptors and Access Control	506	(10)
Account Rights and Privileges	516	(8)
Account Rights	517	(1)
Privileges	518	(5)
Super Privileges	523	(1)
Security Auditing	524	(2)
Logon	526	(7)
Winlogon Initialization	528	(1)
User Logon Steps	529	(4)
Software Restriction Policies	533	(2)
Conclusion	535	(2)
I/O System	537	(78)
I/O System Components	537	(4)
The I/O Manager	539	(1)
Typical I/O Processing	540	(1)
Device Drivers	541	(20)
Types of Device Drivers	541	(7)
Structure of a Driver	548	(2)
Driver Objects and Device Objects	550	(5)
Opening Devices	555	(6)
I/O Processing	561	(29)
Types of I/O	561	(3)
I/O Request Packets	564	(5)
I/O Request to a Single-Layered Driver	569	(8)
I/O Requests to Layered Drivers	577	(8)
I/O Completion Ports	585	(4)
Driver Verifier	589	(1)
The Plug and Play (PnP) Manager	590	(17)
Level of Plug and Play Support	591	(1)
Driver Support for Plug and Play	592	(2)
Driver Loading, Initialization, and Installation	594	(9)
Driver Installation	603	(4)
The Power Manager	607	(6)
Power Manager Operation	609	(1)
Driver Power Operation	610	(3)
Driver Control of Device Power	613	(1)
Conclusion	613	(2)
Storage Management	615	(40)
Storage Terminology	615	(1)
Disk Drivers	616	(6)
Ntldrx	616	(1)
Disk Class, Port, and Miniport Drivers	617	(3)
Disk Device Objects	620	(2)
Partition Manager	622	(1)
Volume Management	622	(32)
Basic Disks	624	(2)
Dynamic Disks	626	(6)
Multipartition Volume Management	632	(6)
The Volume Namespace	638	(8)
Volume I/O Operations	646	(2)
Virtual Disk Service	648	(1)
Volume Shadow Copy Service	649	(5)
Conclusion	654	(1)
Cache Manager	655	(34)
Key Features of the Cache Manager	655	(5)
Single, Centralized System Cache	656	(1)
The Memory Manager	656	(1)
Cache Coherency	656	(2)
Virtual Block Caching	658	(1)
Stream-Based Caching	658	(1)
Recoverable File System Support	658	(2)
Cache Virtual Memory Management	660	(2)
Cache Size	662	(6)
LargeSystemCache	662	(1)
Cache Virtual Size	663	(2)
Cache Working Set Size	665	(2)
Cache Physical Size	667	(1)
Cache Data Structures	668	(6)
Systemwide Cache Data Structures	669	(1)
Per-File Cache Data Structures	670	(4)
File System Interfaces	674	(5)
Copying to and from the Cache	676	(1)
Caching with the Mapping and Pinning Interfaces	677	(1)
Caching with the Direct Memory Access Interfaces	678	(1)
Fast I/O	679	(3)
Read Ahead and Write Behind	682	(6)
Intelligent Read-Ahead	682	(1)
Write-Back Caching and Lazy Writing	683	(3)
Write Throttling	686	(1)
System Threads	687	(1)
Conclusion	688	(1)
File Systems	689	(98)
Windows File System Formats	690	(4)
CDFS	690	(1)
UDF	691	(1)
FAT12, FAT16, and FAT32	691	(3)
NTFS	694	(1)
File System Driver Architecture	694	(17)
Local FSDs	695	(1)
Remote FSDs	696	(4)
File System Operation	700	(5)
File System Filter Drivers	705	(6)
Troubleshooting File System Problems	711	(6)
Filemon Basic vs. Advanced Modes	711	(1)
Filemon Troubleshooting Techniques	712	(5)
NTFS Design Goals and Features	717	(12)
High-End File System Requirements	717	(2)
Advanced Features of NTFS	719	(10)
NTFS File System Driver	729	(3)
NTFS On-Disk Structure	732	(26)
Volumes	732	(1)
Clusters	732	(1)
Master File Table	733	(6)
File Reference Numbers	739	(1)
File Records	740	(2)
Filenames	742	(2)
Resident and Nonresident Attributes	744	(3)
Data Compression and Sparse Files	747	(5)
The Change Journal File	752	(1)
Indexing	753	(1)
Object IDs	754	(1)
Quota Tracking	755	(1)
Consolidated Security	756	(2)
Reparse Points	758	(1)
NTFS Recovery Support	758	(17)
Evolution of File System Design	759	(2)
Logging	761	(6)
Recovery	767	(4)
NTFS Bad-Cluster Recovery	771	(4)
Encrypting File System Security	775	(10)
Encrypting a File for the First Time	778	(5)
The Decryption Process	783	(1)
Backing Up Encrypted Files	784	(1)
Conclusion	785	(2)
Networking	787	(58)
Windows Networking Architecture	787	(4)
The OSI Reference Model	787	(2)
Windows Networking Components	789	(2)
Networking APIs	791	(24)
Windows Sockets	791	(7)
Remote Procedure Call	798	(5)
Web Access APIs	803	(1)
Named Pipes and Mailslots	804	(7)
NetBIOS	811	(2)
Other Networking APIs	813	(2)
Multiple Redirector Support	815	(5)
Multiple Provider Router	816	(2)
Multiple UNC Provider	818	(2)
Name Resolution	820	(1)
Domain Name System	820	(1)
Windows Internet Name Service	820	(1)
Protocol Drivers	821	(7)
TCP/IP Extensions	824	(4)
NDIS Drivers	828	(10)
Variations on the NDIS Miniport	832	(1)
Connection-Oriented NDIS	832	(3)
Remote NDIS	835	(1)
QOS	836	(2)
Binding	838	(1)
Layered Network Services	839	(5)
Remote Access	839	(1)
Active Directory	840	(1)
Network Load Balancing	841	(2)
File Replication Service	843	(1)
Distributed File System	843	(1)
Conclusion	844	(1)
Crash Dump Analysis	845	(26)
Why Does Windows Crash?	845	(1)
The Blue Screen	846	(3)
Crash Dump Files	849	(4)
Crash Dump Generation	852	(1)
Windows Error Reporting	853	(1)
Online Crash Analysis	854	(1)
Basic Crash Dump Analysis	855	(5)
Notmyfault	855	(1)
Basic Crash Dump Analysis	856	(2)
Verbose Analysis	858	(2)
Using Crash Troubleshooting Tools	860	(4)
Buffer Overrun and Special Pool	861	(2)
Code Overwrite and System Code Write Protection	863	(1)
Advanced Crash Dump Analysis	864	(7)
Stack Trashes	865	(1)
Hung or Unresponsive Systems	866	(3)
When There Is No Crash Dump	869	(2)
Glossary	871	(24)
Index	895