Overview

Memory in modern operating systems is not mapped directly to physical memory (i.e the RAM). Instead, virtual memory addresses are used by processes that are mapped to physical memory addresses. There are several reasons for this but ultimately the goal is to save as much physical memory as possible. Virtual memory may be mapped to physical memory but can also be stored on disk. With virtual memory addressing it becomes possible for multiple processes to share the same physical address while having a unique virtual memory address.

See the image below from the https://learn.microsoft.com/en-us/sysinternals/resources/windows-internals book.

Windows uses a flat address space to create the illusion of a giant, private memory space for each program. This means programs see a continuous memory layout even though it might be spread across both RAM and hard disk. The memory manager, working with hardware, translates virtual addresses used by programs into actual physical locations where data is stored. By controlling this translation and access, Windows prevents programs from interfering with each other or the operating system itself.

Virtual memory relies on the concept of Memory paging which divides memory into chunks of 4kb called “pages” in the Windows operating system. As real memory is limited compared to running programs’ virtual needs, the memory manager “pages” unused data to disk. This frees up RAM for other processes or the system itself. If a program needs something that’s been paged out, the virtual memory manager brings it back from disk on demand.

The beauty of paging is its transparency. Programs don’t need to be modified or even aware of this juggling act. The memory manager, with the help of special hardware, seamlessly translates virtual addresses (the program’s view) to physical addresses (where the data actually resides). When a program needs a page that’s been sent to the hard drive, the memory manager simply swaps it back into RAM, keeping the program running smoothly without any hiccups.

Virtual Memory Sizes

The size of the virtual memory differs across the different architectures. In 32bit systems, the total virtual address space has a theoretical maximum of 4 GB whereas in 64bit systems the can extend up to 2^64 bytes, which is an astronomically large number (18.4 million terabytes), however, the actual amount of virtual memory available to a process is typically constrained by practical limitations, such as the amount of physical RAM installed on the system and the configuration settings.. In the Windows OS, the virtual address space space is divided into 2 halves.

The halves are simply referred to as Kernel/System address space (Higher addresses) and the User address space(Lower Addresses). The lower half of this address space is located to processes for their private storage and the upper half is allocated to the OS itself for its own protected memory utilization.

x86/32-bit Systems

• Total Maximum virtual memory size → 4 GB
• Address Range → 0x00000000 – 0x FFFFFFFF
• Kernel Address Space → 0x80000000 – 0xFFFFFFFF
• User Address Space → 0x00000000 – 0x7FFFFFFF

Address Windowing Extensions (AWE)

Address Windowing Extensions (AWE) is a technology in Microsoft Windows that was primarily designed for 32-bit versions of the operating system. The main purpose of AWE is to allow 32-bit Windows applications to access more physical memory than the default 4 GB limit imposed by the 32-bit memory address space.

In a standard 32-bit Windows environment, each process has a 4 GB virtual address space, and this address space is typically split between the application and the operating system. This limitation poses challenges for applications that need to work with large datasets or require more memory.

AWE enables applications, particularly those with demanding memory requirements like databases or certain scientific applications, to use more than 4 GB of physical memory. Here’s how it works:

1. Physical Address Extension (PAE): AWE relies on PAE, a feature of certain 32-bit processors, to address more than 4 GB of physical memory. PAE extends the physical address space from 32 bits to 36 bits, allowing access to up to 64 GB of RAM.
2. Windowed Mapping: AWE uses a mechanism called “windowed mapping” to map portions of the physical memory into the 32-bit virtual address space of an application. This allows the application to use more than 4 GB of memory in a windowed fashion.

It’s important to note that AWE is specific to 32-bit versions of Windows and is not applicable to 64-bit versions. In 64-bit systems, applications can directly address much larger amounts of memory without the need for AWE or similar workarounds.

increaseuserva option

The IncreaseUserVa setting, or more commonly known as the “IncreaseUserVa” boot option, is related to the Address Windowing Extensions (AWE) feature in Windows. This option allows you to increase the user-mode virtual address space available to applications on 32-bit versions of Windows.

By default, 32-bit Windows allocates a 2 GB virtual address space for user-mode processes and 2 GB for the system. The “IncreaseUserVa” boot option allows you to adjust this balance, potentially giving more virtual address space to user-mode applications. This can be useful for applications that require more memory for their data, such as large databases.

Here’s a brief overview of how to use the “IncreaseUserVa” boot option:

1. Accessing Boot Configuration Data (BCD): You can use the bcdedit command-line tool to modify boot configuration settings. Open a Command Prompt with administrative privileges.
2. Check Current Settings:bcdedit /vThis will display the current boot configuration settings.
3. Modify the IncreaseUserVa Option:bcdedit /set increaseuserva <value>Replace “value” with the amount of additional virtual address space you want to allocate to user-mode applications. The value is in megabytes, and commonly used values include 2048 (for a total of 3 GB for user-mode processes) or 3072 (for a total of 3.5 GB for user-mode processes).
4. Reboot: After making changes, you need to restart your computer for the changes to take effect.

To leverage the increased address space for your program, the large address space‒aware flag must be set in the header of the executable image as shown below. During compile time, the /LARGEADDRESSAWARE option should be set which tells the linker that the application can handle addresses larger than 2gb.

In the 64-bit compilers, this option is enabled by default. In the 32-bit compilers, /LARGEADDRESSAWARE:NO is enabled if /LARGEADDRESSAWARE is not otherwise specified on the linker line.

https://learn.microsoft.com/en-us/cpp/build/reference/largeaddressaware-handle-large-addresses?view=msvc-170

x64/64-bit Systems

• can extend up to 2^64 bytes
• Current CPU architectures only support 48 bits addressing
  • 2^48 = 2^40+2^8 = 256 TB
  • 128TB kernel & User address spaces each
• Address Range → 0x000000000000 – 0x FFFFFFFFFFFF
• Kernel Address Space → 0xFFFF80000000 – 0xFFFFFFFFFFFF
• User Address Space → 0x000000000000 – 0x7FFFFFFFFFFF