The Page File, do I need one?

[ubuysa]

Following the 'Understanding RAM' post I've received a PM about the page file, how big to make it, where to put it, and whether you even need one at all. Here's my views on the page file then....

From the 'Understanding RAM' posts you'll know already that virtual memory pages can reside either in RAM or in the page file. If the pages in RAM are being regularly used they will stay in RAM, but if RAM is being heavily used those pages in RAM that are not being regularly used can be paged out to the page file (freeing up the RAM they were occupying). This means that the page file isn't like any other disk file, it's actually part of your memory subsystem and is thus just as important to you as RAM.

The downside to paging out unreferenced pages is that when (or if) a paged-out page is later accessed, Windows has to allocate a new page in RAM, locate the paged-out page in the page file, and bring that page into the newly allocated RAM page. This takes time (it's a disk read after all) and the executing thread is suspended whilst this is going on (the thread is actually in a wait on the page-in, when that's done the thread continues executing). You see this as sluggish performance in the application concerned, so whilst paging is a good thing for RAM management it's a bad thing for performance. (Note that the page-out doesn't cause any performance issues because the page was not being used at the time. Note also that Windows pages in the requested page and several pages around it to try and avoid additional page faults).

Since unreferenced pages are typically only paged-out to the page file when RAM is heavily used, and since most of us these days have 8GB, 16GB, or more, installed RAM (which is almost never fully used) there won't be any actual paging to the page file going on.

Since a page file takes up at least the same number of GB as installed RAM, if there is lots of RAM installed you can do without a paging file and use that disk space for something else, right? The answer to that is a definite...maybe.

A 16GB (say) page file is a lot of disk space to consume for no apparent purpose, but these days we are installing 1TB, 2TB, or even bigger, disk arrays. 16GB on a 1TB disk is less than 2% of the total disk space. Even if your only disk is 256GB, as it might be on some smaller laptops, a 16GB page file is only about 6% of the available disk space.

Here's why you need one...

Open up the Task Manager, click the Performance tab, and then click the Memory icon on the left. Below the two graphs are some numbers, look at the Committed numbers. Committed memory is virtual storage space that Windows has approved for use by processes. The first of the two numbers is the amount of virtual storage currently approved for use, the second is the maximum amount of virtual storage that Windows will ever allow at one time on this system. The maximum is simply the size of installed RAM plus the size of the page file - this is because ultimately virtual storage pages have to exist somewhere real, and that means either in RAM or in the page file.

This is important because all processes run in virtual storage, on 64-bit Windows a process can address up to 128TB of virtual storage and although this is a staggering amount the use of any of that virtual storage must be approved by Windows. If for example, the two committed numbers were '6.4/31.9 GB', that means that Windows has currently approved the use of 6.4GB of virtual storage in total and that Windows will never approve more than 31.9GB of virtual storage. That means that on this system no process can ever access more than 31.9GB of virtual storage - even though 64-bit addressing allows up to 128TB.

If you have no page file (because you have 16GB of RAM installed and you think you don't need one) then those two committed numbers would read '6.4GB/15.9 GB', which means that no process could ever access more than 15.9 GB of virtual storage - on a system that allows up to 128TB per process. Even worse, it actually means the sum of all the virtual storage being used by all processes cannot be more than 15.9GB. Note that the page file size can be dynamically increased, up to a pre-determined maximum, once this maximum is reached no more virtual storage can be obtained.

You might argue that 15.9GB is a lot of memory, but a few hundred browser pages, a couple of databases, and a bunch of other common applications could start to get very close to that, or even bust it. With no page file, exhausting RAM will result in a system crash. A page file increases the amount of virtual storage that Windows can approve, so even if you do run more applications than you actually have RAM, the system doesn't crash.

Talking about system crashes, you probably know that a system crash will generate a dump file. Although you may not know how to read a Windows dump file yourself, you do know that there are a number of people on here who do, so uploading the dump file to these fora could help get your system crash resolved.

Guess where system dump files are written? Yep, the page file. That's another reason why you need one!

Here's where to put the page file...

The page file is memory remember, so if your system is paging constantly you want the page file on the fastest disk you have. That means if you have an SSD you want the page file on the SSD. It is true that this may shorten the life of the SSD (by a small, but probably measurable amount) but the performance benefit of the rapid page-in far outweighs the lifespan of the SSD considerations. Incidentally, if you are running a system that is paging constantly you should plan for a RAM upgrade as soon as convenient (or reduce the number of concurrently executing applications). Note that you can have up to 16 page files, each on a separate volume, and Windows will use them all. On a heavily paging system multiple page files will improve performance a little.

If you are not paging very much (or at all) because you have plenty of RAM you don't need the page file on your SSD, so in this case the lifespan considerations of the SSD outweigh any performance benefit. In this case place the page file on a large HDD, it won't offer terribly good performance if you do start paging, but the system won't crash. If the system needs to write a dump everything is suspended whilst that happens anyway so the speed of the page file on the HDD isn't terribly relevant.

Here's how big to make the page file...

First, ignore all the common advice about making it 1.5 x RAM, that may have been a useful yardstick back in the days of XP but it's meaningless now. Windows 10 has a much better algorithm for automatically setting the page file size than previous versions of Windows did. You probably know that the size of the page file can be increased and decreased dynamically, and Windows 10 has improved algorithms in this area too.

To set the page file size (and location) enter 'sysdm.cpl' in the Run box, click the Advanced tab, click the Performance setting button, click the Advanced tab, and finally click the Change button.

My advice is to allow Windows to manage the size of the page file on Windows 10, especially if it's on a large HDD and you have plenty of RAM. In this situation you'll find Windows sets the minimum page file size to be a tiny bit larger than your installed RAM, this is so that it can hold a full memory dump. The maximum page file size will probably be around 3 x RAM.

If you really want to set an accurate maximum page file size yourself in order to minimise the page file size then you need to start every application you're ever likely to use, along with all the data sets and files they use, so that you're using as much virtual storage as you'll ever need. This is your maximum commit value, which you can read on the Memory page in Task Manager as we saw earlier. If you subtract the size of your installed RAM from this what's left is your maximum page file size (I'd round it up a bit just to be on the safe side). Make your minimum page file size the size of installed RAM (and set your maximum to this too if your calculated value is less than this) so that it can hold a full memory dump.

Note that pagefile.sys is a hidden file so you won't normally see it. You'll need to uncheck the 'Hide protected operating system files' checkbox in File Explorer Options to see it.

 

[ubuysa]

Windows 10 (and 8/8.1) has a second system managed 'paging' file (also hidden) called swapfile.sys, this is used (at the moment) only by UWP apps. A Universal Windows Platform (UWP) app is one that runs on any hardware platform supporting Windows, on desktop versions of Windows (and that's what I'm assuming we're all using) UWP apps are what we used to call Metro Apps, or Modern Apps.

To see why a swapfile.sys 'paging' file is needed start the built-in Windows Calculator app - this is a UWP app. Now start Task Manager, click the Details tab and scroll down to find the Calculator.exe entry - click it to highlight it so you can see it more easily.

In the status column you can see that Calculator.exe is Running, this means it's ready to execute and on the ready queue for logical processors. Even though you're not actually doing anything with it yet the application still receives control of a CPU now and then (to check whether you've entered anything) and that means it's referencing it's core pages in RAM. These pages thus stay in RAM even on a system that is heavily using all of it's RAM - this is good, it means that when you do start using the calculator it's immediately responsive - because it's core pages are in RAM and not paged-out.

Now minimise the calculator to the taskbar. Notice that it's status has now changed to Suspended. A Suspended status means that no threads in the process are on the ready queue for any processor. UWP apps are normally suspended when minimised to save CPU cycles, you clearly can't use the calculator whilst it's minimised so why waste CPU cycles running it?

Suspending a UWP app is good news for CPU performance (that's why it's done) but it's bad news for the UWP app. This is because if the UWP app is not executing then it can't reference any of it's pages in RAM and their unreferenced interval count will be getting higher and higher. If RAM is being heavily used the UWP apps pages will eventually be selected to be paged-out to the pagefile, and quite likely one at a time. The older unreferenced pages will be paged-out first and later other pages will be paged-out, and still later more pages (if RAM is under heavy use).

Now click the calculator icon in the taskbar to restore the app. Notice that in Task Manager it's status has returned to Running and you can start using it. However, because of the long suspend time (and assuming RAM is being heavily used), most of the calculator pages are out on the page file, so it now suffers page fault after page fault to bring all it's pages back in to RAM. This requires several page-in operations, one after the other, because the pages were paged-out singly and are scattered all over the page file. What you'll notice is that the calculator is unresponsive for some seconds as all it's pages are restored to RAM. And that's a problem.

The solution is a process called 'swapping' and (no surprise here) it uses the swapfile.sys 'paging' file. When a UWP app is suspended the Memory Manager is called, it writes all of the UWP app's pages out in contiguous locations to swapfile.sys in a single page-out operation. This is called a swap-out.

Now, when you restore the UWP app and it begins running again the Memory Manager is called and it locates all the pages of the app in the swap file and brings them all back in using a single swap-in page read. Typically this has completed in the time between you clicking on the UWP app in the taskbar and moving the cursor to use the app, so you don't notice it.

Swapping allows the system to suspend a UWP app and thus save CPU cycles, and yet protects the pages of the UWP app from being randomly paged-out. This means the UWP app is not inconvenienced by the suspension.

You have no control in Windows to manage swapfile.sys, it's typically located as a hidden file in the root of your system drive (as with pagefile.sys). The system managed minimum size is 16MB on an SSD and 256MB on an HDD, the maximum system managed size is 1.5 x RAM or 10% of the partition size, whichever is smaller (it grows and shrinks dynamically just like the page file).

We've seen that in systems with lots of RAM there is benefit in moving pagefile.sys off an SSD drive to prolong it's life a little more, it is also possible to move swapfile.sys to a different drive, it just takes a bit more work. Note that on a system with lots of RAM the swap-out is likely never actually done, the UWP app's pages are simply allocated to Standby memory in RAM, so they will stay in RAM as long as their Standby RAM space is not needed (in which case they will be physically swapped out). Note also that swapfile.sys is only used for UWP apps, so if you're not a big UWP app user (or if you have lots of RAM) then there is little point in moving it.

Basically what you need to do is create a new swapfile.sys on your HDD, make it 256MB in size, it must be in the root of the drive where the (or a) pagefile.sys file is located. You then create a symbolic link from swapfile.sys on your system drive (SSD) pointing to the real swapfile.sys on whichever HDD you located it. A symbolic link on your SSD takes up no disk space at all (a tiny bit of MFT space is all) and no reads or writes will be done to the SSD, they will all go to the HDD. There is of course a performance hit in all this, the swap-in when a UWP process becomes ready will take longer from the HDD, but not so long that you'll need to worry (you probably won't even notice).

Here's how to move the swap file (should you want to)....

1. Go to the location where you set the page file size and set no paging file on all drives. Then reboot. Both pagefile.sys and swapfile.sys should now be gone, if they're still there then just delete them (they won't be being used).

2. Now you need to create a symbolic link on the C:\ drive pointing to wherever you want the swap file to be - on my system it's on the D: drive which is my HDD. If you have a choice of HDDs pick the fastest and least busy drive. Open an elevated command prompt and enter the command "mklink c:\swapfile.sys d:\swapfile.sys" (without the quotes). Obviously you might want to change the drive letter of the destination drive to suit.

3. Now go back into where you set the page file size and set either system managed, or whatever size you like, for the page file on the same drive where the swap file links points. Now reboot.

You'll see an entry for swapfile.sys on your SSD still, but it's just a symbolic link (0 bytes), if you look at the destination drive you'll see that swapfile.sys is now on there (along with pagefile.sys of course).

Although, as you can now tell, swapping is not a big deal, especially on systems with lots of RAM, at least you now know what swapfile.sys is for.