Overclock.net banner
1 - 20 of 328 Posts

·
Premium Member
Joined
·
10,774 Posts
Discussion Starter · #1 ·
Hey All

I've seen this recommendation time and time again and I for one do not care for it really. I think this topic could have the potential to spark much debate, but I am going to have a vain attempt at explaining why it is actually counter intuitive to place your page file on a RAM disk, or other virtual storage device that creates a backing store using RAM.

The first thing we need to understand when explaining this issue is how Windows manages memory. You need to know that programs do not directly access your physical memory sticks. The Windows memory manager works closely with the CPU to provide what is called "virtual memory". This essentially gives each process (including paged-pool drivers) on your system their own virtual address space in which to play in. This virtual memory abstraction is in place because it means we don't actually have to worry about how much physical memory is installed on the system (you'll see how the page file comes into play with this shortly).

For 32-bit Windows the default address space for user-mode applications is 2GB. The Windows kernel itself fits into the remaining 2GB. For 64-bit Windows the default address space for user-mode applications is 8TB and the kernel has 8TB also. (Note: This is a simplified explanation).

Because the point of this thread is to not explain how virtual memory works, but rather why you shouldn't place your page file on a RAM disk, I will only quickly iterate some underpinnings about virtual memory: When a program allocates or uses virtual memory from its address space, the Windows memory manager maintains a look up table (and the CPU copies it with what is called the translation look-aside buffer, or TLB) of virtual addresses to physical addresses in RAM.

I will quickly also explain what happens when a virtual memory address cannot be resolved to the physical address required. This is called a page fault, and can either be soft, or hard. A soft fault means that the memory page requested is located somewhere ELSE in RAM (as opposed to the applications physical working set). What Windows does then is moves the page into the physical working set of the process of the current thread's context and be done with it. A hard fault occurs when the memory page requested is not located in RAM at all. In the case of application code, the Windows memory manager will bring it in from the application's executable on disk. In the case of private data, it will bring it in from the page file. In the case of memory mapped files, it will bring it in from the location on disk where the file is mapped from.

Whew. Okay, but you still haven't told us why putting the page file on a RAM disk is bad. Unfortunately, understanding most of the above is required for the explanation. But now that we've got that out of the way, let's get our feet wet with why.

First, lets start with defining one of the major roles of the page file itself: (Paraphrased from Mark Russinovich [http://blogs.technet.com/markrussinovich/archive/2008/11/17/3155406.aspx])

"One of the roles that the paging file plays is defining the system's commit limit and how processes contribute to the commit charge."

So what does that mean? Each application contributes to what is called commit charge on the system, and is essentially the amount of virtual memory that can be allocated by all processes as a whole on the system. For example if you had 1GB of RAM and a 2GB paging file, the system commit limit would be 3GB, as this is the total amount of virtual memory allocation that the Windows memory manager can satisfy at any given point. This is where the dreaded "out of memory" error springs from (not so much nowadays). If you were using 2.5GB of that commit limit already and a program tried to use/commit 600MB of memory, Windows will error out because it can't commit to the application that it actually has someplace to store those pages! (i.e. you can't fit 600MB into the remaining 500MB of system commit).

Seeing as we've run out of system commit in our example above, what you would do is either (preferred) buy more RAM, or (preferred for the sake of this thread) increase the size of your page file.

Now what happens if we move the page file to a RAM disk? Lets stick with our crappy system of 1GB of RAM and a 2GB paging file (min/max). Already, we can't put the paging file into a RAM disk! There isn't enough RAM! Okay, so lets make our system a little more modern with a 64-bit OS, we'll give it 4GB of RAM with a 2GB paging file. The total commit limit is 6GB. Let's pretend we have all of that available commit (impossible in the real world, but this makes it simple for the explanation).

The 2GB paging file is now on a RAM disk which is 2GB in size. The RAM disk driver has reserved 2GB of virtual memory. Windows has said "yes, I will reserve this memory for you for some time in the future".
Note that reservation alone does NOT contribute to commit usage, but in the case of the 2GB page file (because the page file is always completely allocated), the RAM disk is definitely using (committed) all of that memory it asked for/reserved. Its commit charge on the system as a whole is 2GB. The amount of system commit available is now only 4GB.

Note also that there are other memory usages that don't contribute to system commit either, but they are beyond the scope of this thread.

Let's say you fire up "HugeMemoryHogApp.exe", which at start up tries to allocate 3GB of virtual memory and touches/uses/commits it all. This succeeds, and brings our system commit down to a mere 1GB. What the hell, I had 6GB system commit available and only ran that one application which committed 3GB of memory, I should still have 3GB! Ah, but you see, the RAM disk has committed 2GB of its own to store the page file... in RAM! So effectively it is like having no page file at all. Without the page file you'd have 4GB total system commit, and that big allocation of 3GB brings system commit down to... guess what, 1GB (we've seen that number before, haven't we)!

The bottom line: You cannot extend something (virtual memory/system commit) by storing that extension (page file) in the thing you want to extend (virtual memory/system commit)!

EDIT: If my explanation is not good enough to convince anyone, I went ahead and asked Windows kernel developer (now on the Windows Azure team), Mark Russinovich for his opinion on the matter... here is the correspondence:
Quote:
From: Mark Russinovich [ --snip-- ]
Sent: Tuesday, 3 January 2012 10:44 PM
To: --snip--
Subject: RE: TechNet Blogs: Contact request: Page file on a RAM disk

Yes, putting the pagefile on a RAM disk is ridiculous.

From: --snip--
Sent: Monday, January 02, 2012 11:38 PM
To: Mark Russinovich
Subject: TechNet Blogs: Contact request: Page file on a RAM disk

Subject: Page file on a RAM disk

Hey Mark! I was wondering if I could potentially ask you a question regarding the page file? I have quite an understanding of how Windows manages memory and how the page file contributes to things like system commit, etc, but time and time again I see the recommendation to place your page file on a RAM disk.

This just seems counter-intuitive to me, and if one of the roles of the page file is defining the system commit limit, then doesn't it seem silly to extend the amount of virtual memory that can be allocated by allocating virtual memory (which is what the RAM disk driver would have to do...)?

What could be other potential repercussions of storing the page file on a RAM disk?

Hope you can spare 5 minutes or so... I bet you're working hard on Windows Azure!

Cheers!
It is so ridiculous he didn't even bother to list any additional repercussions! Hopefully this convinces most people
smile.gif
 

·
Premium Member
Joined
·
10,774 Posts
Discussion Starter · #3 ·
Quote:
Originally Posted by dragosmp View Post

Nice explanation. Dunno if you agree, but in stead of allocating Pagefile on a Ramdisk the user might do away with the pagefile altogether and in this case when a soft.exe demands x MB RAM & y MB of Virtual memory what will happen is Win will allocate simply x+y RAM. This way the maximum commit charge is the RAM.
Thanks
biggrin.gif


And yep, indeed I do agree. As explained, system commit can actually end up being the size of RAM anyway with a page file stored on a RAM disk!

Only thing I'd like to correct is that "soft.exe" can't demand "physical" memory (RAM) - only virtual memory
smile.gif
 

·
Storage Nut
Joined
·
21,146 Posts
Nice post! Thanks for actually putting a good explanation out rather than just saying don't do it.
smile.gif
 

·
Premium Member
Joined
·
3,285 Posts
Good post.

I partitioned a few GB of space off an HDD that I use only for backup space that I dedicated to Pagefile. It's a SATA 6Gb/s 1.5TB drive. What are your thoughts on this?
 

·
Premium Member
Joined
·
10,774 Posts
Discussion Starter · #7 ·
Quote:
Originally Posted by grishkathefool View Post

Good post.
I partitioned a few GB of space off an HDD that I use only for backup space that I dedicated to Pagefile. It's a SATA 6Gb/s 1.5TB drive. What are your thoughts on this?
Yep, that's definitely fine
smile.gif

Edit: Thanks for your title change suggestion, have simplified it a little!
 

·
Premium Member
Joined
·
10,774 Posts
Discussion Starter · #10 ·
Quote:
Originally Posted by Erick View Post

But what if,
someone has let's say 32GB ram.
Dont you think moving the pagefile to a RAM DISK is a good ideia ?
Much faster data transferes, and no problem with low memory commit.
You explanation is great for low memory systems, but what about systems with huge memories?
No, because in the case of 32GB of RAM, paging will hardly occur. There's plenty for process working sets, and the memory manager will move least frequently used pages into the standby list, which is in RAM instead of having to write them out to disk. Note that modified pages will get moved to the modified page list, and eventually written to disk when memory pressure becomes high. With 32GB of RAM, this will hardly happen.

You are just taking up RAM for something that probably won't be accessed frequently.
Look at it this way: If placing the page file on a RAM disk with 32GB of RAM is a good idea because of faster data transfer, then why not just use RAM instead! (Which is what will happen).
 

·
Premium Member
Joined
·
10,774 Posts
Discussion Starter · #12 ·
Quote:
Originally Posted by Erick View Post

but aren't there some programs that use page file independent of the size of the RAM?
I used to play anno 1404, that would only use about 1.5gb RAm and 10gb of pagefile, instead of using 4gb ram and 6gb pagefile.
Now with anno 2070 i havent checked to see what it does.
No. Programs don't use the page file directly. They don't get to choose where their memory is stored. They simply ask Windows for a chunk of memory, and it either satisfies the request if it fits within system commit, or fails if it does not. (Admittedly this is a very watered down explanation).

This is the important distinction between virtual memory and physical memory.

What you were most likely seeing was 1.5GB of virtual memory committed, and 10GB total virtual address space usage (this wouldn't be possible on 32-bit systems, though as the process address space is only 2GB regardless!). It may have reserved 10GB virtual memory, but only actually touched 1.5GB of it. Only when a program touches its reservation (write/read) does Windows commit it.
 

·
Premium Member
Joined
·
10,774 Posts
Discussion Starter · #14 ·
Quote:
Originally Posted by Erick View Post

I see... thanks!
No problem... I will admit it gets bloody confusing though! The fact that Task Manager also fails to represent certain system counters properly (they use weird and confusing labels) doesn't really help... I think the guy who wrote Task Manager tried to dumb it down a little bit too much.
biggrin.gif
 

·
Premium Member
Joined
·
13,796 Posts
Quote:
Originally Posted by Erick View Post

Yeah very confusing.
Just so to be clear, I have 16gb RAM disable or not pagefile?
Just leave page file on but lower it to something like 1GB. If you get low memory errors, increase page file. If you are just using it for gaming/casual uses, page file and ram will not be used much at all with 16GB ram.
 

·
Audiofool
Joined
·
3,486 Posts
Very informative. Thanks. Rep +
 

·
Premium Member
Joined
·
10,774 Posts
Discussion Starter · #18 ·
Quote:
Originally Posted by CJRhoades View Post

Very informative. Thanks. Rep +
Thanks
thumb.gif

Quote:
Originally Posted by Riou View Post

Just leave page file on but lower it to something like 1GB. If you get low memory errors, increase page file. If you are just using it for gaming/casual uses, page file and ram will not be used much at all with 16GB ram.
+1 to that
smile.gif
 

·
Registered
Joined
·
732 Posts
Quote:
Originally Posted by tompsonn View Post

No, because in the case of 32GB of RAM, paging will hardly occur. There's plenty for process working sets, and the memory manager will move least frequently used pages into the standby list, which is in RAM instead of having to write them out to disk. Note that modified pages will get moved to the modified page list, and eventually written to disk when memory pressure becomes high. With 32GB of RAM, this will hardly happen.
You are just taking up RAM for something that probably won't be accessed frequently.
Look at it this way: If placing the page file on a RAM disk with 32GB of RAM is a good idea because of faster data transfer, then why not just use RAM instead! (Which is what will happen).
Programs can directly effect paging or require it much like Dawn of War. I had 8gigs of memory and dropped my pagefile to 512MB and DoW refused to run saying it needs 1.5GB of pagefile. So I switched over to my 4GB RAM system that has exactly 1GB of Pagefile and the same error occurred. Changed the 4GB RAM system pagefile to 1.5GB and the game ran. Did the same to my 8GB ram system and again, the game ran. Some games are picky.
 

·
Premium Member
Joined
·
10,774 Posts
Discussion Starter · #20 ·
Quote:
Originally Posted by Sarec View Post

Programs can directly effect paging or require it much like Dawn of War. I had 8gigs of memory and dropped my pagefile to 512MB and DoW refused to run saying it needs 1.5GB of pagefile. So I switched over to my 4GB RAM system that has exactly 1GB of Pagefile and the same error occurred. Changed the 4GB RAM system pagefile to 1.5GB and the game ran. Did the same to my 8GB ram system and again, the game ran. Some games are picky.
Yes you're right, they can do that - although they really shouldn't be. Note that they were most likely just checking the size of the current page file - they still aren't allowed to directly allocate from it. A better way would have been to check if system commit was high enough. An even better way would be to just try and allocate the required amount of memory and let the Windows memory manager do the dirty work
smile.gif
 
1 - 20 of 328 Posts
This is an older thread, you may not receive a response, and could be reviving an old thread. Please consider creating a new thread.
Top