Overclock.net › Forums › Industry News › Software News › [W8B] Building the next generation file system for Windows: ReFS
New Posts  All Forums:Forum Nav:

[W8B] Building the next generation file system for Windows: ReFS

post #1 of 87
Thread Starter 
Building the next generation file system for Windows: ReFS

312

Quote:
We wanted to continue our dialog about data storage by talking about the next generation file system being introduced in Windows 8. Today, NTFS is the most widely used, advanced, and feature rich file system in broad use. But when you’re reimagining Windows, as we are for Windows 8, we don’t rest on past successes, and so with Windows 8 we are also introducing a newly engineered file system. ReFS, (which stands for Resilient File System), is built on the foundations of NTFS, so it maintains crucial compatibility while at the same time it has been architected and engineered for a new generation of storage technologies and scenarios. In Windows 8, ReFS will be introduced only as part of Windows Server 8, which is the same approach we have used for each and every file system introduction. Of course at the application level, ReFS stored data will be accessible from clients just as NTFS data would be. As you read this, let’s not forget that NTFS is by far the industry’s leading technology for file systems on PCs.

This detailed architectural post was authored by Surendra Verma, a development manager on our Storage and File System team, though, as with every feature, a lot of folks contributed. We have also used the FAQ approach again in this post.
--Steven

PS: Don't forget to track us on @buildwindows8 where we were providing some updates from CES.

The key goals of ReFS are:

•Maintain a high degree of compatibility with a subset of NTFS features that are widely adopted while deprecating others that provide limited value at the cost of system complexity and footprint.
•Verify and auto-correct data. Data can get corrupted due to a number of reasons and therefore must be verified and, when possible, corrected automatically. Metadata must not be written in place to avoid the possibility of “torn writes,” which we will talk about in more detail below.
•Optimize for extreme scale. Use scalable structures for everything. Don’t assume that disk-checking algorithms, in particular, can scale to the size of the entire file system.
•Never take the file system offline. Assume that in the event of corruptions, it is advantageous to isolate the fault while allowing access to the rest of the volume. This is done while salvaging the maximum amount of data possible, all done live.
•Provide a full end-to-end resiliency architecture when used in conjunction with the Storage Spaces feature, which was co-designed and built in conjunction with ReFS.


The key features of ReFS are as follows (note that some of these features are provided in conjunction with Storage Spaces).:

•Metadata integrity with checksums
•Integrity streams providing optional user data integrity
•Allocate on write transactional model for robust disk updates (also known as copy on write)
•Large volume, file and directory sizes
•Storage pooling and virtualization makes file system creation and management easy
•Data striping for performance (bandwidth can be managed) and redundancy for fault tolerance
•Disk scrubbing for protection against latent disk errors
•Resiliency to corruptions with "salvage" for maximum volume availability in all cases
•Shared storage pools across machines for additional failure tolerance and load balancing

311


Source: Windows 8 Blog

http://blogs.msdn.com/b/b8/archive/2012/01/16/building-the-next-generation-file-system-for-windows-refs.aspx
Edited by Eduardv - 1/16/12 at 4:08pm
Black Dragon
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel Core i7 2600k @ 4.3 GHZ ASRock Z68 Extreme3 Gen3 (PCI-E 3.0, USB 3.0, B... EVGA GTX 580 1.5 GB GDDR5 G.SKILL Ripjaws X ( 4 x 2GB) @ 1866 MHZ 8-9-8-2... 
Hard DriveCoolingOSMonitor
OCZ Vertex 3 MAX IOPS 120 GB Firmware 2.21 Noctua NH-D14 Windows 8.1 X64 Alienware Optx AW2310 (1920 x 1080p) 120 HZ 
KeyboardPowerCaseMouse
Comfort Curve Keyboard 2000 Corsair AX850 PLUS GOLD Certified Silverstone FT02B-W Razer Deathadder 3500 DPi 
Mouse Pad
Steel Series QCK+ 
  hide details  
Reply
Black Dragon
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel Core i7 2600k @ 4.3 GHZ ASRock Z68 Extreme3 Gen3 (PCI-E 3.0, USB 3.0, B... EVGA GTX 580 1.5 GB GDDR5 G.SKILL Ripjaws X ( 4 x 2GB) @ 1866 MHZ 8-9-8-2... 
Hard DriveCoolingOSMonitor
OCZ Vertex 3 MAX IOPS 120 GB Firmware 2.21 Noctua NH-D14 Windows 8.1 X64 Alienware Optx AW2310 (1920 x 1080p) 120 HZ 
KeyboardPowerCaseMouse
Comfort Curve Keyboard 2000 Corsair AX850 PLUS GOLD Certified Silverstone FT02B-W Razer Deathadder 3500 DPi 
Mouse Pad
Steel Series QCK+ 
  hide details  
Reply
post #2 of 87
Nice, raid without the raid I guess....
Or am I reading this wrong?
post #3 of 87
Thread Starter 
Quote:
Originally Posted by HardheadedMurphy View Post

Nice, raid without the raid I guess....
Or am I reading this wrong?

If you are talking about data Striping,the yes,a file can be allocated in 2 physical drives, hopefully with RAID?This provides more data access throughput, which avoids causing the processor to idle wait for data access.However i can be wrong, I am interested in this hoever i dont know much about it smile.gif

What i see more interesting is:

•Disk scrubbing for protection against latent disk errors
•Resiliency to corruptions with "salvage" for maximum volume availability in all cases
Black Dragon
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel Core i7 2600k @ 4.3 GHZ ASRock Z68 Extreme3 Gen3 (PCI-E 3.0, USB 3.0, B... EVGA GTX 580 1.5 GB GDDR5 G.SKILL Ripjaws X ( 4 x 2GB) @ 1866 MHZ 8-9-8-2... 
Hard DriveCoolingOSMonitor
OCZ Vertex 3 MAX IOPS 120 GB Firmware 2.21 Noctua NH-D14 Windows 8.1 X64 Alienware Optx AW2310 (1920 x 1080p) 120 HZ 
KeyboardPowerCaseMouse
Comfort Curve Keyboard 2000 Corsair AX850 PLUS GOLD Certified Silverstone FT02B-W Razer Deathadder 3500 DPi 
Mouse Pad
Steel Series QCK+ 
  hide details  
Reply
Black Dragon
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel Core i7 2600k @ 4.3 GHZ ASRock Z68 Extreme3 Gen3 (PCI-E 3.0, USB 3.0, B... EVGA GTX 580 1.5 GB GDDR5 G.SKILL Ripjaws X ( 4 x 2GB) @ 1866 MHZ 8-9-8-2... 
Hard DriveCoolingOSMonitor
OCZ Vertex 3 MAX IOPS 120 GB Firmware 2.21 Noctua NH-D14 Windows 8.1 X64 Alienware Optx AW2310 (1920 x 1080p) 120 HZ 
KeyboardPowerCaseMouse
Comfort Curve Keyboard 2000 Corsair AX850 PLUS GOLD Certified Silverstone FT02B-W Razer Deathadder 3500 DPi 
Mouse Pad
Steel Series QCK+ 
  hide details  
Reply
post #4 of 87
Introduced in Windows Server 8? Will there be a way to use it with consumer Windows 8?
Neutrino
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel Core i7 6700K Asus Z170I PRO GAMING Gigabyte GTX 1070 Mini ITX Corsair 16GB (2 x 8GB) DDR4-3200MHz 
Hard DriveCoolingCoolingOS
Crucial MX300 750GB Scythe Big Shuriken 2 rev. B Cryorig XT140 fan Windows 10 Pro 
KeyboardPowerCaseMouse
Topre Realforce 87u 55g Corsair SF450 Lazer3D LZ7 Logitech G303 
Mouse PadAudio
Steelseries 4HD Audio Technica ATH-M50 
  hide details  
Reply
Neutrino
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel Core i7 6700K Asus Z170I PRO GAMING Gigabyte GTX 1070 Mini ITX Corsair 16GB (2 x 8GB) DDR4-3200MHz 
Hard DriveCoolingCoolingOS
Crucial MX300 750GB Scythe Big Shuriken 2 rev. B Cryorig XT140 fan Windows 10 Pro 
KeyboardPowerCaseMouse
Topre Realforce 87u 55g Corsair SF450 Lazer3D LZ7 Logitech G303 
Mouse PadAudio
Steelseries 4HD Audio Technica ATH-M50 
  hide details  
Reply
post #5 of 87
Thread Starter 
Quote:
Originally Posted by Vowels View Post

Introduced in Windows Server 8? Will there be a way to use it with consumer Windows 8?

Yes,Windows 8 will be ReFs capable.
Black Dragon
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel Core i7 2600k @ 4.3 GHZ ASRock Z68 Extreme3 Gen3 (PCI-E 3.0, USB 3.0, B... EVGA GTX 580 1.5 GB GDDR5 G.SKILL Ripjaws X ( 4 x 2GB) @ 1866 MHZ 8-9-8-2... 
Hard DriveCoolingOSMonitor
OCZ Vertex 3 MAX IOPS 120 GB Firmware 2.21 Noctua NH-D14 Windows 8.1 X64 Alienware Optx AW2310 (1920 x 1080p) 120 HZ 
KeyboardPowerCaseMouse
Comfort Curve Keyboard 2000 Corsair AX850 PLUS GOLD Certified Silverstone FT02B-W Razer Deathadder 3500 DPi 
Mouse Pad
Steel Series QCK+ 
  hide details  
Reply
Black Dragon
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel Core i7 2600k @ 4.3 GHZ ASRock Z68 Extreme3 Gen3 (PCI-E 3.0, USB 3.0, B... EVGA GTX 580 1.5 GB GDDR5 G.SKILL Ripjaws X ( 4 x 2GB) @ 1866 MHZ 8-9-8-2... 
Hard DriveCoolingOSMonitor
OCZ Vertex 3 MAX IOPS 120 GB Firmware 2.21 Noctua NH-D14 Windows 8.1 X64 Alienware Optx AW2310 (1920 x 1080p) 120 HZ 
KeyboardPowerCaseMouse
Comfort Curve Keyboard 2000 Corsair AX850 PLUS GOLD Certified Silverstone FT02B-W Razer Deathadder 3500 DPi 
Mouse Pad
Steel Series QCK+ 
  hide details  
Reply
post #6 of 87
Quote:
Originally Posted by Eduardv View Post

Yes,Windows 8 will be ReFs capable.

Isn't the consumer version of 8 using NTFS still with some hooks to be able to access and read ReFS? It won't be the full blown file system until probably windows 9 if I read that right.
post #7 of 87
Thread Starter 
Quote:
Originally Posted by sticks435 View Post

Isn't the consumer version of 8 using NTFS still with some hooks to be able to access and read ReFS? It won't be the full blown file system until probably windows 9 if I read that right.

I am afraid so,they wont push the entire filesystem at once,it will be done with incremental upgrades.However we will se good portion in Windows 8,to the part in wich we can creat ReFs volumes.

For compatibility reasosn Windows 7 will be able to read ReFS volumes but it cant access it.
Black Dragon
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel Core i7 2600k @ 4.3 GHZ ASRock Z68 Extreme3 Gen3 (PCI-E 3.0, USB 3.0, B... EVGA GTX 580 1.5 GB GDDR5 G.SKILL Ripjaws X ( 4 x 2GB) @ 1866 MHZ 8-9-8-2... 
Hard DriveCoolingOSMonitor
OCZ Vertex 3 MAX IOPS 120 GB Firmware 2.21 Noctua NH-D14 Windows 8.1 X64 Alienware Optx AW2310 (1920 x 1080p) 120 HZ 
KeyboardPowerCaseMouse
Comfort Curve Keyboard 2000 Corsair AX850 PLUS GOLD Certified Silverstone FT02B-W Razer Deathadder 3500 DPi 
Mouse Pad
Steel Series QCK+ 
  hide details  
Reply
Black Dragon
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel Core i7 2600k @ 4.3 GHZ ASRock Z68 Extreme3 Gen3 (PCI-E 3.0, USB 3.0, B... EVGA GTX 580 1.5 GB GDDR5 G.SKILL Ripjaws X ( 4 x 2GB) @ 1866 MHZ 8-9-8-2... 
Hard DriveCoolingOSMonitor
OCZ Vertex 3 MAX IOPS 120 GB Firmware 2.21 Noctua NH-D14 Windows 8.1 X64 Alienware Optx AW2310 (1920 x 1080p) 120 HZ 
KeyboardPowerCaseMouse
Comfort Curve Keyboard 2000 Corsair AX850 PLUS GOLD Certified Silverstone FT02B-W Razer Deathadder 3500 DPi 
Mouse Pad
Steel Series QCK+ 
  hide details  
Reply
post #8 of 87
Reason enough to buy Windows 8 just to play with this. I wonder if it would be possible to run a separate drive with the new file system to take advantage of it while still running Windows 8 on NTFS?
My Rig
(13 items)
 
  
CPUMotherboardGraphicsRAM
Phenom II X3 720/(X4 @ 3.0 GHz) GIGABYTE MA770T-UD3P Sapphire Radeon HD 5770 (950/1435) 2x2GB G.Skill Ripjaw DDR3-1600 
Hard DriveOSMonitorPower
WD 1TB 7200 32MB Windows 7 Ultimate (x64) Acer H213H (1080p) Corsair 750TX 
MouseMouse Pad
Logitech G5 fUnc Surface 1030 
  hide details  
Reply
My Rig
(13 items)
 
  
CPUMotherboardGraphicsRAM
Phenom II X3 720/(X4 @ 3.0 GHz) GIGABYTE MA770T-UD3P Sapphire Radeon HD 5770 (950/1435) 2x2GB G.Skill Ripjaw DDR3-1600 
Hard DriveOSMonitorPower
WD 1TB 7200 32MB Windows 7 Ultimate (x64) Acer H213H (1080p) Corsair 750TX 
MouseMouse Pad
Logitech G5 fUnc Surface 1030 
  hide details  
Reply
post #9 of 87
Every time a new windows version is announced, they claim to overhaul the file structure... I'm still waiting, windowx XP!
Arcane
(33 items)
 
Sanctum
(9 items)
 
Fukurou
(15 items)
 
CPUMotherboardGraphicsRAM
Ryzen 7 1700 MSI B350M Mortar Arctic RX Vega 64 G.Skill TridentZ  
Hard DriveHard DriveOSPower
Samsung EVO 960 Consatellation Windows 10 Pro Corsair HX 1050 
Case
Raijintek Aeneas White Window 
CPUMotherboardRAMHard Drive
Core i5-661 GA-H55-USB3 12GB DDR3 1333 WD Caviar Black AALS 640GB 
Hard DriveHard DriveOptical DriveCooling
WD Caviar Green EARS 2TB Vertex Turbo 60GB (SSD) Samsung Blu-ray, Samsung DVD Burner H50 (With push/pull nocturas) 
OSMonitorKeyboardPower
Windows 10 Professional 64 LG 47inch LED LCD Saitek Eclipse Lite-touch Wireless Seasonic X750 Gold 
CaseAudio
CM690 II Passive bookshelfspeakers + amp. 
  hide details  
Reply
Arcane
(33 items)
 
Sanctum
(9 items)
 
Fukurou
(15 items)
 
CPUMotherboardGraphicsRAM
Ryzen 7 1700 MSI B350M Mortar Arctic RX Vega 64 G.Skill TridentZ  
Hard DriveHard DriveOSPower
Samsung EVO 960 Consatellation Windows 10 Pro Corsair HX 1050 
Case
Raijintek Aeneas White Window 
CPUMotherboardRAMHard Drive
Core i5-661 GA-H55-USB3 12GB DDR3 1333 WD Caviar Black AALS 640GB 
Hard DriveHard DriveOptical DriveCooling
WD Caviar Green EARS 2TB Vertex Turbo 60GB (SSD) Samsung Blu-ray, Samsung DVD Burner H50 (With push/pull nocturas) 
OSMonitorKeyboardPower
Windows 10 Professional 64 LG 47inch LED LCD Saitek Eclipse Lite-touch Wireless Seasonic X750 Gold 
CaseAudio
CM690 II Passive bookshelfspeakers + amp. 
  hide details  
Reply
post #10 of 87
Its like it has built-in TRIM-like support. I'm so glad I get Windows for free through my engineering department at school! Personally, I don't see anything wrong with the current file system. I thought the biggest under-the-hood news for Windows 8 was the registry was getting a revamp.
Po' Pimpin'
(11 items)
 
  
CPUMotherboardRAMHard Drive
i5 2500k @ stock Biostar TZ68K+ [A3] 4GB  Sandforce 1222 64GB SSD 
Optical DriveCoolingOSMonitor
LG 22x DVD-+RW  Stock Windows 7 x64 Acer S211HL 1080p 
PowerCaseMouse
600w Diablotek Linkworld Electronic Inland 
  hide details  
Reply
Po' Pimpin'
(11 items)
 
  
CPUMotherboardRAMHard Drive
i5 2500k @ stock Biostar TZ68K+ [A3] 4GB  Sandforce 1222 64GB SSD 
Optical DriveCoolingOSMonitor
LG 22x DVD-+RW  Stock Windows 7 x64 Acer S211HL 1080p 
PowerCaseMouse
600w Diablotek Linkworld Electronic Inland 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Software News
Overclock.net › Forums › Industry News › Software News › [W8B] Building the next generation file system for Windows: ReFS