{Guide} Create a Gaming Virtual Machine

dizzy4 · Jan 25, 2012

Dizzy's Guide to Creating a Gaming VM
Beta version 0.20.1

Virtualization is the future of computing and is already being used in many capacities today. In writing this guide I hope to educate our community to it's benefits, usage and limitations. The more support it has, the further developed it will become. That is also my goal for this guide; to see it become simplified and offer broader support for hardware and software.

Some Benefits:

Reset the VM without having to hard reset the system (WIP)
Uninterrupted sessions of other VMs
Ability to run the latest games on a VM
Being able to use all that extra Memory and those Cores

Before we begin, be sure to make sure your system is VT-d or AMD-Vi (IOMMU) capable.

VT-d is an extension on top of the normal Virtualization instructions provided by intel, and IOMMU (AMD-Vi) is the AMD version. These are not the same as VT-x and AMD-V. They are extra and you need to be sure you have them on your system before attempting this guide.

For Intel this requires:

(www.ark.intel.com)

CPU with VT-x Socket 775 and newer. Vpro rated chipset with VT-d in BIOS is also required by 775 CPUs (Most likely Q45)
or
Core i5/i7 or Xeon Chip with VT-x and VT-d. Only Non-K, except C2 stepping 3930k and 3960x chips have the right instructions
and
Motherboard with VT-d option in BIOS. (ASrock and Zotac have a good selection)

For AMD this requires:

Any reasonably new AMD processor (AM2 and newer or socket 940 Opteron and newer)
and
Proper chipset. 880fx and 990fx or most server boards should have IOMMU. Check with the manufacturer or in the manual
and
IOMMU (AMD-Vi) option enabled in BIOS (ASrock or ASUS 880/990FX and most server boards)

Video card is your choice. I suggest using a newer Radeon for now because it has good compatibility with Xen passthrough and HD audio on the card which means you won't need to pass through the motherboard audio to get sound from the VM. Of course, you could pass motherboard or another discrete Audio device through too.

Step 1:

Fedora 20 Download -- Choose your flavor (Warning! LXDE does not currently have GUI bluetooth support)

Step 2:

See your motherboard manual for more info. This will be in northbridge options and listed as VT-d for intel and IOMMU or AMD-Vi for AMD. (Note that VT-x and AMD-V are not enough for this tutorial)

Disclaimer -- Back up any and all data you wish to keep before attempting this guide. I am working on a version that preserves data, but for now this is where I am.

Step 3:

Boot the CD which is a live cd for F20 and run the install program from the applications list. Follow the prompts. Most importantly, choose "Specialized Storage Devices" which will let you set up software/ firmware RAID or otherwise and select the install medium. Next, choose a computer name. I chose the name xenhost.xenhome. The Root account is your administrator, so be sure to set a safe password.

Step 3a:

This is if you wish to contain your virtual machine disks as files, not as partitions.

Select the disk on which you wish to install Fedora.
Select other or custom partitioning
Select recommended layout
Delete the /home partition
Resize the / partition to the largest it will go
Continue the installation

Step 3b:

For more advanced users:

You can create an LVM setup and manage your VMs disks this way or you could even add custom mount points for each VM. Please do not attempt unless you are a power user or want to do some research on linux partioning and LVM storage.

I created a user called xenhost. Don't forget to check the box to add the user to administrators.

For this next session you will be using the terminal program provided with linux. You will be asked to use 'su' and 'sudo' commands which will require a password. When the prompt comes up, type your password. It is being entered even when no '*' show up.

Step 4:

Updating the system is very easy.

1. Download and install my kernel + headers with passthrough built in (I find that building it into the kernel is much more reliable and simpler than loading a module)
-- Kernel 3.12.8-300
-- Kernel-headers 3.12.8-300

2. After downloading the attachments just double-click them and install both

3. Install Xen (4.3.1 currently) In a terminal:

Code:

Code:

$ sudo yum install xen

4. Update your GRUB2 bootloader

in a terminal:

Code:

Code:

grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg

in a terminal:

Code:

Code:

grub2-mkconfig -o /boot/grub2/grub.cfg

Disable selinux:

Code:

Code:

$ sudo nano /etc/selinux/config

change the line that says "enforcing" to "permissive" or "disabled" -- I suggest disabled for new users
You can use whichever text editor you want. If you run Gnome, replace nano with gedit and if you run xfce or LXDE replace nano with leafpad
Save and Exit.
-- At this point your system is able to be booted with xen as the hypervisor --

Step 5:

This step depends on you having a windows install disk and the means to Create a .iso file of your install disk. name the file win7ult.iso (or whatever you want to call it) and transfer it to your home folder in the xenhost linux install. This is critical for the configuration file later.

Step 5a:

I have started using this method because it makes migrating and backing up virtual machines very easy.

Determine the size of the main disk for the VM (I keep mine on SSD and use a virtual server to host other files)
Create a directory to house your file(s)

Code:
Code:
```
$ mkdir ~/virtual_disks
```
You can name it anything you want

Create blank files to house your VMs

Code:

Code:

$ cd ~/virtual_disks
$ dd if=/dev/zero of=~/virtual_disks/VM_diskname bs=1M count=size_in_Megabytes

Repeat for all the VMs you wish to host

Step 5b:

I have started using the other method because it makes migrating and backing up virtual machines very easy. I know that LVMs are great too, but I am not an LVM master.

Please read the comments in the configuration file example to see how to use physical devices (LVM or physical partitions) as storage.

-- Virtual Drives should now be allocated --

Code:

$ sudo yum install pciutils
$ lspci
00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)
00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09)
00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)
00:1a.0 USB Controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05)
00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 05)
00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5)
00:1c.3 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 4 (rev b5)
00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b5)
00:1c.5 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 6 (rev b5)
00:1d.0 USB Controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05)
00:1f.0 ISA bridge: Intel Corporation H67 Express Chipset Family LPC Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family 6 port SATA AHCI Controller (rev 05)
00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 05)
01:00.0 VGA compatible controller: ATI Technologies Inc Cypress [Radeon HD 5800 Series]
01:00.1 Audio device: ATI Technologies Inc Cypress HDMI Audio [Radeon HD 5800 Series]
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)
04:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8191SEvB Wireless LAN Controller (rev 10)
05:00.0 USB Controller: Device 1b6f:7023 (rev 01)

That is a sample of what my PCI devices look like. You can see the first thing on each line is the address of the PCI device. Write down the devices you wish to allocate to each virtual machine. Since I run 2 graphical virtual machines simultaneously, I have given each a video device and each a USB controller. I also dedicated my extra gigabit port to my media/file server and my wireless controller to my windows guest. The network bridge can handle all this, but I wanted extra fallback and throughput.

Code:

$ ifconfig

Take notice of the entry that is your ethernet. Mine was titled p4p1. Be sure to write down the number after ether which is the MAC address.

Code:

Code:

$ sudo nano /etc/sysconfig/network-scripts/ifcfg-p4p1

edit the file to look like this. Replace the DEVICE= with the device and HWADDR with the ether you wrote down

Code:

Code:

DEVICE=p4p1
HWADDR=94:de:80:xx:xx:xx
ONBOOT=yes
BRIDGE=xenbr0
DELAY=0
NM_CONTROLLED=no
BOOTPROTO=none
TYPE=Ethernet

then create the bridge

Code:

Code:

$ sudo nano /etc/sysconfig/network-scripts/ifcfg-xenbr0

Edit the file to look like this:

Code:

Code:

DEVICE=xenbr0
TYPE=Bridge
BOOTPROTO=dhcp
ONBOOT=yes
DELAY=0
NM_CONTROLLED=no
BRIDGE_STP=on

Then restart the network

Code:

Code:

$ sudo service network restart

Code:

$ gedit /etc/xen/win8.sxp

Copy and paste this code below into the file

Code:

Code:

# =====================================================================
# Example HVM guest configuration
# =====================================================================

# This configures an HVM rather than PV guest
builder = "hvm"

# Use this device model! Upstream qemu has poor support for GFX passthrough -- esp. intel
device_model_version = "qemu-xen-traditional"

# Guest name
name = "VM_name"

# Enable Microsoft Hyper-V compatibile paravirtualisation /
# enlightenment interfaces. Turning this on can improve Windows guest
# performance and is therefore recommended
viridian = 1

# Initial memory allocation (MB)
memory = 8224

# Number of VCPUS
vcpus = 7
acpi=1
apic=1
#pae=1 
#PAE is 32-bit only

# Network devices -- You can leave this vif = [ '' ] if you want DHCP to do the work
vif = [ 'bridge=xenbr0,ip=192.168.40.41' ]

# Disk Devices -- Replace filename and OS_install.iso with the names you chose earlier
disk = [ '/home/xenhost/virtual_drives/filename,raw,hda,rw', '/home/xenhost/OS_install.iso,hdb:cdrom,rw' ]
boot = "dc"

#Passthrough devices are inserted, but will be commented out for the time being because we need to install first.

#gfx_passthru=1
#pci = [ '01:00.0', '01:00.1', '00:1a.0', '04:00.0' ]

#nographic = 1
vnc = 1

Save and exit

Step 6:

No PCI devices will be passed through for the install phase of this, step 7 will instruct on how to install drivers and pass devices through properly.
Install a VNC viewer.

Code:

Code:

$ sudo yum install tigervnc

Start the VM and connect to its VNC port.

Code:

Code:

$ sudo xl create win8.sxp
$ vncviewer localhost

Complete the install of the VM (in this case windows) and make sure it's stable (fully cycled through the reboots)

Repeat this for each VM

This step may cause you not to have graphical access to the Domain-0 anymore. Especially if you have only 1 video card or plan on assigning them all. This can easily be solved by connecting to the computer from another network device via SSH. Enable SSH:

Code:

Code:

$ sudo chkconfig sshd on

Tunnel in using SSH and x11 forwarding

Connecting From Windows:

Start by downloading xming and putty
Follow a default install of Xming and then run putty. Then follow these steps:

Input your Xen machine's IP address

Then open the ssh + menu and click on x11. Make the fields look like this photo:

After clicking open, a terminal window will appear and ask for your login. Use xenhost like you did from within linux. Then login as su like you did before and run the virt-manager command. You should end up with something like this:

Allowing OSX to Connect:

Install x11 or XQuartz (OSX 10.6.3 or newer, older versions or vanilla x11 will work)

Connecting from Linux or OSX:

Replace IP with Xen host IP

Code:

Code:

Paul-Youngs-MacBook-Pro:~ paul$ ssh -X [email protected][/SPOILER]
[email protected]'s password: 
Last login: Tue Jan 24 22:38:03 2012 from 192.168.1.5

[B]Connecting from Host with Secondary Video Card:[/B]
[SPOILER=Simply]Make sure you are running your main display from the secondary card. If this means you have to leave the display unplugged from the primary, it is ok. In the terminal:
[code][[email protected] ~]$ su
Password: 
[[email protected] xenhost]# virt-manager

Code:

$ sudo nano /etc/default/grub

Insert a line:

Code:

Code:

GRUB_CMDLINE_XEN="iommu=1"

Add onto the end of the line GRUB_CMDLINE_LINUX=

Code:

Code:

xen-pciback.hide=(01:00.0)(01:00.1)

Add all the entries for devices you wish to pass through. Remember, if you want direct access to the computer, leave one video device unhidden and one USB PCI controller unhidden (unless you use PS/2).

Code:

Code:

$ sudo grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg

When you reboot, be sure to select the proper entry for fedora WITH xen from the GRUB menu

Step 7:

Connect to your Domain-0 after rebooting (directly or over the network as laid out)

Start your windows virtual machine and connect via VNC:

Code:

Code:

$ vncviewer localhost

Pass through the radeon GPU and Audio device with this command:

Code:

Code:

$ sudo xl pci-attach VM_name 01:00.0
$ sudo xl pci-attach VM_name 01:00.1

Replace VM_name with the name you gave it in the configuration file.
Download the latest catalyst driver and install. Then shutdown windows from the VM and turn it off from the console:

Code:

Code:

sudo xl destroy VM_name

Edit your configuration and uncomment the pci = [] line (remove #)

Be sure to reboot the entire system and restart the VM. Wait a few minutes for windows to update the devices and you should have a fully graphical VM!

This is MUCH easier. All you have to do is uncomment (remove #) in the pci = [] line and the gfx_passthru=1 line

Then start the VM:

Code:

Code:

$ sudo xl create win8.sxp

You can actually complete a full install with the integrated chips because they are able to be reset without a hard reboot. Continue using this VM normally.

NOTE: If installing GPLPV drivers, do so before activating windows!!!

Tada!

Now if you had a second card, you could make a second fully functional system! You might want to create another system to use as a minecraft server. No problem, just install another VM to use for that and allocate however much you want to it. Use virt-manager or configuration scripts to create whatever you want -- the possibilities are endless.

Thanks for sticking in there!

Help me by trying this and reporting any errors. I want to make this easier and more compatible too, so any suggestions are welcome.

Best,
Dizzy4

P.S. If you are a linux mint user, please take a look at PowerHouse's guide

It might be more complex than this, but it seems to work very well. Either guide when used properly will give good results.

dizzy4 · Jan 25, 2012

Latest News:Alpha 0.0.3: Windows 8 passthough seems to work just fine! A little trickier to install, but hey it's working! Expect better support with a beta version.

0.0.3 -- Added preliminary Windows 8 support and instructed to turn off selinux
0.0.2 -- Removed some unnecessary steps, development libraries and tools no longer needed
0.0.1 -- First version of guide, first alpha

Here is the emulated IDE performance on the Force3 logical volume

and here is my Samsung 470 series with a patriot USB 3.0 enclosure tested on the etron usb3.0 controller that was passed through

My system Hardware is the rig called Test Chamber ITX.

i7-2600 (non-k)
ASrock H67M ITX/HT
Reference HD5850
Xen 4.1.2
Fedora 16
Windows 7 64bit

evermooingcow · Jan 26, 2012

Nice job getting it fully running

+rep
Too bad not many people here seem interested in things like this.

I've been thinking about doing this too but I'm still waiting on the Radeon 7950 or 7870. Really hoping the new cards work.

Sean Webster · Jan 27, 2012

Nice guide. Bookmarked for sure.

One thing though is that the bright green, red and yellow texts hurts my eyes and is slightly hard to read, so i'd suggest you set it black. Or just make them all red and make them bold as well. And maybe use spoilers in your guide to conceal most of it till the user expands it for use? As an example you can refer to my Windows 7 Install guide if it would help.

dizzy4 · Jan 27, 2012

Quote:

Originally Posted by evermooingcow

Nice job getting it fully running

+rep
Too bad not many people here seem interested in things like this.
I've been thinking about doing this too but I'm still waiting on the Radeon 7950 or 7870. Really hoping the new cards work.

They should work no problem. The components within the new cards have improved, but the way they work is pretty much the same so I would be 99% sure they would work just fine.
Quote:

Originally Posted by Sean Webster

Nice guide. Bookmarked for sure.

One thing though is that the bright green, red and yellow texts hurts my eyes and is slightly hard to read, so i'd suggest you set it black. Or just make them all red and make them bold as well. And maybe use spoilers in your guide to conceal most of it till the user expands it for use? As an example you can refer to my Windows 7 Install guide if it would help.

Thanks for the advice! I hope this cleans it up a little. Of course I will be cleaning it up slowly as I go and I am hoping to revise the guide and even make this about 4 steps less if I can.

subassy · Jan 30, 2012

Quote:

Virtualization is the future of computing and is already being used in many capacities today. In writing this guide I hope to educate our community to it's benefits, usage and limitations. The more support it has, the further developed it will become. That is also my goal for this guide; to see it become simplified and offer broader support for hardware and software.

And yet no one seemed enthused about my proposed dedicated VM sub-forum...sorry, still bitter.

Awesome guide btw. If I hadn't been randomly browsing forums I would never have seen it. Too bad there's no way to "sym link" it to the servers forum. I bet a lot of people in there would love this

lloyd mcclendon · Feb 4, 2012

so ... how does it actually perform?

from what i've heard, the fps and such is all pretty decent. the problem is the input thread lags a few ms behind the display ... so your inputs aren't really in sync with what you see on the screen. any thoughts on that?

i didn't know xen had this feature ... will take a look. i wonder when we'll see the same thing on the kvm side. Spice is still quite unstable and so far has been a disappointment, but has some promise.

dizzy4 · Feb 6, 2012

Quote:

Originally Posted by lloyd mcclendon

so ... how does it actually perform?
from what i've heard, the fps and such is all pretty decent. the problem is the input thread lags a few ms behind the display ... so your inputs aren't really in sync with what you see on the screen. any thoughts on that?
i didn't know xen had this feature ... will take a look. i wonder when we'll see the same thing on the kvm side. Spice is still quite unstable and so far has been a disappointment, but has some promise.

When the USB controller is passed through to the VM as well there is actually no more lag then there would be natively in windows. Maybe you are thinking of using a virtualized USB port, which I would not suggest for a VM. The performance hit is very slight because the video card is actually being fully controlled by the guest operating system so there is no need for spice at that point.

On another note, this guide is about to get a lot simpler. I am working on a custom Linux Live USB image to boot from that will let anyone try this without installing anything. I am also writing a shell script that will walk users through VM setup and pass the devices through properly while being easy to understand from a beginner's perspective. So look for a huge update in a few days

Plan9 · Feb 7, 2012

I absolutely love the concept, but I'm a little confused as to why someone would want to do this as opposed to running Windows natively when you need physical access to the machine anyway.

Something like this would have been handy when I was trying to get Netflix working on Linux, but alas I just couldn't get the performance I wanted (albeit I never tried Xen and had very modest graphics cards so it was all software rendering). I might give Netflix / LoveFilm another try via this method if I get any spare time in the future

lloyd mcclendon · Feb 7, 2012

"why someone would want to do this as opposed to running Windows natively"

Well, obviously _if_ it works just as well there's no need to boot into windows. I never cared for a dual boot, huge waste of time. This way everything is kept in linux, and you have some throw away windows VM for gaming. If the performance is as good as the OP is claming, this is 1000 times better than wine, and 100000 times better than a dual boot. So if it works, i don't know why you wouldn't want to do this. Really this is HUGE news - how many linux users crawl back to windows entirely or dual boot just for gaming. And wine .. it hurts to say this, but it's practically impossible for that project to keep up with the crazyness that the MS developers create.

Apparently KVM does offer the VT-D PCI pass through ... but i'm a little fuzzy on whether or not it actually works right yet. I'll be trying to get one of my gentoo vms with xorg on it to use this... and if that actually works I should be able to get my XP VM showing good FPS as well.

I guess maybe I don't understand this - if I give a windows guest pass through access to my graphics card, and install the windows nvidia drivers package etc - what happens to the host (and the 12 other VMs) using the same graphics card?

dizzy4 · Feb 7, 2012

Quote:

Originally Posted by lloyd mcclendon

"why someone would want to do this as opposed to running Windows natively"
Well, obviously _if_ it works just as well there's no need to boot into windows. I never cared for a dual boot, huge waste of time. This way everything is kept in linux, and you have some throw away windows VM for gaming. If the performance is as good as the OP is claming, this is 1000 times better than wine, and 100000 times better than a dual boot. So if it works, i don't know why you wouldn't want to do this. Really this is HUGE news - how many linux users crawl back to windows entirely or dual boot just for gaming. And wine .. it hurts to say this, but it's practically impossible for that project to keep up with the crazyness that the MS developers create.
Apparently KVM does offer the VT-D PCI pass through ... but i'm a little fuzzy on whether or not it actually works right yet. I'll be trying to get one of my gentoo vms with xorg on it to use this... and if that actually works I should be able to get my XP VM showing good FPS as well.

It does solve a lot of headaches, but it can also create some more in the process. This far from being perfected (the guide and the technology), but it really is the future of computing. KVM does offer VT-d passthrough, but is also a type 2 hypervisor which translates to poorer performance than Xen. There is a small performance loss because it still is a VM, but compared to what we usually think of gaming on a VM this is amazing. For instance, your GTX 570s in SLI would still have enough power to play pretty much anything and a 5% loss of performance might be worth it to not have to dualboot or use Wine. Some nVidia cards are reported to work with Xen and VT-d enabled, but others are more tricky (non-reference). The current unstable Xen release is supposed to have more support for nVidia cards.
Quote:

Originally Posted by lloyd mcclendon

I guess maybe I don't understand this - if I give a windows guest pass through access to my graphics card, and install the windows nvidia drivers package etc - what happens to the host (and the 12 other VMs) using the same graphics card?

The display will go blank because the host no longer owns the resources of the card. This could be fixed by writing a script to reattach the card to the host once the gaming VM shuts down. Another fix is to get a weak little card to run your non-gaming VMs and to keep the host with a dedicated card. This is why having my i7-2600 is so good; The host and other VMs get to use the integrated graphics while the windows VM uses the HD 5850.

lloyd mcclendon · Feb 8, 2012

Quote:

Originally Posted by dizzy4

The display will go blank because the host no longer owns the resources of the card.

ok i didn't catch that at all. That kind of explains Plan9's post then. Somehow I was expecting the card to be "shared" but obviously that is nonsense. So is it possible to hot detach / reattach a GPU while either system remains running? Reading about this here it sounds like you can do it for some pci devices, but I would think a GPU is pretty fundamental to remain the same while the machine is running. If not you'd need at least two cards.

I'll see if I can get something going and throw my welfare ATI card in there for some testing. Then if that works i'll consider removing the SLI bridge and running them as separate cards (eventually get a 3rd... and 4th card), and also a KVM switch.

I picked up a killawatt meter and this thing only pulls between 400-500W from the wall ... so 4 cards is more feasible than I thought.

lloyd mcclendon · Feb 8, 2012

alright well i tried this out last night ... and no luck. KVM has long had support for PCI pass through. You simply add a "PCI host device" in the VM definition. I had seen this before but never really knew what it was. very cool. This all works by using existing kernel code and theoretically should be* faster than xen which is more userland code. _If_ it would work ...

It all uses the intel_iommu and DMA remapping parts of the kernel. Either this is just really new and still has some bugs to shake out, or it's a real problem of hardware compatibility. Booting with intel_iommu=on caused the marvell sata controller on my board to cause a kernel panic at boot. Moving the plugs for two of my drives to the other controller and disabling this controller got past that and the system boots. However when I try to startx, the nvidia driver won't load... something about DMA remappings and bits already being set.

Either this card / this version of the driver / my config is incorrect. I will try to investigate that a bit further later ..

I think if i could get X to start using these cards, I could easily pass the ATI card I plugged in directly to a VM and it would be perfect. Once I get X started there's a few steps of mapping the busID / IRQ using the pci_stub module ... If that works I'll be dedicating at least one 570 to a windows VM for gaming and ditch this wine crap.

SoulCleaver · Feb 16, 2012

Hi there,

Firstly awesome guide, found it very useful re setup of Xen, there is a few issues I have and am hoping you can help me address them (struggling to find information/guides relevant to this OS/your build):

I am running an i870 (no on chip GPU so figuring this makes a difference tbh)

Trying to start a VM with a hdd image I made (I didnt use partitions as you did due to space issues on my small test drive) I got permissions issues (also got for ISO/CD didn't matter much where I shoved the ISO either

I ran 'setenforce 0' to remove the issue prior to trying passthrough of the GPU etc

When you perform a passthrough of hardware you initially get an error stating PCI-BACK doesn't have/own the hardware, I have found out how to fix this adhoc and that's fine tbh but thought I shoud post the fix for those whom don't know

'sudo lspci' (lists hardware ids) primary gpu should usually list as 01:00.0 my secondary listed as 06:00.0 ans 06:00.1 (sound for secondary card) this was an ATI card, take a note of the ID for the hardware in the above format you will need to unbind this hardware to use it on a VM

'lspci -n' - This gets you the vendor IDs (find the hardware id as per above step in this list and the code listed like xxxx:xxxx is what you seek)

3rdly to unbind the hardware I had to run the following

'echo "xxxx xxxx" > /sys/bus/pci/drivers/pci-stub/new_id
echo "0000:06:00.0" > /sys/bus/pci/devices/0000:06:00.0/driver/unbind
echo "0000:06:00.0" > /sys/bus/pci/drivers/pci-stub/bind'

substitute xxxx xxxx for the values you got from step 2 (xxxx:xxxx format (remove the : and replace with a space as above)

After doing all that I could boot and install an OS (Windows XP and Windows 7 were both setup as testers)

Passed through a dedicated Gigabit NIC and this works fantastically (unbind etc needed as above to make this work however) in either OS

Either OS can see the GPU (tried with an ATI 4350 and a Nvidia 560TI), and it installs the drivers but I get error code 43 (seems common when googling), issue has something to do with patches needing to be installed and this is where I get extremely lost

Any ideas re the patching? Also any tips on how to permanently resolve the other issues (not having to unbind on dom0 boot would be awesome)

BTW I did the unbind etc with the primary card and this caused major issues with the dom0 seems if I ran this via ssh/putty as its the primary GPU it freezes/crashes but can reload virt-manager, though nothing seems to work after this also (any tips here would also be cool)

If I can manage to get this working I can finally wipe and partition my SSD (thus making 2x VMs on it one for me and one for Wife with full GPU passthrough from my pair of 560Ti's

Thanks for all your help in advance and congratulations on an AWESOME guide,

dizzy4 · Feb 16, 2012

It sounds like you are having a few hardware quirks, but I am glad to see it working for you! I am working on a configuration and a start-up script that will do all this for the user. Right now, I am about 85% done so stay posted. I am also working on a custom live-usb that will allow this without storage device changes that will include these scripts. SELinux can be a little bit cumbersome and I also usually disable it for use with Xen, so no worries there.

In regards to your PCI-BACK question, I have found a good amount of issues with it and my work-around was to get libvirt to disable it using virt-manager. This can also be done using the virsh console command. The script will take care of this for you, but I will mention how so you can get it up and running in the mean time. virsh nodedev-list will give you a list of all your devices. Find your PCI devices and use this command: virsh nodedev-dettach pci_0000_01_00_0. The device should match what you saw in the list and should work. Notice that "dettach" is misspelled... kind of silly, but that is just how the command is used. The next step would be to do: xm pci-attach win7a 01:00.0. This will attach it to domain 'win7a' and can be done before or after starting the VM. I hope that simplifies your unbind woes.

Nvidia support seems to be somewhat limited now. I hear that some people have had luck using certain reference cards and most of the older 8000 series cards. I am still unsure about patching, but it wouldn't be easy and would most likely require a rebuild of RPMs or a recompile from the source code. Supposedly Xen 4.2 includes the patches, but is still marked as unstable. I will see about including that in the next version of the guide.

I also experience problems when removing the primary video card. For me this is my integrated chip. The OS still seems functional, but will throw the occasional error and sometimes I need to reconnect via SSH. I also have a lot of issues passing through the iGPU to use for another guest. It has do do with the frame buffer of the iGPU being shared with main memory, but 4.1.2 is supposed to be patched to support it. I am spending much of my free time looking into these issues, so stay posted for that too.

P.S. Welcome to OCN!

P.P.S. If anyone wants to help, i would be happy to know how to get my Live-USB to boot Xen before booting into Fedora. This would make it easier for a lot of people who want to try this.

Best,
Dizzy4

RussianHak · Feb 28, 2012

Interesting thread, Booked for sure!

dizzy4 · Mar 2, 2012

Small update! I removed unnecessary install components and fixed an issue I found with selinux -- turned it off

Also, I added some preliminary Windows 8 support! I just tried it with the public preview and it works pretty well. Things will only get better, but this is good news

I am also working on the first beta release! It will be huge and make things about 4 steps less and provide a couple scripts that will make things faster and easier. Most importantly, it will allow users to try this without changing their disk drives!

Peon · Mar 4, 2012

Quote:

Originally Posted by dizzy4

Video card is your choice. I suggest using a newer Radeon for [...] HD audio on the card which means you won't need to pass through the motherboard audio to get sound from the VM. Of course, you could pass motherboard or another discrete Audio device through too.

Could you elaborate a little more on this? AFAIK Nvidia cards also have HD audio, otherwise HDMI wouldn't work properly.

Is it the difference in implementation, in that AMD cards have an mobo-style onboard sound card integrated into them whereas Nvidia cards pass a digital audio stream directly to the receiver, that makes the difference here? If that's the case, I guess Xen can't handle digital audio devices?

dizzy4 · Mar 4, 2012

Quote:

Originally Posted by Peon

Could you elaborate a little more on this? AFAIK Nvidia cards also have HD audio, otherwise HDMI wouldn't work properly.
Is it the difference in implementation, in that AMD cards have an mobo-style onboard sound card integrated into them whereas Nvidia cards pass a digital audio stream directly to the receiver, that makes the difference here? If that's the case, I guess Xen can't handle digital audio devices?

If what you say is true, the fix might be as simple as passing through the onboard sound as well. If both were assigned to the VM it should work alright. I was also wondering if the virtualized sound could be passed through. Maybe that is where the problem is. Through reading I also found that there are several people trying to patch Xen to make it work and version 4.2 will integrate some of those.

So what I would try would be: Install your domU with virtualized graphics then change your guest config file to include nographic=1. From there restart the system with the card passed through AND the audio passed through. That might force it to work properly.

Another idea I just had is to whether or not passthrough is working for you at all. Have you gotten any device to work in a domU? If you are having trouble binding the device to pci-back there is also a workaround using libvirt that I explained to SoulCleaver.

Best,
Dizzy4

lloyd mcclendon · Mar 4, 2012

in my system here the video and audio parts of the card are separate line items in lspci ...

03:00.0 VGA compatible controller: nVidia Corporation Device 1081 (rev a1)
03:00.1 Audio device: nVidia Corporation Device 0e09 (rev a1)
04:00.0 VGA compatible controller: nVidia Corporation Device 1081 (rev a1)
04:00.1 Audio device: nVidia Corporation Device 0e09 (rev a1)

so you would have to just have 2 pci passthrough devices, 1 for the video @ 03:00.0 and one for the audio @ 03:00.1 ... you could also use a virtual sound card, which will fallback to the onboard one, although this isn't true passthrough so it will lag a few ms behind and your audio will be out of sync.

btw since it looks like kvm is going to be about a year or so behind on this, i did go ahead and get all the xen stuff patched in and installed. i've yet to have the time to give it a try.. i will soon

{Guide} Create a Gaming Virtual Machine

Top Contributors this Month