Having trouble getting Linux guests to work
I'm not sure why as I know I've been able to use virt-manager in the past for some of the distro's I'm trying, if anyone could perhaps help figure this out it'd be appreciated.
I have tried a similar XML config to my Windows VM tweaked for Linux, as well as a default XML config. Oddly regardless of my disk types for I/O of install media or virtual drive to install to, the graphics used with VNC display seems to be the cause of snail pace performance. QXL refuses to work(error might have been about my QEMU package not supporting it, host is Manjaro/Arch). Cirrus is the only reliable one for getting native expected performance, not sure why. I have been working with the Q35 chipset(and I think that might be necessary for GPU passthrough with OVMF?), perhaps this is the cause.
My host OS has resumed from a suspend, tsc
as the clock source becomes unstable and the default clock source is changed to hpet
, not sure what the default is for VMs created with virt-manager/libvirt, I know that hpet was disabled via XML config for my Windows VM initially which caused massive stuttering/lag when my host OS used that instead of tsc.
My host OS might have updated it's kernel and other packages since last boot, I suppose there is a chance that may be another cause that rebooting my system could correct.
None of the Manjaro(latest or some older iso's I have) seem to be able to boot their live media(either via sata or scsi(virtio) mounted iso, or USB disk). They get started and I see a bunch of lines in the terminal which if I'm not mistaken is progress/status of the boot process from systemd. It tends to get stuck on TLP or Plymouth, 30 mins+ waiting and no dice. I know I have had this working in VitualBox and with virt-manager on another machine in the past(minus passthrough, though even without the GPU attached it has the same problem.)
![Update Update](https://i.ebayimg.com/thumbs/images/g/e2IAAOSwc2FZ83uw/s-l200.jpg)
I've tried an old iso of KDE Neon I had around, this booted in and I was able to install it to disk. However when trying to boot the installed OS from the disk it'd be ignored regardless of sata or scsi(virtio), I'd get dropped to UEFI/OVMF shell or if the livemedia was still enabled for boot it'd load into that.
I've tried KaOS and this actually works fine, it's just rather lacking in packages missing out on synergy and qemu-guest-agent which I'd really like to have going, can make do without synergy with USB redirection for my mouse/keyboard receiver(unplugging and replugging back into the port to return to guest). SPICE isn't an option right now as my host(Manjaro/Arch) doesn't provide it with qemu by default for some reason.
Antergos installer seems buggy, gives me an unhelpful error number to report of none
when trying to get a package list to download at the start of installation(after going through the setup wizard). This was with the minimal install iso(500ish MB).
ApricityOS(another Arch derivative) booted and installed fine, could boot OS from disk via the window VNC but when removing that and booting with only my GPU passed through and nvidia non-free drivers installed it gets stuck with the screen showing only Loading Linux linux.. Loading initial ramdisk ..
.
Fedora booted/installed fine, on the nvidia GPU(live media) though while I can use the mouse it's not visible where the cursor is unless something like a button highlights on mouse over, the cursor visual is stuck on the top left.On installed drive boot with GPU passthrough it loaded slower(seems only fast with Cirrus). It finished booting but gave a full screen error on white bg and centered text 'Oh no! Something has gone wrong.'. Not sure if the driver install and update instructions for Fedora I got from here is the cause. I guess that rules out Fedora as well.
I have Ubuntu downloading, considering how Fedora is fairly VM friendly hopefully it works. If no luck with that, perhaps openSUSE..
EDIT: Just switched back to my Linux VM config that was similar to the Windows one, Fedora booted from the installed disk properly this time. I guess the NVIDIA workaround is required for Linux too? Perhaps it wouldn't boot because of that and nouveau being blacklisted. Neon boots on nvidia GPU too now as does apricity. Definitely seems like the boot problems were due to missing workaround. Manjaro still having trouble booting those install media for some reason.
I'm trying to get opengl working for headless offscreen rendering on a Amazon p2 instance with Ubuntu Ubuntu Server 16.04.
After instance creation I've installed the appropriate nvidia drivers according to this amazon article, and they seem to be working as expected:
I'm then trying the steps from this related question:
Which does start X but running glxinfo
does not work:
Changing the BusID in Xorg.conf as mentioned in the related stackoverflow question did not help.
I've googled quite intensively and it's unclear to me how to proceed next.. Any help would be appreciated
2 Answers
The official documentation about this is http://www.nvidia.com/content/PDF/remote-viz-tesla-gpus.pdf 'REMOTE VISUALIZATION ON SERVER-CLASS TESLA GPUS'. Read page 15. You have to add BusID to nvidia-xconfig.
First, run nvidia-xconfig --query-gpu-info
. This is my p2 instance result.
Then you need to run this.
You can run Xorg server by sudo Xorg :1
.
The DISPLAY=:1 glxinfo
result.
When using NVidia please consider the information obsolete that is given in the related question you linked.
For about a year now the NVidia drivers support true headless operation without an X server running. See this exhaustive article given on the Nvidia developer blog: https://devblogs.nvidia.com/parallelforall/egl-eye-opengl-visualization-without-x-server/
Not the answer you're looking for? Browse other questions tagged ubuntuopenglamazon-ec2gpuheadless or ask your own question.
- 2 System Requirements
- 3 Host Environment Setup
- 3.2 Build Kernel
- 3.3 Build Qemu
- 4 Guest Setup
- 4.1 Linux Guest Setup
- 4.2 Windows Guest Setup
- 5 Basic usage
- 5.6 Create VM
- 6 Features Supported
Intel GVT-d is one flavor of graphics virtualization approaches also names as Intel-Graphics-Device pass-through feature. It based on Intel VT-d technology, and some special graphics related configuration than NIC pass-through. This flavor allows direct assignment of an entire GPU’s prowess to a single user, passing the native driver capabilities through the hypervisor without any limitations.
2.1 Operating System Requirements
Ubuntu 18.04 has been fully validated as host, other Linux operating system like RHEL with 4.X kernel is also OK.
2.2 Hardware Requirements
For client platforms, 5th, 6th or 7th Generation Intel® Core Processor Graphics is required.For server platforms, E3_v4, E3_v5 or E3_v6 Xeon Processor Graphics is required.
2.3 Library Dependence
Use Ubuntu as an example, there are some software package needed by host environment setup, as below:
apt-get install git libfdt-dev libpixman-1-dev libssl-dev vim socat libsdl1.2-dev libspice-server-dev autoconf libtool xtightvncviewer tightvncserver x11vnc libsdl1.2-dev uuid-runtime uuid uml-utilities bridge-utils python-dev liblzma-dev libc6-dev
3.1 Source Repositories
3.2 Build Kernel
git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
cd linux-stable
git checkout master
With the default kernel, Linux VM will trigger GPU hang/GPU reset while running 2D /3D application, which is a known issue with hypervisor KVM, so we can add the disable stolen memory patch to resolve this problem (If you don't want to add this patch to kernel, you also can enable stolen memory patch in Qemu side, as section 3.3).
echo '|make oldconfig
make -j8 && make modules_install && make install
3.3 Build Qemu
Please follow chapter 3.3.1 for GVT-d KVM setup and chapter 3.3.2 for GVT-d Xen setup.
3.3.1 Build Qemu for GVT-d KVM
git clone git://git.qemu.org/qemu.git
cd qemu
git checkout master
git revert 93587e3af3a // enable stolen memory patch if you didn't do it in kernel side
./configure --prefix=/usr
--target-list=x86_64-softmmu
make -j8
make install
3.3.2 Build Qemu & Xen for GVT-d Xen
git clone git://xenbits.xen.org/xen.git
cd xen
git checkout master
./configure
a. In order to run GVT-d well in 5th Gen or E3_v4 Xeon platform, we need to edit the rombios file with below changes:
make xen -j8
make tools –j8
b. In order to GVT-d run well in 6th / 7th Gen or E3_v5 / E3_v6 platform, we need to edit the qemu-xen-tradtional-dir file with below changes:
Then you need to build tools again.
make tools -j8
make install-xen
make install-tools
Cirrus Logic Gd 5446
3.4 Grub Update
You need to add the GVT-d KVM and GVT-d Xen corrsponing menuentry in your grub.cfg, or you can add it to /etc/default/grub is better. (Since grub.cfg will be auto regenerated by linux distro version upgrade or kernel package update).
For GVT-d KVM, Make sure the 'intel_iommu=on' exist in your menuentry, below is an example on ubuntu:
NOTICE: Since some Windows guest 3rd party application / tools (like GPU-Z / Passmark9.0) will trigger MSR read / write directly, if it access the unhanded msr register, guest will trigger BSOD soon. So we added the 'kvm.ignore_msrs=1' into grub for workaround.
For GVT-d Xen, make sure the 'modprobe.blacklist=i915 xen-pciback.passthrough=1 xen-pciback.hide=(00:02.0)' exist in your menuentry, below is an example on ubuntu:
4.1 Linux Guest Setup
4.1.1 Linux Kernel Support
You can prepare any version Ubuntu or any other Linux guest OS image manually. Then you can change the guest kernel to the same kernel as host, you also can download the stable version from https://www.kernel.org/ or use Ubuntu18.04 guest OS kernel directly.
4.1.2 Linux Intel GFX Card Support
We need to make the special X11 configuration change for Linux guest to use the Intel graphics card as a display card .
The emulated graphics card (linke 'QXL', 'Cirrus')not support OpenCL / OpenGL hardware acceleration, we must make this change.
Here we use Ubuntu as example, we save the following section at /etc/X11/xorg.conf file:
BTW: The PCI ID was chosen by looking at the output of the 'lspci' command.
root@gvtd-host:~# lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.2 USB controller: Intel Corporation 82371SB PIIX3 USB [Natoma/Triton II] (rev 01)
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
While Valmiki’s diction is sparse and direct, Kamban’s exuberant prose sparkles with wit and inventiveness. Kamban, on the other hand, never allows the reader to forget the godhood of Rama. His Ravana too, though flawed, is a heroic figure. Kamba ramayanam with meaning. Though cast in the heroic mould of a Purushotama or ‘the best among men’, Valmiki’s Rama is still a man.
00:02.0 VGA compatible controller: Cirrus Logic GD 5446
00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 03)
00:04.0 VGA compatible controller: Intel Corporation Broadwell-U Integrated Graphics (rev 0a)
English conversation practice by grant taylor pdf download. Dec 27, 2017 - Sep 23, 2017 - 10 min - Uploaded by MyPCUserHow to Download Google Books for Free in PDF Fully without using any Software| Part-3. English Conversation Practice (English) 1st Edition (Paperback) This book provides simple. Grant Taylor. Darshansingh how to read pdf book.
NOTICE: After you change the display card to Intel graphics card, Qemu will comes to black screen, you only can see guest desktop by remote protocal.
4.1.3 Linux Remote Protocol Support
We suggest to use the X11VNC as the remote protocol to validation which has been validated by us.
Use Ubuntu as example, to support X11VNC, you must install the xhost in guest first, the xhost program is used to add and delete host names or user names to the list allowed to make connections to the X server, you must make some configuration for lightdm, the example as below:
After you complete the lightdm configuration, you can run the X11VNC service in guest by following commands or similar commands:
x11vnc -display :0 -forever -bg -repeat -nowf -o ~/.vnc/x11vnc.log -rfbport 5900
Then connect to the guest by host with guest X support by typing the following command:
vncviewer $guestIP
NOTICE: $guestIP means guest IP address, you can got the guest IP address by 'nmap' / 'arp-scan' or any other network related tools in host.
Except X11VNC, there are some other alternatives, which were not validated by us, you also can use it as remote protocol for a try:
- X2Go and based on nx-libs (it seems QVD uses it too)
- QVD (open source version)
- NoMachine: Closed source.
- OpenNX. Client only. Not updated since 2016-10-01.
4.2 Windows Guest Setup
4.2.1 Windows Version Support
Win7-32 / Win7-64 / Win8.1-64 /Win10-RS4-64 are validated.
4.2.2 Windows GFX Driver Support
After you boot up Windows guest, you can see two graphics card in 'Device manager'
The virtual GFX card 'QXL' or 'Cirrus'
The Intel GFX card
Then you must install the Intel GFX driver for the Intel Graphics card to support OpenCL / OpenGL / DirectX9 / DirectX10/ DirectX11 / DirectX12 hardware acceleration .
You can get the Windows driver from graphics drivers section of the Intel website. Depending on your Intel processor, the latest available driver versions as below:
English horror movies in hindi dubbed. Kabylake & Skylake Win10-RS4-64: 25.20.100.6326Kabylake & Skylake Win7-32 / Win7-64 / Win8.1-64: 15.45.23.4860Broadwell: 15.40.41.5058
NOTICE: After you install Windows GFX driver, you can see guest desktop by both QEMU and remote protocol. QEMU display the emulated driver and the remote protocol display the Intel GFX driver. BTW, There is a known issue that some 3D workload cannot run with hardware acceleration while both GFX card is available, so suggest to disable the emulated GFX card in 'Device Manager' after you make sure the Intel GFX card is working.
4.2.3 Windows Remote Protocol Support
We suggest to use Tightvnc as the remote protocol to validation which has been validated by us.
Then connect to the guest by host with guest X support by typing the following command:
vncviewer $guestIP
NOTICE: $guestIP means guest IP address, you can got the guest IP address by 'nmap' / 'arp-scan' or any other network related tools in host.
All Windows Remote protocol are supported, such as TightVNC, HP RGS, RDP.
Except Tightvnc, there are some other alternatives, which were not validated by us, you also can use it as remote protocol for a try:
- VNC (Virtual Network Computing)
- TurboVNC: They recommend to use the UltraVNC server on Windows.
- TigerVNC: Which is similar as TightVNC.
- UltraVNC: Which is similar as TightVNC.
- SSVNC: Which is similar as TightVNC.
- HP's RGS (Remote Graphics Software), which is an enterprise-level application.
- Microsoft's RDP (Remote Desktop Protocol).
- FreeRDP: The best Open Source RDP implementation.
- Remmina (GTK+ 3): Based on FreeRDP. It supports the RDP, VNC, NX, XDMCP and SSH protocols.
- WinConn: Promising but abandoned project. Just put here as reference, not useful for end users (the readers of this document).
- KDE's KRDC (Remote Display Client): It supports both VNC and RDP protocols.
- Gnome's Vinagre
- NX
- NoMachine:
5.1 Preparing a graphic card device for passthrough for GVT-d KVM
root@gvtd-host:~# modprobe vfio
root@gvtd-host:~# modprobe vfio_pci
root@gvtd-host:~# lspci -D -nn
0000:00:00.0 Host bridge [0600]: Intel Corporation Broadwell-U Host Bridge - DMI [8086:1614] (rev 06)
0000:00:02.0 VGA compatible controller [0300]: Intel Corporation Broadwell-U Integrated Graphics [8086:1622] (rev 08)
Thus it can be seen,VGA device (0000:00:02.0,vendor & device ID 8086:1622)
root@gvtd-host:~# echo 0000:00:02.0 > /sys/bus/pci/devices/0000:00:02.0/driver/unbind
root@gvtd-host:~# echo 8086 1622 > /sys/bus/pci/drivers/vfio-pci/new_id
5.2 Preparing a graphic card device for passthrough for GVT-d Xen
root@gvtd-host:~# xl pci-assignable-add 00:02.0
root@gvtd-host:~# xl pci-assignable-list 0000:00:02.0
5.3 Network Bridge
For GVT-d KVM, in order to let guest using the same network segment as host, you must create the network bridge. Besides, in order to let Qemu using the network bridge you created, you also need create a scripte as below (We saved it at /etc/qemu-ifup):
Detail you can reference the the 'Configuring Guest Networking' wiki page http://www.linux-kvm.org/page/Networking
For GVT-d Xen you can just create the network bridge by yourself.
5.6 Create VM
5.6.1 Create GVT-d KVM VM
We have two methods to create VM, one is UPT mode (guest cannot local display in monitor), the other is legacy mode (guest can local display in monitor), the 'legacy mode' was recommended.
The example of create VM script as below :
UPT mode:
Legacy mode:
5.6.2 Create GVT-d Xen VM
You can create hvm configuration file by modifying /etc/xen/xlexample.hvm like below:
After that, you can create XenGT VM via the below command:
6.1 Highlights Features
- General:
- Supported Linux distributions (64bit): Ubuntu, RHEL, SUSE, etc
- Supported Windows guest: Windows7 (64bit), Windows8.1 (64bit),Windows10 RedStone (64bit)
- Single display with resolution up to 1920x1080p
- 3D/2D:
- OpenGL4.5 for Linux guest
- OpenGL4.4 for Windows guest
- DirectX9, 10, 11, 12 for Windows guest
- Compute:
- OpenCL2.0 for Windows/Linux guest
- Media:
- Intel Media Service Studio (2015R5)
- Remote display protocol:
- X11VNC for Linux guest
- TightVNC, HP RGS, RDP for Windows guest
6.2 Validated Benchmarks
- 3D / 2D:
- 3Dmark06, 3Dmark11, WebGL, Passmark, Unigine-tropics, Unigine-heaven, Phoronix 3D, Cairo 2D, etc
- Media:
- Multiple thread decode/transcode for JPEG, MPEG2, H264, HEVC, VC1, VP8, etc
- OpenCL:
- LuxMark, Beignet
Having trouble getting Linux guests to work
I'm not sure why as I know I've been able to use virt-manager in the past for some of the distro's I'm trying, if anyone could perhaps help figure this out it'd be appreciated.
I have tried a similar XML config to my Windows VM tweaked for Linux, as well as a default XML config. Oddly regardless of my disk types for I/O of install media or virtual drive to install to, the graphics used with VNC display seems to be the cause of snail pace performance. QXL refuses to work(error might have been about my QEMU package not supporting it, host is Manjaro/Arch). Cirrus is the only reliable one for getting native expected performance, not sure why. I have been working with the Q35 chipset(and I think that might be necessary for GPU passthrough with OVMF?), perhaps this is the cause.
My host OS has resumed from a suspend, tsc
as the clock source becomes unstable and the default clock source is changed to hpet
, not sure what the default is for VMs created with virt-manager/libvirt, I know that hpet was disabled via XML config for my Windows VM initially which caused massive stuttering/lag when my host OS used that instead of tsc.
My host OS might have updated it's kernel and other packages since last boot, I suppose there is a chance that may be another cause that rebooting my system could correct.
Cirrus Logic Mac Drivers
None of the Manjaro(latest or some older iso's I have) seem to be able to boot their live media(either via sata or scsi(virtio) mounted iso, or USB disk). They get started and I see a bunch of lines in the terminal which if I'm not mistaken is progress/status of the boot process from systemd. It tends to get stuck on TLP or Plymouth, 30 mins+ waiting and no dice. I know I have had this working in VitualBox and with virt-manager on another machine in the past(minus passthrough, though even without the GPU attached it has the same problem.)
I've tried an old iso of KDE Neon I had around, this booted in and I was able to install it to disk. However when trying to boot the installed OS from the disk it'd be ignored regardless of sata or scsi(virtio), I'd get dropped to UEFI/OVMF shell or if the livemedia was still enabled for boot it'd load into that.
I've tried KaOS and this actually works fine, it's just rather lacking in packages missing out on synergy and qemu-guest-agent which I'd really like to have going, can make do without synergy with USB redirection for my mouse/keyboard receiver(unplugging and replugging back into the port to return to guest). SPICE isn't an option right now as my host(Manjaro/Arch) doesn't provide it with qemu by default for some reason.
Antergos installer seems buggy, gives me an unhelpful error number to report of none
when trying to get a package list to download at the start of installation(after going through the setup wizard). This was with the minimal install iso(500ish MB).
ApricityOS(another Arch derivative) booted and installed fine, could boot OS from disk via the window VNC but when removing that and booting with only my GPU passed through and nvidia non-free drivers installed it gets stuck with the screen showing only Loading Linux linux.. Loading initial ramdisk ..
.
Fedora booted/installed fine, on the nvidia GPU(live media) though while I can use the mouse it's not visible where the cursor is unless something like a button highlights on mouse over, the cursor visual is stuck on the top left.On installed drive boot with GPU passthrough it loaded slower(seems only fast with Cirrus). It finished booting but gave a full screen error on white bg and centered text 'Oh no! Something has gone wrong.'. Not sure if the driver install and update instructions for Fedora I got from here is the cause. I guess that rules out Fedora as well.
I have Ubuntu downloading, considering how Fedora is fairly VM friendly hopefully it works. If no luck with that, perhaps openSUSE..
EDIT: Just switched back to my Linux VM config that was similar to the Windows one, Fedora booted from the installed disk properly this time. I guess the NVIDIA workaround is required for Linux too? Perhaps it wouldn't boot because of that and nouveau being blacklisted. Neon boots on nvidia GPU too now as does apricity. Definitely seems like the boot problems were due to missing workaround. Manjaro still having trouble booting those install media for some reason.