April 2, 2010: I released a new patch which brings some improvements, adds support for newer versions of the Linux kernel and fixes all aforementioned problems. The new patch is available here.

January 18, 2010: Radu Cotescu integrated this patch to the latest version of his script, that now applies the patch automatically in Ubuntu, Fedora and openSUSE. The script is available here.

It has been a while since VMware updates VMware Server to add support for newer versions of the Linux kernel. This is a problem for users of newer distributions such as Ubuntu 9.10 (Karmic Koala) and Fedora 12 (Constantine) who want to use VMware Server.

To resolve this, some unofficial patches that update VMware Server to add support for newer versions of the Linux kernel have been released. However, these patches do not properly add support and have several problems, some of them even require the Linux kernel to be recompiled.

I released a patch for VMware Server 2.0.2 that properly add support for newer versions of the Linux kernel and does not require it to be recompiled. This patch was tested on Ubuntu 9.10 and Fedora 12.

The patch has the following features:

  • Add support for versions of the Linux Kernel that implement COW credentials.
  • Add support for net_device_ops structure.
  • Remove references to init_mm structure by removing APIC support for 2.6.25 and later (APIC code currently uses the macro pgd_offset_k).
  • Remove references to dev->priv by using netdev_priv().
  • Suppress GCC warnings.
  • Fix other known issues.

The following are instructions on how to apply the patch:

Download the VMware Server (VMware-server-2.0.2-203138.i386.tar.gz).

Download the VMware Server update patch:

$ wget -N http://risesecurity.org/~rcvalle/VMware-server-2.0.2-203138-update.patch

Extract VMware Server:

$ tar -xzf VMware-server-2.0.2-203138.i386.tar.gz

Extract VMware Server modules:

Change working directory to vmware-server-distrib/lib/modules/source/
$ tar -xf vmci.tar
$ tar -xf vmmon.tar
$ tar -xf vmnet.tar
$ tar -xf vsock.tar

Apply the patch:

Change working directory to vmware-server-distrib/
patch -p1 < ../VMware-server-2.0.2-203138-update.patch

Archive VMware Server modules again:

Change working directory to vmware-server-distrib/lib/modules/source/
$ rm -f vmci.tar
$ rm -f vmmon.tar
$ rm -f vmnet.tar
$ rm -f vsock.tar
$ tar -cf vmci.tar vmci-only/
$ tar -cf vmmon.tar vmmon-only/
$ tar -cf vmnet.tar vmnet-only/
$ tar -cf vsock.tar vsock-only/

Run installer script as root:

Change working directory to vmware-server-distrib/
$ sudo ./vmware-install.pl

Note for Ubuntu users:

When asked for the current administrative user for VMware Server, specify your user as a different administrator.

Note about VMware Remote Console Plug-in:

The VMware Remote Console Plug-in does not work properly on Ubuntu 9.10, Fedora 12 and other newer distributions. A workaround for this is to set the environment variable VMWARE_USE_SHIPPED_GTK before running the VMware Remote Console Plug-in. To set this environment variable at login time, add the following line to your ~/.profile:

export VMWARE_USE_SHIPPED_GTK=yes