Monday 1 April 2013

OpenCL, Python and Ubuntu 12.10

Installing PyOpenCL, the python OpenCL wrapper on Ubuntu 12.10 could have been just as easy as installing the according package python-pyopencl but this is currently not possible because of a packaging bug that was reported and confirmed before the release of 12.10 but not fixed since. I am not going to comment on this but document my work around instead.

What is OpenCL?

OpenCL is the short name for Open Computing Language - an open standard that specifies a vendor agnostic language and cross-platform framework for developing code that is can be executed on parallel computing enabled platforms, such as GPUs (graphics cards), FPGAs (field programmable gate arrays) or newer generation multicore CPUs.

The most typical use-case is using the graphics device for non-graphics computation tasks. Typical computation intensive tasks can be broken down into multiple sub-tasks that can be executed independent of each other, in parallel. The many pourpose built processing pipelines, special shift register circuits, shaders and vertext processors inegrated into modern graphics devices are equipped with a multitiude of 256bit registers, and excel at high volume calculation of matrix operations, interpolations and special tranformations (e.g. fourier, laplace). OpenCL enables one to unleash this raw parallel/stream processing power to be used for generic purposes, other than gaming.

Currently, OpenCL support is provided by proprietary drivers for Nvidia GPUs, AMD GPUs and CPUs and Intel CPUs. Although these drivers are not open source components, OpenCL, being an open standard, enables one to avoid coding against vendor specific APIs and create software mostly portable across computing platforms. (One well known example of such vendor specific APIs is provided by Nvidia's Compute Unified Device Architecture - CUDA - but recent CUDA versions expose OpenCL interfaces as well.)

As of the time of this writing, OpenCL is my preferred platform for computation intensive, time critical applications. It is worth to be aware of, and tinker around with it for fun and profit.

What is the bug?

In a nutshell, the package cannot installed, the bug is a package dependency on a non-existent package, namely opencl-icd that is not available in any of the official or partner repositories and also not provided by other package. As the but was reported and confirmed one month before the final release of 12.10 I am rather surprised it has not been fixed yet, especially as fixing the problematic dependency boils down require merely one line to be changed in the package metadata.

I have quickly read up on the format of debian packages to make sure I understand the issue correctly and found the following alternatives for solving the current problem:

  • Modify the package python-pyopencl by removing the dependency on opencl-icd. This is a low hanging fruit but would leads to broken packages in the long run as any update to the package would overwrite my local changes. This issue can be dodged by putting the package on hold.
  • The strategic solution would be to alter the metadata of packages nvidia-current and fglrx so these packages would provide opencl-icd as a virtual package. This is not my preferred approach for a local workaround, as leads to maintenance issues on package updates. The concerns and workarounds of the previous alternative apply here as well.
  • My preferred approach was creating a dummy package python-pyopencl that merely depends on the appropriate drivers but does not contain any real file itself, only metadata. This approach will shield me from package update issues.

Creating and installing the dummy package


$ mkdir /tmp/dummy && cd /tmp/dummy
$ tar czf data.tar.gz * # creating an empty archive, will output some errors
$ tar tzf data.tar.gz   # this verifies the archive is empty and can be read without errors
$ touch md5sums
$ cat <<EOF>> control
Package: opencl-icd
Source: opencl-icd
Version: 1.0.0-1
Architecture: amd64
Maintainer: Tibor Bősze <tibor.boesze@gmail.com>
Installed-Size: 4
Depends: fglrx | nvidia-current | intel-ocl-sdk
Section: universe/libs
Priority: optional
Homepage: http://tuxicate.blogspot.com/
Description: Dummy package
 This is a dummy package to workaround bug 1048036
EOF
$ tar czf control.tar.gz md5sums control
$ echo "2.0" > debian-binary
$ ar rcs opencl-icd_1.0.0-1_amd64.deb debian-binary control.tar.gz data.tar.gz
$ dpkg-deb --verbose --info opencl-icd_1.0.0-1_amd64.deb # verify
$ sudo dpkg -i opencl-icd_1.0.0-1_amd64.deb # install

The package, once installed, will appear under Obsolete and Locally Created Packages in aptitude, so it is easy to track and uninstall if needed. Now, python-pyopencl can be installed and used without issues.

One could modify the dependencies (the line starting with Depends:) as required, I have added Nvidia or AMD Radeon/FireGL graphics drivers, or the Intel OpelCL SDK. The Intel SDK can be downloaded in rpm for Intel, and converted to a deb package with alien.

6 comments:

  1. Hey Tibor!

    Thanks for this post! Great insight, yet how would you amend the last two lines to Nvidia drivers?

    Also, do you insall python-pyopencl after this?

    R

    ReplyDelete
    Replies
    1. Hi,

      as you can see, the package depends on fglrx OR nvidia-current OR intel-ocl-sdk. In your case just make sure nvidia-current is installed before running the commands I listed - this way you will not get errors about unmet dependencies when running dpkg. Once opencl-icd is installed, simply install python-pyopencl via your favourite package manager frontend (I use aptitude).

      Hope this helps,
         T

      Delete
  2. Hi Tibor!

    Thanks for your help with this! It seems that I installed he nvidia-current driver, yet for some reason I cant make the opencl-icd work properly - i think i already created a version with the amd64 line in it - thus making it totally wrong on my system.

    Would you be able to tell me how to delete this and start everything from scratch?

    Thanks!

    ReplyDelete
    Replies
    1. Hi,

      The amd64 line would matter if you ran a 32-bit operating system. If unsure launch 'System Monitor' and check the 'System' tab. If the first line under Ubuntu should reads 'Release 12.10 (quantal) 64-bit' you are fine with amd64.

      To undo all changes once the package is installed, just follow what I have written above: 'The package, once installed, will appear under Obsolete and Locally Created Packages in aptitude, so it is easy to track and uninstall if needed.' Just type 'sudo aptitude purge opencl-icd', or use any other package manager frontend to purge the package.

      Hope this helps,
         T

      Delete
  3. If you were here I'd hug you
    A..manly one though *cough* *cough*

    ReplyDelete
    Replies
    1. Thanks for the feedback, I am glad you found this useful.

      Cheers,
         T

      Delete