Switching from compile-on-demand to binary packages

For many years installing and using cPanel & WHM has involved compiling software on-demand. Want Apache and PHP? Run /scripts/easyapache, which builds those and all dependencies from source. Want perl installed? Download and run the perl installer from httpupdate.cpanel.net, which installs perl from source. While compiling from source has its benefits, it also has its drawbacks.

The drawbacks are more obvious when you consider what cPanel & WHM attempts to do with compile-on-demand. We are automating this on thousands of servers, each with their own subtle differences (and some no-so-subtle). A non-trivial portion of our development effort, bug fixing and support requests are consumed by getting source compilation working in environments under other peoples (yours) control. We decided this needed to change. Hence we are switching our model from compile-on-demand to delivery of pre-compiled RPM packages.

What is involved in this change

For many years we’ve provided applications via RPM. For example MySQL, Dovecot, and Exim. We will soon be adding over 800 additional RPMs. These RPMs provide:

  • perl 5.14
  • all the perl modules that scripts/checkperlmodules formerly installed
  • PHP 5.3 for cPanel & WHM
  • rrdtool
  • all the pear packages needed for Horde, Roundcube, phpMyAdmin, Logaholic and Squirrelmail
  • the encoders used by internal PHP
  • nearly everything in /usr/local/cpanel/src

The vast majority of the RPMs are for perl and CPAN modules (801 at last count).

New RPMs will have the naming convention of cpanel-APPLICATION-APPMAJORVERSION-APPVERSION-CPANELPATCH-CPANELVERSION. For example:

cpanel-perl-514-5.14.2-0.cp1136

Broken into its parts we have:

  • cpanel-perl: The application is perl
  • 514: perl major version of 5.14
  • 5.14.2: The specific version of perl in the RPM
  • -0: We have not applied any patches, therefore we use 0
  • cp1136: Built for cPanel & WHM 11.36

All the new RPMs will be installed to /usr/local/cpanel, rather than to the operating system locations (e.g. /usr ). Whenever possible we fulfill dependencies (e.g. libraries) using what the operating system provides. If that is not possible, we provide the dependency ourselves with an RPM.

Along with an expanded set of RPMs we are also providing a tool you can use to reinstall RPMs. This is useful if something is modifying files managed by a package. The tool will tell you the packages that need reinstalled and allows you to perform the reinstall at your convenience.

How Something is installed

Does this mean you will find scripts like perlup, internalphpup and so forth? No. In fact all the scripts that install the current RPMs (e.g mysqlup, eximup) no longer do anything. Rather they output a message and point you to documentation. Instead RPMs are tied to the product version and are installed by our “rpm.versions” system. This system behaves somewhat like yum, or apt-get, in that you are able to configure repositories and install software from them. The rpm.versions system was introduced in 11.30. To handle hundreds of RPMs, it needed to change significantly. More information on that later.

By and large RPM installation is handled by upcp. For now this is acceptable as the only RPMs we are delivering are required by the product. Eventually we want to provide optional RPMs (e.g. git, subversion, etc) that you can elect to install. When we reach that stage tools, both command line and graphical, will be developed to facilitate package installation outside of upcp.

Source Compiles

All the source RPMs will be provided on our mirrors. Customers that want to apply patches to the software we provide, or just like compiling software they use, will need to obtain the source RPM and rebuild it. If you want to do this for more than one server, then you will be able to use our rpm.version system so your cPanel & WHM servers install your custom package from a central location.

What is not Involved in this Change

Not everything will switch to delivery via RPM packages. EasyApache will still build Apache et al. from source. cPanel & WHM itself will not be delivered via RPM in this release.

The Benefits

Fresh installs will see an average 30% reduction (or better) in installation time. This is due to installing perl, and necessary CPAN modules, from RPM rather than building them from source.

Upgrades should also see a reduction in time, in the long term, as new applications and dependency updates will be fulfilled via RPM packages, rather than building from source during the upgrade.

The rpm.version system gives you a simple way of deploying custom software directly as part of the cPanel & WHM installation and upgrade.

Removal of problems stemming from automated source builds in wildly diverse environments.

Re-installing perl is a matter of re-installing a package, rather than building from source and forcing the reinstallation of a bunch of CPAN module.

Applications and dependencies delivered with cPanel & WHM will be in one location, /usr/local/cpanel, rather than strewn all over the file system(s). The long term goal is to have everything installed to /usr/local/cpanel. The RPMs existing prior to this change (e.g. MySQL, exim, etc) still install to their current locations however.

More

There is far more to these changes than what is detailed here. In the coming weeks we’ll share more information on:

  • perl 5.14 in cPanel & WHM
  • Changes to the internal PHP
  • rpm.versions system
  • Changes to how cPanel & WHM are updated to make it more robust

Please discuss this article on our forum.

Posted in: News, Release Announcements | Tagged: ,