rgrunber

Tag: Eclipse plugins

Eclipse p2 Droplets in Rawhide

So p2 Droplets have been in Rawhide for a little while now, and since then we’ve converted most Eclipse plugins to build using the new format. With the exception of some cases that will be done manually, pretty much everything building with the XMvn macros (%mvn_build, %mvn_install) is guaranteed to be a p2 Droplet after a rebuild. We still support the old format (Dropins), and an installation on rawhide can detect both types, but the goal is to switch completely to Droplets.

If you’re unfamiliar with the term, you can think of p2 Droplets as a new way of packaging one or more Eclipse features. It contains the same jars and feature folders as before but with an additional file (fragment.info) containing some extra data.

The main reason for the switch was that Dropins (using the p2 Reconciler) has been deprecated for a while and has made it much more difficult to diagnose issues when it fails. Eclipse’s first launch actually caches a lot of data to make subsequent ones much faster, and the Reconciler can be a large chunk of that time. I have actually run into cases where installing just an extra package can take the first startup from around 10 seconds, to 3 minutes!

Testing was done with the following setup :

  • Fedora Rawhide Virtual Machine
  • ~700 OSGi bundles on the system
  • ~430 of these bundles to be tested through the Dropins and Droplets approaches
for i in {1..5}; do rm -rf $HOME/.eclipse/ ; (echo exit; echo y;) | /usr/bin/time -f "%E" eclipse -noexit -console ; done;

To test under Dropins, we simply took all the bundles packaged as Droplets and removed their ‘fragment.info’ file, along with the ‘p2.fragments’ line in ‘/etc/eclipse.ini’, just to be sure we completely disabled the new logic.

So what were the results ? Under Dropins, the average startup time was 17.63 seconds, and under Droplets, it was 9.44 seconds. The variance was very small, and nearly the same for both cases so there’s strong evidence that Droplets will be saving a lot of time on Eclipse’s first startup. Of course all subsequent launches (where $HOME/.eclipse is kept) would be much faster (~2s).

So one might ask what’s happened with all the work the Reconciler used to do.

  1. Fedora maintainers do a good job of making sure packages are using the latest version of a library even when upstream isn’t. As a result, things like the Reconciler aren’t as necessary in attempting to satisfy dependencies across different versions
  2. A lot of the logic to calculate dependencies and produce a self-sustaining package is done properly at build-time, and rightfully so because that’s where the provides/requires are generated.

We’ve been working on this change, along with the other aspects of simplifying Eclipse plugin packaging for a while now so it’s nice to see some easy wins immediately from adoption.

Improving Eclipse Platform Stability On Rawhide

The Eclipse platform on Fedora Rawhide can be pretty unstable at times. Every update to one of its dependencies requires a rebuild. As a result, it has been on our TODO list for a while to work out some way of making Eclipse more resilient to these kind of dependency updates (at least in cases where a rebuild shouldn’t be required). Looking upstream, there are quite a few bugs relating to this topic (410710, 410785, 408138) .

For simple rebuilds where project metadata likely hasn’t changed there’s fix for symbolic links in place.

Another common breakage happens when dependencies that are listed as plugins within a feature get updated.

Unlike regular Eclipse plugins that might contribute certain capabilities, a feature represents a set of plugins. One can think of them as RPM meta-packages and some features are used especially by end-users to install a larger set of plugins (eg. org.eclipse.cdt.feature.group is “C/C++ Development Tools”). A feature lists the set of plugins it provides with the <plugin> tag and may also specify dependencies with the <requires> tag. The difference is that the <plugin> tag locks onto the exact version discovered at build time, and may only resolve against that exact version. The <requires> tag on the other hand allows you some flexibility in terms of dependencies with ranges and some compatibility levels.

Sometimes one might see features listing dependencies as content. Does the JDT provide, or own org.junit and org.hamcrest ? Do we really mean to say that changing the versions of those plugins implies a completely different feature ? Clearly this isn’t the case, and it would make much more sense to use <requires> but the former practice seems quite common.

I don’t think this is done out of lack of understanding, but because projects also want to include their dependencies into the repositories they deploy. The <plugin> definition accomplishes this. Using <requires> one would also need to list the plugins to be included in the repository definition (site.xml/category.xml) and some platform projects have to jump through some additional hoops to change this file so it’s not a huge surprise that many go for the simpler option. Sadly, this causes some problems for us in Fedora land.

Luckily there’s now fix for platform features that should help reduce the number of rebuilds.

Extending the p2 Repository Format in Fedora

We ship a lot of OSGi bundles in Fedora.

$ repoquery --repofrompath=foo,http://kojipkgs.fedoraproject.org/repos/f21-build/latest/x86_64/ --repoid=foo -q --whatprovides "osgi(*)" | wc -l
693

We already allow packagers to have requirements based on the OSGi metadata to some extent. We’re supporting Require-Bundle in specfiles through use of the “osgi(…)” provide but even with this, building Eclipse plugins has always seemed like a giant hack, and even more so when compared to the latest Java Maven Packaging Examples. Building with Tycho requires that your OSGi dependencies be in a p2 repository (update site). We obviously can’t just point to the eclipse.org update sites for our Fedora builds since all of our dependencies must be from packages within the Fedora infrastructure that were built from sources. Initially the simplest way to achieve this was to look on the system for all OSGi bundles, publish them to a p2 repository, and then feed this directly into Tycho’s build target platform.

This approach certainly works (and that’s what the copy-platform-all script did) but it still requires performing the same steps every time one needs to feed OSGi bundles to Tycho or some other application that uses p2 repositories. The end-goal is to have tighter integration with tools like XMvn, and to make packaging an easier task. To do this we really need to have our system locations be recognized as p2 repositories so that we could also take advantage of p2 APIs.

When I got back from EclipseCon NA 2014, I started playing around with this idea of having filesystem directories being recognized as a p2 repository.

Imagine :

$ eclipse -application org.eclipse.equinox.p2.director -repository fedora:/usr/share/java/ -installIU org.swtchart

with the result being that ‘org.swtchart’ from somewhere in ‘/usr/share/java’ gets installed into the eclipse instance. This was easy to implement and fairly simple to integrate with Tycho so that we only inject a few system locations and get proper dependency resolution.

I’ve named this fedoraproject-p2 mainly because Fedora’s requirements are the driving force but I could see this being useful in many other cases.

This has been used for some time now on rawhide, and with the addition of things like the xmvn-p2-installer-plugin, it won’t be long until Eclipse plugin packaging will be drastically simpler.