www.bsiag.com
 

Eclipse Scout builds with CBI

bob_and_scouty

After many hours of

  • acquiring knowledge for the SVN-Git migration
  • preparing and doing the SVN-Git migration
  • getting acquainted with and learning Apache Maven and the Tycho plugins
  • reading poms of other projects (Platform Build [1], Minerva [2])
  • structuring and restructuring the Eclipse Scout bundles and features
  • normalising lineendings for Git (several times)

Eclipse Scout builds since Kepler M3 CBI-style. So Eclipse Scout is ready for 2013 and 2113 and further into the future – Eclipse Scout is LTS ready [3].

The Adventure Begins

This adventure started with more and more customer of BSI asking about Maven builds and Eclipse Platform switching to Maven/Tycho. Several customers used already Maven to build J2EE projects and wished that our project will be built with Maven or at least not break other Maven builds. For getting LTS ready the Eclipse Foundation suggests or even defines using Maven/Tycho. And finally early 2012 Eclipse Platform started migrating the builds to Maven/Tycho. These three points made it clear for us, that this is the right path to take. Tycho had and may still have some issues — but we have and will find and solve them together with all projects migrating now or soon. Soon Tycho will be as great as we’d all like it to be. Look at the bugfixes and features already implemented in the last two releases [4,5] — and I’m sure more will follow until Kepler. Nice work Tycho Team!

End of june 2012 I started migrating our ant-custom-pde-build-bash-script builds to CBI-style Maven/Tycho builds [6]. This work is based on the migration of the Scout sources from SVN to Git [7].

Migrating from SVN to Git

Although the source history of Eclipse Scout isn’t that long and just starts two years ago [8] we wanted to keep this history and migrate from SVN to Git and not just push the whole source tree into a fresh Git repository. Also I wished to have a valid migration path for other SVN repositories in our company. The migration guide from eclipse.org suggests using svn2git [9]. svn2git does a great job extracting the history of an SVN repository to a git repository. What it lacked was the ability to update a bare Git Repository. Unfortunately Kevin Menard seemed to be under water with other projects or unavailable for other reasons. The svn2git project seemed unattained (Kevin picked up attaining svn2git in october 2012). So I forked it [10] and merged some other forks which added needed functionality and added the ability to update a bare Git repository from SVN (which reminds me to write a pull request).

I was now able to create a Git Clone from a SVN Repository and update it if needed. This laid the groundwork for migrating any SVN Repository to Git at BSI including our Eclipse Scout SVN Repository at eclipse.org. Now began the study how to resolve the old mistakes of committing big binaries into SVN. As SVN doesn’t clone the whole history, this doesn’t hurt a lot. But by moving to Git every binary and its history hurts. Sure, we have now huge disks and Gigabit LAN/WAN, but as everyone wants to have a SSD space gets limited again and sometimes the network isn’t as fast as we’d like to. So relieving the repository of its unnecessary binaries will help. Here is the script I use to get rid of big binaries (any improvements are welcome).

Preparations are done and we where ready for migration. Just one tiny problem: Git isn’t as easy to work with as SVN. Sure many things like branching and merging are faster and easier than svn IF you know how to work with Git. But the daily workflow is different and beside the current project work the developer has to learn and to adapt how to work with Git. So we decided to migrate Eclipse Scout in two steps. First, only the branch for the Kepler release. After Juno SR2 we will move all the other branches and tags to the Eclipse Git Repository and make the SVN readonly.

Shortly after opening the bugzilla request [7] I could push the Kepler branch to the Eclipse Git repositories [11,12]. The groundwork was done for the CBI-build.

To make it possible to merge changes made in the Juno branch to the new Git Kepler branch we have now the following setup. On our SVN repository server is an update script which updates the Git bare repositories from the SVN repositories at Eclipse.org and also from internal repositories. This is done with help of svn2git. From the eclipse repositories the Juno branches are pushed to Github [13] and from here I can add them to the “Remote Repositories” in EGit. Now I’m able to merge new commits from Juno to Kepler.

Migrating from PDE-build to CBI-build

As the process of migrating to Git took quite some time I started learning Maven and migrating the Scout bundles and  features between the migration steps. It helped me a lot, that the Eclipse Platform had already done quite some work for the CBI build. From their parent project I copied a lot which gave me a kickstart. From here, I learned how to manage the maven plugins and create profiles for the singing process.

Build Structure

Like the Platform build we have for the Runtime and SDK bundles two different Git repositories which gave us the possibility to create an aggregator project which contains the parent pom with the build information and where we can add other releng bundles / pom projects. The Runtime and SDK bundles have their own parent which have the aggregator (releng) parent as parent project.

Eclipse Scout Maven Structure

Target Platform

As Eclipse Scout has many dependencies to other Eclipse Projects and 3rd party libraries we need to build p2 repositories. With Tycho it is possible to place the repository informations into the parent pom or use a target file. We chose the second option because the target files can also be used from the IDE to develop and compile Eclipse Scout. As we have customers who use Eclipse Scout based products within IBM Lotus Notes Eclipse Scout has to be compatible with Eclipse >=3.4. To achieve this, there are target files for all platforms from 3.4 up to the current staging.

ScoutTargets

Challenges

As described by Daniel Wiehl [14] we need two jars (javax.jms_1.1.0.jar, javax.mail.jre16_1.4.3.jar) in the endorsed folder of the JRE. These two jars are also needed at build time. As the build has to work on any machine we can’t rely on having those two jars in place. This means that we have to include and add them into the build process somehow. The next gist shows how you can modify the bootclasspath of the java compiler. There we can add the two jars. The problem is, that this method overwrites the bootclasspath calculated by Tycho (there is already a fixed bug 394387 [15] concerning this. Howerver, I wasn’t able to check it out so far).

Another issue regarding the bootclasspath is that a class in Eclipse Scout makes use of internal stuff located in the tools.jar. The tools.jar isn’t part of the JRE but of the JDK. (The functionality isn’t used in the runtime, its part of the JAX-WS part and used to build sources from a WSDL.) As the tools.jar isn’t in the JRE consequently it isn’t taken into the bootclasspath by default. So we somehow have to “manually” write it into the bootclasspath as the two jars above. What we face here is another problem which I’m not sure is solved by bug 394387 [15]. For the tools.jar we need the path to the java.home directory and from there  (relative) to the JDK and its lib folder. java.home now is the running JRE (which runs maven) and not the one we want to use at compile time (BREE). It works for now, as the used classes and methods aren’t different but this is a ticking time bomb. I hope the aforementioned bug can solve this.

Contributing to Eclipse Scout

According to the main goal of the CBI initiative [16] I hope we have made it easier to contribute to Eclipse Scout. I will update the Contribution Guidelines [17]. Surely the Build is much easier to run on any machine. All things that need a special configuration are put into profiles so a “mvn clean install” can be done by anyone. No prerequisites are necessary.

If you wan’t to know more details to anything concerning the Eclipse Scout CBI Build please do not hesitate to ask via forum or twitter. I’ll be happy to answer the questions.

[1] http://wiki.eclipse.org/Platform-releng/Platform_Build
[2] http://wiki.eclipse.org/Minerva
[3] http://wiki.eclipse.org/LTS
[4] http://wiki.eclipse.org/Tycho/Release_Notes/0.16
[5] http://wiki.eclipse.org/Tycho/Release_Notes/0.17
[6] https://bugs.eclipse.org/bugs/show_bug.cgi?id=384958
[7] https://bugs.eclipse.org/bugs/show_bug.cgi?id=384960
[8] http://git.eclipse.org/c/scout/org.eclipse.scout.rt.git/commit/?id=3f93b1c1c7b1208477e198a91ac3f8f23365dfcd
[9] https://github.com/nirvdrum/svn2git
[10] https://github.com/sleicht/svn2git
[11] http://git.eclipse.org/c/scout/org.eclipse.scout.rt.git
[12] http://git.eclipse.org/c/scout/org.eclipse.scout.sdk.git
[13] https://github.com/BSI-Business-Systems-Integration-AG
[14] http://www.eclipse.org/forums/index.php/m/647259/
[15] https://bugs.eclipse.org/bugs/show_bug.cgi?id=394387
[16] http://wiki.eclipse.org/CBI#Primary
[17] http://wiki.eclipse.org/Scout/Contribution

Scout Links: Project Home, Forum, Wiki, Twitter, Instagram

Diesen Eintrag kommentieren
Wir sind sehr an einer offenen Diskussion interessiert, behalten uns aber vor, beleidigende
Kommentare sowie solche, die offensichtlich zwecks Suchmaschinenoptimierung abgegeben
werden, zu editieren oder zu löschen. Mehr dazu in unseren Kommentarregeln.