Planet Forges

Jenkins und die APT-Repositories Ⅱ

2011-11-29T15:46:37Z

As reported earlier we’ve got some kind of Jenkins/APT integration, with automatically generating as many repositories as a job desires.

News are that the builds host has moved, so the URIs to the repositories have changed. The new syntax is https://ci-something.lan.tarent.de/ for the Jenkins ⇒ “deb https://ci-something-debs.lan.tarent.de/jobname/ distribution suite …” and currently only usable in the company-internal network.

We’ve also got some more magic mksh code to automate the entire process – check the code out from SCM (required, as Jenkins’ svn checkouts are broken), build a Debian source package, NEW! ask cowbuilder to compile it in a clean chroot environment, and call mvndput.sh for APT repository publication. Sample projects are ci-evolvis/virtualscreen (git) and ci-dev/portal-setup (svn). Talk to me if you have any questions.

This allows for a one-line “run a shell command” build step!

Everybody else is, of course, invited to take and re-use our code, and maybe even improve upon it and submit that back. It’s all Open Source, after all.

tl;dr Jenkins Jobs now have integration with cowbuilder. There’s a new script to automate the whole build pipeline. The APT repositories have moved with the recent move.

FusionForge, October 2011

2011-10-28T14:45:04Z

As usual, a brief roundup of what happened in FusionForge land this month.

5.1.1 released. Nothing particularly Earth-shattering, mostly an acknowledgement of all the bugfixes that had been committed onto the branch since 5.1 was initially released. There are more already, including a large bunch of fixes to the search system. It's not part of an upstream FusionForge release, but the patches are in Debian unstable (and should trickle to testing in a few days).
Much work has happened on the database upgrading system. It's not committed yet, because it needs more testing first, but it'll go to trunk, meaning that FusionForge 5.2 and following will (at last) have the same handling of database schema changes whatever the installation method (*.rpm, *.deb or from sources).
An indirect consequence of that is that we'll use some features of PostgreSQL that are only available starting with PostgreSQL 8.3, so earlier versions will not be supported anymore. Since all major distros have at least 8.3 (more often 8.4 or 9.x), this shouldn't be much of a problem. (Note for CentOS 5 users: the postgresql package is 8.1, but there's a postgresql84 for 8.4.)
The Coclico project is officially over. It's been a great two years, and we're all happy of the dynamics that have been brought about by it. We're not going to drop the results on the floor, either, and many of the features that were developed as part of Coclico are already being used by production forges.
The server previously hosting Coclico has been moved to a VM, with the nice side effect that the SVN repository of FusionForge is now accessible on the standard ports even on IPv4 (IPv6 has worked for years ;-).
A plugin integrating MoinMoinWiki into FusionForge has been pushed to trunk, and it will be part of 5.2.
Speaking of 5.2: we don't have a release date yet, but we've agreed that the stabilization branch should start in early December, with a feature freeze on the 1st. So if you have a local feature that you'd like to see in upstream FusionForge 5.2, now is the time to bring it into shape and submit it.

Conférence aux journées Mathrice sur les forges logicielles

2011-10-06T09:38:59Z

J’ai fait une présentation le 5 octobre dans le cadre des journées mathrice à Lyon, sur le sujet des forges de développement logiciel.

C’était l’occasion de faire un point rapide sur le panorama des forges, nos efforts sur l’interopérabilité dans COCLICO (qui est maintenant terminé), et de mentionner brièvement quelques grandes manoeuvres en cours sur le sujet des forges, notamment dans l’enseignement supérieur et la recherche.

Voici mes slides :

Presentation forges logicielles à mathrice

Merci aux organisateurs pour l’accueil sympathique, et les échanges intéressants.

BerliOS forge will be closed on 31.12.2011

2011-10-02T16:37:57Z

I’have just read that the BerliOS software forge will be closed the last day of the current year, eleven years ago it was launched. I would like to offer the entire message in my blog to preserve the last decision of a project that have bosted the development of libre software in Europe.

Now, it’s time to start thinking about the best way to move the data from the BerliOS repository to other repository. The team besides the project ForgePlucker started to work on this subject some time ago. It could be interesting for BerliOS users.

Here you are, the message posted by the BerliOS team:

BerliOS will be closed on 31.12.2011


Dear BerliOS developers and users,
BerliOS was founded 10 years ago as one of the first repositories in Europe. It was developed and maintained by Fraunhofer FOKUS. As an European, non-proprietary project BerliOS pursued the goal to support the various Open Source players and provide a neutral mediator function. In 2011 over 4800 projects have been hosted on BerliOS, with 50,000 registered users and over 2.6 million file downloads each month. We are proud that with BerliOS we have brought the idea of an OSS repository to Europe. Meanwhile, the concept has prevailed and there are many good alternatives.
Unfortunately, as a research institute Fraunhofer FOKUS has only few opportunities to operate a repository like BerliOS. Such a project will only work with a follow-up financing, or with sponsors or partners taking over the repository. In the field of OSS this is a difficult undertaking. In a recent survey the community indicated some support in funds and manpower which we would like to thank you for. Unfortunately, the result is not enough to put the project on a sustainable financial basis. In addition the search for sponsors or partners was not successful.
Open Source is understood by Fraunhofer FOKUS as a paradigm for future-oriented intelligent use of IT. It hurts us all the more that we are forced to discontinue the hosting for BerliOS by 31.12.2011.
- As a developer, you should export your BerliOS project into another repository. For alternatives see http://en.wikipedia.org/wiki/Comparison_of_open_source_software_hosting_facilities.
- On our site you will find a guide on how to get your project data out of the portal and migrate it in a different platform, see http://developer.berlios.de/docman/display_doc.php?docid=2056&group_id=2.
Fraunhofer FOKUS has a strong commitment to Open Source and interoperability, and is involved in numerous successful OSS projects. The institute focuses on the development of quality standards for Open Source software and in particular on the technical, semantic and organizational interoperability between OSS components and between open source and closed source software. Example of our OSS activities including our management of the German Competence Center QualiPSo.
We thank all who have used BerliOS over the years.

Fraunhofer FOKUS www.fokus.fraunhofer.de

UPDATE: BerliOS continues – non-profit association is founded

fusionforge 5.1.1

2011-09-29T11:57:00Z

Minor release in the 5.1.x series. Nothing earth-shattering, only three months of accumulated bugfixes and translation updates.

Fusionforge, September 2011

2011-09-28T20:30:04Z

A summary of my FusionForge-ish activities this month:

FusionForge 5.1 is now available in Debian testing ("wheezy"). 5.0 is still in Debian stable ("squeeze"), so support continues; nevertheless, the improvements are significant enough that I recommend new users to run the version from testing if possible. Note that you can install the packages from testing onto a system running stable, either by hand or with APT pinning (there are only a handful of packages you need to grab from testing to satisfy the dependencies).
This was made possible by some bugfixing in the 5.1 branch of FusionForge. Nothing major in itself, but the accumulated fixes will probably be worth a 5.1.1 release soon.
I didn't do much on the trunk apart from merging from 5.1, but others did: several new plugins appeared, notably as part of the Coclico project (which is approaching its end fast).
Also, not yet pushed to trunk are two local branches I have: one is a plugin to integrate MoinMoinWiki (with permissions managed through the FusionForge roles), developed with a client of mine. The other will end up in reuniting the FusionForge database upgrade process across installation methods (the *.rpm packages and the source installation currently use a PHP script, the *.deb ones use a Perl script with transactions). It'll probably break many things when initially committed, so I'll keep it on a local branch until I feel it's mature enough.
I updated the "sandbox virtual machine" image; it now uses Debian wheezy, and the testsuite now runs a more recent version of Selenium; more importantly, the testsuite now passes.

Compact preview of resources in FusionForge

2011-09-19T15:07:16Z

I’ve been working on the ‘compactpreview’ plugin (in FusionForge’s SVN trunk), in order to support some javascript popups that can be used to display some “compact preview” of users and projects.

As can be see in the screencast below (also here) there are 2 types of compact preview popups :

those for local resources of the forge, that are displayed directly, as queried by the JS code on the /users/ or /projects/ pages with a specific “application/x-fusionforge-compact+html“content-type (required in the Accept HTTP header).
those compatible with the OSLC compact preview specifications, that can be displayed, should any other application want to display a compact preview. Again, these are served with content-negociation for “application/x-oslc-compact+xml“, which returns a short RDF document, which points to a script of the forge in charge of rendering the HTML compact preview.

The latter is demonstrated in the screencast, for display of popups for remote projects linked to a fusionforge project with the ‘extsubproj‘ plugin I already blogged about.

This code is still new, but will hopefully extend to other forge resources.

In the meantime, I’d be glad to see other forges implement similar mechanisms.

FusionForge news, June 2010

2011-09-16T11:59:34Z

Another month, another update, but nothing spectacular to be announced in FusionForge-world. We're still working on finishing the transition to the new configuration system, we're testing the migration to a simpler and more flexible set of Apache configuration files, and work is in progress on the RPM packaging. And so on.

Possibly the most newsworthy item is the FusionForge presence at next month's Libre Software Meeting in Bordeaux (the “RMLL” in French). I'll do a FusionForge, one year and a half later talk summarising the status and progress of FusionForge, and there'll also be a *forge devroom where we'll mingle with people interested in all kinds of software forges. Come see us if you're around!

FusionForge developers/users meeting coming up

2011-09-16T11:59:34Z

News is slow this month on the FusionForge development front. We're all busy gathering all the things that we want to go into the next release, but there's no big news from the code. However, there is something of interest.

You may have heard about the Coclico project, which is an initiative aiming at collaboration and convergence between several forge engines, most notably FusionForge, Codendi and Novaforge. That project was started last October, and it holds regular meetings with its members. The next meeting is scheduled for the 2nd of February in Paris, and we thought we could host an open meeting on the 3rd for non-Coclico members, a bit like the forge meeting we had last year (which is when FusionForge was officially born), but with an emphasis on what Coclico did so far. Since most of the FusionForge hackers are in Western Europe, and several are in Paris (especially if we add those who go to Paris for the Coclico meeting), we thought it would also be a good opportunity to gather for a technical and social meeting.

It seems the Coclico open session didn't generate much interest this time (at least, it hasn't so far), so I proposed to hijack the room for this FusionForge meeting, and I didn't hear any objections. I have several themes I'd like to discuss with people, and possibly start implementing during that day:

database maintenance and schema: unification of the upgrade scripts (including for plugins), cleanup of obsolete stuff, addition of missing constraints, and so on;
configuration system: my initial prototype didn't raise many objections (at least in its scope), now what to do with the next steps?
packaging and installation system: what needs to be done to keep the three ways of installation (manual, *.deb, *.rpm) in sync with as little work as possible?
permissions system: clarification of what happens currently, ideas for evolution;
plugins and interaction with external software: do we lack stuff that would make this easier?
roadmap, long-term plans, this sort of things;
other things that users may want to discuss with hackers?
possibly drink a beer or two;

…and so on. These are in no way specific to FusionForge, and in fact I think it would be great if hackers/users of other forges were present, because we could benefit a great deal from their experience and plans. But if we find ourselves amongst FF people only, I think these would be good to discuss, possibly write some code for, and go home with a clearer picture of where our efforts should focus in the near future.

I'd therefore like to invite interested people to mark the 3rd of February on their agendas. The meeting will take place in Issy-les-Moulineaux (near Paris, within reach of the tube). If you're interested, please get in touch with us (#FusionForge on the FreeNode IRC network, or the fusionforge-general mailing-list), so we can have a rough estimate of how many people to expect. The meeting room is provided by France Télécom, and they're probably going to need numbers if not names. Further details will be announced when known.

FusionForge news, March 2010

2011-09-16T11:59:34Z

Here's another quick update on the status of FusionForge.

We released version 4.8.3. Nothing earth-shattering, but a collection of bugfixes that had accumulated on the 4.8 branch. If you're running a patched version, you might want to merge.

We also published the second release candidate for 5.0. It's not final yet (there have been a few commits on that branch since then), but we're running out of known bugs. We're currently down to zero open bugs targeting 5.0, so the actual release is probably going to happen in a matter of days. 5.0rc2 is currently available in Debian experimental for those who want to test it, and the final 5.0 will be uploaded to unstable, and hopefully migrate to Squeeze in due time.

Stay tuned…

FusionForge news, September 2009

2011-09-16T11:59:34Z

Here's another round of the semi-regular bulletin about FusionForge.

First item: FusionForge 4.8.1 was released this week. It's not exactly an important update, but the 4.8 branch had been accumulating fixes over time and we felt that it would be good to push these fixes out. If you don't encounter particular problems, there's probably no need to upgrade in a hurry.

A follow-up for the rewrite of the SCM subsystem: I now consider the Bazaar and Git plugins complete. The missing part, in both cases, was a proper integration of a repository browser and the collection of commit statistics; since one of my clients wants to use Bazaar and another one wants Git, both features have been completed recently. The code still lives on a branch based off 4.8 (for people who need a 4.8-based instance), but it's also been pushed into trunk so the next release will have it natively.

Another branch I've been working on (for clients) was about making the Mediawiki plugin able to handle one wiki per project rather than one shared wiki. This is now possible with yet another 4.8-based branch, where the wiki creation is completely automated. A nice feature is that the FusionForge identification is used as a basis for Mediawiki, with different groups on the wiki depending on project membership and role in the forge. That allows specifying wiki permissions in a simple way, for instance to say that only project members can create new pages, authenticated users can only edit existing pages, and non-authenticated users are read-only. This code will be pushed to trunk in the coming weeks.

Thanks to Alain Peyrat, we now have a buildbot running Hudson for unit tests and a few other things. The coverage isn't complete yet, but we hope to increase it as time passes. It's already proven useful, by ensuring at least correctness of PHP syntax, encoding and line-endings.

I think that's about it for this time. Business as usual.

FusionForge 5.0

2011-09-16T11:59:34Z

Fourteen months after the renaming of the Free/Open Source code of GForge 4.x to the new “FusionForge” name, we're pleased to announce version 5.0. As mentioned in the release notes, this is still an incremental step over version 4.8 rather than a revolution, but the changes are important enough, and numerous enough, that we felt it justified to bump the major version number.

Major improvements, beyond a host of bugfixes, include:

a rewrite of the version control integration (with support for Bazaar, Darcs, Git and Mercurial in addition to the “traditional” CVS and Subversion);
a much better integration of Mediawiki (one wiki, with its own set of permissions, per project);
a cleaner database layer, more robust against SQL injections;
configurable display for the trackers;
more powerful tracker engine, with configurable workflows;
a rework of the default theme, with better accessibility.

FusionForge 5.0 now also includes new plugins that were previously only “floating around” (or completely private):

projectlabels gives a simple way of adding bits of HTML onto project description pages, so the forge admin can, for instance, highlight a “project of the month”;
extratabs allows a project to define new tabs in its pages, pointing at external resources;
globalsearch is a first step in the “federation of forges” concept, whereby a project search can be conducted on several forges at once;
contribtracker allows a forge to prominently display major contributors to projects, to give them visibility beyond the simple commit logs.

These plugins, as well as a large part of the improvements in the trackers and the rewritten Mediawiki plugin, are a direct consequence of the “upstreaming” of work having been done in private instances of forges. We're happy to note that this goal of ours (to merge local patches into the central repository when it makes sense) seems to be working well. For the record, this 5.0 release includes work and plugins that were reintegrated from sources such as Alcatel-Lucent, Adullact and AdaCore.

This release is also the first to have had the benefit of automated testing during the whole cycle. Coverage isn't 100 % yet, but the existing unit tests and functional tests help us be confident in the quality of the release. We'll keep adding more tests as time passes, of course.

Looking back at the initial goals stated when the project started, we seem to be on the right track:

stable release pushed out: check (this is the third one, not counting minor releases);
new plugins merged: check;
automated testing: check;
external contributions merged: check;
explicit governance model and release process: sort-of (there's still a cabal, but it's partially documented).

We still need to work on the database schema and the cross-distro part, as well as cross-forge interoperability. The good news is that work is happening on these fronts already. And with almost 2500 commits, we truly seem to have accomplished at least one of the (implicit) goals: to bring development back to a healthy state. And we're far from being out of ideas for the future, so there's a lot of good stuff still cooking!

FusionForge news, October 2009

2011-09-16T11:59:34Z

This month hasn't seen many big changes happen in FusionForge. Notable improvements include an initial search engine for Word files, fixes to the automated builds and tests, and lots of bugfixes.

The biggest news is probably the start of the Coclico project, an initiative bringing together developers and users of several existing forges in order to reduce the gap (and ideally unify the codebase across the forks) and work together in some fields where cooperation is important. Subjects include a generalisation of the current identity/permission/authentication models and systems, data exchange and migration, interoperability, integration of agile development methods inside the forge, and better integration with the desktop applications such as IDEs. The participants include NovaForge, Codendi, and of course FusionForge. The project only officially started early this month, but we hope to be able to demonstrate results soon.

Business as usual apart from that.

FusionForge news, May 2010

2011-09-16T11:59:34Z

The usual semi-regular bits of news from the FusionForge project. We continue being quite active, with several hundred commits each month. The momentum doesn't seem to stop even after the 5.0 release last month. A large part of the activity stems from the Coclico project, which has several work-packages related to convergence of code across forges (mostly between FusionForge and Codendi). This convergence comes in three flavours:

some new features are developed in common for both forges; as an example, the Mailman and ForumML plugins recently committed can now run unmodified in FusionForge and in Codendi (or at least that's the goal);
some features existing only on the Codendi side are ported across to FusionForge; this includes the Codendi “widgets” system, which allow drag-and-drop customization of some of the web pages, and the Hudson plugin;
finally, some of the core code is rewritten so that a common API and abstract data model can be used by higher-level pieces of code; the configuration system has been almost completely converted to a simple API, and the role-based access control system has also been rewritten into a clean model that extends both FusionForge's previous RBAC system and Codendi's, so each forge will provide new features for access control (the immediate gain for FusionForge is the ability to grant different permissions to anonymous visitors and to visitors that are logged in even if not members of the current project).

All in all, a fairly busy period for FusionForge. The current trunk is evolving rather fast, with some long-overdue rewrites being underway. Interesting times.

FusionForge news, November 2010

2011-09-16T11:59:34Z

Okay, so it's been five months since the last update. Sorry about that. I guess I could force myself to more regular updates.

What have you missed? “Only” the branching of what will eventually (soon?) become FusionForge 5.1. Quoting from the default home page, and in no particular order:

New Funky theme.
New UI and features for the document manager (download as zip, locking, referencing documents by URL).
New progress bar displaying completion state of trackers using a custom status field.
Improved sorting in trackers.
More flexible and more powerful role-based access control system.
New unobtrusive tooltip system based on JQuery and Tipsy to replace old help window.
New plugins: Blocks, to add free HTML blocks on top of each tool of the project; Gravatar, to display user faces; OSLC, implementing the OSLC-CM API for tracker interoperability with external tools.
scmgit plugin: Personal Git repositories for project members.
Simplified configuration system, using standard *.ini files.
Reorganised, modular Apache configuration.
RPM packaging.

We've also added many tests to our testsuite, and we'd welcome more. If you're interested in the new features coming to FusionForge, now would be a good time to try them and report the bugs you find. And if you don't want to upgrade your production server just yet, you can use a VM image that builds and installs everything automatically inside a VirtualBox (or another virtualization system). See details of my VM image for easy testing mailing-list post. That VM covers trunk rather than the 5.1 branch, but since the branching is still rather recent there isn't much difference yet.

Another tidbit of information: since I recently realized the IPv4 horizon is less than a hundred days away, I've been engaging in a flurry of migrations from IPv4-only to dual-stack, one result of which is that fusionforge.org is now accessible via IPv6. Yay for less NAT and reverse-proxying.

And I think that's it for now. There's one obvious piece of news I'm looking forward to announce, but it has to actually happen first…

GForge in Debian, February 2008

2011-09-16T11:59:32Z

Quick status update: not much happened due to a variety of reasons, but there is still some progress to report.

The most important piece of news is that the Mediawiki plugin should be on its way to Debian sid by the time you read this, as the new gforge-plugin-mediawiki binary package (it'll have to go through NEW, but that seems to be rather fast these days). Testing and reporting and bugfixing are most welcome, of course.

I also went through a round of cleanups in the packaging. No more Lintian overrides, far fewer Lintian errors and warnings, and some fixes for PostgreSQL 8.3 compatibility.

GForge is now FusionForge

2011-09-16T11:59:32Z

Executive summary

To avoid confusion with the proprietary versions of GForge (known as GForge Advanced Server, GForge Express Edition and GForge Community Edition), the free/libre/opensource codebase will from now on be separately maintained under the name FusionForge by the main developers of the free GForge 4.x codebase. Since this is mostly a renaming, the migration path for existing users will be smooth.

Longer version, with details

After the initial forking from the Sourceforge codebase, the development of GForge has long been hosted, and many enhancements directly developed, by the GForge Group (GForge, LLC), with regular contributions from outsiders. The results of these evolutions were public and free, subject to the GNU GPL.

In parallel, the GForge Group wrote a proprietary re-implementation of GForge, which it sold under the name "GForge Advanced Server", or "GForge AS" for short. This re-implementation added some features for "the enterprise", but was not contributed wholesale to the GForge codebase under a free license. Although some of the features were contributed to the public, the GForge Group concentrated its efforts on its (proprietary software) business model, with more versions appearing, such as "GForge Express Edition" and more recently "GForge Community Edition". As a result, it became increasingly harder for the public to know which version was which without doing extensive research (indeed, some users mistakenly installed one version instead of the other). A consequence was that the free software codebase suffered from a loss in visibility, which lowered its momentum to the point that there haven't been any moderately important releases since the (currently stable) 4.5.x series was announced in late 2005.

So, in order to clarify things, avoid further confusion, and regain some of the lost momentum, it was decided by a group of leading contributors that the free software version of the GForge codebase would from now on be developed under the FusionForge name, and its development would be hosted on FusionForge.org.

So is this a fork?

Well, we don't know yet. It could arguably be called one, since we're taking the code and running away with it under a new name. However, we believe it's not a fork unless both roads continue their own way (more of a oddly-shaped bend). What happens to the GForge codebase developed by the GForge Group at gforge.org remains to be seen, although for the sake of our users we will backport security fixes to the gforge.org Subversion repository (at least for the 4.5.x series and the unreleased 4.6 and 4.7 pre-series) for some time. The bulk of the development will move on to FusionForge and the repositories at FusionForge.org, though, and users are encouraged to migrate at their own pace. Since we're basically continuing the evolution rather than starting from scratch, the migration path should be rather smooth.

So why the FusionForge name?

Because there were actually lots of locally-patched versions of GForge (and Sourceforge), and we felt it was a waste of resources that should be fixed. It seems many people and organisations took these codebases at some point in time and evolved them for their own needs. Sadly, many of the changes were not contributed back or even published, so lots of efforts were duplicated. Fortunately, many of the people managing these locally-patched forges are now realising that "out-of-tree" patches and features require quite some manpower to maintain. Some formal inter-project discussion is already taking place, and we hope to achieve actual merging of most of the interesting features that have been developed here and there into a common base that can be reused locally with minimal changes. We'd like to "un-fork" as much as possible.

We also expect that, by using standard components and tools, we'll facilitate the work of potential contributors, thereby reducing the risk of a new era of fragmentation.

And who are we anyway?

We're Christian Bayle, Roland Mas and Alain Peyrat, long-time contributors to GForge and responsible for over 95% of the commits over the past two years, as well as a few relative newcomers. Christian and Roland have been maintaining the Debian packaging since the "Debian-SF" era, and Alain has been focusing on code quality. The three of us have, for various reasons, a vested interest in maintaining a lively codebase in a healthy ecosystem.

What are our plans?

Our short-term goals, as currently planned, include:

pushing a stable FusionForge release out of the door;
cleaning up and auditing the database schema to ensure consistency and increased performance;
merging in new plugins, in particular for new version control systems;
continuing streamlining the installation process to make it more portable across distributions.

Longer term goals are less well defined, but we're thinking about the following:

increasing code quality by the use of modern automated tools;
encouraging a lively stream of external contributions, to reduce the gap between the official version and the locally-patched ones;
defining an explicit governance model and release process for the FusionForge project;
as a consequence, a more frequent and predictable release schedule;
increasing data portability and interoperability with other forges, to reduce lock-in for users and projects.

Some of these items should be facilitated by our switch to a distributed version control system and a new coordinated workflow. Also, the Debian i18n team has been kind enough to offer to host our translation effort on their Pootle server, which means translators will have a much easier time doing their job.

We hope to hear from users and contributors alike in the near future.

For more information, we can be reached via our fusionforge-general mailing-list (see our lists), which is also suitable for general discussions. We can also be found on IRC (#fusionforge on the Freenode network).

FusionForge news, February 2010

2011-09-16T11:59:32Z

This is getting old news, and others have blogged about them before I did, but here's my summary of the recent activity in and around FusionForge.

The early February meeting was a success, and gathered about twenty people on the first day and a dozen or so on the second day (not planned initially). My impression is that there was a healthy mix of FusionForge hackers, FusionForge users, and people from other forge communities (Codendi, NovaForge, and even one representative from nFORGE, from South Korea). I'm not going to repeat all that was said then, especially since the proceedings are online. Beyond the technical points, I'll just advertise PlanetForge again, since everyone present agreed we had lots to share and that this site would be a good and relatively neutral place. If you're into forges, I recommend joining us in that community.

On the purely FusionForge front, news are good too. Most of the major pieces we want to see in the next release (which is probably going to be called 5.0) are in place. The last blocker we had was the merge of the rework of the default theme for better accessibility and easier maintenance and customisability (most of the theming now happens in CSS). This merge has been completed this week, and although there are still a few rough edges, it's mostly done. We'll try to fix most of these rough edges soonish, then start a stabilisation branch towards 5.0, so more experimental work can start again on trunk. For the impatient and the curious, there's a list of new features on the fusionforge.org homepage, and the site is now running code from trunk.

Of course, we're eager to get testers for that, which is why I prepared snapshot packages. They are currently stuck in NEW on their way to the official Debian experimental repository due to the renaming of the source package and the introduction of plenty of new binary packages, but they can already be obtained from my unofficial repository at people.debian.org. The packages are built for Debian unstable, but they seem to run just fine on Lenny if you grab mediawiki from backports (only required for the Mediawiki plugin, of course), and libnusoap-php and php-htmlpurifier from Debian testing (they don't drag any extra dependencies).

I'll end this note by reminding people of the announcement I did three months ago: as of this week, Debian Etch is no longer officially supported security-wise, and so neither is GForge 4.5. As far as I know, I was the last person doing that, and my incentives have gone away on the day Etch ceased to be supported, since it was also the day the Adullact forge finally migrated from Etch with GForge 4.5 to Lenny with FusionForge 4.8. If you're still using 4.5, well… I think you should be aware of that.

That more or less wraps it up for now. The next announcement is likely to be about a release candidate…

GForge/FusionForge update

2011-09-16T11:59:32Z

I normally don't relay security announces for GForge or FusionForge on this blog, but I will make an exception this time: Alain Peyrat found several places in the code with insufficient input sanitizing, which can cause cross-site scripting vulnerabilities (CVE-2009-3303). It's been fixed in the 4.7 and 4.8 branches as well as the trunk of FusionForge (and in Debian Sid and Squeeze), and updated Debian packages for GForge 4.5 and 4.7rc2 have been released for users of the Etch and Lenny distributions.

The reason I make an exception for announcing this here is to remind people that I appear to be the only one maintaining code for GForge 4.5. I do that for two reasons: first, because I'm the maintainer of the package in Debian, and Debian Etch has GForge 4.5, and Etch is supported for security fixes; second, because I also admin/maintain an instance for a client of mine, so I need to backport the fixes anyway, and making them public is no bother. Both of these reasons are going to vanish sometime in the not too distant future: security support for Etch will end in February, 2010, and I hope to have migrated my client's forge to FusionForge 4.8 by then too. A direct consequence is that I will probably stop maintenance for GForge 4.5 in the coming months (at least I'll stop doing it in my free time).

So if you're still using GForge 4.5, you should really consider upgrading to something supported, either GForge AS (free download from the GForge Group) or FusionForge (free as in Free Software). Both have an upgrade path. Obviously I think FusionForge is a better choice, but my position is probably biased.

Amis lecteurs de 01...

2011-09-16T11:59:32Z

J'ai l'immense honneur de figurer dans les pages de l'illustre magazine 01 Informatique de cette semaine, dans un encadré en marge d'un article sur les forges déclenché par une actualité (Sun a sorti une forge). Wouhou, c'est la gloire, je suis devenu un people et les statistiques de fréquentation de mon site pro autant que de ce blog vont exploser, et je vais être courtisé par les décideurs qui lisent 01 !

Je saisis donc l'occasion pour apporter quelques rectifications à cet article de Yann Serra.

Ne vous fiez pas au titre de l'encadré (« 2 questions à Roland Mas »), ni aux guillemets : il ne s'agit pas de questions qui m'ont été posées, ni encore moins de citations de mes réponses.
Apparemment je n'ai pas assez insisté lors de l'entretien téléphonique sur le fait qu'une forge est une plate-forme collaborative, pas juste un site de téléchargement. Du coup, la première question perd tout son sens (ce qui explique que la « réponse » supposée soit confuse.
Sur la deuxième « question »... la réponse qui m'est attribuée ne répond pas à la question, et pour cause : Sun n'a jusqu'à présent fait que des annonces sur ce qui différencie Kenai des autres forges. “We're more than just a forge”, mais on ne sait pas quoi. “More about these features will be revealed as we get closer to their availability.” Difficile donc de juger sans savoir.
Je n'ai pas non plus assez insisté sur l'aspect Ruby, visiblement. Ma position là-dessus est que les choix d'implémentation de Kenai n'ont pas la moindre espèce d'importance sur le succès de la forge kenai.com tant qu'elle est uniquement hébergée par Sun. Ce serait du Cobol ou de l'Intercal que ça ne changerait rien. Ils ne deviendront pertinents que si le logiciel Kenai est publié sous une licence libre, parce qu'ils constitueront, ou non, des barrières à l'entrée pour des contributeurs potentiels qui feront vivre la communauté.

Je ne crache pas sur le journaliste, ç'aurait pu être bien pire. Je suis juste déçu que mes propos aient été complètement réinterprétés, et je déplore ouvertement que l'article laisse entendre que Sun ait relancé une quelconque bataille, alors que pour l'instant rien ne différencie dans le bon sens Kenai d'une autre forge.

Mais bon, je suis un people, il faut que je m'y habitue, pour être préparé le jour où on me fiancera à Grace Kelly ou à la Castafiore. En attendant, vous pouvez trouver des paquets Debian de GForge sur http://people.debian.org/~lolando/. Avec les autres people.

More GForge progress

2011-09-16T11:59:32Z

I'm on a roll...

Gettext transition is fully done now (old functions have even been removed).
French translation 100% complete! Woo!
Packages now use ucf. So there.
Installation process streamlined: if you just want the web interface, you'll only need to aptitude install gforge-web-apache2, and the only interactions it'll require are to type the admin password (twice) and to accept (or integrate by hand) a change in the PostgreSQL config file.
It even works on Debian GNU/kFreeBSD!

Plans for the near future include continuing to clean up upstream code and maintainer scripts, making sure the installation process is as simple as possible (even for other subpackages), splitting out a few plugins into their own packages. And the big placeholders-in-prepared-SQL-queries audit I mentioned last time, but it may happen progressively rather than in one big go.

Faire-part de naissance : FusionForge

2011-09-16T11:59:32Z

(Ceci est juste un résumé en français de l'annonce complète en anglais.)

Au début, il y avait SourceForge, un logiciel libre développé par une boîte qui s'appelait (à l'époque) VA Linux Systems. Ce logiciel était utilisé par plein de gens, et développé de manière plus ou moins collaborative. Un jour, VA Linux Systems a décidé que les nouvelles versions de SourceForge (à partir de la version 3) ne seraient plus libres (et qu'elles s'appelleraient Sourceforge Enterprise Edition). Plusieurs personnes sont donc parties avec la dernière version libre (celle qui aurait pu devenir la version 2.6), avec dans l'idée de maintenir ce code.

Il y a donc eu GForge, un logiciel libre développé entre autres par une boîte qui s'appelle le GForge Group. Ce logiciel était utilisé par plein de gens, et développé de manière collaborative, notamment par votre serviteur. Un jour, le GForge Group a décidé que les nouvelles versions de GForge (à partir de la version 5) ne seraient plus libres (et qu'elles s'appelleraient GForge Advanced Server). Mais comme l'hébergement du projet était maintenu ouvert, les versions 4.x ont pu être maintenues de manière libre et collaborative.

Pendant ce temps, l'appellation "GForge Advanced Server" s'est faite de plus en plus rare, et des noms plus équivoques sont apparus : "GForge AS", "GForge Express Edition" (ou "GForge EE") et, plus récemment, "GForge Community Edition", aucun de ces logiciels n'étant libre (même si certains sont disponibles en téléchargement gratuit et qu'on peut même jeter un œil aux sources).

Comme cette ambiguïté prêtait à confusion (certains utilisateurs ont installé une version propriétaire en croyant installer un logiciel libre), les principaux développeurs de la version libre de GForge (Christian Bayle, Alain Peyrat et moi-même) ont décidé de... partir avec le code, et de renommer le projet de développement et de maintenance de ce code libre.

Le résultat s'appelle FusionForge, et il est hébergé sur FusionForge.org. Nous avons plein d'idées, mais un des buts majeurs (qui a justifié le nom) est que nous allons chercher à réintégrer dans le code commun des fonctionnalités qui ont été développées localement par des utilisateurs mais non publiées. Ça tombe bien, nous sommes déjà en relation avec plusieurs de ces utilisateurs institutionnels qui semblent intéressés par cette convergence.

Call for translations for GForge

2011-09-16T11:59:32Z

Stuff happens quietly on the GForge front, but after some time we decided we're getting bored with not releasing. Since we seem to have run out of major problems in the codebase, the long-awaited GForge 4.7 release is probably round the corner.

And so, since GForge migrated from its own in-house translation system to the more conventional gettext API, I'd like to take the opportunity to issue a call for translations, knowing that potential translators won't be too disturbed by unusual tools and formats.

You can grab the current state of the translations from the GForge repository browser. Or, for more long-term involvement, checkout the code through Subversion or through Bzr (my gateway branch is available from bzr.debian.org. Current statistics are as follows:

bg.po: 382 translated messages, 174 fuzzy translations, 1813 untranslated messages.
ca.po: 1592 translated messages, 281 fuzzy translations, 496 untranslated messages.
de.po: 1627 translated messages, 272 fuzzy translations, 470 untranslated messages.
el.po: 0 translated messages, 2369 untranslated messages.
eo.po: 0 translated messages, 2369 untranslated messages.
es.po: 1600 translated messages, 236 fuzzy translations, 533 untranslated messages.
eu.po: 1392 translated messages, 272 fuzzy translations, 705 untranslated messages.
fr.po: 2368 translated messages, 1 untranslated message.
he.po: 0 translated messages, 2369 untranslated messages.
id.po: 0 translated messages, 2369 untranslated messages.
it.po: 1781 translated messages, 303 fuzzy translations, 285 untranslated messages.
ja.po: 246 translated messages, 118 fuzzy translations, 2005 untranslated messages.
ko.po: 1292 translated messages, 264 fuzzy translations, 813 untranslated messages.
la.po: 0 translated messages, 2369 untranslated messages.
nb.po: 0 translated messages, 2369 untranslated messages.
nl.po: 1621 translated messages, 287 fuzzy translations, 461 untranslated messages.
pl.po: 0 translated messages, 2369 untranslated messages.
pt_BR.po: 1292 translated messages, 262 fuzzy translations, 815 untranslated messages.
pt.po: 0 translated messages, 2369 untranslated messages.
ru.po: 302 translated messages, 142 fuzzy translations, 1925 untranslated messages.
sv.po: 1289 translated messages, 261 fuzzy translations, 819 untranslated messages.
th.po: 0 translated messages, 2369 untranslated messages.
zh_CN.po: 1691 translated messages, 300 fuzzy translations, 378 untranslated messages.
zh_TW.po: 1664 translated messages, 289 fuzzy translations, 416 untranslated messages.

Results as patches to our patch tracker or the gforge-devel ML please.

(Note to Debian-related readers: this translation work will be directly useful on Alioth when we upgrade it.)

GForge security patch, and a new feed

2011-09-16T11:59:32Z

First, and most important: while researching a functional bug for a client, I found a rather important security problem in GForge. All versions (starting from 3.1) are vulnerable to an SQL injection problem due to missing input sanitisation. Debian packages have already been fixed and released, and the patches have been committed to the upstream Subversion repository, so non-Debian users are encouraged to grab the patches from there. For instance, the patches for the 4.5.* branch can be obtained from the ViewVC page. For reference, the CVE ID for this problem is CVE-2008-0173.

Secondly, there's a new "gforge" tag on this blog, to filter posts that relate to GForge. I mainly created it in response to the existence of a feed aggregator focusing on forges and variants, but you can also subscribe to it directly if you only want to hear about Gforge and not about my other Free Software activities. I'll also use it to announce security patches like this one.

GForge, janvier 2008

2011-09-16T11:59:32Z

Voici quelques nouvelles de GForge en français, pour changer, parce qu'il y a visiblement un fort contingent d'utilisateurs francophones de GForge. Si le blabla ne vous intéresse pas, sautez quelques paragraphes, y'a une annonce qui peut vous intéresser.

Je m'en doutais un peu à vrai dire : je savais déjà qu'il y avait des instances de GForge en usage dans un certain nombre d'entreprises et d'administrations françaises. Je déplorais d'ailleurs que ces usages soient privés, voire secrets... On n'en entendait parler que par la bande, au détour d'une conversation. Et chacun avait ses bricolages locaux, et ses améliorations personnelles, dont personne ne profitait.

Heureusement, les différents utilisateurs francophones de GForge (de forges en général) ont fini par plus ou moins se retrouver, et des discussions ont commencé sur la liste Picolibre Forges. On s'aperçoit donc que de nombreux utilisateurs existent, qu'ils ont souvent des besoins communs, et que certains ont même déjà des solutions à apporter à certains de ces besoins. Il se pourrait bien que ces utilisateurs (et ces développeurs) se mettent à communiquer et à relancer une vraie dynamique de communauté autour de GForge (pour l'instant, il y a une poignée de développeurs, et quelques utilisateurs qui ne communiquent pas, donc j'hésite à appeler ça une communauté). Quelques-uns de ces utilisateurs et moi-même nous sommes rencontrés à la conférence Qualipso la semaine dernière, normalement nous devrions nous retrouver au salon Solutions Linux la semaine prochaine, et une réunion spéciale forges est même organisée après ça. Donc, ça prend forme.

Concrètement, ce que j'espère principalement est que les modifications de chacun seront partagées, de sorte qu'elles puissent être intégrées au cœur de GForge, et portées sur une version plus récente (puisque l'immense majorité des utilisateurs actuels sont basés sur une version 4.5.x patchée). Le « tronc » Subversion de GForge devrait donc intégrer, dans un futur que j'espère pas trop lointain, les évolutions suivantes :

Intégration du bug-tracker Mantis : au moins deux (et vraisemblablement trois) entités ont déjà réalisé cette intégration, et j'essaie de récupérer les patches pour que tout le monde en profite. Pourquoi tout le monde se focalise sur l'intégration d'un nouveau tracker au lieu d'exploiter la flexibilité de celui de GForge pour l'étendre, ça me dépasse, mais je ne suis pas là pour juger. De même, je trouve vraiment dommage que ce développement ait été fait deux ou trois fois de manière indépendante et sans concertation. Vous avez dit gaspillage de temps humain ?
Ajout d'un système d'intégration continue (on me dit « Maven »). Là encore, normalement ça a déjà été fait, il ne devrait plus rester qu'à publier les patches et les porter vers l'état actuel du code.
C'est peut-être lié à l'item précédent (je manque de détails), mais on devrait aussi voir apparaître une intégration dans GForge d'un système de tests automatisés.

...et je ne doute pas que d'autres utilisateurs, qui ont eux aussi ajouté leurs propres fonctionnalités sans rien dire à personne, vont aussi se révéler au grand jour et collaborer avec la communauté (n'est-ce pas ?). Peut-être même que des gens vont remettre au goût du jour l'empaquetage RPM, abandonné depuis plusieurs années.

Histoire de ne pas être en reste, je fais ici l'annonce publique suivante : le plugin Mediawiki pour GForge est enfin publié. Ce plugin fait suite à une intégration faite « avec des contraintes de temps assez serrées » (comprendre « un peu à l'arrache ») pour un client, et à une autre intégration faite plus proprement pour un autre client. Le dépôt SVN de gforge.org contient donc présentement le code qui va bien, et la prochaine version des paquets Debian qui seront publiés fournira un nouveau paquet binaire appelé gforge-plugin-mediawiki. Je dispose également d'une version du plugin pour GForge 4.5.x, mais comme Mediawiki nécessite PHP 5, il faut également extraire de ma branche client la conversion PHP 4 → PHP 5 de GForge 4.5 (et en retirer les fonctionnalités réellement spécifiques au client), ce qui explique que ce n'est pas encore publié sur mon dépôt APT (ni déployé sur Alioth). J'y travaille, promis.

Lecture on “Jailbreaking the Forges : project export/import efforts”

2011-09-15T21:18:51Z

I’~~ll be speaking next week~~‘ve been at Open World Forum, in the OSDCfr track in with a speech titled “Jailbreaking the Forges : project export/import efforts”

Here are the slides (as PDF – 1.3 Mb)

Jailbreaking the Forges : project export/import efforts

Here’s a copy of the presentation I wrote.

Software forge are “data jails” in that development projects established in a forge may suffer from data lock-in if they have to, or want to, change of hosting solution.

Some of the tools allow easily to fork or move a project’s code (such as DVCS like Git, Bzr or Hg), but for other tools like bugtrackers, mailing-list managers or wikis, it’s much harder to extract data from one forge and transport it to another one. Also, users and their privileges, as well as many other metadata (who did what, and when) may suffer from such migrations.

Even though most projects don’t fell such lock-in as a high risk (even in FLOSS projects which value freedom of information, strangely), history as shown that in case of outages, hosting platforms can be quite a trap to projects.

Other hazards may happen, like unresponsive admins, forks in a community, archiving old projects while being able to restore them, do migrations, or just the wish to move to newer, cooler hosting platforms.

Despite 10 years of forge usage, it is only recently that few progress have been made in implementing standard exchange data formats and supporting tools, allowing us to envision a possible solution to these lock-in issues.

We’ll present the ForgePlucker project (initially started by esr after a few popular blog posts on the subject), and further efforts lead in the COCLICO project to provide an open and extensible standard exchange format for projects data export and import. In addition to forgeplucker, we’ll demonstrate the FusionForge import tools used as an archive/restoration mechanism.

We’ll then call for other forge implementors and advanced users to join us, for more efforts on this topic, in order to gather all the tools that are needed to make possible migrations of projects from forges to forges.

OAuth support in FusionForge, and the forge can tweets

2011-09-14T07:24:51Z

We have been working in the frame of COCLICO on implementing support of the OAuth protocol, both as an OAuth provider/server (for instance for the needs of authentication to the OSLC server‘s Web Services), and an OAuth consumer/client.

That OAuth consumer has been put to work in order to connect to twitter’s API, so that the forge is now able to push news to tweeter and other IM services.

OAuth will, beyond twitter, allow the forge to interoperate with other web services supporting connections on behalf of users, instead of using fake accounts, or storing passwords in the databases.

All these are committed in FusionForge’s SVN trunk.

Dynamically querying external (sub) projects properties with RDF in FusionForge

2011-08-29T13:35:32Z

I’ve been working recently on two plugins for FusionForge. The work is somehow a POC for some COCLICO dynamic interoperability work-package, but some outcomes may actually be of use someday in real life, who knows

The first plugin is called extsubproj, and allows the definition of links to external subprojects (i.e. hosted on another forge), in the properties of a FusionForge project. It’s basically managing a set of stored URLs and displaying them in the top project’s summary page. Nothing fancy, so far, and the code is not yet finished, nor pushed to FusionForge’s trunk yet.

The second plugin, called doaprdf, allows the publication, by the forge, on the same URL as the project summary page (those URLs are standardized in FusionForge in the form : http://.../projects/projname), of a RDF+XML description of some of the project’s metadata, using the DOAP dialect. This works with content-negotiation, following principles of the Linked Data paradigm, so that the same URL, when requested for HTML, renders a Web page (the forge project’s summary page) meant for humans^geeks, and when queried with a special content type Accept HTTP header (Accept: application/rdf+xml), meant for machines.

Now, when you combine these two, you gain the possibility of having extsubproj display not only the suprojects’ URLs, but also some of their meta-data, for instance a link in the form of fetched doap:name.

The POC illustrates how one may then construct a hierarchy of (public so far) projects and sub-projects accross the buondaries for different forges databases, and display them in a similar manner as local projects (for instance, what the FusionForge plugin projects-hierarchy provides), through dynamic query of the remote project’s properties fetched on demand and modeled in a generic dialect (RDF with common ontologies such as DOAP).

Note that a similar FusionForge plugin “foafprofile" is being developped too for users profiles, using RDF and FOAF.

Stay tuned for more content in the same vein.

Links collection about software forges: status, criticism and new ideas

2011-08-24T18:40:14Z

During the last two years it is quite common to hear about new software forges, but I’m not going to talk about forges proliferation in this post, what I would to like to discuss it what the next step in collaborative development environments is. Thus in order to get the big picture I spent some hours looking for scientific and “informal” publications, now I think I have a good starting point and it would be great if you can offer feedback or even improve it.

The list of links contains papers, blog entries, presentations and reports. Some of them point out very interesting problems like the “data jail”, others re-think the concept of software forges like the paper named “The networked forge” and finally, some of the developers of the main software forges present what they do and what they want to do during next years.

Papers:

“The Networked Forge: New Environments for Libre Software Development” by Jesus M Gonzalez-Barahona, Andrés Martínez, Alvaro Polo, Juan José Hierro, Marcos Reyes, J Soriano (2008)
“Open Collaboration within Corporations Using Software Forges” by Dirk Riehle, John Ellenberger, Tamir Menahem, Boris Mikhailovski, Yuri Natchetoi, Barak Naveh, Thomas Odenwald (2009)
“Collaboration in Software Engineering: A Roadmap” by Jim Whitehead (2007)

Blog entries:

“COCLICO project’s efforts towards better forges interoperability (long)” by Olivier Berger (2010)
“Three Systemic Problems with Open-Source Hosting Sites” by Eric S. Raymond (2009)
Excerpt of a report made for the ministry of education calling for more forges pooling (sharing of maintainance costs, etc.) in France, by Olivier Berger (2011)
“About software forges” by Carlo Daffara (2010)
“Review: Free Software Project Hosting” by John Goerzen (2009)
“[Discussions] Quick & dirty report from the OWF 2010 think tank on Open Forges” by Olivier Berger (2010)
“GitHub, Collaboration, and Haters” by Elizabeth Naramore (2011)
“What Black Duck Can Tell Us About GitHub, Language Fragmentation and More” by Stephen O’Grady (2011)
“Everything you ever wanted to know about software forges (code forges), June 2011″ by David Williams (2011)

Presentations:

“Choosing a software forge” by Joseph Roumier (RMLL 2010)
“Fusionforge, one year and a half later” by Roland Mas (RMLL/LSM 2010)
“The #Launchpad collaboration platform” by Jonathan Lange (#LSM2010)
“A new #Savane” by Sylvain Beucler (#RMLL2010 / #LSM2010)
“The OSS Forge Ecosystem” by Nathan Oostendorp from sourceforge (2010)
“Coclico project” by Olivier Berger (2010)
“Specifications for interoperability slides” by Olivier Berger (2010)
“Spago4Q – Quality of OSS Forges” Davide Dalle Carbonare at fOSSa Conference (2010)

Reports:

“Informe Técnico: Forjas: entornos de desarrollo colaborativo. Su integración en el ámbito empresarial. 2009” by LibreSoft (2009, in Spanish)
“The business value of open collaboration” by IBM (2010)

I’ll submit this information to the planetforge wiki, as soon as it is posted there I will include here the link.

[This entry is part of the work I do in LibreSoft and it is also available in my blog at libresoft.es]

Evolvis-Wikis und die Suchmaschine

2011-08-11T12:29:44Z

(Dieses Posting wurde ursprünglich im internen Blog veröffentlicht, ist jetzt jedoch hierhin umgezogen. Firmeninterne Information wurde gekürzt, ist aber für Mitarbeiter im Mailinglistenarchiv ersichtlich.)

Neues aus der Adminstube (und vom Evolvis-Team) für unsere Projektmanager:

Im Rahmen der tarent-Suche werden die Wikis indiziert; dies ist natürlich nicht immer gewünscht, auch wenn die Suche nur Mitarbeitern zugänglich ist. Darum gibt es jetzt eine Möglichkeit, pro Projekt das Wiki öffentlich oder privat zu setzen. Im Moment werden nur die Wikis auf evolvis.org und dev.tarent.de indiziert, ab dem 20.06.2011 aber auch auf den Kundenevolvinēs – daher stellt eure Wikis bitte ein!

Wir werden jedes Wochenende den Suchindex leeren und neu aufbauen, das heißt, zukünftige Änderungen von öffentlich auf privat werden erst in der Folgewoche aktiv!

Die Einstellung betrifft den XML-Export, die tarent-Suche und die Sichtbarkeit für nicht eingeloggte Besucher. Wenn ihr die Einsicht eines Wiki vor Mitarbeitern, die keine Projektmitglieder sind, verstecken wollt, müssen wir weiterhin die Rechtefreigaben von Hand anpassen.

Die aktuelle Konfiguration ist wie folgt: [… gekürzt …]

Jenkins und die APT-Repositories

2011-08-11T12:29:05Z

(Dieses Posting wurde ursprünglich im internen Blog veröffentlicht, ist jedoch hierhin umgezogen. Es sind keine sensitiven Informationen enthalten, allerdings können die Links nur im Firmennetz angesteuert werden.)

Neues aus der Adminstube (und vom Evolvis-Team) für unsere Entwickler, die wir, nachdem wir die Projektmanager schon bedient haben, auch nicht zu kurz kommen lassen wollen:

Wenn ihr einen Jenkins-Job habt, der Debian-Pakete (oder für Derivate) erstellt, könnt ihr ab sofort (mindestens) ein Repository pro Job haben, in dem ihr die Pakete „veröffentlichen“ könnt. Diese Repositories werden automatisch signiert, und durch tarent-keyring werden die Signaturen bereits seit Dienstag verteilt, sodaß auch SecureAPT funktioniert.

Im folgenden wird die Arbeitsweise am Beispiel von Saschas WIP-Portalinstaller erklärt.

Ausgangspunkt ist ein Jenkins-Job, in diesem Fall auf dem test-hudson, hier mit dem Namen „portal-setup“, der unter „Build“ den Punkt „Execute shell“ aktiv hat. Hier wird durch diverse Kommandos ein Debian-Paket (Source und Binary) gebaut, das wirklich wichtige hierbei ist folgender Ausschnitt:

cd portal-setup-$pv dpkg-buildpackage -rfakeroot -us -uc cd .. #rm -rf portal-setup-$pv

Der Befehl dpkg-buildpackage übernimmt das Bauen und schmeißt die erstellten Pakete (Source *.dsc und Binaries *.deb) und, ganz wichtig, die *.changes-Datei, ins Elternverzeichnis (daher die cd-Aufrufe). Nun möchte Sascha, daß diese Pakete einfach von Kollegen getestet werden, und ändert daher den Codeschnipsel so ab:

cd portal-setup-$pv dpkg-buildpackage -rfakeroot -us -uc cd .. #rm -rf portal-setup-$uv portal-setup-$pv

/opt/mvn-debs/mvndput.sh portal-setup squeeze main *.changes

Im ersten Teil bleibt alles beim alten, aber ein neuer Befehl ist am Schluß hinzugekommen. Was macht der?

Nun, /opt/mvn-debs/mvndput.sh ist die Magie, die unterhalb des Pfades /opt/mvn-debs ein Debian-Repository mit dem Jobnamen portal-setup anlegt. Wir publizieren Pakete in die „dist“ squeeze, und darin in die „suite“ main. Das ist hier lediglich eine Konvention; normalerweise heißt die dist so wie die Debian-Version (sarge, dapper, etch, hardy, lenny, squeeze, …), für die das Paket gedacht ist, und die suite ist eine Unterkategorie – also zum Beispiel main/contrib/non-free oder main/restricted/universe/multiverse oder wtf/tarent/evolvis (in unserem Adminrepo). Man kann die aber auch frei Schnauze nennen (ich hab zum Beispiel im m68k-Repo eine dist namens „cross“, unterhalb derer eine Cross-Toolchain liegt).

Das letzte Argument ist *.changes, was die Shell für uns expandiert: alle Dateien, die auf „.changes“ enden, werden dort (ASCIIbetisch sortiert) der Reihe nach aufgelistet.

Das Skript prüft nun zunächst die Namen und Dateien auf Plausibilität (es sind schließlich immer nur gewisse Zeichen erlaubt) und schiebt dann vermittels dput ein Release (also eine *.changes + alle drin enthaltenen *.deb + dazugehörige *.dsc, *.diff.gz/*.debian.tgz, *.orig.tar.gz, oder *.tar.gz bei native packages) ins APT-Repository und läßt danach den Index regenerieren und PGP-signieren. dput ist ein schlaues Tool, es schreibt nämlich nach getaner Arbeit eine *.upload-Datei, in welcher dieser Fakt verzeichnet wird, sodaß Pakete auch wenn man den workspace nicht jedes Mal wegschmeißt nur einmalig hochgeladen werden. (Es existiert allerdings hier kein Schutz davor, den workspace zu leeren und ein Paket mit derselben Version aber anderem Inhalt ein zweites Mal hochzuladen. Nur die Systeme, die das installieren wollen, mögen einen dann ggf. nicht mehr so gerne.)

mvndput.sh ruft nun also mvndebri.sh auf, was allerdings mehr macht als nur das Äquivalent von Debians dak oder CentOS’ createrepo. Es erstellt nämlich auch einen Index.

http://test-hudson-debs.bonn.tarent.de/portal-setup/debidx.htm heißt der Gute, und ist eine Auflistung aller Pakete nach dist, suite, Source und Binary im Repository. Weiterhin steht oben nochmal, um welches Repository es sich handelt, und welche dists und suites verfügbar sind. Der Name bildet sich aus „http://“ + Jenkins-Systemname + „-debs.bonn.tarent.de/“ + Jenkins-Jobname + „/debidx.htm“ – wenn man den Dateinamen wegläßt kann man das auch direkt als Verzeichnisstruktur anschauen.

Der Clou: die passende sources.list-Zeile steht auch schon da! (Allerdings muß man die suiten ggf. auf die, die wirklich benötigt werden, reduzieren.)

deb http://test-hudson-debs.bonn.tarent.de/portal-setup squeeze main

Das funktioniert auf allen acht Jenkins-Systemen, allerdings natürlich nur aus dem tarent-Netz heraus – dafür ohne https oder aufwendige Authentifizierung. Einfach so.

Bitte achtet darauf, den Namen des Jenkins-Jobs als erstes Argument zu mvndput.sh zu verwenden, ggf. gefolgt von einer Kennzeichnung, falls ihr mehr als ein Repo pro Job braucht (warum, weiß ich nicht, aber es geht). Da alles als maven-User läuft findet keine Abgrenzung statt.

Wenn ihr Pakete aufräumen möchtet, so loggt euch (als User maven) auf dem entsprechenden Jenkins ein und schaut auf der Verzeichnisebene nach; einen direkten Link gibt’s auch, wenn man das [dir] im tabellarischen Index anklickt, zum Beispiel http://test-hudson-debs.bonn.tarent.de/portal-setup/dists/squeeze/main/Pkgs/portal-setup/, was im Dateisystem dem Verzeichnis /opt/mvn-debs/portal-setup/dists/squeeze/main/Pkgs/portal-setup/ entspricht.

Nachdem ihr händisch Änderungen dort vorgenommen habt, müßt ihr natürlich den Index neu generieren lassen, das geht dann so:

mksh /opt/mvn-debs/mvndebri.sh /opt/mvn-debs portal-setup squeeze

Wenn ihr nicht nur eine dist angefaßt habt, könnt ihr die auch auflisten:

mksh /opt/mvn-debs/mvndebri.sh /opt/mvn-debs portal-setup hardy squeeze

Oder einfach weglassen, dann macht er alle:

mksh /opt/mvn-debs/mvndebri.sh /opt/mvn-debs portal-setup

Das ist dann auch der Unterschied zwischen dists und suites: letztere teilen sich ein Release-File, erstere nicht, nur den XHTML-Index.

So, ich hoffe, ihr findet das so nützlich wie Sascha und so arbeits- und zeitersparend wie ich ☺

PS: Der Code ist mittlerweile auch publiziert. Wer solch ein Szenario noch woanders aufsetzen möchte ist uns herzlich willkommen.

35, a.k.a. it’s time for a blog posting again ☺

2011-07-20T09:12:02Z

Evolvis 4.8.35 has been released this week. This is an amazing upgrade within the 4.8 series of development, to bridge the time gap until such time as the 5.1 series can be used.

As announced earlier, our APT Repository contains packages targetting Debian Lenny (amd64, i386), including side packages and backports needed for a standard EvolvisForge installation, add one of these lines to your /etc/apt/sources.list to use it. These packages might work on Debian squeeze, maybe even *buntu, but will probably have issues with multiarch on Debian unstable.

First the big caveat: support for old-style (Jutta Horstmann) external Wiki instances has been removed, since we migrated all Evolvis installations to use gforge-plugin-mediawiki a while ago already.

Now the big news: bug trackers (old and new) contain several predefined search queries useful for Software Engineering and Quality Assurance. There’s a new standard tracker type called “Funktionsreferenz” (functionality specification) on every newly created project. The Extratabs plugin, backported from 5.1 then enhanced, allows adding arbitrary tabs as link or IFRAME (embedding content) to any project. And the DatePicker component allows easily entering a date, or a date plus time-of-day, using an ECMAscript pop-up while also accepting simple strings, for instance from Lynx users. The format used for displaying dates can be configured in “My Account” between d.m.y, y-m-d and m/d/y; the software accepts all three formats, always, nevertheless. Furthermore, the ECMAscript DatePicker widget is available in a number of languages for all three formats – English, German (Deutsch), Spanish (Español), French (Français), Italian (Italiano), Dutch (Nederlands), Polish (Polsku), Portuguese (Portugues), Romanian (Românesc), Bulgarian (Български), Russian (Русский), Hungarian (Magyar), Norwegian (but I’m not sure if it’s Nynorsk or Bokmål).

Now the small improvements: a robots.txt file is present by default, allowing wget and asking all crawlers to slow down; system administrators may configure it (in gforge.conf) to switch to one disallowing search engine spiders. Newly created Tasks have a default duration of two weeks (standard Scrum timeframe), not one week. /usr/share/gforge/bin/scm-newsubrepo.php supports easily adding new git repositories to a project already using them.

And finally, the most important / visible bug fixes: Sorting tables in Tasks works again, and the search drop-down boxen in Tracker are now sorted. The modify task/tracker forms have more “Submit” buttons and have been slightly rearranged to improve User Experience. “Copy+Close” a task works again. Some links, labels and translations (German only) have been corrected. Project members are now added to the default mailing lists correctly, i.e. to the -discuss group always, to the -commits group if they have SCM Write permissions. Custom Tracker fields of type Checkbox no longer have the “100 None” option. Tasks do not show up on “My Page” several times now, once is enough ;-)

Last but not least, a look at the future: we’re working on our Jenkins Plugin, which we might provide for Evolvis 4.8 if it’s functioning quickly enough. We’re also working on getting Evolvis 5.1 into a usable shape, and porting all Evolvis 4.8 functionality to it. Much has been integrated in the recently released FusionForge 5.1 already. A new functionality to mark bugs as duplicate, followup or having a simple, nontyped relationship to another tracker item is being worked on, as are improvements (new items, better look’n’feel) to the “My Page”. And that’s just the big news.

A German language User’s Guide / Handbook is also being worked on. It’s a Wiki, so feel free to help ☺ (although the licence is CC-BY-NC-SA and thus non-free, sorry for that, but the software itself is GPLv2+). And as usual, there’s a full changelog and you can look at the svn revision log.

We wish you an enjoyable software development and lifecycle management, your tarent solutions GmbH Evolvis Project Team

Second CodingTeam meeting

2011-07-07T02:58:06Z

Another CodingTeam meeting was organized in Paris. Last time we were only 5, and this time, we were 7. If you are living in Paris, watch the forum for a probable future meeting!

The appointment was in downtown Paris (Châtelet) at 6 PM and our meeting ended around midnight. We talked a lot about various subjects like:

the daily server problems on codingteam.net and the recent call for contributions to rent a better server (the full call message can be read in french and english on the codingteam.net homepage ; and, by the way, we need your help!)
the lack of importation mechanism in CodingTeam and the necessity to have a system that allow to save our data and to switch to another software forge without data loss
the need to write an installation script for CodingTeam
the futures plans for CodingTeam and the planned releases
the progress of the correction of a few bugs
the fact that parisian sell Breton galette under the ridiculous name of "salty crêpes"
and so many other topics!

For your information, after the flammekueche of the first meeting, we chose pizzas this time.

Here are a few pictures of us:

From left to right: Mathieu (mathieui), Emmanuel (manu) and Yann (Asterix, the Gajim developer)

From left to right: Emmanuel (Link Mauve, who just came from somewhat an original exhibition) and Mathieu (mathieui, again!)

From left to right: Harold (Melsophos), Florent (louiz') and Erwan (xbright)

So it was a very pleasant meeting between good, rich and sexy people (the next time we'll try to make better photos of us)! Hope to see you in the future!

fusionforge 5.1

2011-07-01T20:17:00Z

Release notes for FusionForge 5.1
---------------------------------

FusionForge 5.1 is another incremental step over 5.0, although it's a
large step in some regards, with about 3000 commits since version 5.0.

The FusionForge team would like to dedicate this release to the memory
of Alexandre Neymann, who passed away in March 2011. Alexandre was
one of the founding members of the FusionForge project in 2009, and
one of its most active developers. His death has meant a great loss
to us all.

On the user interface front, the project summary pages and users'
personal pages are now built out of blocks called "widgets" that can
be rearranged at will; these widgets (taken from Codendi) allow
greater flexibility on how the pages look. A new theme, called Funky,
is now available, with a more modern feel (contributed by Capgemini).
In the same field, the old help window has been replaced with a new
unobtrusive tooltip system (Alcatel-Lucent). A less visible aspect is
that the generated webpages are now much closer to full XHTML
compliance.

Among the improvements in features, the document manager has been
vastly rewritten (by Capgemini again), with more features and an
improved usability; more improvements are planned for the next
release, but this should bring the docman to something we're no longer
ashamed of :-)

Many improvements in the trackers have been contributed (by
Alcatel-Lucent), including a progress bar and improvements in sorting.

The permissions system has been enhanced and made more flexible, with
the new ability to have several roles at once in the same project and
to share roles across projects (contributed by Roland Mas as part of
the Coclico project).

New projects can now cloned from one of a set of configurable
templates, thus allowing forges to have standard shapes for their
common project organizations (Roland Mas, Coclico).

New plugins have been written or merged from Codendi as part of the
Coclico project: Mailman, Forumml, Hudson and SoapAdmin. Not all are
production-ready yet, but they're made available for the adventurous.
Other new plugins include a gravatar plugin (Alain Peyrat) and a
"blocks" plugin allowing to add free descriptions in several places
(Alcatel-Lucent).

Behind the scenes, the configuration system has also been made
simpler, it now uses standard *.ini files that are taken into account
immediately; the Apache configuration files are also splitted out into
independent components, rather than a large generated file. (Both by
Roland Mas.)

Another improvement making installation easier is the availability
packages in RPM format for Red Hat Enterprise Linux and CentOS, in
addition to the *.deb packages for Debian-based distributions.
Installation from source is still supported, of course.

As usual, this release provides scripts to migrate what needs to be
migrated to the new schemes (in this case, for the database schema,
the configuration variables and the existing permissions). Care
should still be taken to check the results of these migrations, as a
safety net.

Forges mutualisées, dans le rapport “L’industrie du logiciel”

2011-06-28T13:12:52Z

Je viens de parcourir l’excellent rapport sur “L’Industrie du Logiciel” (en France : une analyse et des propositions pour l’enseignement et la recherche).

Il contient, entre autres sujets de réflexion et propositions, une analyse des modes de financement des forges de développement de logiciel du monde académique.

Je vous livre l’extrait en question (ai replacé les liens en notes de bas de page comme des liens dans le texte):

6.3 Des modes de financement des infrastructures de recherche peu adaptés au logiciel

Dans l’effort actuel de financement de la recherche, certains instruments ont été conçus pour soutenir des investissements importants dans des infrastructures collectives en se basant sur le modèle issu des grandes installations de la physique ou de l’astrophysique. L’État alloue des crédits exceptionnels, parfois considérables pour financer la construction d’un grand équipement de recherche (un accélérateur de particules, un télescope, …) laissant aux “opérateurs de la recherche” (CNRS notamment) le soin de recruter et d’affecter le personnel qui va l’utiliser.

Ce modèle d’intervention de l’État est inadapté pour financer les “forges à logiciel” modernes qui sont indispensables aux grands développements collaboratifs des technologies logicielles de demain. En effet, la part principale de coût de ces infrastructures est le coût des personnels. En particulier, elles ne trouvent leur pleine efficacité que si des ingénieurs de recherche en nombre suffisant y sont affectés, permettant de finaliser et professionnaliser les prototypes logiciels, de maintenir les outils et plateformes logicielles de développement et de former les utilisateurs nouveaux entrants.

L’essor du Cloud Computing permettrait par exemple de construire des forges ou des centres de données mutualisés qui réduiraient énormément les couts et augmenteraient l’efficacité du développement, mais des initiatives de ce genre n’entrent pas dans la définition des infrastructures de recherche actuellement financées, les montants d’investissement étant trop faible, et la part de fonctionnement trop importante.

Les forges existantes fonctionnent plus ou moins en vase clos. La forge du CRU accepte par exemple la création de projets venant de l’ensemble des universités, mais elle refuse les projets étudiants et fonctionne à l’intérieur du monde universitaire, sans ouverture réelle sur le monde des entreprises. La forge de l’INRIA est une des plus significatives de France, et elle rend un service appréciable aux équipes propres ou associées à cet institut.

Le besoin de créer une forge au niveau du CNRS a été exprimé formellement et une proposition de réfléchir à une forge au niveau enseignement supérieur et recherche existe aussi .

Ces outils existants et ces projets ne prennent pas (ou imparfaitement) en compte les besoins d’articulation entre le monde académique et celui des entreprises pour faire éclore les innovations technologiques logicielles ; ils ont aussi tendance à oublier que le développement collaboratif est un besoin qui ne se limite pas au code, mais se retrouve aussi dans l’écriture de documents techniques.

L’innovation logicielle prenant place dans des “écosystèmes” décloisonnés, ces forges doivent être accessibles à tous les acteurs d’un développement innovant, chercheurs d’organismes différents, PME partenaires, pôles de compétitivité, etc.

Laisser le soin aux opérateurs (CNRS, INRIA, CEA, universités) de déployer séparément de telles infrastructures sans que celles-ci ne soient mutualisées est donc contre-productif.

Il serait très préférable d’avoir une structure mutualisée offrant des « vues » spécialisées par types de développements (par ex.: mondes virtuels 3D et jeux, internet d’objets, etc.) ayant des besoins différents, mais ouvertes à tous les acteurs concernés, et qui puissent :

héberger non seulement le développement collaboratif du code, mais aussi de la documentation technique associée, voire même l’écriture collaborative d’articles scientifiques dans le domaine ;

prévoir explicitement l’hébergement de projets étudiants, et leur possible évolution dans le temps vers un cadre plus institutionnel ou international ;

prévoir explicitement un lien et un échange de métadonnées avec les autres forges, notamment industrielles ou internationales ;

permettre de suivre l’activité des contributeurs dans le temps, ce qui peut aider les étudiants à constituer facilement un bilan objectif de leurs compétences logicielles pour leur curriculum.

La mutualisation ne vise pas seulement à partager les coûts des investissements matériels (assez modestes si l’on utilise des serveurs virtualisés), mais surtout à mutualiser ces compétences qui à terme forment une part essentielle de la valeur de ces plateformes communes. Dans cet esprit, il nous parait souhaitable que l’alliance ALLISTENE, en liaison avec les pôles de compétitivité, organise une réflexion sur la création et le fonctionnement d’une ou plusieurs forges nationales, fédératives et ouvertes à tous les acteurs concernés, et la mise en place, indispensable, d’un programme de recherche multidisciplinaire sur les environnements collaboratifs qui guide l’évolution dans le temps de ces forges.

Proposition n°12 : Confier à ALLISTENE, en liaison avec les pôles de compétitivité, une mission de préfiguration d’infrastructures de développement collaboratif, adaptées à un type de développement particulier, et dotées des moyens nécessaires en personnel pour mener leurs activités de manière pérenne.

Bien évidamment, l’analyse me semble très pertinente, et la proposition me semble aller dans le bon sens, même si je ne sais pas si ALLISTENE est le meilleur porteur pour cela.

mksh R40 is out, so what now?

2011-06-14T12:02:33Z

mksh R40 has been released Sunday 12nd June. So, what can you do with The MirBSD Korn Shell?

Have a look at the collection of shell snippets, which admittedly is only the beginning – most scripts don’t make use of the cool new features yet. But they eventually will. The Shell-Toolkit project is definitely worth a visit! (Admittedly not using my most ❦ dear SCM, but with a DVCS, every checkout is a clone, i.e. a backup, and none of the contributors must fear a central repo being taken down, which, for such a loose collection, is desirable.)

How (not to) encode MIME headers

2011-05-26T12:36:08Z

I was just tracking down why some mails seem to have garbled Subjects.

It looked like this in Alpine:
Subject: [BOFH commits] r2040: fix directory struct =?UTF-8?Q?ure=E2=86=B5=20unix?=/mirror/=?UTF-8?Q?=20=E2=86=92=20mirror?=. bonn

The raw header sight was this:
Subject: [BOFH commits] =?utf-8?q?r2040=3A__fix_directory_struct__=3D=3FUT?= =?utf-8?b?Ri04P1E/dXJlPUUyPTg2PUI1PTIwdW5peD89L21pcnJvci89P1VURi04P1E/?= =?utf-8?b?PTIwPUUyPTg2PTkyPTIwbWlycm9yPz0uIGJvbm4=?=

Ein Schelm, wer Böses dabei denkt… First suspect was, of course, Mailman – after decoding, this showed the classical signs of a double-encode. (After ruling out general header brokenness, but no, 76 chars is ok.) I thus hand-crafted an eMail with the correct header line and sent that out:
Subject: [BOFH commits] =?utf-8?q?r2040=3A_fix_directory_structure?= =?utf-8?b?4oa1IHVuaXgvbWlycm9yLyDihpIgbWlycm9yLmJvbm4=?=

Huh? Mailman and Python weren’t the culprit, thus – this is correct mangling. Okay, let’s dive into the Perl code that actually sends out the eMails. To make a long story short, have a look at this, then RFC 2047:
tglase@tglase:~ $ perl -MEncode -e '$subject = "r2040: fix directory structure↵ unix/mirror/ → mirror.bonn"; Encode::from_to($subject, "UTF-8", "MIME-Q"); print "{Subject: ".$subject."}\n";' {Subject: r2040:=?UTF-8?Q?=20fix=20directory=20struct?= =?UTF-8?Q?ure=E2=86=B5=20unix?=/mirror/=?UTF-8?Q?=20=E2=86=92=20mirror?=. bonn}

Amazingly enough, PHP’s mb_encode_mimeheader, despite being talked to trash in the comments on its online documentation, does manage to get it right:
tglase@tglase:~ $ php -r 'mb_internal_encoding("UTF-8");echo "{".mb_encode_mimeheader("Subject: r2040: fix directory structure↵ unix/mirror/ → mirror.bonn", "UTF-8", "Q")."}\n";' {Subject: r2040: fix directory =?UTF-8?Q?structure=E2=86=B5=20unix/mirror/?= =?UTF-8?Q?=20=E2=86=92=20mirror=2Ebonn?=}

Wow. Now, the Perl guys I know told me to use Perl’s Mail tools… which are much too high-level though, for all I have and want is the subject string and an RFC 822 header line. I told them I’m not above doing this, and so I did. The 3P languages can really be annoying.

Why’s Perl’s output wrong anyway? I don’t know for sure, but I think the atoms must be separated, so unquoting /mirror/ in the middle, with no spaces around it, are wrong. (Besides, Encode::from_to can’t do the job right anyway, as it misses the name of the header, which is included in the 76 chars allowed for the first line. BAD • Broken As Desdigned.)

Disclaimer: I don’t really know any Perl, I fight my way through PHP and, barely, Python. (But I can code.)

2 presentations at Solutions Linux 2011

2011-05-19T15:44:53Z

2 presentations have been made at Solutions Linux 2011. See : http://www.coclico-project.org/index.php/Coclico:Current_events

Survey results, future prospects

2011-04-18T19:10:25Z

It's now the time to publish the results of our survey. I want to thank you all for your participation, as we get nearly 80 answers to the "CodingTeam survey 2011".

Even so our survey suffered from a few weaknesses like the fact that a few questions not addressed to all of our users were mandatory, the result provide us a lot of useful informations. The problem with our mandatory questions (features scores) is that the scores are devalued and there were confusion in the responses.

So, let's start!

The CodingTeam software forge is a young project and it's used by young people! 57.89% of our users report being between 15 and 25 years. There are 38.16% of our users between 26 and 60 years. On the other side, there are fewer users that are more than 60 years (2.63%) or less than 15 years (1.32%). Thus, we can see that the majority of our users are young, with nearly 60 percent of them under 25.

In the beginning, CodingTeam was an attempt to provide a good forge to the French-speaking community, and we can see that 67.87% of the users are from France. Other well represented countries are Belgium (8%), Switzerland (6.67%), Brazil (2.67%), Germany (2.67%) and Slovakia (2.67%). There are also users from Australia, China, Czech republic, Ireland, Madagascar, Russia and Vietnam. Therefore, it's an international project, although we can bemoan the lack of users from United States, United Kingdom, Spain etc. Note that we know a few users from India too.

It's interesting to take a deeper look to the SCM usage question. We asked if our users were using a (distributed) version control system or not, and which one. It appears that 21.34% of the interviewed CodingTeam users don't use an SCM. Thus, we can conceive that the work we did on the HCI allows users who are not developers to use our software forge: translators, graphic artist, packagers… It may be the sign that we successfuly created the easy-to-use interface we wanted. The others 78.66% are using various systems:

Subversion (50%)
Git (21.43%)
Mercurial (16.07%)
Bazaar (5.36%)
CVS (5.36%)
Darcs (1.78%)

It also appears that 33.96% of users discovered the project by a friend (many thanks to them!), 28.30% by a project using it, 20.75% by a news article and 16.99% by an Internet research.

On the communication subject, we asked our users how they want to be advised of project news. Thus, 59.375% of them would accept to receive a newsletter from time to time. This may not be done because of the large 40.625% of our users that don't want at all. It's also interesting to see that only 54.69% of our users knew that the CodingTeam blog exists, the others just ignored it!

Thereby, our users give a rating of 6.52/10 on the quality of our project communication. We already know that it was one of our weakness and we will take into account the remarks made by them, like:

the usage of the French in addition to English
modernization of the appearance
ATOM feed (hey, that already exists, just follow this blog, man!)
interesting tutorials/links

Another user complained about the fact that the communication is too oriented toward the group of French-speaking people on the XMPP chatroom. This can explain the large majority of our users from France. We should focus other countries and languages to reveal our software forge across oceans!

It's very interesting to know that the CodingTeam software forge is mainly use on CodingTeam.net (73.69%), but it's also used elsewhere:

users own forge (19.30%)
forge installed for a single project (3.49%)
companies (1.76%)
research laboratories (1.76%)

Our users seems to be satisfied with CodingTeam! The software forge gets a global score of 7.75/10.

The translation system is rated 6.55/10 (the next release will bring a few improvements as the reduction of the loading time). The bug tracker gets a 6.88/10 (again, the next release will bring improvements like pre-selected filters). The documentation system is rated 6.50/10 (users seem to don't like CamelCase and ergonomics).

The top rated features are the SCM support (7.29/10) and the communication tools (7.13/10).

The feedbacks sent by our users are also very interesting. Most of the opinion are positive (encouragement, congratulations) and a few features requests.

It is not surprising to see that among the 39.44% of users that are using another software forge, the most used are Launchpad and Teamforge/SourceForge.net (both 18.42%), followed by Redmine and Trac (both 15.79%), Github (10.54%), InDefero and FusionForge (both 5.26%) and BitBucket, CodePlex, Google Code and Savane.

We asked for the opinion of our users on certain probable features for the future of the project. Here are the podium:

more statistics (27.45%)
unbelievable technology from outer-space (26.47%)
agenda and calendar (25.51%)
website hosting (21.57%)

A lot of work have been done on the development version of CodingTeam on the statistics. We are glad to announce that projects will soon benefit from detailled statistics! The current stastistics page will be extended with a global activity report (commits, bug tracker, forum topics, translations and documentation in a table and a graph), and a source code analysis!

We also plan to send someone in space to bring back a piece of magic Moon rock, that will give us the ability to provide a software forge for galaxies and mice!

More seriously, our users asked for a few features, like:

ability to fork a project
installation script (planned)
work on the appearance
German translation
API to communicate with CodingTeam (take a look at this post)
overview: late bugs, notifications

Among these features requested, some will be part of the next release:

Git (it seems to be more popular than Justin Bieber)
timeline by developer
teams associated with projects

Finally, we collected a lot of useful informations on our users and how they see and use our software forge. I also started to develop what users have asked in this survey. So it was very important to take the quizz, and again, thank you all!

See you for the next release!

CodingTeam Survey 2011

2011-04-17T11:36:06Z

We need you for the CodingTeam survey 2011!

In order to know more about you (users or administrators of the CodingTeam software forge), we decided to launch a great survey. This survey is bilingual: you can answer in french or in english. The survey only takes a few minutes and will end in about a month.

We did that so we can improve CodingTeam, learn more about our users and their needs. The survey is online and every users of the CodingTeam software forge is free to fill it!

We invite you all to complete this survey and especially to talk about it to all your friends who are using CodingTeam!

Knowing your views, your habits, your wishes, will allow us to better guide future development, set the right priorities, make CodingTeam the great tool you need! So take the time to fill it! This is for your highest good.

Take the quizz! French and english.

JSON

2011-04-13T16:22:38Z

I’ve pimped the minijson code in EvolvisForge to be a full-blown JSON encoder and decoder (lexer/parser) now. You wouldn’t believe just how much is broken in PHP (its own json_encode handles floats wrong in most locales, and mb_check_encoding doesn’t check the encoding of a multibyte string… at least, unlike Python, PHP manages to get 8bit okay though).

Anyway, thought you want to know. If you ever need a Pure PHP JSON encoder/decoder, you know where to look. It’s 100% my code (although written during dayjob, so the exploitation rights are with tarent GmbH). Current licence is GPLv2+ or AGPLv3+. If you spot any bugs, you know where to report them. I think it should be good though. (Do note that JSON is case-sensitive, so “NULL”, “True” and “\N” or “\U20AC” are not valid, “null”, “true”, “\n” and “\u20AC” or “\u20ac” are.)

I plan to store the “art_cust123” information in the user_prefs table in JSON now, instead of ‘|’-separated values, to avoid problems like these we had with broken database format and subsequent corruption of values, by using a dictionary (JSON Object) instead.

The ECMA 262 standard (ECMAscript and JSON) is freely available, though. JSON is also additionally specified in RFC 4627 which differs slightly (see my notes for details, mostly the goal element).

Administrativa (Evolvis Blogs)

2011-04-13T16:16:25Z

The blogs have been moved from *.blogs.evolvis.org to *.blog.tarent.de to provide proper SSL certificates. This also means that the use of blogs for people who are not employees of tarent GmbH, tarent AG or one of its subsidiaries is currently not available. (If that should become necessary, please contact us.

The feed has moved to http://evolvisforge.blog.tarent.de/feed/ for this blog (similarily to the others), but just in case, you’ll get redirected (although only to the https version, which Planetplanet doesn’t seem to like).

Planet Evolvis will be set up anew using Planet Venus software shortly.

All in all, the Greater Evolvis Platform is undergoing updates and improvements.

fusionforge 5.0.3

2011-04-04T18:57:00Z

Maintenance release, bugfixes and translations only.

yet another software forge: openMyWork

2011-03-25T12:08:38Z

Via acs I’ve seen that the Basque Government is promoting yet another software forge, its name is openMyWork. I haven’t seen either a list of features or a roadmap so the only thing I can offer is the description and the link.

openMyWorks is a project that includes the forges of the Basque Government. It is composed by openMyForge (the forge) and openMyDesktop (a desktop aplication). The software is developed using the web framework Web2py

[This post is also available in my blog at libresoft.es]

How to get your FusionForge development environment, the easy way

2011-03-23T00:56:46Z

Last night I was trying to run my latest version of FusionForge and everything was ruined. I did it in the hard way using sources and bypassing some checks to install them in Debian. Now I know I’ve been wasting time, Roland Mas created some months ago a great method to get a FusionForge development environment with a minimum effort, it is based on a VirtualBox installation with scripts ready to update the source code from the repos and install them as Debian packages. Brilliant.

Here you have the steps I followed:

aptitude install linux-headers-2.6-$(uname -r|sed ‘s,[^-]*-[^-]*-,,’) virtualbox-ose
virtualbox-ose should be available now: /etc/init.d/virtualbox-ose start
Download FFSandbox.ovf and ffsandbox.vmdk from http://fusionforge.fusionforge.org/sandbox/
Open “VirtualBox OSE” and import the .ovf file. When I choose the .ovf it imports automagically the .vmdk
At this point you already have a Debian with FusionForge, but it is not the latest code (yet)
Have a look at the IP address of your virtual machine and enter with a ssh connection: ssh root@192.168.1.102
execute /root/scripts/update.sh which will update the sources
from FusionForge’s Subversion repository and also the currently installed .deb
packages.
execute /root/scripts/update.sh to build the packages from
sources
finally execute /root/scripts/install.sh to install
an up-to-date FusionForge should be running on http://forge.local/.

Thanks to obergix for the tip.

FusionForge en -> es hack session

2011-02-14T12:01:54Z

Last Friday some workmates and I spent a couple of hours working in the translation to Spanish of FusionForge. We translated around 700 strings and ate 4 pizzas. It is not a bad average at all. I’ll go on with the translation in my free time but it is possible that we repeat the “FusionForge en -> es pizza hack session” during the following months, it is a great excuse to enjoy a different meal with some friends.

Saludos de parte de los traductores españoles por un día / Greetings from some of the Spanish translators for a day

pizza hacking session

In this photo: Roberto Andradas, Dani Izquierdo, me and Alvaro Olmedo

[This post is also available in my blog at libresoft.es]

Integrating JUnit tests inside a PHPUnit+Selenium test suite

2011-02-08T12:01:25Z

I’ve spent some time recently integrating the OSLC open source test suite (from the OSLC open source support project)into the FusionForge test suite.

FusionForge‘s test suite uses PHPUnit to drive Selenium (RC) “end user” like tests. Selenium is a tool that makes such tests possible by piloting an instance of FireFox browsing the Web interface of the forge, in a controlled environment.

The OSLC test suite consists in a series of JUnit tests, which is driven by Maven (initially started from inside Eclipse, then also from command-line after I found the nasty command line parameters and changes in the pom.xml file that were required .

The whole of the test suite may not pass for our implementation of OSLC-CM in FusionForge, so some tests fail, but I don’t bother too much as this is “normal”, and all we’re caring for at the moment is mainly non-regression. There seemed to be no way to exclude some of the tests from the suite for the moment, but fortunately, that doesn’t matter, since the way I have integrated both test suites happens to allow the verification of only success on some of the tests.

So, the way they are integrated is through execution of the JUnit suite during one of the test cases of the Selenium suite (using a system( ) PHP call), which generates an HTML report (using the Maven SureFire reports plugin), which can then be viewed in the Web pages of the tested forge, so that Selenium + PHPUnit assertions can verify the content of the test report.

This is a bit hackish, but all in all suites our needs so far. Next step is to see if it works in other people’s test environments, including the automated executions in Hudson.

cp Redmine Chiliproject; chmod g+w Chiliproject

2011-02-05T20:00:40Z

A couple of days ago some people from the Redmine community announced the creation of a fork, its name is Chiliproject. For those of you that don’t know Redmine, it is a libre software project management very popular during the last two years.

Redmine is an active project, the creation of a fork could take away contributors from it and the forked project will have a strong and popular competitor, the reasons to create a fork in those cases must be very strong. The Chiliproject leaders explain it in a post, they basically complain about the community management which from their point of view have to be more open.

… Integration of community-created patches were too sporadic, lacked a clear methodology, and was interfering with the effectiveness of the Redmine project for its users. Over the past two years, several members of Redmine’s community worked to resolve management bottlenecks through clear suggestions and contributions. They also attempted to broaden and open up the development process to more contributors. But efforts via public and private forums to discuss the goals and future direction with the project manager of Redmine failed, as the current project manager did not share these priorities

These are the reasons why some people from the Redmine community decided to create a new project, they want to put in practice a more transparent and open governance model following the “ideals of Free and Open Source Software ethics, governance and development practices”. During the following months we will witness a very hard competition where two similar projects will use two different approaches to manage its community. Will Chiliproject be able to attract more code contributors than Redmine? Won’t Redmine lose a bigger part of the community in favour of Chiliproject? These questions will have an answer by the end of the year.

References:

https://www.chiliproject.org/projects/chiliproject/wiki/Why_Fork

Evolvis: git and scm, SOAP WSDL, Jenkins, …

2011-01-21T16:19:06Z

tarent GmbH now offers both git and Subversion as Source Code Management systems in their Evolvis platform.

The scmsvn plugin of EvolvisForge was amended with the scmgit plugin, backported from a newer FusionForge release, while work to upgrade the main forge code to the soon-to-be-released FusionForge is ongoing in parallel. Of course, most features, such as commit mails (in git, they’re really sent out after a “git push” from the developer), are already available for both; others will follow after the code base upgrade for simplicity. Already, git repositories can be used for all regular tasks – although it’s currently not possible to have both git and svn repositories in a project.

During the migration (as well as regular development), many features of Evolvis will show up in regular FusionForge to benefit the broad community of Forge users, courtesy of the Open Source policy of tarent GmbH, who has contracted the FusionForge project leader to improve both projects, following the mantra of “Fairtrade Software”.

Among other changes, the SOAP WSDL has been corrected and is now versioned, and other areas (especially the Tasks and Tracker) have been improved.

There’s a release announcement of the new version at Evolvis, in case you want to know the details.

Finally, an Evolvis developer has set foot not only in the Mediawiki packaging team at Debian but also the Mailman packaging team. Expect many, if not all, improvements to show up there within some time.

Did you know that Hudson is now called Jenkins? This is a change in name only, due to trademark concerns, but rest assured the Evolvis platform will continue to offer Continuous Integration in a usable fashion, no matter the technology behind (there was Continuum, now Hudson, soon Jenkins).

In our evergoing quest to improve the Evolvis platform, we wish you a pleasant user and developer experience!
//The Evolvis team

Coclico project new RSS news feed

2010-12-08T17:05:33Z

We'll be publising news about the COCLICO project at the website's RSS feed from now on. This will allow subscription and syndication to other agregators, like PlanetForge's. Of course, we use the project's FusionForge news feature for this ;-)

New blog for the project using the news app

2010-12-08T17:05:29Z

Read and contribute the project's blog/news at : https://forge.projet-coclico.org/news/?group_id=14 as part of the project's "website". The RSS is at : http://forge.projet-coclico.org/export/rss20_news.php?group_id=14

Screencasts for export/import of FusionForge projects

2010-12-08T17:05:24Z

We're working (as part of our Work package 2 on interoperability) on export/import features and specifying a standard format so that it's possible to export projects from one forge to another. Screencasts of early results for FusionForge have been produced by the INRIA team, for a presentation at fOSSa 2010 made by Sebastien Campion. More details in Sebastien's post.

The OpenForge API

2010-11-02T01:16:37Z

Some time ago, during year 2008, Tim and I wrote the first version of the OpenForge API. The goal of this API was to allow communication between software forge. Thus, the 0.1 came with three main features :

retrieve project data
retrieve user data
search for projects and users

Tim is the developper and the administrator beside ShareSource, another public software forge. So, our idea was to connect our forges, allowing our users to, for instance, search for projects registered on CodingTeam.net through ShareSource.

Nowadays, CodingTeam supports the full specification and all the three functionnalities of the OpenForge API are available inside our project. We can also notice that the most exciting features, the ability to search for projects and users in a network of forges from another, is not used at all. In fact, only CodingTeam supports this API and thus only CodingTeam forges can connect their search results (this can be activated in the administration panel, and by the way, if you are running a CodingTeam forge and want your search results to appear on CodingTeam.net, just contact me).

In fact, we can say that OpenForge is a failure. The innovant part is not used and the simple parts are barely used. Beyond that, we can try to explain the relative unsuccess of our API.

The main problem is that the simple parts of OpenForge (data retrieve) is somewhat too simple. See by yourself:

retrieving project data from Bluemindo
retrieving user data from me

Example of an external search on two other forges (XML output example):

A lot of ideas have been debated on our developement room since OpenForge exists and there are a lot of possibility to improve it. I think it is reasonable to wonder if it's in the CodingTeam interest to keep these implementations in the codebase, since OpenForge is not really used by other projects (or even by users) and as the goal of OpenForge being the wish to connect software forges. Or maybe, it's time to work on this API.

It would be great to open the development of the API. Everyone should be able to give his opinion and his ideas.

There are many development possibilities that we thought about. We could use existing data representation like DOAP for projects and FOAF or vCard for users. We could switch to a really public and open development of the API. We could imagine that OpenForge supports importation and exportation of projects, bugs, SCM, wikis… Last but not least, OpenForge could also be used to post or edit content directly from the API.

Maybe OpenForge wasn't a very good idea. Maybe it can be improved to be very useful. We need help and comments.

FusionForge Virtual released

2010-10-29T11:01:10Z

We are releasing today FusionForge Virtual, it extends the features of FusionForge to display some of the projects in “different” forges with different URL, web domains and layout. Actually one forge instance can offer several views sharing the same database, that’s why we called them virtual forges.

Repositories like OSOR or Morfeo want to attract people and entities to come to their platform and some of the entities want to have their own layout, url and logos next to the projects. Due to the fact that all the data is stored in the same database is not a big deal to create different views of the same projects, that’s why we developed the virtual forges support. A couple of years ago we did something very similar for the Morfeo community and CENATIC with the platform based on Gforge and during the last months we polished up the code to create a newer version based on FusionForge 5.0.1

There is a demo site available at http://forge.ffvirtual5.libresoft.es/, there you’ll see three virtual forges that offer different views with subsets of projects with a customized layout and URL.

[This post is also available in my blog at libresoft.es]

fusionforge 5.0.2

2010-10-23T14:42:00Z

Maintenance release, bugfixes and translations only.
Fixes for Subversion over HTTP.

Quick report from OSLC meetup in Paris last week

2010-10-04T17:07:19Z

Last week, as part of the COCLICO project‘s efforts to work on forges interoperability issues, we invited partners and contacts for a short meetup in Paris, with a special guest, Steve Speicher, lead of the OSLC-CM domain work-group.

Steve was in Paris to speak at the Open World Forum 2010 in the forges interoperability track. As time slots at OWF were quite short, we proposed to have this meetup a few days earlier to be able to have more detailed discussions and demos.

Steve has presented the following slides : OSLC Specifications for Interoperability and a screencast of AJAX interaction between an OSLC-CM server and a consumer.

On our side, we made a quick demo (a screencast actually) of our Mantis add-on for OSLC-CM.

We have discussed several technical aspects of OSLC, and also the community of actors working on the specifications. Regarding the french speaking community, there doesn’t seem to be an urgent need to have a specific structure setup so far, but we will nevertheless probably continue sharing content in french whenever we have some time.

I’ve been very glad to meet Steve, and hopefully, there are gonna be other times for new meetups. Why not a formal OSLC conference some day ?

In the meantime, feel free to tell us if you’d like to discuss OSLC and forges interoperability.

Quick report from OSLC meetup in Paris last week

2010-10-04T17:07:19Z

Steve has presented the following slides : OSLC Specifications for Interoperability and a screencast of AJAX interaction between an OSLC-CM server and a consumer.

On our side, we made a quick demo (a screencast actually) of our Mantis add-on for OSLC-CM.

I’ve been very glad to meet Steve, and hopefully, there are gonna be other times for new meetups. Why not a formal OSLC conference some day ?

In the meantime, feel free to tell us if you’d like to discuss OSLC and forges interoperability.

Back from the Open Forges Summit 2010

2010-10-04T11:46:27Z

Last Friday some workmates from LibreSoft and I were in the Open Forges Summit, we couldn’t be there more than a couple of hours because we returned the same day but it was really interesting hearing a discussion about the problem of the data lock-in among people from sourceforge, codeplex, fusionforge, berlios and other places.
Some of the problems that came up in the discussion are summed up by Eric S. Raymond in this recommended blog entry “Three Systemic Problems with Open-Source Hosting Sites“, IMHO these should be the target for the next years in FLOSS forges, but at the end forges are an agregation of useful tools and some of them are now toooo big for being flexible. Maybe it wouldn’t be that bad to stop and re-think the workflow to give them a social approach and get its data distributed (and even standardized)

COCLICO project’s efforts towards better forges interoperability (long)

2010-10-03T20:08:18Z

Abstract

I’ve given a talk in the recent Open Forges Think Tank track of Open World Forum, which was organized by Christian Rémy from Bull, also a partner in the COCLICO project (btw, thanks Christian, this was a great track, with several interesting presentations and a great panel).

I’ve had the privilege to speak on behalf of the whole COCLICO project, in the afternoon session which was focused on forges interoperability.

This article will somehow be a transcript of what I’ve said (or intended to say), with the accompanying slides available here.

In this quite long piece, I’ll first recap some of the context elements about the COCLICO project. Then I will describe the interoperability issues that I’ve tried and focused on in my presentation, including the issues of project lock-in in the forges. I’ve tried also to describe the current ideas we’ve elaborated in the project to address these issues of interoperability (including our plans for open standards elaboration for forges interoperability). I finally conclude with a proposal to join the PlanetForge community for all interested parties.

Unfortunately, not all of these ideas are currently yet properly documented on the COCLICO website, so I hope this article will serve as a useful reference for what COCLICO is doing, still being a subjective piece of my own views, not necessarily representing those of other COCLICO participants, nor a precise description of what we’ll manage to achieve in COCLICO or PlanetForge.

Foreword

The COCLICO project is addressing many different topics about the software development forges, but interoperability issues are present in many of its workpackages. So I hope our views can help shed some light on current problems, and that we will be able to propose promising developments to address these in a concrete manner.

Right before me in the track, there was a very interesting round table/panel discussion which exhibited many different aspects of forges interoperability (I’ll try to summarize the discussions in another installment). One of the major problems that was exhibited in that discussion, is that there is a lack of standards that could help address the different interoperability issues, for instance, the lack of a standard format for backup/dump/export of forge projects data (but same stands for dynamic interoperability between tools).

After a very inspiring panel discussion for most participants (I hope), I’ve tried to provide some more links illustrating some of the elements that were discussed and on which the COCLICO project has been working for a year now. Most are big concerns for me at the moment, so hopefully my slides’ content and the ideas exposed by panel participants were quite complementary.

Also, I tried and provided a few ideas about the potential of some emerging (in the forges domain) standards or techniques that may prove helpful in solving interoperability issues. Among these is the use of the OSLC standard, which was the topic of the next speaker’s presentation (nicely devised schedule of talks).

About the COCLICO project

Those impatient readers or those who already heard about the COCLICO project may skip this part.

COCLICO is a 2 years project funded under the french collaborative R&D cluster programmes (pôle de compétitivité) System@tic (Paris Region) and Minalogic (Grenoble Region), which is arriving at mid-term now (fall 2010).

Our consortium groups 9 participants mainly in Paris and Grenoble, with several large industrial companies (Bull, Orange Labs, Xerox), SMEs (CELI France, Bearstech, Gnurandal and Objet Direct) together with two academic institutions (INRIA and Institut TELECOM, more precisely our team at Telecom SudParis). Most have past expertise in the forges domain.

The COCLICO project has a strong focus on Free/Libre/Open Source software (FLOSS) as per the charter of the funding programme System@tic’s special interest group on FLOSS (GTLL). IMHO, that should sound natural whenever public spending subsidies software development, anyway.

The goals of the project are (excerpt from our site) : “The Coclico project aims to reinforce software forges communities by structuring an open source ecosystem for which a critical mass exists in France.”

That said, we’re aiming international impact, of course, at least for what relates to technology, hoping that only the market development aspects will be substantially better in France than anywhere else, if we’re smart enough .

More precisely, the COCLICO Project’s goals include (again repeating the website) :

Re-dynamization of the Open Source development community around open source historic forge code base (FusionForge and Codendi)
Definition of an open integration model
Data integrity and confidentiality
Exchange of data in real-time between various forges
Features for industrial use and quality assurance
- traceability of information,
- support of software engineering methodologies,
- interaction with the user’s workstation.
etc.

Interoperability issues and project lock-in

Let’s get back to the current interoperability issues of software development forges.

Forges have been a blessing for many communities, both on the Net, and inside organizations. They helped solve a great deal of problems, but they show some limitations, and their use brings new risks.

Among these risks, I can see one major problem, namely the projects lock-in to their hosting forges. It is not the only problem users or forge owners are facing, and definitely not one that may deserve everyone’s attention, compared to other issues that may be experienced in everyday forges use. But I think that too much locking of projects data into the tools used to host the project can have damaging consequences. It may not be perceived as a major problem by most but has definitely been a problem for some users at particular instants in their communities history, and thus deserves that we try and address it the best way.

When users are not satisfied with the tools they use and they have little hope to be able to change tools easily, this generates enough frustration and anger to have a substantial impact on the quality of the community relations, on the motivation and productivity of the projects. This is even more important for FLOSS communities, and in general all communities of volunteers, which rely mainly on the good will of all participants, where great care must be taken to preserve the cohesion, and to avoid that tools become a problem rather than a means.

Whenever forges don’t allow projects to adapt the way they use the tools to their needs and the evolution of their community, forges can harm these projects. And moving a project from one forge to another one, for instance because new tools are available on the new one, isn’t easy nor exactly possible in most cases today.

So even if projects are happy with the current tools they’re using on one particular forge, they should have the option to change those tools at any point in time without fearing too much disturbance.

I see the possibility of changing hosting facilities for projects/communities as a fundamental freedom that should be preserved, even if there is not the everyday need to exercise such freedom. As such, interoperability of forges in such aspects can become as important as the legal protection that FLOSS licenses offer to such projects.

Not everyone has the need to fork a project, or patch a source code, but licenses guarantee that one may do it should it become necessary, for FLOSS users and developers. And the same reasoning should apply for the freedom to fork a community, for instance, so that a community has the means to leave a project hosted on one forge, and create a new project, taking with it the artifacts (data and meta-data) that constitute this online community, without having to start again from scratch. Such needs are rare, but do happen in FLOSS communities.

It is ultimately the responsibility of forge owners to preserve such freedom for the users of their tools, and for the communities hosted on their forges.

Making backups doesn’t mean that you’ll crash data and need to restore, but not making backups surely won’t help in the event that a crash occurs.

Who’s backing up community relationships, and all the history of projects that have been woven through collaboration on the many tools hosted on a software forge, then ?

Then, having the possibility to export/backup contents of all the artefacts and relations that constitute a project in a forge is quite important. Even more important is the ability to restore these into another set of tools on another forge. Technical and semantical interoperability then enters the field at this point.

Unfortunately, at the moment there hasn’t been any standard format proposed for such exports or backups to be made. That’s one of the problems we’re trying to address in the COCLICO project.

As I said before, this is not the only interoperability issue that forge users and owners face, but it is surely one worth tackling. Hopefully, by trying to solve this problem and others, common elements of a solution can appear, leading to broader interoperability solutions and general standards.

Who’s caring actually ?

Those who have been in the forges business for a long time may remember this project from 2003, the CoopX initiative, which never actually delivered, but was the first public attempt to try and work on a standard for export of forges data, to my knowledge.

Now, in 2009-2010, the subject seems to be somewhat still unsolved, but still a problem for people.

For instance, Eric S. Raymond (ESR) blogged about “Three Systemic Problems with Open-Source Hosting Sites“, where he draws a very bad situation in that :

Hosting Sites Are Data Jails
Hosting Sites have Poor Scriptability and finally
Hosting Sites Have Inadequate Support for Immigration (in the provicative style of ESR).

Another prominent voice (at least in the Perl community), Jesse Vincent tries to warn us against the problems of hosted infrastructures for users who lose control over their tools. The bottomline of his speech (“Web 2.0 is Sharecropping” conference) is “If you don’t own your tools, you’re going to be in a whole mess of trouble.“. Said by the CEO of a bugtracker hosting company, that seems quite believable, I guess

Another influential voice these days must be heard also in this part of the problem analysis, I think, namely prof. Eben Moglen who warns us against the far too large power gathered by the owners of the social web sites and the owners of the cloud (see for instance his conference “Freedom in the Cloud“. After all, what’s differentiating a software development community from any other community of “social networking” websites users ?

Forges can become jails for our communities of developers and users in a maner similar to those at Facebook, twitter and other sites… but fortunately on a lower scale, since the network effects are less important here. Also, the importance of privacy issues is lower, but still, too much concentration and lock-in should be fought against, maybe with complementary techniques and tools.

But these ideas, that have been present in the mind of the proponents of software freedom since more than a decade, are not always familiar to those who work in the forges area (and I acknowledge there may be more urgent matters). Still, these lockin and hosting issues happen from time to time, either for a single project leader (like ESR with gpsd and BerliOS) or for a larger community.

One of the latest such hazards I’ve noticed, is stated in the FAQ of the newly created Document Foundation that established itself in order to give the OpenOffice.org community a more independant structure (and a new name for the software : LibreOffice).

It goes like this (http://www.documentfoundation.org/faq/ as of sept. 30 2010):

Q: Why are you building a new web infrastructure?

A: Since Oracle’s takeover of Sun Microsystems, the Community has been under “notice to quit” from our previous Collabnet infrastructure. With today’s announcement of a Foundation, we now have an entity which can own our emerging new infrastructure.

I’m not so sure about the details (comments welcome), but I can feel the additional burden on the newly born foundation, to have to setup a new hosting infrastructure and copy part of the knowledge base that its community has built over years.

Forge proliferation

There are many hosting services for communities wishing to establish their development infrastructure somewhere on the Net. And public hosting sites are not the majority of such forges, with the even larger number of self-hosted communities and projects.

There happens to be a growing number of forge software offerings (many as FLOSS), to be installed, and offering different variants of community support and software developement tools.

Still, even though it is ever easier to choose a forge at a project’s birth, it has not become easier to move one project from one forge to another over years. Maybe the situation improves a little bit with the advent of distributed version control tools such as Git or Bzr which natively support cloning of a whole repository of source code. Still, other than for source code, it’s still a pain to move a mailing list or a forum for instance (not to speak about release systems / download areas).

It is great to see many new forges appearing every month, but it would even be greater if they did not reproduce the errors of their predecessors, and would some day support a way out for their hosted projects, preserving users freedom of change.

Our efforts

Just in case, to make it clear, COCLICO is not going to reinvent yet another forge. Actually, instead, we try and work at reducing forges fragmentation, by allowing some convergence of implementations between FusionForge and Codendi, on the plugin system for instance. Both Codendi and FusionForge have a common code-base ancestry in the venerable PHP code of the original SourceForge dating circa 2000. So it is quite feasible to share some PHP code and internal libraries, the most basic way to provide interoperability

Due to the French context and the natural pre-existing links among participants to the project, FusionForge has been targetted from the very beginning as a primary receptacle for many of the COCLICO efforts. FusionForge’s lead developers live and work in France, and a good share is onboard COCLICO. FusionForge has thus benefitted from the added momentum that the COCLICO launch represented for its former users. Many of those that had previously deployed GForge instances, and that had customized them to integrate with their IT, were afraid of the lack of momentum of the GForge community, and feared there would be no way to preserve their investment in that forge. The birth of the FusionForge community with the goal of re-animating the development, and the launch of COCLICO, which would be active in bringing new efforts around FusionForge, helped a lot of other parties to rally the band. Indeed, the pace of development (illustrated by commits to the code) has dramatically increased since COCLICO was started.

We also work on implementing an import/export toolbox and a standard dump format for forges, in order to try and solve the lock-in issue (based on an extended version of ForgePlucker; more about this later).

We also try to study and provide some bits of implementation in order to help users of the venerable forges to inter-operate wit more modern tools, or in more modern contexts. This includes :

more REST APIs (around OSLC-CM for instance, for the trackers)
more modern semantic representation formats for data hosted in the forges (as RDF or RDFa, which provides inherent extensibility)
we investigate (based on these REST and RDF Web techniques) the possibility of more Linked Data oriented tools in the forges (see below)
we also work on integrating new tools and methodologies (Continuous Integration, tests, Eclipse, etc.)

Another important aspect of our activity is to raise awareness of these issues in the community of forge owners/providers and users. Events such as the forges think tank at OWF 2010 or the Forges track at RMLL/LSM 2010 are examples of this effort.

As physical meetings and workshops are not enough and contacts and work needs to be continued online, we established a proposed online community “PlanetForge” to try and reach as many interested parties as possible (see bellow). Of course, the goal is to have much more participants than just the COCLICO partners in PlanetForge, so that our dissemination and standardisation efforts can meet others’ and a lively community can work together in a lasting manner.

Other efforts

Data portability

Data portability seems a very interesting initiative whose goal is to solve such lock-in problems like the one we exhibited for the forges in this piece.

However it is not obvious to us if it may be the right place for discussing Forge data portability. Maybe joint efforts could be made between us (in COCLICO and the PlanetForge community) and participants to DataPortability.

Initial contacts seemed to produce no real echo, but that may just be caused by bad timing or misunderstanding. In any case, we’re very open to any more contacts with the initiative, should anyone help with such contacts (feel free to comment bellow this article).

ForgePlucker

ForgePlucker, a tool developed by Eric S. Raymond (ESR), is interesting in many aspects.

Its goal is to help users locked-out of their project, to regain access to data hosted on a forge. Of course that is just one of the use cases where data export from a forge needs to be performed, but certainly a valid one (see the above mentioned blog post and the forgeplucker announcement by ESR for the very detailed rationale).

There’s code available in ForgePlucker’s repository, but that would still require a lot of work (and many improvements) in order to provide a full backup of a forge project. And of course developing connectors for all the many forge variants would require much effort, not to mention the importers for all different target forges.

Our plan in COCLICO is to extend and improve the current core of ForgePlucker, to produce an Export / Import toolbox that may become generic enough to solve the lock-in issues for many users (still requiring them to contribute to it some custom connectors).

The format of the dumps produced by the current initial version of ForgePlucker is JSON, but with a non-semantic ad-hoc content. We intent to improve this to render it more semantic, by including reference to particular ontologies, with the RDF model.

Whenever possible we’d then use existing ontologies and standards instead of ForgePlucker’s own custom ontology, such as FOAF, DC, DOAP, or OSLC core (for OSLC, see below).

Linked Open Social Web

We intend to use the same approach as for the dump format, of using the RDF model to describe forges data and meta-data, reusing and extending existing ontologies, together with REST interfaces to demonstrate how it is possible to include such data in Semantic Web formats in the very interfaces of the forges.

The goal will be to make sure that development artifacts and communities that are hosted in (public) forges can become first class resources participating the Linked Open Data cloud (more details about the Linked Data approach on http://linkeddata.org/)

We intend to see this leveraged by a similar approach adopted in the OSLC compatible tools (see bellow).

In order to achieve such goals, we propose an ontology that represents data/meta-data and associated links, in a way which models the existing structure of tools and data found in forges.

We intend to propose this ontology to extend OSLC data representation standards whenever suitable, to make forge resources available to the Linked Open Data cloud.

Of course, many artifacts found in the forges resemble those found in other social sites or collaboration platforms, mainly when it relates to the people participating on the sites, as groups or communities, discussing and solving problems, sharing content together.

It is interesting to note that in the same way as development becomes more distributed (following the example of the FLOSS projects), and provides new tools supporting such distributed work (as Git emerges vs the centralized model imposed by Subversion (SVN)), social networks become more and more distributed also. Of course, solving the lock-in issues can be done through intrinsic distribution.

As SVN loses ground against Git or Bazaar (BZR), traditional RDBMS see No-SQL database systems emerging, and some predict that FaceBook may see a competitor in Diaspora, the tendency may be for more and more social sites and hosting platforms (forges included) to become more distributed, more inter-linked, more Social Semantic Web (Web 3.0), and so “no-forges” may be the future ?

We can see already some interesting attempts in such directions, with projects like SD, Fossil, or the new Savane rewrite, as well as the next generation forges proposed by QualiPSo, and on which COCLICO participants are working also nowadays.

Still, more traditional (monolithic) forges are in great need of dynamic interoperability, to allow the plugging of tools to render new services around the development activities.

Dynamic interoperability

Another aspect on which COCLICO is actively working is dynamic interoperability of forges with other tools.

Some emerging standards are worth citing as we try and implement support for them in existing forges like FusionForge and Codendi in COCLICO.

First, Open Services for Lifecycle Collaboration (OSLC) which is a proposed standard, elaborated as an Open standard mainly by commercial editors in the ALM sector. Despite its origin that may trigger suspicion from hard-core FLOSS addicts, it appears to be a very nice open standard, elaborated by an open community, and with, hopefully, many good technical and methodological principles. Thus, we see it as a very good candidate for establishing an interoperability standard in the forges domain.

About the technological aspects, it is notable that all specs are based on the use of Web standards, like REST, RDF, or Ajax interactions.

We have already started implementing OSLC-CM, the Change Management domains specification of OSLC for the Mantis bugtracker in the prior HELIOS project. We continue these efforts in order to implement an OSLC-CM compatible REST server for FusionForge trackers (reusing the same codebase as for Mantis, also a PHP app).

We do hope that OSLC Core, the most basic specs that underline all the other domain-specific OSLC specs can be a building block for future forges-related specifications and standards. Of course it seems reasonable to also contribute to OSLC whichever standardization elements come from our experiments in the forges area in COCLICO, as we had already started to do during HELIOS for the OSLC-CM V2 specs. At least, at Institut TELECOM, we intend to continue to contribute to OSLC specs, and implement the standard in FLOSS applications.

Some other standards or specifications seem quite interesting and we are going to try and support them in forges during COCLICO, like WebID (aka FOAF+SSL), or OAuth (which is recommended by OSLC, by the way).

We do hope that other participants to the PlanetForge community will join us in testing, validating, and promoting such standards for greater interoperability of the products they use or develop.

The PlanetForge community

Finally, as was aready mentioned several times above, we hope to help all users, editors, implementors, and owners/admins of forges to exchange and collaborate more, mainly regarding standardization, for instance in order to preserve users freedoms, as well as to better integrate different tools with each-other.

We propose to host such inter-forges and inter-communities collaboration around forges under the PlanetForge umbrella, hosted at http://planetforge.org/.

At the moment, the following on-line tools are available :

a News agregator (planet RSS agregator) at http://planet.planetforge.org/
a microblogging hashtag/community /
- at Identica : !planetforge
- at twitter : #planetforge
mailing-lists
- the discussion mailing-list in english : discussions@planetforge.org
- a french speaking mailing-list also
a wiki (running MediaWiki) to share content (specs, use cases, useful pointers) at : http://wiki.planetforge.

In addition, we intend to organize, again under this PlanetForge umbrella, series of events allowing direct physical contact between all participants, in the line with previous meetups held at events like RMLL/LSM or OWF.

We do hope people interested by the subject of forges will participate to PlanetForge, exchanging and sharing ideas, experience, collaborating around tools, and of course participating to standardization efforts for more interoperability.

Conclusion

I hope this long article will have provided a few useful bits of information, and some hints on how the current interoperability issues of the forges may be addressed properly, and hopefully an interesting description of what COCLICO is trying to do in this respect. I sincerely hope others will join our efforts in the frame of the PlanetForge community, for the larger benefit of all actors.

Special thanks to Madhumita for english proofreading.

My presentation about COCLICO and forges interoperability at OWF 2010

2010-10-02T16:59:29Z

I’ve given a talk on Friday 1st of October at Open World Forum 2010 (OWF) in the (not necessary all so much) Open Forges Think Tank track on behalf of the COCLICO project and some of our (or mine) ideas on forges interoperability.

I’ll try and prepare a more detailed textual version of all these ideas (worth adding to the project deliverables maybe ?), but here’s already a link to the slides (attention: 5 Mb). Actually I’ve slightly modified them after my speech before posting them, as I added some more comments in last minute that may have deserved a bullet of their own. EDIT: I’ve finally posted a textual version of my ideas.

Here’s also a crappy preview online, as it seems slideshare doesn’t like the fonts used by this beamer presentation :

Coclico project – Forges Interoperability (OWF 2010)