apache.numprojects -= 1; apache.karaf.intro = “Welcome!”

For only the fourth time in our history at last month’s June board meeting we passed resolutions that effectively reduced the total number of Apache projects by one.

  • As was widely expected, the board terminated the Apache iBATIS project, and sent it to the Apache Attic. This recognizes that we don’t expect there to be an active Apache iBATIS community, and that we don’t expect there to be any new development in that project for a while. The Apache Attic will continue to provide all the project’s resources on a read-only basis for any existing users. (Note: current users may also be interested in the external fork over at mybatis.org)
  • The board also terminated the little-known Apache Quetzalcoatl project and moved it to the Attic. “Quetz” had been charged with developing the mod_python module, but it never really took off as an organized Apache project. Current users may be interested in finding the sources over at modpython.org
  • In happier news, the board voted to promote an Apache Felix subproject named Karaf to top-level status. Apache Karaf is a small OSGi based runtime which provides a lightweight container onto which various components and applications can be deployed. The Felix PMC had seen that there was sufficient community around just the Karaf subproject that it deserved to have it’s own project.

So that’s two projects down, but one project up for the month of June.

iBATIS and Quetz both join previously retired projects in the Attic, HiveMind, Shale, AxKit, Xang, Beehive, and Jakarta Taglibs. Each are projects that had lost an effective Apache community able to actively develop them.

In the past, the ASF has also terminated a handful of other projects before the Attic was opened in 2008; those include Apache Commons (the first version) and Apache Avalon, both terminated for community issues. The ASF also once had an Apache PHP project that was terminated; in that case it was a happy and mutual separation of the PHP Group from the ASF.

Resolutions for creating and terminating Apache projects are passed by the board, typically at monthly meetings, and our public records of formal board actions are always available.

Stay tuned for news of the upcoming Annual Member’s Meeting of the ASF being held in mid-July, where we’ll also be electing a new board of directors.

mybatis.org forks from Apache iBATIS

The bulk of the active development community for the Apache iBATIS project has decided to fork the code, and move their development efforts to the new project they’ve founded called mybatis. While this may be old news to iBATIS/mybatis users, I thought it deserved a wider mention, both about the fork and some of the thoughts behind it.

What is mybatis?

mybatis.org is a new, independent project run by Clinton Begin and most of the PMC members and active committers of the Apache iBATIS project. It’s a fork of all the iBATIS code under a new name. They’ve recently released mybatis 3.0.1 GA, which will be the primary focus of development for the new mybatis team. They currently ship mybatis releases using the Apache License.

Since it’s a fork of the existing iBATIS code, it provides the same advanced SQL data mapping functionality that existing users expect. “MyBatis couples objects with stored procedures or SQL statements using a XML descriptor. Simplicity is the biggest advantage of the MyBatis data mapper over object relational mapping tools.”

Who is mybatis?

Clinton Begin, the original creator of iBATIS in 2001, is leading the project. Along with him are the bulk of the active developers of both mybatis (for Java) and mybatis.net. They are using Google Code to host the projects, along with source code, mailing lists and bug trackers. The project is independent, and is not directly associated with the ASF.

What will happen to Apache iBATIS?

The ASF is a public charity with the mission of provide software for the public good. Thus, we will continue to provide the complete source code under the Apache license in the future. In terms of the project as a whole, it’s expected that due to lack of community still participating at the ASF, it will be moved to the Apache Attic during the upcoming mid-June board meeting. That will mean the PMC and officer positions associated with Apache iBATIS will be dissolved, and some time thereafter, the website, mailing lists, and code for the project will be moved to the Attic for future read only reference.

Projects are moved to the Attic when there isn’t a sufficiently active and healthy community (primarily at the PMC level) to provide oversight to the project and it’s releases. The decision to move projects to the Attic is not a technical one – it’s purely a community issue. A project’s code and past history are always freely available to users, although in a read-only state. If, in the future, a sufficiently diverse community wishes to re-activate the project and follow the Apache Way, we welcome that.

What’s the difference? Why did mybatis want to fork?

Obviously, you’ll need to ask the mybatis team for the answer to this question. From emails I’ve exchanged with Clinton, and seen from other members of the mybatis team, my impression is that the difference wasn’t specifically technical, more organizational. They simply have a different vision for how to develop open source software projects than the ASF does.

ASF projects are expected to follow a number of procedures roughly known as “The Apache Way”. We believe that software projects with a diverse community; that use a consensus process to make decisions; and that do all their work in the open results in the best overall quality and longest lasting projects. While the ASF expects it’s projects to follow the Apache Way, we certainly understand that not everyone believes that our way of software development is the right one for everyone or for all projects.

While there were some unfortunate miscommunications (not necessarily anyone in particular’s fault) in the process of the fork, I hope there is no ill-will between the ASF and the mybatis team. While it’s sad that the ASF is losing a project (and the community with it), we wish the mybatis project luck in their work.

Is it OK for them to fork the project?

Of course! The Apache License is written to provide the maximum freedom to the users of our software. This means you are free to re-use any of the code we produce in almost any way you wish, including building atop it to create a proprietary product, or by forking the code and re-architecting the whole thing. While we always appreciate developers who choose to contribute fixes and new code back to our projects, we’re perfectly happy for people to take our code and do their own thing with it, either privately, or shared with the world.

So, good luck to mybatis, and mark this as (I think) the first time the ASF has had a coherent community choose to fork themselves and move elsewhere in a productive manner!

Congratulations to six new Apache projects!

In last week’s monthly meeting of the Board of Directors of the ASF, we approved the creation of six new Top Level Projects (TLPs) at the ASF. This is the most new TLPs ever created at once, followed only by the meeting of November, 2008 where 5 new TLPs were created (CouchDB, Buildr, the Attic, Qpid, and Abdera).

In this particular case, much of the growth comes from within existing projects, wherein subprojects communities within Hadoop and Lucene have matured sufficiently to deserve to manage their own fates, and to create their own Project Mangement Committees (PMCs) to take charge. To put this in another perspective, this is also reflective of the ASF’s growth; before this meeting we had over 70 TLPs and over 30 Incubator podlings, so an addition of 6 new TLPs is less than 10% growth for the month.

We should congratulate the Apache Traffic Server community first, since they went through the Incubation process and successfully graduated from an Incubator Podling into their own TLP. Soon to be served (once the website migration is complete) from http://trafficserver.apache.org/, Apache Traffic Server is fast, scalable and extensible HTTP/1.1 compliant caching proxy server. Congratulations to the whole team in showing a strong and diverse community around this new product.

Next up come three subprojects within the well-known Apache Lucene project which have grown organically from modules within Lucene to be diverse and active projects within their own right. You may recognize some of these product names from the Lucene world.

  • Apache Mahout, which is building a system for creating scalable and effective machine learning libraries which can perform recommendation mining, clustering, classification, and grouping into itemsets.
  • Apache Tika is a toolkit for detecting and extracting metadata and structured text content from various documents using existing parser libraries.
  • Apache Nutch, integratable with both Lucene and Hadoop, adds web-specific crawling, fetching, and organization features.

The Apache Hadoop project – another wildly distributed computing technology – has also grown two of it’s subprojects to the point where they deserve their own fame.

  • Apache Avro is a fast data serialization system that includes rich and dynamic schemas in all it’s processing.
  • Apache HBase is the Hadoop database – designed to provide random, realtime read/write access to Big Data – billions of records – using commodity hardware.

Why did these subprojects spin out to become their own TLPs? The driving factor is not the technology, but rather the community and oversight aspects of how the ASF organizes it’s mostly self-running projects.

From the oversight perspective, the ASF Board relies on every project’s PMC to manage their project’s operations within the broad guidelines of the Apache Way, and to report their project’s progress and issues to the board. This means that there must be enough PMC members who can actively monitor and participate in their project’s activities, and can especially show due diligence and responsibility in voting on any official product releases the project makes. With the rapid growth in both community and technology areas in the Hadoop and Lucene projects, it’s a difficult job for the PMCs to truly understand and help manage all the subprojects they’ve created or added over the past two years.

While the scope of oversight may have hinted that some subprojects should be promoted to TLP status, the gating factor is community. Does a subproject have a strong and diverse enough community to provide their own, independent PMC that can manage their own affairs? Becoming a TLP is both a benefit and a responsibility: the community through it’s new, more focused PMC can better run itself; however the new PMC is also expected to provide accurate reports and responsible oversight of their community and product releases.

Congratulations to all six new projects! Please note that as the websites are updated, each project will be moving it’s home page to http://projectname.apache.org in the near future.

Winter holiday ASF news roundup

There have been a number of exciting updates at and around the ASF over the winter holidays – which I enjoyed quite well with a long spell offline. No more; here are some of the key Apache news from the past month.

Reminder: as the official Foundation blog states, the only content officially produced by the ASF as an organization happens at apache.org domains. This blog is written by Shane, is solely my work, and represents neither the official policy nor views of either the ASF or my employer.

ApacheCon Oakland roundup

There was a ton of great news at ApacheCon US 2009 in Oakland this year, including some terrific blog posts about it and the Apache Way in general.

Watch for a wrapup of the many different videos that came out of the conference soon!

Getting to ApacheCon + Resources

Discount codes are still available for registering for ApacheCon, and we have plenty of free events. Tips on actually getting here:

  • Pack layers; the weather is a beautiful sunny 70degF days/50degF nights.
  • From SFO and OAK, take the BART light rail trains to the 12th Street Oakland City Center stop. From OAK, you’ll first take the AirBART bus to the BART station, which requires $3 exact change on the bus.
  • The Bay Bridge is open once again, so you can drive, although the BART is highly recommended unless you need assistance.
  • The Marriott hotel tower is kittycorner/diagonally across the street from the 12th street BART station – it’s closer than you think.
  • All conference events are in the Convention Center, which is in the same building as the Marriott: walk through the reception area around and past the escalators to the second floor conference rooms.
  • Sessions and Meetups are held on the first floor main conference halls, and in the 200-series rooms on the second floor of the convention center.

Roundup of information about ApacheCon:

ApacheCon discounts and free MeetUps

The hotel room rate discount has been extended through 19-October – this Monday – so there’s still time to get the group room rate. Some other useful posts and bits about ApacheCon have been posted this week.

  • Our CrowdVine calendar is up! You can connect with other attendees and create your own “my schedule” there to get a private ics feed of events you want to attend.
  • CrowdVine also provides a great aggregation of attendee’s blogs, tweets, and photos too.
  • OStatic by GigaOM is running a guest editor series on ASF people, including Jim (Chairman), Justin (President), and Shane (Director) so far, with more planned.
  • There are plenty of evening Meetups planned on Monday, Tuesday, and Thursday – free to attend by all.
  • If you haven’t registered yet, there are still a few Apachecon discount codes available for another day or so – if you do a little digging or tweeting, I’m sure you’ll be able to score one!

ApacheCon room discount; Training interviews

The group discount rate ($169/night) for ApacheCon attendees staying at the conference hotel is scheduled to end today – so be sure to reserve your hotel room now if you’re planning to stay at the Marriott where the conference is held!

Our keynote speakers have been announced: Brian Behlendorf, Kirrlily Robert and the Apache Pioneers’ Panel.

The blog traffic is now really picking up about ApacheCon US 2009.

ApacheCon Blog Roundup

Folks have just started blogging in earnest about what they’ll be doing at ApacheCon this year in Oakland – besides our awesome birthday party. Here are a few of the best posts.

Check out OStatic as well for some cool upcoming Apache related postings.

ApacheCon registration discount ends Friday!

There’s still time to save over $200 for registering early for ApacheCon – just sign up before the end of Friday, 25-September!

Check out our technology-focused schedules each day, created and organized by our PMCs and communities directly.

Day Business, Community, and OFBiz (new!) Running Servers: httpd and Tomcat Providing Content on the Web Search Everything Beyond Web Services
Wednesday Business – what ASF projects mean for your business Tomcat – learn about Tomcat 7 from it’s developers Content Technology – Tapestry, Velocity, Wicket, Roller, Shindig Hadoop – use in the cloud, and subprojects Hive, Pig, HBase Tuscany & Synapse – they pull together your service stack
Thursday Community – strong communities make great projects HTTPD.conf – for admins and users of httpd Content Technology – Jackrabbit, Sling, CouchDB, POI Lucene – actually using all those search results with Solr, Nutch Web Services – Axis2 and WS in the real world
Friday OFBiz (new schedule being published!) HTTPD – for module users anddevelopers Geronimo & Directory – enterprise and open source LDAP and J2EE products Lucene – building intelligent and realtime search apps Felix (OSGi) – not your grandfather’s service platform

Follow @apachecon for the latest info, and for other great news about ApacheCon.