Tag: governance

Meritocracy and Hierarchy at Apache

Apache is actually governed in two ways: by the meritocracy of the Apache Way, and the hierarchy of the Apache Software Foundation. We strive to govern ourselves as much by meritocracy (and consensus) as possible, and as little by hierarchy as is practical for non-profit corporation. This has a number of both practical effects, as well as a number of social corollaries.

The ASF prides itself in running as a community-based meritocracy of peers. That is, within each project community, people participate as individuals and gain merit as measured by that community. The community recognizes your merit by voting people in as committers or PMC members. A new committer can vote on certain kinds of code changes, and can checkin their own code. PMC members vote on new committers and product releases.

In Apache projects the underlying cliché is correct: those who are doing the work get to make the decisions. A healthy community will seek consensus over the direction of the project, and typically only uses [VOTE]s to solve thorny technical issues that don’t come to a clear consensus. Similarly, PMC members each have one vote on elections for new committers or PMC members; decisions within projects are made by the community as a whole. Technical governance is always done by meritocracy.

Even within the Foundation – i.e. the Delaware registered non-profit corporation that makes the ASF a legal entity – we strive to govern by meritocracy as well. Nominating new members and electing the board of directors are the obvious rights of members, and these are discussed and judged based on individuals’ merit towards the various Apache projects and to the ASF as a whole.

Similarly while the board appoints individuals to serve as corporate officers with specific delegated powers, all Apache members are allowed to subscribe to any mailing lists and participate across various ASF activities. Various members step up to work on policy, corporate, legal, treasury, and many other tasks that are needed in any corporation, always working with the relevant officers. Again, those who do the work decide how it gets done.

In these organizational tasks, it’s not always as clear how to reach consensus on contentious issues. Social issues or corporate governance questions don’t often have as simple a metric to measure against as technical issues do. Code either works or it doesn’t; corporate policy often has a very different kind of judgement. Similarly, with an active membership (with many very passionate individuals!), finding consensus on non-technical questions amongst 100+ individuals is often difficult. Here, hierarchy steps in to govern.

In most cases this works smoothly. Either the relevant officer will make a policy decision and publish it, or the board will pass a resolution or clearly ask for a change. While the whole community – the membership, the various PMCs of Apache projects – may not have a consensus on the decision, it is still policy, and projects will generally work to make the change.

In some cases, this doesn’t. A key way that the board ensures there is sufficient oversight of all Apache projects is by requiring reporting: monthly for officers; quarterly for PMCs / projects. There have been cases where PMCs have not reported regularly, or have not provided sufficient details to the board’s satisfaction in their reports. In these cases, the board will require a new or updated report. This is not a question of merit, or getting a consensus: this is an order. Non-response to a board request or not following an officer’s policy will and does result in the board stepping in to force changes within the project. As they say in politics, officers serve at the pleasure of the board.

It seems that sometimes this switch from meritocracy – where everyone in the PMC has an equal say, and they reach consensus – to hierarchy – where the board says “jump” – sometimes catches people by surprise. Sometimes, it seems that some PMC members just never really thought about the fact that the ASF is a legal corporation, and must follow certain rules. Other times, this surprise is clearly exacerbated by poor communication in specific cases between the board and projects. While cases like this really aren’t a choice – the board can and will dissolve or reconstitute projects (always as a last resort!) – it is important to clearly explain the issues and why this kind of hierarchy is needed in some cases.

Has anyone else had this feeling? Of switching between meritocracy, and consensus among the community, to hierarchy, and the board simply making a decision? How can we improve the handling of future issues like this, to ensure we keep the maximum meritocratic freedom within our projects, while still ensuring the legal shield of the ASF is secure and can function for the long term?

I’ve been thinking about this lately since I recently drew up a new org chart here for the ASF as a whole. Note that the hierarchy part only comes in on organizational, legal, and similar matters – technical decisions are always in the meritocratic hands of the PMCs and projects. More reading can be found on some of these excellent slides about Apache and open source.

Apache projects are technically independent

This post has been improved and posted as guidance to all Apache projects on the Community Development website – please read the new version there!

In the earlier essay I wrote on Apache project independence (since promoted to an article on the Apache Community Development website) I missed a critical bit of independence that I think we should add to the ComDev site.

Apache projects are technically independent

While Apache PMCs provide a report of their operations to the Board on a quarterly basis, they are technically independent of the Board. PMCs are free to collaboratively decide their project’s technical direction however they choose. While the Board holds PMC chairs and PMCs accountable for ensuring that their methods are appropriate – following the Apache Way and the few other rules at Apache – the Board does not exercise any technical direction over Apache projects.

This is a critical point to understand. There is no top-down direction at the ASF in terms of what technology we implement or what kinds of projects we seek out or support. All technical choices are made by the individual committers and PMC members within their project communities – i.e. the individuals actually doing the coding decide where the coding will go.

This applies equally to the Apache Incubator, as the gateway of new project proposals at Apache. The ASF has noagenda of what projects it wants to find, nor do we have a corporate strategy for new project acquisition. Project communities that want to come to the ASF are welcome – and will be reviewed in the Incubation process for community health and diversity, and for IP clearance and licensing. But not for technical merit.

The essence is: if there is a sufficiently diverse community who chooses to live by the Apache Way, the ASF is happy to host them. The fact that there is sufficient diverse community means that people are interested in the technology. Similarly, the Apache Attic awaits projects that may seem interesting technically, but fail to have a sufficiently diverse and active community to maintain them.

So – worthy to add to the ComDev page?

Apache Governance: Membership

This post has been improved and published on the ASF’s website as the Governance overview of Membership – please read it there!

This is the third in a series of essays describing Apache governance. The ASF is a non-profit, membership-based corporation; as such the Members of the ASF have a role similar to shareholders in a publicly traded corporation.

In this essay, the term “member” specifically means someone who is a Member of the Foundation itself, as elected per the corporate bylaws.

Organization

The ASF is a membership corporation, so Members serve a similar role as shareholders do in publicly traded corporations. Members may propose new candidate members, may cast one vote in new candidate elections, and may cast one vote in board elections.

Members are not empowered to speak officially on behalf of the ASF as a whole, nor do they have any special rights to influence technical direction of any Apache projects that they are not otherwise elected as a committer or PMC member on. While members often volunteer and serve in many capacities both at the Foundation level and within various Apache projects, their only specific organizational rights are to vote in board and member elections.

The Apache Incubator is special: any ASF member may request to be added to the Incubator PMC without a vote. Within the Incubator PMC, members may serve as official mentors or champions to incoming podlings, as well as voting on Incubator policy and releases of podlings undergoing incubation. In this way, members work to mentor podling communities and guide them to the Apache Way and eventual graduation as a top-level Apache project.

Legal

Members act as shareholders of the corporation. As such, each member has a single vote on electing directors to the board; similarly, members may vote on new nominees to membership. Members are eligible to nominate new candidate members and to nominate individuals to the board.

Organizationally, members do not have specific standing within any Apache projects or Incubator podlings. However most members are active as individuals within multiple Apache projects on their own technical or social merits, and once elected as a member often find more ways to get involved in more Apache projects.

While the board and relevant officers are directly responsible for providing oversight to the many Apache projects, members often work within many Apache projects to help ensure projects run smoothly and follow the Apache Way.

Communication

The central place for member-focused announcements and discussions is the privately archived members@ mailing list. This is used for a wide variety of purposes: both proposing and discussing new technical or policy ideas within the ASF, announcing marriages or births within member’s families or other major social events, and any formal announcements to the membership from the board or corporate officers.

By policy, members have the right to inspect and review the archives of all mailing lists at Apache. This policy is designed to ensure that every member can independently inspect all corporate operations and the operations of all Apache projects. Thus members may review all private mailing lists (for example) about internal legal affairs, fundraising, security issues, as well as any private@ lists used by our Apache projects’ PMCs.

While members may send messages to any private lists, they do not automatically receive any special merit in terms of influencing the technical direction of Apache projects. Merit in Apache projects is gained within each individual project community, and membership does not convey any other special privileges.

Meetings

The board typically holds an Annual Members Meeting, where a new board of directors is elected, and new member candidates may be voted upon. Meetings are currently held on a private IRC channel, as an alternative to a traditional conference call, where any reports are read and members may ask questions. The meeting has a 48-hour recess for voting, and then reconvenes to announce results and complete the meeting: this allows members who may not be able to personally attend the first portion of the meeting to attend the second half, as well as to conduct voting over email using our own secure voter tool. Members who are unable to attend any of the 2 day meeting period may provide for another member to proxy their attendance and votes.

The board often holds interim Special Members Meetings in between Annual Meetings, primarily to give members a chance to nominate new candidate members and be voted in. Members also use Special and Annual meetings to raise additional questions or issues about Foundation operations – although members usually just raise questions to the board or to any ASF officer at any time over our usual mailing lists. Members and invited guests are welcome at all Members meetings.

Given the distributed and volunteer nature of the ASF, official in-person meetings of the Membership are no longer held. All Foundation or project business is conducted on normal mailing lists – although some of the lists are private. Members do often meet in small groups in person – although this is for social reasons, and often involves a meal or drinks. Many members and committers also traditionally attend the ASF’s annual ApacheCon conference.

Merit

All members have the ability to nominate new individuals as candidates for membership. The amount and types of merit that existing members look for in Apache committers varies, but always includes some significant technical or other contribution to one or more of our projects, as well as a clear interest and understanding of the Apache Way. In most cases, these contributions and traits are displayed over a significant period of time, usually over a year of engagement in one or more Apache projects.

Note that it is not required to be a committer before being considered for membership, however as a practical matter, in the vast majority of cases any potentially worthy individual has already become a committer on some Apache project. At least two individuals have been elected as members without being committers first, in each case for non-coding contributions in mentoring on the Apache Way or other organizational work.

Many newly elected members are surprised (pleasantly!) to be told they’ve been elected. Typically, being nominated and elected as a new member is something that happens well after the nominee has a clear track record and a positive influence on Apache projects for some time; in hindsight, a frequent comment is “isn’t So-and-So a member already? They do such good work!”. Individuals asking to be made a member (or worse, insisting you should be elected!) is culturally frowned upon. In a perhaps counter-intuitive way, being considered for membership is something that requires real effort acting over a measurable time, but without making it obvious that you’re seeking recognition.

Candidate members are voted on as a simple majority vote (more yes’s than no’s) at Annual or Special member’s meetings. Voting is performed using custom-written voting software with secret ballots.

Within the Membership, merit is equal; all members have an equal vote and ability to propose change within the ASF. Membership is a notably helpful factor in being considered for the board or an appointment to officer positions at the ASF, although the most important factor is demonstrated merit within the particular project or the ASF as a whole.

Community

The ability of members to influence the ASF is simultaneously major and immaterial. On one hand, members vote on new candidate members and more importantly vote on the Board of Directors, which clearly affects strategic policies of the ASF. On the other hand, membership grants individuals no other special merit within any of our projects and is not an official position within the legal corporation itself.

The perceived importance of membership is likewise a dichotomy. Many members are quite modest about their membership, and a frequently heard comment is that “oh, they really deserve to be recognized more than I did”. For many, being elected a member was not a goal or title that individuals were pursuing; rather, they were naturally doing their own work (at an Apache project), and were then recognized for it. Many members list their affiliation on their resume (or LinkedIn, etc.); some do not.

Recognition of membership varies widely outside of the ASF: some software companies don’t seem to care what the ASF is or what membership means; a few software companies have strongly encouraged their employees to become committers or members, and gaining one of those roles is a notable prestige bonus within those companies. Likewise, a number of members or officers of the ASF are frequently sought out as speakers at open source conferences.

Members share both the community of all other members, as well as often serving as bridges across the various Apache projects they are already involved in. Many members also volunteer extra time to serve as mentors, both within the ASF working with our projects and communities, and outside of it, mentoring others or speaking at conferences.

Technical

The membership as a whole and individually does not provide technical direction for any Apache projects directly; every PMC is free to manage it’s technical direction independently. PMCs are the governing body for their project and are expected to manage the project’s technology in the best interest of the whole project community, independent of outside commercial influence.

This is similar to how the board does not set technical direction for projects: this is a key reflection of how the ASF is intentionally structured to provide maximum freedom to its projects. The board and the ASF membership are happy to provide a home to any software project communities that are willing to follow the Apache Way. The mission of the ASF is to provide software for the common good: we are happy to help like-minded communities to provide that software; are confident that communities will form around software that is useful, and understand that there are many different ways to effectively and collaboratively build software.

In the case of members, nearly all members participate in a number of Apache projects – as individuals based on their merit within the specific projects. Simply being elected a member does not confer any additional abilities in terms of other projects. Many members chose to serve as ambassadors of the Apache Way, and usually get involved in more projects as time goes on.

Resources

Apache Governance: Project Management Committees

For the latest on what Apache PMCs are and how they work, please see the updated version of this on the official Apache Governance Overview of PMCs!

This is the second in a series of essays describing Apache governance. Fundamentally, the most important organization at the ASF is a PMC or Project Management Committee. These are the actual groups that decide what software our projects release, and as such do the bulk of the actual work of the ASF.

Continue reading Apache Governance: Project Management Committees

Apache Governance: Board of Directors

This post has been improved and turned into the ASF’s official Corporate Governance overview of the Board!

This is the first in a series of essays describing Apache governance. While there are a number of existing documents that explain some of the “hows”, I’m hoping to fill in some of the key information about the “whys” of how the Apache Software Foundation (ASF) actually works day-to-day. Look for more articles on things like the Membership, PMCs, projects, and more coming up!

Continue reading Apache Governance: Board of Directors

Apache projects are independent and non-commercial

This post has been improved and posted as guidance to all Apache projects on the Community Development website – please read the new version there!

While not all aspects of the Apache Way are practiced the same way by all projects at the ASF, there are a number of rules that Apache projects are required to follow – things like complying with PMC release voting, legal policy, brand policy, using mailing lists, etc., which are documented in various places. There are a few rules that I think may not be documented as succinctly as they should be.

A primary purpose of the basic requirements the ASF places on its projects is to help ensure long-lived and stable projects by having a broad enough community to maintain the project even in the potential absence of any individual volunteer or any sea change at a major vendor in that area. The Apache project governance model is explicitly based on a diverse community. This is different from other governance models, like the “benevolent dictator” idea or the often corporate-backed model that Eclipse uses.

Apache projects are independent

This is implicit in the fact that the Project Management Committee (PMC) runs the project and the fact that PMC members are expected to contribute to the project as individuals, wearing their “PMC hat”. The concept of hats means that when a PMC member votes on project matters, they are casting their vote as an individual acting in the best interests of that PMC, and not as an employee or representative of some third party. There are also certain expectations of diversity within a PMC; the board may apply extra scrutiny to PMCs with low diversity (i.e. PMCs that are dominated by people with a common employer). Similarly, the ASF does not allow corporations to participate directly in project management, only individuals.

There are two important aspects to this independence: project management, and project use by end users.

Apache projects are managed independently

Apache projects should be managed independently, and PMCs must ensure that they are acting in the best interests of the project as a whole. Note that it is similarly important that the PMC clearly show this independence within their project community. The perception of existing and new participants within the community that the PMC is run independently and without favoring any specific third parties over others is important, to allow new contributors to feel comfortable both joining the community and contributing their work. A community that obviously favors one specific vendor in some exclusive way will often discourage new contributors from competing vendors, which is an issue for the long term health of the project.

Apache products may be used independently

All Apache projects must release their code under the Apache License, which clearly specifies the minimum restrictions that users of Apache software must agree to. Apache software is all about being able to use it for virtually whatever our users want – open source, proprietary, secret – we’re happy to have users take our software (although not our name) for virtually any purpose. While our legal guidelines allow certain other software licenses to be used for specific dependencies, the software we release always uses our license.

Extending this idea, users of Apache software should be able to find our software, learn how to use it, and actually apply it to all its common use cases solely by going to the Apache project’s own website. Apache projects should provide sufficient documentation, install features, basic user help (through mailing lists) and services for the common use cases to the user, without them having to rely on third parties. It is important that our users can both make use of our software freely – both in terms of not having to pay for the software, as well as not having to worry about IP claims or other more restrictive licenses on either the software or the configurations or other common materials required to actually use the software.

Apache projects are non-commercial

The ASF’s mission is to produce software for the public good. All Apache software is always available for free, and solely under the Apache License. While our projects manage the technical implementation of their individual software products independently, Apache software is released from the ASF and is always meant to serve the public good.

We’re happy to have third parties, including for-profit corporations, take our software and use it for their own purposes – even when in some cases it may technically compete with Apache software. However, it is important in these cases to ensure that the brand and reputation of the Apache project is not misused by third parties for their own purposes. It is important for the longevity and community health of our projects that they get the appropriate credit for producing our freely available software.

 

 

Reminder: “The postings on this site are my own and don’t necessarily represent positions, strategies or opinions of either my employer nor the ASF.”

Apache new(s): officers, projects, commits

A brief roundup of recent Apache news:

Please welcome a number of new top level projects that have recently been created:

  • Please welcome the Apache jUDDI project, a Java implementation of the UDDI v3 specification, which graduated in August from the Web Services project to be come a top level project.
  • Welcome also the Apache Pig project, a platform for analyzing large data sets often used with Apache Hadoop – where it recently decided to move from being a subproject to a TLP in it’s own right.
  • Likewise, the Apache Hive project has also split off from Apache Hadoop as well, providing a data warehouse providing a simple query language called Hive QL.
  • The Apache Shiro project recently graduated from incubation to become a TLP, and provides a security framework for authentication, authorization, cryptography, and session management.

Finally, I’d like to extend the appropriate thanks and appreciation (which are very large!) to our recently outgoing executive officers:

  • Justin Erenkrantz, our outgoing President, who’s literally traveled the world speaking at events and companies on the ASF’s behalf.
  • Sander Striker, our outgoing Executive Vice President, who’s also been a help in outreach and our infrastructure in many areas.