Where Is The ASF Going? Director Q&A

With Apache board elections coming up soon, an ASF Member came up with a great set of questions for all director candidates. With permission, I’m sharing those questions here, and providing my answers as well.

I’ve also posted my own Director Position Statement for this year (and past years!).

Continue reading Where Is The ASF Going? Director Q&A

What Apache Needs In Foundation Members

As the ASF’s Annual Member’s Meeting approaches this month, the Membership has an opportunity to vote in new individual Members to the Foundation. I’ve written about how member meetings work and have proposed some process improvements.

But the bigger question is: how can the membership better help the ASF succeed? What a Member can do at the ASF is documented, but what should Members consider doing? Where does the ASF need Members to help out, and how?

Continue reading What Apache Needs In Foundation Members

Three React-ions to the Facebook PATENTS License

There are really three aspects to your project’s decision (to use React.js or not based on the BSD+Patents license), and it’s important to consider each of them. You really need to consider which aspects are important to your project’s success — and which ones don’t really matter to you.
(See the updated FAQ about the PATENTS issue on Medium!)

  • Legal — both details of the license and PATENTS file that Facebook offers React.js under, and some realistic situations where the patent clauses might actually come into play (which is certainly rare in court, but it’s the chilling effect of uncertainty that’s the issue)
  • Technology — are other libraries sufficiently functional to provide the features your project needs? Does a project have the capacity to make a change, if they decided to?
  • Community — how does the rest of the open source community-of-communities see the issue, and care about your choices? This includes both future buyers of a startup, as well as future partners, as well as future talent (employees) or contributors (open source developers).

Continue reading Three React-ions to the Facebook PATENTS License

What Apache needs in a Board

The ASF is holding it’s annual member’s meeting soon, where we will elect a new 9-member Board of Directors for a one-year term.  I’ve been honored with a nomination to run for the board again, as have a number of other excellent Member candidates.  While I’m writing my nomination statement – my 2016 director statement and earlier ones are posted – I’ve been thinking about what Apache really needs in a board to manage the growth of our projects and to improve our operations.

Continue reading What Apache needs in a Board

Behind the scenes at Apache: Corporate Org Chart

You probably use contribute to several Apache projects.  But do you know what goes on behind the scenes at the ASF?  Besides all the work of the 200+ project communities, the ASF has an annual budget of about one $million USD to fund the services our projects use.  How we manage providing these services – and governing the corporation behind the projects – continues to change and improve.

Continue reading Behind the scenes at Apache: Corporate Org Chart

How Apache *really* works

How much do you know about the Apache Software Foundation (ASF) and the many Apache projects we host? Did you know we’re holding our annual Members meeting to elect our board of directors and new Members in just a few days?

I’m often surprised by the variety of basic questions and misunderstandings I hear in the software world about how the ASF really works. We’ve written plenty of documentation about the Apache Way and our governance, but let’s try a different approach. I’d like to interview myself to try to explain some things. Continue reading How Apache *really* works

Apache Governance – Projects First

When push comes to shove and full consensus on governance matters at the ASF or at Apache projects isn’t easily found, it’s important to consider what our underlying objectives are. The mission of the ASF is to produce software for the public good. That’s a good start, but like many concise mission statements, it doesn’t tell the whole story.

There are several aspects of how we expect Apache projects to work that we believe are critical to our mission’s success and longevity. These include things like The Apache Way of: volunteer and collaborative led community built software projects; using the permissive Apache license; and having a consistent and stable brand, infrastructure services, and home for all Apache projects.

We – as in the bulk of the Apache membership and especially the directors that we elect to the board (and thus the corporate officers the board appoints to set policy) – believe that these few core organizational and community management principles are so important for our long term mission that we require these behaviors of all Apache projects. Part of having a consistent brand is showing some level of consistent behavior with the products you produce. Having a shared set of infrastructure services likewise simplifies the process for users to find and use our products. Using the permissive Apache license means that we give maximum freedom to the users of our software, meaning that more users are likely to choose it.

But beyond these core principles, ones we have carefully crafted through experience working on Apache projects and have attempted to minimize when prudent, how do we decide where the ASF should go in the future, or what other, more detailed principles might be important to us? While the officers and directors set the official policies of the ASF as an organization, we look to the constructive input of all Apache members when discussing these ideas.

One recurring meme when discussing “What else should the ASF be/do” has been: “Projects first, or Foundation first?” I.e. is the ASF here to serve the needs of our Apache projects, or have Apache projects chosen to come here to follow the direction of the ASF? Should the ASF decide what kinds of projects we want to join, and decide The One True Technology; or should we simply see what projects show up, and try to support them?

If you picture Eclipse, you can see an example of “Foundation first”: the Eclipse board and leadership – mostly through it’s paying corporate sponsors and de facto corporate-led projects – has an overall vision and drive that explicitly provides direction for it’s projects. Eclipse has full-time staff devoted to “…1) IT Infrastructure, 2) IP Management, 3) Development Process, and 4) Ecosystem Development.” Similarly, the Eclipse board is partially made of appointed members from their Strategic Developer and Strategic Consumer corporations. That leads to strong consistency and direction, primarily for the benefit of the corporations directly or indirectly funding Eclipse projects.

“Projects first” means that the ASF exists to serve the needs of any projects who choose to join us, and who willingly agree to follow our basic rules, like using our license, consensus-driven decisions, and running projects independently of corporations. Our mission is not on behalf of the many corporations who pay their employees to donate code to Apache projects. Our mission is to provide software for the public good – that is, the end users of our software, and the larger world around us.

The ASF doesn’t have an agenda beyond that mission. We don’t have a strategic plan for acquiring projects in specific technologies or from specific corporations. We rely on providing a stable and friendly home for like-minded individuals and communities to build software… for the public good. We certainly hope – and time has shown – that this model will attract vibrant communities that will build useful software. There are plenty of Apache Members and Committers who are passionate about sharing their experiences in software development with others, which is great. But as an organization, we don’t have a strategic plan for seeking out new projects: we believe the best process is to let like-minded projects seek us out.

To ensure that Apache projects focus on the public good, we also require our projects to be run independently of undue commercial influence. Apache projects don’t exist to serve as a vehicle for the corporations who may have donated technology or employee developer hours to them – they exist for the broader public good.

This independence is fundamental to the long-term health of our projects, and to the ASF. It both makes the ASF a place where potentially competing corporations can feel comfortable collaborating on technology (which gets us lots of software for our mission) and also seems to attract plenty of individuals who are interested in participating in our projects – even when changing employers. This independence means that an independent Apache project can continue to exist for the benefit of it’s users even if one of the corporations donating technology to it decides to change direction.

But beyond the core requirements here – ones designed to keep project governance in our independent communities – the ASF strives to not add any additional requirements, but rather to provide the best home to like-minded Apache projects that we can. So in that way, we try to remain focused going forward on putting “Projects first”.

Actually, I should amend that. We do have the core project requirements; ones associated with being an Apache project. So I should really say that we put “Apache projects first”. But beyond that Apacheness that the ASF has developed over 13 years and 190 project communities, we strive to not have an agenda – other than to support those Apache project communities who choose to join us as best we can.

People still reading may also be interested in learning how governance at Apache works.

Apache Corporate Policies

The old saying goes that one of the last things done in many open source projects is the documentation. Taking this to the organizational level at the ASF – the corporate level – that’s both all too true, and not at all true.

Questions regularly come up across various ASF lists about all sorts of corporate operations, and there are a fair number of times when it takes a while to get the right answer to the questioner. Luckily, we have a lot of very experienced Members and Officers who are there to ensure the right thing does happen – but I believe that the answers come be a lot quicker if we had better documentation for these corporate operations. Namely, all the stuff that’s not dealing with code.

This is not to say we haven’t documented how the ASF as a corporation works: a lot of effort has gone into our standard operating procedures, and we’ve both covered the basic legalistic bases, as well as put serious thought into minimizing the absolute rules at Apache while providing lots of best practices. These procedures – and the discussions of pros and cons and whys explaining how they came to be that way – are all available… on the mailing lists, somewhere. 😉

The foundation of our corporate policies is The Apache Way. Understanding that, and understanding that the fundamental mission of the ASF is to produce software code for the public good, leads you to many of the policies we practice, both at the ASF and in Apache projects. To start with, here’s a brief laundry list of how to learn about Apache Corporate Policies.

  • Apache Corporate Governance and org chart describe organizationally how the ASF as a corporation is run. This also makes clear the distinction between organizational governance – the board, membership, and officers as a whole; and technical governance – the fact that PMCs independently manage their projects.
  • The Foundation page provides the overall corporate mission, and links to the Board and Membership pages with the nitty gritty of corporate records.
  • The apache.org/dev pages are both a generic project runbook, technical issues FAQ, and partial corporate operations listing. While these are written by committers, for committers and developers, they have a lot of great information on how all sorts of things at the ASF and all Apache projects should work.
  • Another key overview of organizational documentation is the Community Development project site. This aims to be a more human-friendly signpost to find the relevant technical widget or policy information elsewhere.
  • The Apache Incubator is another important place where proper procedures are documented: this is where we try to explain to prospective new communities how an Apache project should be run before the ASF formally accepts them.

Here are some other principles and rules we follow that are important to understand.

  • Transparency. All technical decisions on Apache projects are reflected on publicly archived mailing lists. The phrase “If it didn’t happen on the list, it didn’t happen” is not just an old saying, it’s the rule at Apache.
  • Respect. One of the primary exceptions to transparency are discussions about individuals. Votes on new committers and PMC members are typically held in private lists amongst a whole PMC, since this both allows better honesty of expression, as well as prevents embarrassment in rare cases where a nominee is not voted in (yet).
  • Organizational Oversight. While the ASF relies on volunteers to produce the code that we give away, we keep it organized by having oversight: the PMCs ensure that projects are managed appropriately; the board ensures that PMCs manage themselves, and corporate officers ensure that the ASF’s corporate bits are covered.
  • Member Oversight. Members of the ASF have the final level of oversight of the board and the corporate officers. All members are allowed to review any private mailing list within the ASF, including any private@ list for projects. Members do not receive special status on any lists; merely the ability to oversee all operations. The only exceptions are specifically restricted mailing aliases that are used for legally protected communications or infrastructure root level issues.

Note that while Members have a broad oversight ability within all aspects of the ASF’s operations, organizational matters like this kind of oversight are not transparent outside of the ASF. While we certainly strive to make public as many of the great lessons we’ve learned about governing community-led software projects, the operations of governance are kept primarily within the Membership.

Partly, this is due to all the lawyers. The details of running a million-dollar corporation require a fair amount of legal work. We strive to publish as much information as is practical in things like board minutes; however the actual corporate processes are done privately.

Partly, this is because part of a meritocracy is allowing those who do to do the work. Individuals who are trusted and useful to Apache projects are elected as committers or PMC members, and therefore can help directly govern those projects. Similarly, individuals who show wisdom and effort towards the goals of the ASF may be elected as Members, and therefore can help elect the board who will appoint the officers to ensure the ASF continues running smoothly.

A great place to ask questions about these kinds of issues is the Community Development mailing list.

Meritocracy and Hierarchy at Apache

Apache is actually governed in two ways: by the meritocracy of the Apache Way, and the hierarchy of the Apache Software Foundation. We strive to govern ourselves as much by meritocracy (and consensus) as possible, and as little by hierarchy as is practical for non-profit corporation. This has a number of both practical effects, as well as a number of social corollaries.

The ASF prides itself in running as a community-based meritocracy of peers. That is, within each project community, people participate as individuals and gain merit as measured by that community. The community recognizes your merit by voting people in as committers or PMC members. A new committer can vote on certain kinds of code changes, and can checkin their own code. PMC members vote on new committers and product releases.

In Apache projects the underlying cliché is correct: those who are doing the work get to make the decisions. A healthy community will seek consensus over the direction of the project, and typically only uses [VOTE]s to solve thorny technical issues that don’t come to a clear consensus. Similarly, PMC members each have one vote on elections for new committers or PMC members; decisions within projects are made by the community as a whole. Technical governance is always done by meritocracy.

Even within the Foundation – i.e. the Delaware registered non-profit corporation that makes the ASF a legal entity – we strive to govern by meritocracy as well. Nominating new members and electing the board of directors are the obvious rights of members, and these are discussed and judged based on individuals’ merit towards the various Apache projects and to the ASF as a whole.

Similarly while the board appoints individuals to serve as corporate officers with specific delegated powers, all Apache members are allowed to subscribe to any mailing lists and participate across various ASF activities. Various members step up to work on policy, corporate, legal, treasury, and many other tasks that are needed in any corporation, always working with the relevant officers. Again, those who do the work decide how it gets done.

In these organizational tasks, it’s not always as clear how to reach consensus on contentious issues. Social issues or corporate governance questions don’t often have as simple a metric to measure against as technical issues do. Code either works or it doesn’t; corporate policy often has a very different kind of judgement. Similarly, with an active membership (with many very passionate individuals!), finding consensus on non-technical questions amongst 100+ individuals is often difficult. Here, hierarchy steps in to govern.

In most cases this works smoothly. Either the relevant officer will make a policy decision and publish it, or the board will pass a resolution or clearly ask for a change. While the whole community – the membership, the various PMCs of Apache projects – may not have a consensus on the decision, it is still policy, and projects will generally work to make the change.

In some cases, this doesn’t. A key way that the board ensures there is sufficient oversight of all Apache projects is by requiring reporting: monthly for officers; quarterly for PMCs / projects. There have been cases where PMCs have not reported regularly, or have not provided sufficient details to the board’s satisfaction in their reports. In these cases, the board will require a new or updated report. This is not a question of merit, or getting a consensus: this is an order. Non-response to a board request or not following an officer’s policy will and does result in the board stepping in to force changes within the project. As they say in politics, officers serve at the pleasure of the board.

It seems that sometimes this switch from meritocracy – where everyone in the PMC has an equal say, and they reach consensus – to hierarchy – where the board says “jump” – sometimes catches people by surprise. Sometimes, it seems that some PMC members just never really thought about the fact that the ASF is a legal corporation, and must follow certain rules. Other times, this surprise is clearly exacerbated by poor communication in specific cases between the board and projects. While cases like this really aren’t a choice – the board can and will dissolve or reconstitute projects (always as a last resort!) – it is important to clearly explain the issues and why this kind of hierarchy is needed in some cases.

Has anyone else had this feeling? Of switching between meritocracy, and consensus among the community, to hierarchy, and the board simply making a decision? How can we improve the handling of future issues like this, to ensure we keep the maximum meritocratic freedom within our projects, while still ensuring the legal shield of the ASF is secure and can function for the long term?


I’ve been thinking about this lately since I recently drew up a new org chart here for the ASF as a whole. Note that the hierarchy part only comes in on organizational, legal, and similar matters – technical decisions are always in the meritocratic hands of the PMCs and projects. More reading can be found on some of these excellent slides about Apache and open source.

Apache projects are technically independent

In the earlier essay I wrote on Apache project independence (since promoted to an article on the Apache Community Development website) I missed a critical bit of independence that I think we should add to the ComDev site.

Apache projects are technically independent

While Apache PMCs provide a report of their operations to the Board on a quarterly basis, they are technically independent of the Board. PMCs are free to collaboratively decide their project’s technical direction however they choose. While the Board holds PMC chairs and PMCs accountable for ensuring that their methods are appropriate – following the Apache Way and the few other rules at Apache – the Board does not exercise any technical direction over Apache projects.

This is a critical point to understand. There is no top-down direction at the ASF in terms of what technology we implement or what kinds of projects we seek out or support. All technical choices are made by the individual committers and PMC members within their project communities – i.e. the individuals actually doing the coding decide where the coding will go.

This applies equally to the Apache Incubator, as the gateway of new project proposals at Apache. The ASF has noagenda of what projects it wants to find, nor do we have a corporate strategy for new project acquisition. Project communities that want to come to the ASF are welcome – and will be reviewed in the Incubation process for community health and diversity, and for IP clearance and licensing. But not for technical merit.

The essence is: if there is a sufficiently diverse community who chooses to live by the Apache Way, the ASF is happy to host them. The fact that there is sufficient diverse community means that people are interested in the technology. Similarly, the Apache Attic awaits projects that may seem interesting technically, but fail to have a sufficiently diverse and active community to maintain them.

So – worthy to add to the ComDev page?