You probably use contribute to several Apache projects. But do you know what goes on behind the scenes at the ASF? Besides all the work of the 200+ project communities, the ASF has an annual budget of about one $million USD to fund the services our projects use. How we manage providing these services – and governing the corporation behind the projects – continues to change and improve.
Juggling several speaking engagements coming up, I’m reminded of how hard the job of conference organizers is. Having helped to run ApacheCon as part of a volunteer team for years, I know how hard it is selecting talks, wrangling speaker acceptances (and rejections), and ensuring your final conference schedule is appealing. And wrangling your clunky CFP system and keeping the finicky schedule website updated are two problems that software hasn’t solved yet.
Equally important is how the conference acceptance & organization process works from the speaker’s side. Remember? Those people who make all the content your conference relies on? All those people who you love and appreciate – but don’t who you don’t pay anything – and who you’ll do anything to fix last minute problems for? While we can’t prevent all the last minute problems, there are a few simple steps to improve the speaker communication process to help prevent problems.
Website Brand Review of Apache Hadoop
We’ve all heard of Apache® Hadoop® – well, at least heard of Hadoop, and by now you should realize it’s an Apache project! But when was the last time you took a critical eye to the actual Apache Hadoop project’s homepage?.
Here’s my quick review of the Apache Hadoop project, told purely from the point of view of a new user finding the project website.
What Is Apache Hadoop?
“Apache Hadoop (is) a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models”
“Hadoop is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures.”
There’s a huge amount of volunteer energy that flows around Apache’s Annual Member Meeting every year. Old members and new alike come together and brainstorm all sorts of new ideas, both organizational and technical – and we have plenty of online… discussions, let us say. There is an amazing amount of energy from a lot of very smart people, and when we focus this energy, we make real improvements to the Foundation and sometimes in some of our projects.
As we’ve grown, keeping a full shared understanding of all the details of membership and corporate operations has become much harder. We have some documentation, but we also still have a lot of tribal knowledge and decisions hidden in our mailing list archives. To understand the same things, we need to be able to see what rules or policies we’ve actually decided on – or at least written down.
So here is an overview of all the different roles that people can have with the ASF as either a Foundation or with specific Apache projects. In particular, I’m focusing on the specific agreements we make with individuals, or the explicitly posted policies that we expect people to abide by. For more information on how Apache works, see /dev, /governance, and Community.
The ASF is holding it’s annual Member’s Meeting this week to elect a new board and a number of new Members to the ASF. I’m honored to have been nominated to stand for the board election, and I’m continuing my tradition of publicly posting my vision for Apache each year.
We are lucky to have both a large involved membership, as well as another excellent slate of candidates including a couple of great new faces. No matter how Apache STeVe ends up computing the results, Apache will have a great board for the year to come.
Please read on for my take on what’s important for the ASF’s future…
Our annual Apache:Big Data and ApacheCon:Core events were held recently at the lovely Corinthia Hotel Budapest, and the content and attendees were amazing. The weather was great too, and sightseeing and shopping in Budapest were lovely. Attendance was still good even in the face of time-competing software conferences and the local refugee crisis happening in the region.
While they were booked as separate events, many people stayed for the whole week. Going forward, we will likely have a single event, but be even clearer with the strength of content in specific track days. The broad array of very deep and well-received technical content in the big data space was truly impressive; Apache has over a dozen big data related projects and probably 20 more incoming Incubator podlings, so we certainly have the space covered!
We got some great press coverage and a few independent blog posts with key events at ApacheCon Budapest this year:
- Notes from the keynote: Apache Big Data & ApacheCon Core Europe 2015 | Open Source Insider
- Apache’s Rich Bowen explains the ‘Apache Way’ | IDG Connect
- Apache retains an informal vibe to manage $1m of open-source projects | IT Analysis from V3.co.uk
- Apache community looks to address big data’s ‘unicorn’ problem | IT News from V3.co.uk
- IBM’s Anjul Bhambhri on Apache Spark, women in tech, and Watson | IDG Connect
- Apache Trafodion DBMS Innovation for Big Data with SQL access to Hadoop | Bloor Research
- Hortonworks’ Murthy: Hadoop’s next mission is to be more business friendly | ZDNet
- Talked HTTP/2 at ApacheCon | daniel.haxx.se
- Highlights from Apache Big data Conf | Squid Solutions
Overall, ApacheCon is always a good week for me, but this year it was exceptional. The Corinthia was as lovely as ever, and I finally had time to really take a walk and shop in the central market in Budapest. Plus, Thursday was a special day for me, and somehow everyone at the conference (including the hotel staff) found out, and was wishing me well. Many thanks to the friends who took me to an authentic Hungarian restaurant for dinner! Even the gypsy band playing a version of “Happy Birthday” was fun, and I’m glad I got to bring home the music of Norbert Salasovics!
Our conference producer the Linux Foundation has been really improving how we organize our CFP and put together highly focused tracks on a variety of Apache projects. While it’s hard to put a spotlight on all 200+ projects and initiatives at the ASF, expect to see even better organized content and talks in the ApacheCon to come, with full in-depth tracks on key technologies – along with excellent community and “how does Apache do it all” advice to boot.
Slides for all talks and videos for keynotes should be posted on the event archive websites:
Stay tuned for the CFP for ApacheCon North America, which will be returning to Vancouver, Canada on 9-13 May 2016. Hope to see you there!
The ASF recently held it’s Annual Member’s Meeting where all Members of the Foundation cast ballots in the annual election for the Board. We are lucky to have had a number of excellent candidates for the board as always.
The new board comprises:
- Rich Bowen
- Shane Curcuru
- Bertrand Delacretaz
- Jim Jagielski
- Chris Mattmann
- David Nalley
- Brett Porter (chairman)
- Sam Ruby
- Greg Stein
I also keep a graphical history of the ASF board. The graphic there is a great way to see the slow but steady progress of electing new faces to the board over time. Thanks to all the active Members who voted in the elections!
As the ASF grows in projects, communities, and Members, we’re looking forward to continuing to support our now 165+ top level Apache projects going forward!
Note that a number of new Apache Member nominees were also elected; however we don’t share their names until they’ve all been contacted and have accepted the invitation. Stay tuned in a month for that announcement from @TheASF.
The ASF is holding it’s annual Member’s Meeting this week to elect a new board and a number of new Members to the ASF. I’m honored to have been nominated to stand for the board election, and I’m continuing my tradition of publicly posting my vision for Apache.
We are lucky to have a large roster of excellent director candidates, so no matter how the election turns out we’ll have a stellar board. Given the wide variety of opinions in our candidates, I urge all Apache members to set aside the time this week to carefully consider all the board candidates, as well as all the great new Member nominees. Please vote – and if you’re not free this week, be sure to assign your proxy for the meeting attendance: I and several other Members are happy to proxy for you.
Please read on for my take on what’s important for the ASF’s future…
How much do you know about the Apache Software Foundation (ASF) and the many Apache projects we host? Did you know we’re holding our annual Members meeting to elect our board of directors and new Members in just a few days?
I’m often surprised by the variety of basic questions and misunderstandings I hear in the software world about how the ASF really works. We’ve written plenty of documentation about the Apache Way and our governance, but let’s try a different approach. I’d like to interview myself to try to explain some things.
So, Shane, what *is* Apache? I thought it was that web server?
The ASF is a non-profit, public charity, 501(c)3 membership corporation with the mission of producing software for the public good. The Apache HTTP Server project (to use it’s formal name) is a project community at the ASF that creates the httpd web server, which has powered more active websites than any other server since 2000.
The ASF is the corporation that provides legal, branding, press, fundraising, and infrastructure support, and proven community mentoring to the many Apache projects like the HTTP Server. Think of the ASF as a great big house, where we provide shelter for a lot of different families that write open source software.
Well how many Apache projects are there?
We have over 165 different projects, and about 40 podlings. These 200+ project communities create a wide variety of software products, including Apache Hadoop, Apache Lucene, Apache OpenOffice, Apache CloudStack, and many, many more.
You are almost certainly using multiple Apache products right now as you read this. You may not realize it, but much of the plumbing of the internet uses Apache software to keep servers organized and connected. Most browsers use various Apache products under the hood for a wide variety of utility functions. It is our project communities that actually create the software you’re using — the ASF just helps keep them organized.
How does the ASF organize all these projects?
The ASF provides all the infrastructure an open source project needs: websites, code repositories, mailing lists, bugtracking services, a crack infrastructure team. We also provide all the rest of the services that a project will want, like legal support, access to press releases or analyst contacts, and some fundraising support. The ASF also owns all Apache trademarks on behalf of our projects, to ensure they get the credit they deserve.
Most importantly, the Apache Membership and many of our 4,000+ Apache committers provide the community mentoring and support to keep our projects running smoothly, with an independent project governance. We have many passionate Members with amazing experience in making open souce projects work, and they volunteer to help keep our projects healthy and running strong.
But this is mentoring and guidance, not direction. The ASF does not direct the technical direction of our projects. We let the people doing the work — the project committers and Project Management Committees (PMCs) decide where the code should go.
So the projects direct themselves. But what is “independent project governance“? How do you enforce it?
A critical behavior for any Apache project is independent governance. That means that every project manages their code for the benefit of all users (the public good), and not just for some company or vendor. In particular, the ASF and Apache projects only recognize individuals as committers or Members — never companies.
We expect when committers are working within their Apache project, they are acting for the best interests of the project itself. But we also have checks and balances: all Apache projects report formally to the board of directors quarterly. The board reviews project health — are they acting indepenently, are they publishing software releases, are they voting in new committers. If the board sees behavior that does not show mature Apache project behavior, the board will work within that project community to help the project community correct itself. Many Apache Members also volunteer to mentor our projects in these cases. In extreme cases (very rare), where a project does not follow the Apache Way, the board will unilaterally make changes to correct their course.
Can you clarify who are the board, the Members, and how they relate to projects? Are Members part of all projects?
Imagine Apache as a condominium association with multiple condos together. The ASF as a corporation provides the building. Like some condo associations, we also define a few expected behaviors and appearances for all the condos we offer. We also offer bonus services, like help moving into your condo or fixing things up. Each Apache project lives in one of these condos. We’re happy for you, the project community, to live your own lifestyle within your condo and paint the inside whatever color you want, as long as your public behaviors when you’re here follow our community best practices.
Here, the Apache board is the board of the condo association. They set the core rules and guidelines for the building. The Membership are sort of the owners of the building — not that they can ever sell their shares or make a profit, but they are the only ones who can nominate and elect the board. The board appoints all the officers who set detailed policies and make all the operations of the building work, like trash pickup and elevator maintenance.
Every Apache project condo has at least one Apache Member involved with the community: the Incubator requires that every new project has a few Members interested in that community to help mentor it. But within each project condo, the code direction or decor choice is completely up to the whole project community to decide. Membership in Apache is not transitive to any project: Members need to be elected to your project to have a direct say in it.
The ASF offers a lot of services to projects. How does this all get paid for?
The ASF board approves an annual corporate budget of about one million dollars. Our primary income is from our formal Sponsorship program, where organizations can provide a regular annual donation. As a 501(c)3 charity, we also have many individual donors, and some authors donate royalties from their books about Apache software to the ASF.
Importantly, sponsorship of the ASF does not provide any influence over Apache operations nor the operations of any projects. The ASF’s mission is to serve the public good, and while we very much appreciate our generous sponsors, we do not serve them: we serve our project communities and software users.
Sponsors provide a variety of reasons why they sponsor the ASF, many of which relate to how we host so many different critical software product communities. As one sponsor said, “Apache builds the plumbing of the internet”. Some sponsors and donors simply want to give back to the ASF in appreciation for all the software we provide for free.
So the Sponsors can pay for Apache project development, interesting.
No! Sponsorship funds are purely undirected — we do not accept donations with ties or requirements. By policy, the ASF does not pay for core development on any Apache project. All our budget is used for the support services that allow our project communities to do their work — which is building Apache software.
Do any Apache committers get paid for their work?
Of course — but not by the ASF. Many committers are working on Apache projects on behalf of their employers, who may be software vendors providing support, hosting, or add on products for that Apache project. Some committers are independent consultants, trainers, authors, or the like, who make their own living from helping other people use Apache projects. And a lot of committer work is done simply because that person needs to fix a bug or add a feature that they need for themselves. With so many people using Apache software to run their businesses, most work is self-serving: building code that they need.
The ASF provides a vendor-neutral place where everyone who benefits from Apache software can collaborate to improve that software. The ASF does not have an agenda or direction — we rely on the people using our software to help improve it.
Well… how much do *you* get paid at Apache?
No, seriously — how much do you get paid for this?
Seriously: nothing. Zero. Zip. The ASF has never paid me for my work here, and my current dayjob is wholly unrelated to my Apache activities. I’m here purely as an unpaid volunteer.
How did you get to be at Apache? What drives you to do all this unpaid work?
I first started committing code to the newly formed Apache Xalan project shortly after the ASF was incorporated in November 1999. At that time, I was paid by Lotus/IBM, my employer, to contribute to the Xalan as part of my job. I also got to attend and speak at a few ApacheCons to try to promote our work on Xalan and Xerces.
Over time, my dayjob changed direction, and turned away from Apache, but I was still interested in how the ASF worked. At ApacheCon I had followed a friend into a conference planning meeting to see how it worked, and I walked out of the meeting with assigned tasks for the next ApacheCon. Once I started helping with events, I was hooked. In 2002 I was elected as a Member of the ASF, and got to see how the sausage was made from the inside. In 2004 my job changed to be wholly unrelated to any open source work, but I was already personally invested in the ASF and our many excellent communities.
I volunteer at Apache for several reasons:
- This is how I give back to the world. I’m lucky enough to have a healthy family, nice home, and stable job. I volunteer my extra time to help make it easier for Apache project communities to build more free software for the public good.
- I love the ASF and it’s people. I’ve met so many amazing people at ApacheCon and within our projects, and it feels like much of the Apache Membership is one big family. Sure, we fight plenty, but we also buy each other plenty of free beers and meals.
Helping open source communities get organized and keep their volunteers motivated is something I’m good at, and something I’d love to do more of if I could. Volunteering at Apache is a huge impact to my personal brand and my future job prospects.
Wow, that’s been a great interview Shane! How should we wrap this up?
It’s been a pleasure. Thinking about what motivates me, this is one of the things I love doing: explaining technology and communities to interested audiences. This is also great timing for this interview, because the ASF is having it’s annual corporate meeting where the Membership elects a new board.
We have a truly stellar list of director nominees this year: looking at the candidates the Membership has nominated shows just how talented and friendly all our candidates are, and how any of them would be a help to ensuring the smooth operations of the ASF in the year ahead.
Since I do have the microphone, I will make one short plug to note that I’m also running for a seat on the board. I’ll be posting my director nomination statement — a note detailing why I hope Members will vote for me — soon here on my open source blog, Community Over Code.
Good luck with the election Shane — it sounds like Apache is in good hands no matter who gets on the board.
Yup. While we still have a way to go to make it simple to understand Apache for newcomers, the ASF and our project communities are doing amazing work, and often having a lot of fun doing it. Apache plans to be around for the next 50 years providing a stable home for like-minded project communities of all sorts.
Thanks for the interview — this was great to talk to someone about this!
Shane volunteers as the Vice President of Brand Management for the ASF, although all content here is his own personal opinion. He is not normally an interviewer, but does sometimes play a trademark lawyer on the internet. He hopes you liked this article, which also appears on Medium, and will ask him any questions about Apache that you might have. He promises to stop writing in the third person now.
Open source has come a long way in the past 30 years and is entering the consciousness of most modern cultures. When thinking of open source projects, people categorize them several ways: governance structure, type of product platform, programming language, utility, technical details (language written in), industry sponsored or fully independent, and more.
But what truly defines any open source project, making it a unique entity different from all other open source projects? I would propose that there are three key elements of any open source project that frame, define, and differentiate that project from all others: the code, the community, and the brand.