What is Apache Hive? Website Branding Review

Website Brand Review of Apache Hive

While we’ve all heard about Apache Hadoop®, did you know there are over a dozen big data projects at Apache? We host projects that provide all the different functions your big data stack: databases, storage, streaming, logging, analysis, and more. Apache Hive™ is one of these pieces of the whole big data ecosystem.

Here’s my quick review of the Apache Hive project, told purely from the point of view of a new user finding the project website.

What Is Apache Hive?

“The Apache Hive ™ data warehouse software facilitates querying and managing large datasets residing in distributed storage”.

Continue reading What is Apache Hive? Website Branding Review

What is Apache Flex? Website Branding Review

Website Brand Review of Apache Flex

Many projects come to Apache from software vendors donating them to the Apache community, where the Apache Incubator works to form an open and independent community around the project. Here, Adobe donated both the code and the brand for their Flex project to Apache. Now, the ASF is the steward both to the vibrant Apache Flex community, as well as the new owner of the Flex brand and registered trademark.

Here’s my quick review of the Apache Flex project, told purely from the point of view of a new user finding the project website. While we’re all familiar with Adobe Flash browser plugin, not everyone may be familiar with the Flex environment for building Flash (and other!) applications.

What Is Apache Flex?

Apache Flex® is the open-source framework for building expressive web and mobile applications.

In other words, Flex is a toolkit for building general applications that can be run on a variety of web browsers and mobile platforms that include the Adobe Flash or Adobe AIR runtimes or application containers. Flex is the coding language and environment you use to write applications for the Flash/AIR containers.

No, Really, What Is Apache Flex For?

Continue reading What is Apache Flex? Website Branding Review

What is Apache HBase? Website Branding Review

Website Brand Review of Apache HBase

How do open source projects get popular? By providing some useful functionality that users want to have. How do open source projects thrive over the long term? By turning those users into contributors who then help improve and maintain the project. How well a project showcases themselves on the web is an important part of the adoption and growth cycle.

Here’s my quick review of the Apache HBase project, told purely from the point of view of a new user finding the project website. HBase is a key part of the big data storage stack, so although you may not work directly with it, it’s probably underlying some systems you use.

What Is Apache HBase?

“Apache HBase™ is the Hadoop® database, a distributed, scalable, big data store”.

Continue reading What is Apache HBase? Website Branding Review

What Is Apache Mesos? Website Branding Review

Website Brand Review of Apache Mesos

How do open source projects get popular? By providing some useful functionality that users want to have. How do open source projects thrive over the long term? By turning those users into contributors who then help improve and maintain the project. How well a project showcases themselves on the web is an important part of the adoption and growth cycle.

Here’s my quick review of the Apache Mesos project, told purely from the point of view of a new user finding the project website. Mesos is turning into a major project in the big data and cloud space; not perhaps the obvious popularity of Apache Spark yet, but certainly big.

What Is Apache Mesos?

Apache Mesos abstracts CPU, memory, storage, and other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively.

Continue reading What Is Apache Mesos? Website Branding Review

Apache CMS: Adding static data tables easily?

Is there an easy way to add simple generated data tables from CSV or the like using the Apache CMS system for the apache.org website? I.e. I want to checkin a CSV (or other simple table of data) that certain committers can edit via a spreadsheet, and then display selected rows from that table on an apache.org/foundation/ webpage in some semi-pretty manner.

Did you know that the ASF has their own CMS / static generator / magic update system that runs the apache.org homepage and many Apache project homepages? While it’s more of an Apache infra tool rather than a full Apache top level project, it’s still a full service solution for allowing multiple static website builds that are integrated into our servers.

While there are plenty of great technical CMS systems, when choosing a system for your company, many of the questions are organizational and deployment related. How easy is it for your IT team to manage the core system? How easy is it for various teams (or projects) to store and update their own content, perhaps using different templates in the system? How can you support anonymous editing/patch submission from non-committers? Does it support a safe and processor-respectful static workflow, minimizing the load on production servers while maximizing backups? And how can you do all this with a permissive license, and only hosting your own work?

The Apache CMS – while a bit crufty – supports all these things (although the infra peeps might argue about the maintenance part!) Everything’s stored in SVN, so restoring a backup or bringing the production server back is just checking the tree out again. Many projects use a Markdown variant, although some projects configure in their own static generator tools. The web GUI, while sparse, does have a great tutorial for submitting anonymous patches to Apache websites.

My question is: what’s the simplest way to have an apache.org top level webpage pull in some sort of simple data source? In particular, I don’t want to have to maintain much code, and I only want to add this data table bit within an existing page, without having to run my own whole generation script.

The first specific use case is displaying /foundation/marks/list/registered, a normal a.o page that will display a data table of all the registered trademarks the ASF owns. I’ll checkin a CSV that I get from our counsel that includes all the legal details of our trademarks.

Bonus points for a simple system that:

  • Can pull some columns from a separate table: namely, projects.a.o descriptions from the projects.
  • Can pull my CSV listing trademark numbers from a private repo (committers or foundation).
  • Uses Python or JS and not Perl.

Note: I have cut back my $dayjob recently, so I will actually have time to write some of the code for this work myself now – finally!

What Is Apache Spark? Website Branding Review

Volunteering at the ASF and elsewhere in open source, I think a lot about open source brands. In particular: how do various open source projects – run by a wide variety of typically very geeky volunteers – present themselves publicly to new users? We sometimes spend so much time working on the great new code – and explaining it to other developers we already know – that sometimes I wonder if we’re really showcasing what our great new code can do for new users and contributors.

Here’s my quick review of the Apache Spark project, told purely from the point of view of a new user who just came to the project website. I’m trying to show what I think someone new to the project might think about the project once they get to the homepage. Since Spark is a major project in the big data space, there are a lot of search hits for Spark, including a wide variety of other software vendors.

Continue reading What Is Apache Spark? Website Branding Review

ApacheCon Big Data/Core News Wrapup

Our annual Apache:Big Data and ApacheCon:Core events were held recently at the lovely Corinthia Hotel Budapest, and the content and attendees were amazing.  The weather was great too, and sightseeing and shopping in Budapest were lovely.  Attendance was still good even in the face of time-competing software conferences and the local refugee crisis happening in the region.

While they were booked as separate events, many people stayed for the whole week.  Going forward, we will likely have a single event, but be even clearer with the strength of content in specific track days.  The broad array of very deep and well-received technical content in the big data space was truly impressive; Apache has over a dozen big data related projects and probably 20 more incoming Incubator podlings, so we certainly have the space covered!

We got some great press coverage and a few independent blog posts with key events at ApacheCon Budapest this year:

Overall, ApacheCon is always a good week for me, but this year it was exceptional. The Corinthia was as lovely as ever, and I finally had time to really take a walk and shop in the central market in Budapest. Plus, Thursday was a special day for me, and somehow everyone at the conference (including the hotel staff) found out, and was wishing me well. Many thanks to the friends who took me to an authentic Hungarian restaurant for dinner! Even the gypsy band playing a version of “Happy Birthday” was fun, and I’m glad I got to bring home the music of Norbert Salasovics!

Our conference producer the Linux Foundation has been really improving how we organize our CFP and put together highly focused tracks on a variety of Apache projects.  While it’s hard to put a spotlight on all 200+ projects and initiatives at the ASF, expect to see even better organized content and talks in the ApacheCon to come, with full in-depth tracks on key technologies – along with excellent community and “how does Apache do it all” advice to boot.

Slides for all talks and videos for keynotes should be posted on the event archive websites:

Many of our speakers use Slideshare as well, and the Apache Community Development project has a separate listing of some key Apache Way slides.

Stay tuned for the CFP for ApacheCon North America, which will be returning to Vancouver, Canada on 9-13 May 2016. Hope to see you there!

Congratulations to the 2015 Apache Board of Directors

The ASF recently held it’s Annual Member’s Meeting where all Members of the Foundation cast ballots in the annual election for the Board. We are lucky to have had a number of excellent candidates for the board as always.

The new board comprises:

  • Rich Bowen
  • Shane Curcuru
  • Bertrand Delacretaz
  • Jim Jagielski
  • Chris Mattmann
  • David Nalley
  • Brett Porter (chairman)
  • Sam Ruby
  • Greg Stein

I also keep a graphical history of the ASF board.  The graphic there is a great way to see the slow but steady progress of electing new faces to the board over time.  Thanks to all the active Members who voted in the elections!

As the ASF grows in projects, communities, and Members, we’re looking forward to continuing to support our now 165+ top level Apache projects going forward!

Note that a number of new Apache Member nominees were also elected; however we don’t share their names until they’ve all been contacted and have accepted the invitation.  Stay tuned in a month for that announcement from @TheASF.

Shane’s Apache Director Position Statement 2015

The ASF is holding it’s annual Member’s Meeting this week to elect a new board and a number of new Members to the ASF.  I’m honored to have been nominated to stand for the board election, and I’m continuing my tradition of publicly posting my vision for Apache.

We are lucky to have a large roster of excellent director candidates, so no matter how the election turns out we’ll have a stellar board. Given the wide variety of opinions in our candidates, I urge all Apache members to set aside the time this week to carefully consider all the board candidates, as well as all the great new Member nominees.  Please vote – and if you’re not free this week, be sure to assign your proxy for the meeting attendance: I and several other Members are happy to proxy for you.

Please read on for my take on what’s important for the ASF’s future…

Continue reading Shane’s Apache Director Position Statement 2015

How Apache *really* works

How much do you know about the Apache Software Foundation (ASF) and the many Apache projects we host? Did you know we’re holding our annual Members meeting to elect our board of directors and new Members in just a few days?

I’m often surprised by the variety of basic questions and misunderstandings I hear in the software world about how the ASF really works. We’ve written plenty of documentation about the Apache Way and our governance, but let’s try a different approach. I’d like to interview myself to try to explain some things.

So, Shane, what *is* Apache? I thought it was that web server?

The ASF is a non-profit, public charity, 501(c)3 membership corporation with the mission of producing software for the public good. The Apache HTTP Server project (to use it’s formal name) is a project community at the ASF that creates the httpd web server, which has powered more active websites than any other server since 2000.

The ASF is the corporation that provides legal, branding, press, fundraising, and infrastructure support, and proven community mentoring to the many Apache projects like the HTTP Server. Think of the ASF as a great big house, where we provide shelter for a lot of different families that write open source software.

Well how many Apache projects are there?

We have over 165 different projects, and about 40 podlings. These 200+ project communities create a wide variety of software products, including Apache Hadoop, Apache Lucene, Apache OpenOffice, Apache CloudStack, and many, many more.

You are almost certainly using multiple Apache products right now as you read this. You may not realize it, but much of the plumbing of the internet uses Apache software to keep servers organized and connected. Most browsers use various Apache products under the hood for a wide variety of utility functions. It is our project communities that actually create the software you’re using — the ASF just helps keep them organized.

How does the ASF organize all these projects?

The ASF provides all the infrastructure an open source project needs: websites, code repositories, mailing lists, bugtracking services, a crack infrastructure team. We also provide all the rest of the services that a project will want, like legal support, access to press releases or analyst contacts, and some fundraising support. The ASF also owns all Apache trademarks on behalf of our projects, to ensure they get the credit they deserve.

Most importantly, the Apache Membership and many of our 4,000+ Apache committers provide the community mentoring and support to keep our projects running smoothly, with an independent project governance. We have many passionate Members with amazing experience in making open souce projects work, and they volunteer to help keep our projects healthy and running strong.

But this is mentoring and guidance, not direction. The ASF does not direct the technical direction of our projects. We let the people doing the work — the project committers and Project Management Committees (PMCs) decide where the code should go.

So the projects direct themselves. But what is “independent project governance“? How do you enforce it?

A critical behavior for any Apache project is independent governance. That means that every project manages their code for the benefit of all users (the public good), and not just for some company or vendor. In particular, the ASF and Apache projects only recognize individuals as committers or Members — never companies.

We expect when committers are working within their Apache project, they are acting for the best interests of the project itself. But we also have checks and balances: all Apache projects report formally to the board of directors quarterly. The board reviews project health — are they acting indepenently, are they publishing software releases, are they voting in new committers. If the board sees behavior that does not show mature Apache project behavior, the board will work within that project community to help the project community correct itself. Many Apache Members also volunteer to mentor our projects in these cases. In extreme cases (very rare), where a project does not follow the Apache Way, the board will unilaterally make changes to correct their course.

Can you clarify who are the board, the Members, and how they relate to projects? Are Members part of all projects?

Imagine Apache as a condominium association with multiple condos together. The ASF as a corporation provides the building. Like some condo associations, we also define a few expected behaviors and appearances for all the condos we offer. We also offer bonus services, like help moving into your condo or fixing things up. Each Apache project lives in one of these condos. We’re happy for you, the project community, to live your own lifestyle within your condo and paint the inside whatever color you want, as long as your public behaviors when you’re here follow our community best practices.

Here, the Apache board is the board of the condo association. They set the core rules and guidelines for the building. The Membership are sort of the owners of the building — not that they can ever sell their shares or make a profit, but they are the only ones who can nominate and elect the board. The board appoints all the officers who set detailed policies and make all the operations of the building work, like trash pickup and elevator maintenance.

Every Apache project condo has at least one Apache Member involved with the community: the Incubator requires that every new project has a few Members interested in that community to help mentor it. But within each project condo, the code direction or decor choice is completely up to the whole project community to decide. Membership in Apache is not transitive to any project: Members need to be elected to your project to have a direct say in it.

The ASF offers a lot of services to projects. How does this all get paid for?

The ASF board approves an annual corporate budget of about one million dollars. Our primary income is from our formal Sponsorship program, where organizations can provide a regular annual donation. As a 501(c)3 charity, we also have many individual donors, and some authors donate royalties from their books about Apache software to the ASF.

Importantly, sponsorship of the ASF does not provide any influence over Apache operations nor the operations of any projects. The ASF’s mission is to serve the public good, and while we very much appreciate our generous sponsors, we do not serve them: we serve our project communities and software users.

Sponsors provide a variety of reasons why they sponsor the ASF, many of which relate to how we host so many different critical software product communities. As one sponsor said, “Apache builds the plumbing of the internet”. Some sponsors and donors simply want to give back to the ASF in appreciation for all the software we provide for free.

So the Sponsors can pay for Apache project development, interesting.

No! Sponsorship funds are purely undirected — we do not accept donations with ties or requirements. By policy, the ASF does not pay for core development on any Apache project. All our budget is used for the support services that allow our project communities to do their work — which is building Apache software.

Do any Apache committers get paid for their work?

Of course — but not by the ASF. Many committers are working on Apache projects on behalf of their employers, who may be software vendors providing support, hosting, or add on products for that Apache project. Some committers are independent consultants, trainers, authors, or the like, who make their own living from helping other people use Apache projects. And a lot of committer work is done simply because that person needs to fix a bug or add a feature that they need for themselves. With so many people using Apache software to run their businesses, most work is self-serving: building code that they need.

The ASF provides a vendor-neutral place where everyone who benefits from Apache software can collaborate to improve that software. The ASF does not have an agenda or direction — we rely on the people using our software to help improve it.

Well… how much do *you* get paid at Apache?


No, seriously — how much do you get paid for this?

Seriously: nothing. Zero. Zip. The ASF has never paid me for my work here, and my current dayjob is wholly unrelated to my Apache activities. I’m here purely as an unpaid volunteer.

How did you get to be at Apache? What drives you to do all this unpaid work?

I first started committing code to the newly formed Apache Xalan project shortly after the ASF was incorporated in November 1999. At that time, I was paid by Lotus/IBM, my employer, to contribute to the Xalan as part of my job. I also got to attend and speak at a few ApacheCons to try to promote our work on Xalan and Xerces.

Over time, my dayjob changed direction, and turned away from Apache, but I was still interested in how the ASF worked. At ApacheCon I had followed a friend into a conference planning meeting to see how it worked, and I walked out of the meeting with assigned tasks for the next ApacheCon. Once I started helping with events, I was hooked. In 2002 I was elected as a Member of the ASF, and got to see how the sausage was made from the inside. In 2004 my job changed to be wholly unrelated to any open source work, but I was already personally invested in the ASF and our many excellent communities.

I volunteer at Apache for several reasons:

  • This is how I give back to the world. I’m lucky enough to have a healthy family, nice home, and stable job. I volunteer my extra time to help make it easier for Apache project communities to build more free software for the public good.
  • I love the ASF and it’s people. I’ve met so many amazing people at ApacheCon and within our projects, and it feels like much of the Apache Membership is one big family. Sure, we fight plenty, but we also buy each other plenty of free beers and meals.
    Helping open source communities get organized and keep their volunteers motivated is something I’m good at, and something I’d love to do more of if I could. Volunteering at Apache is a huge impact to my personal brand and my future job prospects.

Wow, that’s been a great interview Shane! How should we wrap this up?
It’s been a pleasure. Thinking about what motivates me, this is one of the things I love doing: explaining technology and communities to interested audiences. This is also great timing for this interview, because the ASF is having it’s annual corporate meeting where the Membership elects a new board.

We have a truly stellar list of director nominees this year: looking at the candidates the Membership has nominated shows just how talented and friendly all our candidates are, and how any of them would be a help to ensuring the smooth operations of the ASF in the year ahead.

Since I do have the microphone, I will make one short plug to note that I’m also running for a seat on the board. I’ll be posting my director nomination statement — a note detailing why I hope Members will vote for me — soon here on my open source blog, Community Over Code.

Good luck with the election Shane — it sounds like Apache is in good hands no matter who gets on the board.

Yup. While we still have a way to go to make it simple to understand Apache for newcomers, the ASF and our project communities are doing amazing work, and often having a lot of fun doing it. Apache plans to be around for the next 50 years providing a stable home for like-minded project communities of all sorts.

Thanks for the interview — this was great to talk to someone about this!

Shane volunteers as the Vice President of Brand Management for the ASF, although all content here is his own personal opinion. He is not normally an interviewer, but does sometimes play a trademark lawyer on the internet. He hopes you liked this article, which also appears on Medium, and will ask him any questions about Apache that you might have. He promises to stop writing in the third person now.