Lessons Learned Building Open Source Software
Having spent a significant amount of time conceptualizing and growing Vagrant into a decently successful open source project, I’ve come away learning quite a bit. I haven’t seen many blog posts about open source maintainers commenting on their lessons learned, so I’d like to share them here. These are not only engineering lessons, but lessons involving being an open source maintainer, promoting your project, etc.
Open Source Lessons
These are lessons that are generally applicable to open source projects.
Be friendly. This is the most important thing. You’re going to get bad ideas sent to you, people are going to get angry when something doesn’t work despite you providing your project for free, and you’re going to get terrible pull requests. Just remember that despite all this: these people are using your project, so be respectful, even if they may not be. I’ve only had one issue where I was starting to become clearly agitated, but I’m pretty proud to say that despite all that I think I remained friendly. It is important to remain friendly because as long as you are friendly you are approachable, and open source simply doesn’t work if people can’t approach you for help or contributions.
Don’t have crazy rules for contributions. Don’t worry about style, whitespace, proper indentation, etc. Unless your project is huge, complex rules for contributing only serves to hinder those people who want to contribute. Style, indentation, etc. is very quick to hand-fix. You should instead be grateful and accepting of the changes which are actually coming in, rather than any superficial attributes of it. So how do you contribute to Vagrant? Just fork the code, make your change, and pull request. I don’t care about style, tests, etc. I’m just happy to see contributions.
Documentation is key. I can’t show evidence for this, but I’m not exaggerating when I say that 100% of the first users of Vagrant who talked to me said they did so because it was so well documented. And at conferences I’ve heard more of the same. Another important thing is that writing documentation is probably the most boring thing in the world. If you put it off until the end and write in all in one go you risk becoming horribly depressed. So just slowly build up the documentation. Also, make your documentation forkable for easy contributions.
Have a clear method of communication. IRC, mailing lists, forums. It doesn’t matter what it is, but you need to have a clear way for users to get support or voice their opinions in a place that they can expect a response reasonably quickly (within 48 hours would be nice). For Vagrant, I’ve always had an IRC channel and mailing list. This has worked well. And if your users interface with you more, they gain a stronger trust for the project.
You don’t know everything. At some point, inevitably, you’ll get a feature request where you’ll say “this is useless” to yourself. The major responsibility of the project maintainer is to guide the vision of the project, not to micromanage specific features. Does the feature fit in with the vision? Is it useful for at least a couple people, even if it isn’t for yourself? Merge it. Guide the vision, be open minded, your users know what they want to see more than you do. But you know whether it fits in with the overall goal of the project better than anyone.
Marketing Lessons
So how do you get people to use your project? Here are some things I’ve learned in this space.
Hacker News. The hacker news community likes trying new things, and there are a lot of developers on there (hackers, if you would). So submit your project there, and comment in the thread saying you’re ready to answer any questions. Be friendly, because you will be criticized! Also, just some good tips: avoid submitting news when something crazy happens, submit around 1030 AM GMT-8 which is when HN is most active, don’t link your HN entry directly because upvotes won’t count, don’t try to game the system with fake accounts because you’ll get caught. When Vagrant was first released, it was #1 on Hacker News for most of the day. I received 11,000 unique page views that day (although some of that was from Reddit as well).
Reach out to popular niche blogs. In every community, especially in the Ruby community, there are niche blogs that are popular and like to report on cool projects in a specific language or field. Find these blogs, reach out to them, and get them to write about your project. This will result in two things: If they write about it, your project not only gets more eyeballs to look at it but your idea is better validated! This is the best way to get your early adopters.
Speak at meetups (before conferences). Find local meetups that would find your project interesting, and speak there. Practice speaking if you’re new to it. Don’t give a tutorial on your project, instead sell the crowd on the vision of the project, since they’re probably smart enough to do the tutorial you have with your amazing documentation (right?!). If you’re new to speaking, do not try to speak at conferences right away, because you will be bad, people will remember, and it will be that much harder for you to to ever speak again. Also, meetups give you a place to hone your vision and get important feedback from real people.
Speak at conferences. Start with regional conferences. Next, speak at regional conferences. These conferences are generally smaller but filled with good content, and people will be more forgiving if your talk doesn’t go so well. Also, the bigger conferences are unlikely to accept a talk about a project that is brand new. Now, you’re going to be in front of around 100 to 200 people and have 40 minutes. Also, you’re going to be recorded, so make sure you’re well prepared. Deliver your vision, smile, and take note of the feedback you receive. Use this to further hone your idea and stabilize your project.
Speak at big conferences. Now, I’m talking about conferences like VelocityConf or QCon. These guys will get you in front of a huge crowd of people (500+), and the audience will be extremely intelligent. Give a deeply technical talk that will challenge these folks, and again, sell your vision. Also, if your project is still relatively new, you should have examples of it being used successfully in the wild by people other than you in order to validate the whole thing. These big conferences attract generally important attendees (CIOs, VPs of Engineering, etc.) who will have a big say in getting your project in use at their big companies!
Engineering Lessons
Here are the handful of things I’ve learned about the programming side of things:
Test. I don’t think this has to be said, but since it is so important, I will include it. Tests are not something you can just bolt on to a project later. You must test early and test often. Also, don’t forget integration tests. I added integration tests to Vagrant much later, and they are by far the most valuable tests prior to release. Unit tests will catch my basic errors quickly, but integration tests will find major issues prior to a release.
Support Windows ASAP. Vagrant began supporting Windows pretty early, with John Bender doing much of the initial work. Despite Vagrant not being very feature-rich at the time, it was an absolute nightmare, since many of our dependencies didn’t work on Windows, and because there were many places in the code which expected Linux or simply called methods that didn’t work on Windows. I couldn’t imagine trying to bolt on Windows support later in the product cycle, since it can possibly require huge overhauls in your code base. Additionally, there are a significant number of Windows developers who want to use Linux-style tools. Instead of responding “Why don’t you switch to Linux?” I think its important to accommodate.
Avoid FFI. This is more of a Ruby thing. The Ruby FFI library is just not good for anything more than the simplest C libraries. I spent over 18 months working heavily with FFI. I’d even venture to guess I was one of the heaviest users of FFI. The issue is that the FFI library would routinely update and break, even in patch releases. More than a handful of times I awoke to find that Vagrant simply no longer worked on Windows due to FFI compilation issues, due to a patch release of FFI. Additionally, I found it extremely difficult to work with callbacks and dealing with memory management with FFI. Vagrant was one big memory leak until version 0.9. In the end, I ditched FFI until a better library exists, and now support C extensions again.
Become friends with the maintainers of your dependencies. Any heavy user of a library will find a bug in that library. Throughout the life of Vagrant, I’ve found a bug in every dependency. At this point, I’m reasonably good friends with the maintainers of all my dependencies, so I’m able to quickly ask “is this your bug? how long until its fixed? can you release a fix?” The worst possible output is that there is a bug in a dependency and the maintainer either won’t fix it or won’t push a release containing the fix.
In the end, I still have a lot to learn, but hopefully these tidbits will help some people out there when working on open source.
The Hardest, Most Rewarding Job I’ve Ever Had.
I just released Vagrant 1.0, exactly two years since I showed Vagrant to the world for the first time. I’ve made an official announcement but I think it is only appropriate to share a deeper, more personal story of the road travelled to reach this milestone, and what I’ve learned throughout the process. My goal in sharing is to give others an inside look at the guts of a successful open source project and perhaps offer a different point of view of the open source world.
Today is one of the proudest days of my life. I’ve released the first stable version of Vagrant, the software project I started with John Bender over two years ago while I was still in college. Vagrant is currently used by Mozilla, RackSpace, LivingSocial, Shopify, OpenStack, EventBrite, and many, many more. Both the project and the ideas behind the project have been far more successful than I could’ve ever dreamed of. But the road to this point is an interesting one, filled with highs and lows, and I’d like to share it with you.
The original idea for Vagrant came in 2009, when both John Bender and I were employed by Ruby development shops where it was routine to see a new project every 6 to 8 weeks. It was becoming increasingly frustrating to setup our development environments for every new project, which would be slightly different from previous projects. And it was absolutely infuriating when we had to go back to do maintenance on an old project since it was always a nightmare to get the environment back to that state. In the winter of 2009, I was frustrated enough to try to come up with a solution to this problem.
At this point in my life (and even still today), I viewed open source very romantically. People like Yehuda Katz and John Resig were my idols, because they did their work in the open passionately and successfully. I wanted nothing more than to find my own “jQuery” or “Ruby on Rails.” Basically: I was eager to start something which I thought would change the landscape of some field, just as jQuery and Rails did theirs. I don’t actually believe that Vagrant is as influential as jQuery or Ruby on Rails, but this gives you an idea of how I was thinking at the time.
In Janurary, 2010, I approached John Bender with some initial thoughts for what would eventually become Vagrant. John immediately saw value in the idea and offered to join in on the project, which I gladly accepted. We got started working immediately. It was January 21, 2010.
We both worked furiously, and within a week we had a working prototype that could bring up virtual machines we could SSH into. Below, you can see a screenshot from January 31, 2010, about a week after we started hacking, showing a functional version. Yes, Vagrant started life named “hobo.”
The initial development went fairly smoothly. Much of the essence of what Vagrant is today was molded during those initial weeks, and I’m happy to see that the ideas John and I had have been validated worldwide. Some are starting to show their age, but the fact that so many early decisions have lasted this long I think shows we had the right idea.
Fun fact: John came up with the entire idea of the “box” system about a week before the public release. Prior to that, I had planned to release Vagrant 0.1 only supporting a single Ubuntu image. The “box” system has been one of the most critical pieces in making Vagrant as successful as it is today, so my hat is off to John here.
While Vagrant was ready for release around mid-February, I was concerned that an open source project coming from two unknown/unproven developers would hurt our initial adoption. I decided that to be as successful as we could, we’d have to have amazing documentation and a mascot. Yes, a mascot was critical. I don’t know why, but I just find projects that have mascots to be more trustworthy. So, Vince was born (shown below), and I spent a week only working on documentation. I can’t stress how important this week was.
Vagrant 0.1.0 was released on March 7, 2010 with a mostly positive response. There were a handful of individuals who immediately grasped onto the idea and began using Vagrant with their projects right away. Most of these individuals still use it today, and deserve recognition for being so brave to adopt a new technology so early. Some of these early adopters even border on fanatical, creating things like Vagrant pins they distribute anywhere they go. I love this:

The first few months of Vagrant were reasonably uneventful. Besides the initial rush of early adopters, growth mostly stagnated, and each release of Vagrant was getting averaging around 100 downloads. While I loved the project and still whole-heartedly believed in it, seeing very little growth was hugely discouraging. At this point, I actually started to view Vagrant as a potential “failure.” Despite this, John and I and our respective companies used Vagrant every day and saw the value first hand. And I was still passionate about the project, so I decided to just keep going, believing that if this were truly a good idea, something good would happen.
And something good did happen, something great: Carl Lerche discovered Vagrant. At the time, Carl Lerche worked at Engine Yard and was a Ruby on Rails core developer. He also specifically pair programmed with Yehuda Katz. Carl popped in and out of the Vagrant IRC channel for a few weeks, asking for help and offering ideas here and there, and even contributed a few times. After a few weeks, he private messaged me. I don’t remember the exact words he used, but it was something along the lines of “How would you feel if Engine Yard sponsored Vagrant?” I vividly remember shaking with excitement at this point, despite the uncertainty. Engine Yard had long been known for being huge supporters of open source in the Ruby community, and backed important projects such as Ruby on Rails, JRuby, Rubinius, and a few more. I saw this potential sponsorship opportunity as huge idea validation as well as an outlet to better spread the word about Vagrant.
On October 14, 2010, Engine Yard announced that I had joined their OSS grant program. The specifics of the deal are private, but the basic idea is that they would help me in any way possible, as long as it was reasonable. This was a really exciting day because it was the first time with Vagrant that I could say “Mom and Dad, look! See! I told you I’m not just playing on my computer.” At this point I was still in college, as well, just to put things in perspective.
The Engine Yard sponsorship changed everything. Just having Engine Yard supporting me spurred a huge interest in Vagrant, and blasted Vagrant into “small-time popularity:”
The personal horsepower I put behind Vagrant went up to over 9000 at this point. Since I was still in college, I was spending 8 or more hours per day on Vagrant. At the same time, I was sending speaking proposals everywhere I could to educate people about Vagrant and try to gain some more interest in the project. I spoke at a handful of conferences, pushed many releases, and by March, 2011, the average page views per day on vagrantup.com had gone from around 200 to over 500. Success!
Unfortunately, this success came at a price: burnout. By March, although I refused to admit it for many more months to come, I was completely and utterly burnt out. This is clear to see from the release dates of various Vagrant versions: It took 6 months to release Vagrant 0.7.0 from Vagrant 0.6.0 (although there were various bug fix releases between). This was the lowest point in my personal involvement with the project. During these times, I would let bugs pile up to around 20 or 30 before triaging them all in one go. I’m not proud of this, but it was an important part of the history of Vagrant.
While I was burnt out and busy feeling sorry for myself, Vagrant only grew more and more popular. A great community built around Vagrant, a healthy set of plugins, and I gained a small fan club. It is the community that brought me back. I simply started getting more and more tweets, emails, etc. telling me how Vagrant had changed people’s lives, how they couldn’t imagine working before Vagrant, etc. I was flattered, and the praise was highly motivational. By the fall of 2011, I was working on Vagrant again, though not as much since I was out of college at that point and had a full time job.
In October, 2011, I travelled halfway around the world to Sweden for DevOpsDays, where I gave a talk on DevOps. It was here that something big happened: About 15 seconds into the talk, I introduced myself as the creator of Vagrant, in case anyone would recognize me that way. I thought maybe a handful would care, but instead the entire room, filled with around 200 people, erupted in applause, which you can hear in the video. This single act of kindness, again by the community, showed me just how much people cared about what I was doing, and motivated me even further. I consider this an extremely important moment in Vagrant history, and would be the first of many amazing events I’d witness in the following months.
Vagrant has been a full time job for the past 2 years. I work 8 hours at work not on Vagrant, and then spend at least 4 hours at home working on Vagrant, and typically also work on it on the weekends. I’m incredibly proud to finally ship a 1.0, and I’m proud of what the project has taught me and the community that has grown around it. I hope my story shows how much work, luck, and passion has gone into Vagrant. If I could go back in time, I wouldn’t change a thing, since as they say, “it’s all about the journey!”
I’m looking forward to see where this journey continues to take me.
And finally, last but not least, thank you so, so much to the Vagrant community and early supporters. Patrick Debois, Christian Trabold, Kieran Pilkington, and so, so many more: You make it a joy for me to work on Vagrant every day. Open source is all about the community. And, of course, thank you to Engine Yard for all their support, which continues to be critical in educating the world about Vagrant.
"New" Purely Functional Data Structures
This StackOverflow post covers “new” purely functional data structures created since Okasaki’s book “Purely Functional Data Structures” published in 1998.
“The Essence of Functional Programming” Part 1: The Identity Monad
This is the first of a new series of posts that will take papers that I read and will explain them and attempt to make them more concrete (and less theoretical) by showing real world examples using more industry-standard languages. These papers will be tagged with theory-in-practice. For more information on the “Theory in Practice” series, read the introductory post.
Today’s paper is “The essence of functional programming” by Philip Wadler. This paper is widely known as the paper which popularized monads, and introduces them in a relatively elementary context with a focus on practical programming use.
This is part 1 of a multi-part series. In order to allow the information to soak in and to not consume too much of your time, I’ve decided to split this into multiple parts which I will publish over the coming weeks. These will all be tagged with essence-of-functional-programming.
Part 1 will cover the inspiration for monads as well as introduce the trivial monad.
Prior knowledge required: Basic programming experience with basic experience in functional programming (in any language, whether it be Ruby, Python, C, etc.). The examples will be given in Haskell and Scala.
Theory in Practice
I’m starting a series of blog posts, published weekly, called “Theory in Practice.” These will take papers I read from the world of academia and attempt to reiterate them in simpler, more practical terms. One way I will do this is to take the examples, often in Haskell, and rewrite them in a more practical language, such as Scala. I’ll also add more real world applications, if needed.
Since my interests are currently in the world of programming languages and functional programming, I will start this series with a set of foundational papers in this subject area.
My goal with this series is to open the world of academic papers to a more general audience, as I’ve found that reading these papers has helped me in my day-to-day work greatly, and I hope that my blog posts will have a similar effect on others.
The first part the series will be on “The essence of functional programming” by Philip Wadler, and will be published this week.
Welcome to Ruby. We Have Many Rubies.
Although I consider myself (and I’m proud to be) a polyglot, most of my open source code is Ruby. I’ve been a Rubyist for almost 5 years now and it is still my language of choice for starting projects. Professionally, I’ve begun to use Python in the work-place, and in the process of learning Python I’ve had a hard time understanding the various Python implementations. Additionally, I know friends coming to Ruby who have trouble understanding the reasoning behind all the different Ruby implementations.
As such, I’m going to give a brief overview of the major Rubies available, their purpose, their development status, and my opinion of them. I’ll cover Matz’s Ruby (MRI or CRuby), Ruby Enterprise Edition (REE), JRuby, Rubinius (rbx), MacRuby, IronRuby, and MagLev.
The world of Rubies is unlike other languages, such as Python, where I would guess 95%+ use CPython exclusively. It is not uncommon for a single Ruby developer to use Rubinius at home, JRuby for production deployment, and MacRuby for desktop development.
CloudFormation: The Big Picture
Amazon announced CloudFormation to the public yesterday, and while the general opinion I could glean from various sources shows that people are excited about this new technology, many are still unsure what it is and how it fits into their current cloud workflow. I feel as though I have a firm grasp on CloudFormation and will attempt to answer some questions here.
Note: I’m definitely not a representative of Amazon in any way, and anything here is simply my educated opinion on the matter.
Flexible Local AWS Credentials Management
The Amazon Web Services(AWS) command line tools require certain environmental variables to be set up, such as EC2_HOME for EC2 and AWS_ACCESS_KEY_ID and so on. Most people throw this in their bashrc file and call it good, but this quickly becomes inflexible if you have to manage more and more accounts.
For example, I happen to manage 4 separate AWS accounts (my personal account and 3 other projects).
This post outlines how I manage my credentials in a DRY (Don’t-Repeat-Yourself) and flexible way, such that I could easily add more credentials in the future if I needed to.
Let’s Get Back to Business
Five years ago, and for about two years after that, I was an avid blogger. I maintained a personal blog that focused on PHP (specifically the Zend Framework) and a year later I launched a successful blog on Erlang, known as spawn_link. Both blogs had a readership of over 500 people, which, while not trying to brag, I was proud of.
Though both blogs brought multiple serious job offers, it was my personal blog which got me my first two jobs in the tech industry (note that I was only 17 when I started blogging!). My first job was at Zend as a contractor doing screencasts for the Zend Framework, and my second job, and still my current employer, was with Citrusbyte, a modern web development shop (which introduced me to the Ruby programming language).
Both jobs kept me busy, super busy, and the result was that I stopped blogging!
The past few years I’ve worked with some extremely smart people, had some unbelievable experiences, and have grown — in many ways — at a rapid speed. There is still so much out there I want to do and things to learn, but I think my experiences the past few years and ongoing deserve to be put down to “ink” for others to gain from.
And that’s my goal with this blog: Let’s get back to business, the business of giving back for everything that I’ve gained.


