Thinking about making your project Open Source?

Thu, Feb 20, 2014 at 4:35PM

You may find that you have to do a number of extra things when making your software able to be distributed to others.  Perhaps the biggest ones are:

  • Documentation and automation for installing / upgrading the software.
  • Improve security so you don't cause yourself more problems.
  • Add more training and documentation resources for both developers and end users.
  • Consider building an API to allow easier integration with your software.
  • Learn how to build and communicate with a developer community

My clients never really ask for these things, so I had to work on them as I could over time.  These areas are generally weaker since they have had less attention to them, but I hope to change that.

Other developers have different requirements to use the software then what non-technical clients need, so you have to do more work for free to make it a good open source project before you gain the support of other developers.

Many web developers aren't really in the habit of building software, but instead hack together many things in a rush to build the solution that will satisfy the client requirements most quickly.   If you instead focus on building a single software package to solve these same requirements, you'll often find yourself repeating yourself less frequently and more able to deliver a high quality result at greater profit.   It's hard to incrementally move in this direction, so it's often better to adopt other existing software if you aren't going to commit to building things correctly.  But if you believe in your vision, you can do the extra work, and carve out a niche for yourself among the competition.

Add Install / Upgrade Support

For installation and upgrades, I choose to automate as much as possible in Jetendo CMS so this would be just as easy to upgrade as other popular software.    

I built a schema differencing system for MySQL/MariaDB that can detect anything that doesn't match with installed version of the schema and generate the SQL to fix it.  I created the formats and code to store and compare this data from scratch.   It also can install the database from scripts, and all changes to the database are also done by adding a new scripts now that execute schema and data changes in sequence until it matches the current version, and this includes automated backup/restore in case it fails.    Many folks directly change the database, and building a system like this takes a while.   You don't have to verify the schema like I did, but it does reduce the chances that someone will run into problems post-upgrade that cause the application to no longer function.  I verify before and after installation, and even though there are almost 200 tables, it only takes a few seconds.   I wanted to automate / reduce time needed to support the app due to limited time / resources. 

In addition to automating database upgrades, I made scripts that can install all the source code, cronjobs automatically, so you have fewer steps to update and install.   There is also documentation for building the complete linux OS environment from scratch.   Having documentation to recreate your environment is very important to help installation problems later on.

So far, I've only run my app as a single installation to avoid duplicate efforts maintaining it, but now that I built the install / upgrade process, it wouldn't be much harder to run and maintain separate installs.   I want to begin working with other developers, and now I can enable them to run a separate installation more easily without giving them direct access to the production server.   Forcing everyone to work through an Integration Manager Workflow is the only way I want to share my code, because I care a great deal about the quality and security of the sites I host.  I never wanted to share the code until I had automated upgrades / install. 

Improve Your Security

Relying on "security through obscurity" is a bit impossible when you make your project open source.  Chances are, many developers have serious security problems in their code, that just haven't been discovered yet.  If you have a query vulnerable to SQL injection, or a file upload system that allows arbitrary code to be uploaded and executed, you may have a serious problem when the rest of the world finds out.   While some folks will help you fix these problems, others may use the knowledge to attack your business and affect your brand's reputation.

I add more detail to the process when doing R&D related to security related subjects for all the software I'm using.  If someone gets hacked because of my programming, or they don't understand security and install the app wrong, this would hurt the brand name.  Some other software already has a bad reputation for security, which is mainly due to third party plugins and people not following good practices like upgrading continuously, strong authentication methods, and security best practices on their development/production networks. 

The code might have been secure on, but that isn't enough to protect us from human error.   I tried to create a variety of features that validate the work developers do by verifying and correcting the software's configuration with cron jobs, validation and on demand.   These systems will reduce the chance of a security breach.   The number of security features implemented in Jetendo goes beyond anything I've seen in any single software project.   A CMS does many things, but security is one of the most important features, that is too often disregarded.

Document your software, APIs, and user interfaces

Documentation is not very fun to write, but it's maybe more important then the code when it comes to open source development.   I probably won't have many users until more documentation exists for Jetendo CMS.   Other software is further ahead on documentation and other resources.   This is very expensive to create from scratch.  I built a documentation system into Jetendo, but haven't written much of the text yet.   I built the documentation system to be relevant to the user, so if you are a server admin, you will get different documentation then a content contributor.   This should help the documentation become more useful when it is available.

If you build an API for your software, you'd want to ensure it is documented and somewhat stable (i.e. the function definitions stop changing in a way that breaks compatibility).  Currently, Jetendo CMS really doesn't have a public API that I'd encourage others to use, so It is a bit incomplete in this area.   In the future, I plan on documenting the features that are recommended for use on individual projects, and how they work.   This will help developers stay compatible with future versions of the software more easily.   If you've never built APIs for your software or concerned yourself with minimizing the amount of duplicate code between projects, you may find it harder to allow others to integrate custom themes and plugins.    

Building a Hook or Interface based API in addition to documenting a library of reusable functions and components is pretty much required for any CMS project to succeed.  Jetendo CMS actually does have a fairly consistent API already, but it's pretty much impossible to use an API that isn't documented for another developer.  Asking developers to read the source code for hundreds of functions to figure out how to implement something, is not acceptable.   Documentation and API stability will be the biggest areas of growth for Jetendo CMS in the future.   I won't change the status of the project from "Alpha" to "Beta" until there is a documented API for making custom themes and/or plugins.

What is special about your project?

Beyond just making your code free open source, you have to be able to help others identify with what is unique about your software.  Getting another developer to use your project is much harder sale then giving your CMS to a customer who pays your business for service & support.  I expect to fail in gaining and maintaining users several times until I adapt to what they tell me is wrong with the software from a development point of view.   But after a number of a years, I hope I'll do better because of making these changes.    Anyone I hire would have the same complaints.   Open source will reveal problems that are easier to hide or ignore in your client work.   It should make your own team have greater pride in their work when they can enjoy working on the project and know they are giving back to a larger community.

Jetendo CMS has a number of unique features.   It replaces the need for a Linux control panel for many tasks.   It is security hardened by default for the related software.   It relies on the Railo CFML engine for the bulk of the source code.  It has visual software building tools.  It is one of the only open source full-featured Real Estate web site solutions.   These things are what help me to gain customers.  It is not software that is relevant to everyone.  It is relevant to the people interested in these things.   If you try to be too broad/generic, you probably will fail.

Challenge yourself to do hard things more often

I set a very high standard for my first release of Jetendo CMS, but it's still not good enough.  I need to do a lot more.   Going open source was an intentional choice to reach for achieving some of the hardest goals possible.   Even if I gain only a handful of users, it was still an amazing personal journey to have went through the entire process.   Making money with inferior software & workflow is not very hard to do, but building quality software that solves common web development tasks in an elegant way is incredibly hard.  I have to constantly prioritize for money or for quality.  To get both at the same time is often not practical when building your own solutions to everything.  Sometimes the only way I can earn a profit is if I deliver a project that is 95% the same as a previous project, so that I have more time to build a new feature correctly.  I choose to value the creativity control and personal knowledge gained more then the financial outcome, so I develop my own software.   It's much easier to just hack at copies of other people's software, but I like doing hard things.  It is too boring to simply be constantly busy making money with no time for innovation.

Practice community building

Part of going open source will involve dealing with people around the world who are very different from you.   You'll have to try to avoid your personality quirks from damaging your reputation with the community.   CFML is a fairly small community, so it's kind of like going to a small school. You don't want to make too many people hate you, or you may have no friends.   I'm not a very social person to begin with, but I do like to write, and journal my thoughts.   I am not always as technically accurate as others, but I feel like I develop thoughts about the subjects I concern myself with in an enough detail to be able to say something about them occasionally.   I wrote hundreds of forum posts, dozens of blog posts, and hundreds of off-line emails / documents about my open source aspirations and software development in general.  

I kind of feel like there are just 50 to 100 people in the CFML community who are extremely vocal, and probably thousands of others who follow them, and then a lot more that have no interest in following the social stuff, but just want answers to problems.   I used to be one of the silent developers who never talked about programming online or offline with anyone, but I did read the answers or came up with my own.   About 2 years ago, I started communicating about programming online through my blog, the Railo forums and on other people's blogs like Adam Cameron's CFML blog as part of a plan to become part of the community.   I've contributed a number of bug reports for the Railo project since I started using it and I really enjoy that.  I hope someday I could find the time to do some of the Java changes too.  I learned a lot about what other developers are concerned with, and I've worked towards applying those things to my software.    

I highly recommend making yourself more active in the social aspect of development if you want to do open source software & community building.    Just like anything, you'll need to practice something to be good at it, so start practicing your communication skills by collaborating on a few open source projects you use, and see if this is the right path for you.  It's a bit of a lifestyle change.

It's so easy for me to do little things for my little business, but when my changes could affect hundreds or thousands of people someday, I'm going to have to be more responsible.  I have to challenge myself to be not only be concerned with my paying customers, but other people who may never pay me anything.    I've found that some of the developers demand a level of accuracy that is really hard to achieve.   It's really hard for me to write anything sometimes because they try to cover all the bases.   It's never really about random conversations when you look at programming discussions, everyone wants to have the ultimate solution in the least words now.   I'm not very good at that. 

Other concerns

You also need to do things like choose an open source license, develop a web site for the project, choose where the source code will be hosted, name your project, and get ready to provide support through various free or paid tools.    In the end, you may find you have greater respect for the other software projects out there, and wonder how they managed to get started so long ago.    Don't lose faith though, you can do anything if you focus on it long enough!  Many of the things I've done were planned many years ago, and taking shortcuts only make it more difficult later.  It takes small forward and backwards steps over years to build complex software.

Continue reading my blog to learn more about what I've done in the last year to prepare Jetendo CMS for becoming an open source project and what I've done since then.

Bookmark & Share