Tableside Open Source Collaboration

December is a good month to work on something serendipitous. We started a little experiment to sit a few random developers at a table to build something from scratch together. This wasn't anything big like a hackathon. We wanted to create a cozy (or as the Danish would say, hygge) space for people to get together to work on something fun. In the beginning of the month, I sent out this call for participation for a weekly hack night for December.

Meetup screenshot

To my delight, a couple developers actually came on the first day!

First Git commit with Julia and Mark

Julia, Mark, and I spent our first gathering hashing out the idea for 99issues.

99issues is a Github issues discovery web application for finding easy/newbie issues in your favourite languages for use during dojos or hack days.

One of the deliverables that we got done on the first day was this memes. Priorities!

99issues memes

It was like musical chair for the rest of the month. People came, gone, and returned. Usually we have 3 to 4 people around the table. For each session, one person that were here the previous week would induct the newcomers on what's happening.

As you can imagine, this is a project management challenge. One thing we did well was spent time up-front to discuss user needs, draft requirements, and document design choices. So everyone that came onboard subsequently are clear on the direction. We documented extensively on pen and paper and on Github issues. For each session, we break down work into doable-for-the-day chunks and made use of the usual open source collaboration Gitflow process. Nothing special here. Just divide and conquer.

Another hurdle for us that occurred early on is that as it's a general functional programming group, everyone have their own programming language of choice. Mark is into F#. Julia is into Haskell. Nandan is into Javascript. And RasmusErik is into Clojure.

Seeing that we didn't actually commit any code after the first session, I sneakily bootstrapped the project in ClojureScript and that was the end of story for this language war. (Thanks guys for putting up with my dictatorship!)

After a grand total of 3 hack nights (we skipped 23 Dec), we managed to get the project shipped with a bare scaffolding. We still need to work on the design and flush out the functionality. Pull requests are welcome! I had a lot of fun working side-by-side and getting to know these guys at the pub. We have different background but share a love for programming. I'm looking forward to doing this again next time I'm in town.

Running a server on Microsoft Azure: A UX report

Given the proliferation of open source cluster managers (for example, Mesos and Kubernetes) and PaaS software (for example, Deis and Dokku), the infrastructure layer is being abstracted further away from the application. The result is a cleaner separation between development and operation. The simplest example is probably Dokku. You install Dokku on a server, then git push to deploy your applications. That's it. Just as you would do on Heroku. There are no Puppet or Chef scripts to mess with to deploy your app. Given the freedom that these tools provide for less demanding deploy workflow, I've decided to try another Infrastructure as a Service.

I spinned up a Ubuntu 14.04 virtual machine on Azure, set up Dokku on it, and deployed my applications for a data privacy project. From account sign up to a running stack took less than an hour. There were no surprises, and the process was straightforward. Sign up for an account, then follow the instructions. Once you're in the portal, click on the giant New button to create a virtual machine. The UI itself guides you step-by-step through a horizontally expanding panel. Don't Make Me Think. I like it.

Azure new VM

I have to point out that the UI I'm showing is a preview for the next portal version. Compared to Azure's old UI, the improvements on both function and esthetics are evident. For example, in the old UI, they had a quick create for Ubuntu machines, but using that would mean you could only log in with a password. Which is bad practice. If you want to use an SSH key, you need to go through the detailed settings (screenshot below), which aren't apparent for a user to try. You can't use an RSA key either, and need to upload an X.509 certificate. The new UI has neither of those problems. There are a few small annoying quirks like that in the old UI. I would recommend anyone trying Azure for the first time to use the Preview Portal. You can opt-in at your Account menu on the top right.

Azure old VM

Once your VM is up and running, which took less than a minute for me, the instance view (back to the new portal preview) is informative and clean. Grab the SSH information from there to connect, and you're in. I'm impressed by the fact that information which I would want to see on each page is exactly where it should be. It's like Microsoft made the effort to think about the UX.

Azure VM information

Navigating to the Home dashboard, you can recognize Microsoft's signature Metro design. And guess what, billing information is right there! No need to jump to another part of the site like on AWS.

Azure dashboard

As much as I like the Azure management portal, in reality it might not be the best choice for many ops people. If you're an ops person spending most of your time on the web management console, then you're probably doing it wrong. For my use case of spinning up a server or two to host personal projects every now and then, the management UI is a convenient tool to have. Azure seems to beat AWS on the web design. However, if that's your use case, you probably don't need an IaaS and would get better value with a smaller provider like Linode or Digital Ocean.

I haven't had the chance to use Azure much yet. Anyway,the server is still up and running after a week. I haven't needed to tinker with it , which is a good thing. The obvious negative at the moment is that the free trial only lasts 30 days. Other than that, so far so good.

Posted 19 June 2015 in computing.

My slides from Docker Boston meetup on Using Fig for Developing Microservices

Simple, easy, quick: using Go along with Clojure

We had questions about how Go compares with Clojure at the Boston Clojure Meetup earlier this week. My default answer to any this-vs-that technology debate is -- it depends. But it just so happens that our backend system at Glassy Media consists of an even split of services written between Go and Clojure. However, this isn't due to any technical reasoning. It’s just that using the two languages together feels right for our particular needs.

Rich Hickey made clear the distinction between simple and easy in his famous talk, Simple Made Easy. Clojure makes simple easy. But sometimes I don't want simple and easy. I want quick and easy. That is exactly where Go comes in to complement Clojure.

Simple, Easy, Quick

How Clojure made simple easy is a path well-beaten. The rest of this post is about why I think Go is quick and easy.

Our path to Go

Once upon a time (i.e. a few months ago) our backend cogs and gears consisted of a bunch of command line programs that we would run on our laptop every now and then to churn our data. We then distributed those tools to our non-developer helpers to serve our clients. Those command line tools started out in Python or Node.js. But distributing, maintaining, and updating those programs to other people's laptops eventually became a real pain. So one weekend I rewrote one of our command line programs in Go. On the Monday, I shared a link to a single executable file on Dropbox and people were able to run it directly from there. No more asking people to go on command line and typing pip install or npm install. That was how we started using Go at Glassy Media.

I had no prior experience with Go before that. The fact that I was able to learn the language and then produce something useful over a weekend was not to my credit but lies in the smart tooling, the clear documentation, and the practicality of the language. These features make Go easy to ramp up.

Language Practicality

Take the for loop as an example, you can iterate over a map like so. (code sample taken from Effective Go)

for key, value := range oldMap {
    newMap[key] = value
}

The same range can be used to iterate over an array:

sum := 0
for _, value := range array {
    sum += value
}

No need for additional syntax to do the same thing. And there are no while or do loops either. for, alone does all of that, as shown:

// complete `for` struct
for init; condition; post { }

// only check condition like a `while`
for condition { }

// or, nothing like a `do`
for { }

The Go core syntax is consistently minimal like this.

Clear Documentation

From The Go Programming Language,

 The Go project takes documentation seriously. Documentation is a huge part
 of making software accessible and maintainable. Of course it must be
 well-written and accurate, but it also must be easy to write and to
 maintain. Ideally, it should be coupled to the code itself so the
 documentation evolves along with the code. The easier it is for
 programmers to produce good documentation, the better for everyone.

Go's source code documentation has simple convention and automatic publishing. Take our trivial go-arrays library for manipulating arrays. Here is one of the functions and its docstring:

// Contains check if 's' is in 'coll' string array
func Contains(s string, coll []string) bool {
    ...
}

What's amazing is that a corresponding project documentation page is automatically scraped from our source code and hosted on GoDoc.org with no extra work needed on our part. So as soon as you push a project onto Github, you can request your project documentation on GoDoc.org website. Here is a screenshot of one:

Godoc.org for go-arrays

Because software documentation is baked into the process, I find most of the Go libraries are less frustrating to pick up.

Smart Tooling

I don't know why GoFmt isn't more common in programming. GoFmt is a tool that automatically formats Go source code. It is like jshint or pyflakes but goes a step further and actually fixes your code styling for you too. No more gg=G in Vim.

Consistent source code styling makes life so much easier when digging into other people's projects.

Overall, I enjoy the workflow developing in Go. There is no interactive REPL. What you get is an almost instant go test and go run execution time. After doing everything in Vim, switching between my editor and the command line isn't as annoying as I would have expected.

One reason for this relative ease is that the workflow is different when developing in Go. GoFmt is integrated in Vim (and other typical editors). A second reason is that Go is statically typed, by the time when your code passes GoFmt without error, most of the time it would work as expected. So it's not very often that I need to resort to doing go test to check my code.

Go for web services

Having had Go in production for only a few months, our experience with it is limited. Still, what stands out most with Go is using it to build web services. Take our SMTP callback verification service, for example. The HTTP server code for it is under 50 lines using go-json-rest and a couple core packages. Not only that, Go has mock HTTP server testing functionality built right in.

I am aware that this is subjective. But having used a handful of languages and frameworks to build RESTful web services, Go is by far the easiest for getting a server up and running.

Complementing Clojure with Go

The one time I had to choose between Go vs. Clojure was when the Boston Go Meetup happened on the same night as the Boston Clojure Meetup.

Otherwise, there is no reason why Go and Clojure can't work alongside each other as part of a toolset suite.

Everyone has their tools of choice. Your taste and circumstance are probably different from mine. What matters is that the tools in your toolbox fit the problems that you face. Go and Clojure each answer different needs for us.

Posted 12 May 2014 in computing.

continue   →