My slides from Docker Boston meetup on Using Fig for Developing Microservices

4 Disciplines Necessary in Building a Startup

I was reading this piece by Jeff Atwood on why discipline makes strong developers. Having only been doing this startup thing for less than a year so far, I am learning my way as a startup co-founder as we go. So that article got me thinking on how discipline plays a role in the context of an early-stage startup. Here are the four recurring themes that I find upon closer reflection.

  1. Discipline in time
  2. Discipline in focus
  3. Discipline in communication
  4. Discipline in seeing it through

Discipline in time

It is not easy distinguishing between working hard and working productively. As I've wrote in More Problem Solving, Less Solution Glorifying, I sometimes catch myself building "cool" things rather than just the practical things. Taking a page from the Pareto Principle, we want to focus on the 20% of effort that would deliver 80% of value to our users. It is ok in the first release if things break or don't work that well. If anyone notices it, that means your users are using your product! Ship it first before anything else. This is the discipline in making the best use of your time to deliver visible values.

Discipline in focus

We regularly hire contractors to fill our gaps (design, in particular, is our Archilles' heel). It is tempting to hire even more contractors to do more work so we can get even more things done. Getting more done is good, right? But more can add complexity. It could be complexity within the product. In which case it would unnecessarily complicate the user experience. It could also be complexity in sales. A more complex product is harder for a customer to understand and see the value of. If more doesn't add complexity, then it adds to planning. In which case, you need to be critical of your basis for why you're doing it. This is a counter-intuitive one as startups need to move fast. The nuance, as Eric Ries puts it, is that we want to make informed progress and not just haphazardly shooting in all directions. This is the discipline to remain focused via customer development-driven progress.

Discipline in communication

One way to ensure that we're focusing on the right steps to take is proper communication. The people talking to clients need to communicate what users say and don't say. The people building the product needs to communicate what are the do's and not do's. Only when the two sides can come together can we find the 20% work to deliver 80% values.

Within the engineering team, we lay out the problems and communicate our plans. My job as a team lead is to enable our engineers to solve problems and not point people to build the solution in my head. Jared Spool has a good piece on Moving from Critical Review to Critique. The article is on product development but the idea of i) focus on the problem instead of the to-do's, and ii) ask questions instead of making suggestions, are sound advice in a broader context.

Within the whole company, we need discipline to be able to speak honestly and listen without passing on judgement. That one I'm still struggling on as I have this horrible tendency to be too analytical and diving into details prematurely to new ideas.

Discipline in seeing it through

Building Glassy Media is the hardest thing I've pursued, with the double whammy that there's no way to tell what we're doing even matters. I've often described to my friends that this is a roller coaster ride. In Jessica Livingston's Founders At Work, one of the founders said it best (I'm paraphrasing here):

One day you're all optimistic and things seem to be coming together.
Then the next day, you could be all gloomy and feel like this is going
to go down in flame. Even when nothing has fundamentally changed.

I couldn't agree more. I find that the key is to keep your heads up beyond the next loop to see where you're going, enjoy the ride regardless of the outcome, stay on track, and keep at it going through one obstacle at a time.

Well, that's my take. How does discipline play a role in what you do?

Posted 17 August 2014 in startup.

Simple, Easy, Quick: Using Go along with Clojure

We had questions about how Go compares with Clojure at the Boston Clojure Meetup this week. My default answer to any technology this-vs-that debate is -- it depends. But it just so happens that our backend system at Glassy Media consist of an even split of services written between Go and Clojure. However, this isn't due to any technical reasoning. Using the two languages together feels right to our particular needs.

Rich Hickey made clear the distinction between simple and easy in his famous talk, Simple Made Easy. Clojure made simple easy. But sometimes I don't want simple and easy. I want quick and easy. That is exactly where Go can complement Clojure.

Simple, Easy, Quick

How Clojure made simple easy has been a path well beaten. The rest of this post is why I think Go is quick and easy.

Our Path To Go

Once upon a time (i.e. a few months ago) our backend cogs and gears consisted of a bunch of command line programs that we would run on our laptop every now and then to churn our data. We distributed those tools to our non-developer helpers to serve our clients. Those command line tools started out in Python or Node.js. But distributing, maintaining, and updating those programs to other people's laptops eventually became a real pain. So one weekend I rewrote one of our command line programs in Go. On Monday I shared a link to a single executable file on Dropbox and people were able to run it directly from there. No more asking people to go on command line and typing pip install or npm install. That was how we started using Go at Glassy Media.

I had no prior experience with Go before that. The fact that I was able to learn the language and then produce something useful over a weekend doesn't have to do with me. It is the practicality of the language, clear documentation, and smart tooling that makes Go easy to ramp up.

Language Practicality

Take the for loop as an example, you can iterate over a map like so. (code sample taken from Effective Go)

for key, value := range oldMap {
    newMap[key] = value

The same range can be used to iterate over an array:

sum := 0
for _, value := range array {
    sum += value

No need for additional syntax to do the same thing. And there're no while or do loops too. There is only for to do all of that as shown.

// complete `for` struct
for init; condition; post { }

// only check condition like a `while`
for condition { }

// or, nothing like a `do`
for { }

The Go core syntax is consistently minimal like so.

Clear Documentation

From The Go Programming Language,

 The Go project takes documentation seriously. Documentation is a huge part
 of making software accessible and maintainable. Of course it must be
 well-written and accurate, but it also must be easy to write and to
 maintain. Ideally, it should be coupled to the code itself so the
 documentation evolves along with the code. The easier it is for
 programmers to produce good documentation, the better for everyone.

Go's source code documentation has simple convention and automatic publishing. Take our trivial go-arrays library for manipulating arrays. Here is one of the functions and its docstring.

// Contains check if 's' is in 'coll' string array
func Contains(s string, coll []string) bool {

What's amazing is that a corresponding project documentation page is automatically scraped from our source code and hosted on with no extra work needed on our part. So as soon as you pushed a project onto Github, you can request your project documentation on website. Here is a screenshot of one. for go-arrays

Because software documentation is baked into the process, I find most of the Go libraries are less frustrating to pickup.

Smart Tooling

I don't know why GoFmt isn't more common in programming. GoFmt is a tool that automatically formats Go source code. It is like jshint or pyflakes but goes an extra step further and actually fixes your code styling for you too. No more gg=G in Vim.

Consistent source code styling makes life so much easier when digging into other people's projects.

Overall, I enjoy the workflow developing in Go. There is no interactive REPL. What you get is an almost instant go test and go run execution time. Coming from doing everything in Vim, switching between my editor and the command line isn't as annoying as I would have thought.

One reason for that is because the workflow is different when developing in Go. GoFmt is integrated in Vim (and other typical editors); along with the fact that Go is statically typed, by the time when your code saves without error, most of the time it would work as expected. So it's not that often when I need to resort to doing go test to check my code.

Go for Web Services

Having had Go in production for only a few months, our experience with Go is limited. Still, what stands out the most with Go is using it to build web services. Take our SMTP callback verification service, for example. The HTTP server code for it is under 50 lines using go-json-rest and a couple core packages. Not only that, Go has mock HTTP server testing functionality built right in.

I am aware that this is subjective. But having used a handful of languages and frameworks to build RESTful web services, Go is by far the easiest to get a server up and running.

Complementing Clojure with Go

The one time when I had to choose between Go vs. Clojure was when the Boston Go Meetup happened on the same night as the Boston Clojure Meetup.

Otherwise, there is no reason that Go and Clojure can't work along each other as part of a suite of toolset.

Everyone has their tools of choice. Your taste and circumstance are probably different than mine. What matters is that the tools in your toolbox as a whole fit the problems that you face. Go and Clojure fill different needs for us.

Posted 12 May 2014 in computing.

More problem solving, less solution glorifying

I have been growing fond of this trendy Go programming language lately. We use it at Glassy Media to build services over HTTP and command line. So when we needed a Amazon Mechanical Turk interface I thought why not use Go for that too? I found an existing library in Go for interacting with Amazon Web Services. But its Mechanical Turk component is incomplete. So I dove into the source code and started patching to make it work.

A weekend of coding later, I still haven't finished. Then I realised what I was doing. I was building a cool solution but not actually solving the problem at hand. This might have been beneficial for a capable company to help contribute back to the open source community. But not where I'm at -- a new startup that's stretching thin on resources. We simply couldn't afford to lose focus at this phase. So I switched over to good ole Python. The boto Python interface to AWS worked just fine. The result was that there were no "I did this in Go" blog post to write about but our problem was solved and we moved on to the next task.

This sense of building cool solutions just for the sake of it seem common in tech. Just last week at a Go meetup, someone spoke about Using Reflections in Go. Even Rob Pike, the man himself, wrote that reflection "should be used with care and avoided unless strictly necessary". Somehow that was taken as a challenge.

challenge accepted

I am guilty of glorifying new technologies and putting down on tried and true solutions too. On more occasions than I dare to admit at data conferences, I participated in bashing at SQL databases with the attendees. I can't speak for others, but for me it was due to ignorance. I used SQL a few times before but was put off by the data warehousing aspect of it. Having gotten more familiar with MySQL in particular over the past couple years, SQL can be simple and beautiful too. I have actually come to prefer it in some cases. Use the right tool for the right task, right?

An example of focusing on solving the problem. Here is one of the distilled tech specs we have for a component.

  1. A webpage for users to view a dataset of their private contacts
  2. Sorting capabilities on some of the data fields
  3. Expected number of contacts per user is several hundreds
  4. Limit to no more than 10 - 20 users at beta launch

Most of the efforts went into distilling our problem to make it as easy as possible. Like letting the user edit the data? Nope. UX design? Not until we figure out what data to surface. Because the problem specs are concise, solving this is almost trivial. Our current technical stack to solve this is embarassingly simplistic:

  • HTML/CSS/Javascript for front end hosted on Heroku,
  • Go for REST-based backend on Heroku; and
  • Google Spreadsheet as database.

Nothing fancy. Yes, we didn't even bother with spinning up a database.

People build more complicated things in a hackday than this. That's because the real problem we have is that we think this is needed with no real proof yet. Perhaps all they need is just having CSV files emailed to them? We tried that too. We want to get this out to our customers as soon as possible for them to use and give us feedback. What is the task that this component is doing for our customers in the scope of our product experience? Abstract problems like these are not trivial and validating them is why we are building things in a startup. Not the other way around.

As I was reminiscing earlier this year, effective problem solving is more about defining a clear problem than coming up with a smart solution. "A problem well stated is a problem half-solved." In a pursuit of the latest and greatest, I sometimes find myself swung too far in the pendulum towards glorifying the solution rather than appreciating the effectiveness of clarifying the problem. Sometimes the best solution is to realise not to do something.

At this early stage of our startup, the biggest problem should clearly be shipping and learning. Being a Clojure enthusiast myself, we've only used Clojure for 2 of the dozen or so of projects shipped. Most of the rest are a mix of Python, Go, Javascript, Bash scripts, or even Google Doc macros. However, don't mistaken getting things done quickly with cutting corners or committing spaghetti code. We are adament about keeping our software architecture as simple and decoupled as possible. If that's done right, the individual implementation don't matter because we can always rewrite any of them without impacting the system. That Mechanical Turk project? You can bet that we'll come back to refactor it later if it proves useful. Maybe we'll pick up on goamz again. But at least it's shipped now. I am focusing on the problem and not let building a startup become a technical challenge to satisfy my vanity.

Posted 16 March 2014 in startup.

continue   →