Go proverbs
The following is a transcript of the Go Proverbs talk given by Rob Pike at Gopherfest 2015. I find it useful to reference and could not find a transcript of it online, so I’ve reproduced its contents here.
Introduction.
I think it’s time to start philosophizing a little bit. How many of you have played the game of Go? (Oh good, lots.) In fact, in the time I have to give this talk I could teach the rest of you how to play - not play well, but play - because it’s a very simple game. The name for the language is obviously the same as the game, and that’s not entirely a coincidence, although there’s more to the story.
The thing about learning the game of Go is that it’s very easy to learn and really, really hard to master. It’s not like most Western games, like chess. You don’t learn to say, “Well, if that person goes there, then I go here, and if they go there, then I go here.” It’s not like that. There’s 19² positions on the board. The exponential is just crazy. Instead you have to think about patterns, and territory, and it’s a very different and much more holistic thing to think in.
It’s very hard to teach us left-brained Westerners how to play Go. I’ve known some good ones, but I’m not among them. There’s a principle for teaching, it’s really used for pedagogy, and it’s the idea of a Go proverb. There’s a lovely little book that was translated about 50 years ago from Japanese into English called Go Proverbs Illustrated.
Just to give you an idea of the flavor of them, here’s one: “The comb formation is alive.” That means that those white stones, if you attack, you can actually keep them alive no matter what black does, because you can always create two eyes. Don’t worry whether you understand that or not. It’s just a nice little thing, “the comb formation is alive.”
Or “The carpenter’s square becomes ko.” That’s a really nice little proverb because the black stones in this case, if you attack them as white and play perfectly, you end up with a pattern called ko, which is a very strange - it’s the one weird thing in the game that you have to understand beyond the trivial stuff.
But you see these have a poetic, nice feel to them, but they carry a lot of information for people learning how to play. When you see this thing on the board, you know, “Oh, I know what happens if I have this position because I can create a ko, and that may or may not be a good thing.”
So, I was wondering, can you have Go language proverbs? Can we think about teaching Go, or maybe understanding or explaining Go, with a similar kind of Eastern philosophy approach? And I realize this is very very wet stuff, but bear with me, because in fact, there is already one proverb you all know.
Don't communicate by sharing memory, share memory by communicating.
If you think about it, this is a proverb. It’s a little tiny, pithy statement, slightly poetic, it’s got a sort of repetition in a way. But really understanding what that says is a fairly big deal because you have to understand - what do you mean by “sharing memory by communicating?” Well, you mean passing on a channel the address of a data structure or an object, and also the idea that when you send that object over a channel, if you don’t keep the pointer, then you don’t have access to it anymore. And so safe, concurrent, or even parallel calculations are inherent in this very model. So there’s actually a lot behind there, it’s a nice rich little thing.
And I was thinking, well maybe there’s more of them, so I tried to come up with a few of them and I’ll go through my list, and maybe there’s more you guys can think of.
Concurrency is not parallelism.
If you’ve seen the talk I gave at Waza, you know that concurrency and parallelism are often confused by beginners. They think they’re the same thing. But at least the way you talk about them in the Go community, they’re very different, and important to keep separate.
Concurrency is a way of structuring your program to make it easy to understand and scalable. Parallelism is simply the execution of multiple goroutines in parallel. It’s somewhat less interesting a concept than concurrency.
But there’s a lot in those four words.
Channels orchestrate; mutexes serialize.
A lot of beginners struggle with the idea of, “When do I use a mutex, or a sync.Cond, or an atomic value,” or, “When do I use a channel and a goroutine,” or you know, all this stuff.
One way to think about that is that mutexes are fine-grained, and very small, and they tend to serialize execution. If you put a mutex on a variable, then only one thing can ever happen at a time to that variable. And that’s often very important, and sometimes that’s exactly what you want. But in the big picture of the whole program that’s going on, goroutines and channels give you a way to structure the program and to orchestrate, to arrange how the pieces work to generate the large-scale service or whatever it is you’re building. The select/for loop that everybody knows using channels to orchestrate your program is the canonical example.
The bigger the interface, the weaker the abstraction.
Now, if you come from Java - where everything is bigger - you think of interfaces, typically, as having lots of methods. And people talk about how unusual it is in Go to have interfaces that are not declared to be satisfied - they’re satisfied implicitly.
But that’s not actually the most important thing about Go’s interfaces. The most important thing is the culture around them that’s captured by this proverb, which is that the smaller the interface is, the more useful it is.
io.Reader, io.Writer, and interface{} are the three most important interfaces in the entire ecosystem, and they have an average of two-thirds of a method.
If you think about the way a Java guy would build it, he’d build a larger interface, and it would only generalize a little bit because there might be two implementations of it. How many implementations of io.Reader are there? I’ve written programs with 20 implementations of io.Reader inside. That’s a really powerful structuring plan.
This is really a Go specific idea here, that we want to make little interfaces so that we can build components that share them.
Make the zero value useful.
I think this is something that early Go programmers don’t appreciate. You really want to arrange the API to your package so that the zero value of the types inside it can be useful right out of the box.
Some examples of the standard library that show how that important that is are things like bytes.Buffer and sync.Mutex. You can put one of those things inside your structure without allocating it, or declare a variable of that type, and just use it. You don’t have to call an init function or have to think about it.
It matters even more when you’re building complex data structures that have these things inside them. Because the zero value of a bytes.Buffer is valid and ready to use, if you put that inside another structure, and you make all the other parts of it also valid zero, bytes.Buffer doesn’t break that. And so it’s a way to structure your whole data to make sure that the programs are much easier to use.
Now of course, there’s nothing wrong with calling a constructor. It’s a perfectly good thing to do; it’s often necessary. But when you can avoid it and make the program nicer, it just means there’s less API there. And that’s always a good thing… unless you’re a Java programmer.
interface{} says nothing.
It actually carries no information. Just yesterday or the day before, I saw someone had a bug where they’d written a piece of code where the function argument was interface{}, and so they called it, and it just worked. They refactored some code and the program broke mysteriously. Because it couldn’t be statically checked! Because the function with the interface{} argument didn’t force any guarantees on the caller. You might as well program in Python if you’re doing that.
interface{} is also something you see in almost every question on Stack Overflow about Go. There’s always interface{}. Or a map[string]interface{}, which is the least useful data structure there is, because it’s a map of names to meaningless objects.
So when you’re programming and you have interface{}, think very hard about whether that’s really what you want, or if there isn’t just a little something you could put into an interface with an actual method that really is necessary to capture the thing you’re really talking about.
Sometimes you absolutely need it, there’s no question about it. But it’s way overused, especially by beginners.
Gofmt's style is no one's favorite, yet gofmt is everyone's favorite.
gofmt is, of course, a big deal in the Go community. A lot of people say, especially beginners, “I want to move the braces,” or “Why tabs instead of spaces?” or whatever.
Who cares. Shut up.
gofmt’s style is nobody’s favorite. The way that it formats isn’t even the way Robert Griesemer likes code to look, and he wrote the program! Because we argue about it, right?
But the thing is that if you ask experienced Go programmers their favorite thing about Go, a significant fraction, maybe even the majority, will say their favorite feature is gofmt. “Yeah, I don’t like how it formats, but I really like that it formats.”
A little copying is better than a little dependency.
When I first went to Google, one of the first things that was told to me that scared me, was somebody who said, “We really care most importantly about code reuse.” And I thought that was a weird thing to say.
I found what they meant was, if you could avoid writing one line by doing #include, you should do that. And I learned that that was a very, very bad idea, and Google, internally, is still suffering from that decision.
You can actually make your programs compile faster, be easier to maintain, and simpler, if you keep the dependency tree really, really small. And one of the ways you can do that is sometimes say, “You know what, I don’t need that whole library. All I need is these three lines of code that I can just copy and it’s fine.”
There’s even an example of that in the standard library. strconv has an IsPrint function so you can tell whether a thing is printable or needs to be escaped under certain conditions. One way to get that is to import the unicode package and and say “Does this match the IsPrint category?” But that involves something like 150KB of data. That’s a huge dependency for such a trivial question.
strconv actually has its own implementation of IsPrint, which is tiny - it’s about five, ten lines of code - and that saves all that dependency overhead. But then, of course, there’s a test, so that every time strconv is tested, it guarantees that the unicode and strconv packages agree on that definition. The test has a unicode dependency, but the package doesn’t.
And that’s a really good example of making code more compact and less dependent on other pieces. Don’t be afraid to copy when it makes sense.
Syscall must always be guarded with build tags.
This happened today, actually. Somebody was complaining that syscall needed to be changed because of a problem: it wasn’t portable between two different architectures. And you know, syscall isn’t portable: that’s the point. It’s a system-specific package.
If you import syscall into your package, you must have a build tag for the particular architecture and operating system that that syscall invocation is valid for. If you think you have a portable thing, you’re using the wrong package. You should be using os or something else. You should always guard your syscall uses with build tags.
Not very poetic, but it works.
Cgo must always be guarded with build tags.
For exactly the same reason! If you’re calling C, God knows what it does. Under the covers, it’s very, very non-portable. I know we think of C as a portable language in the old world, but it’s really not, and the open-source community has made sure that it’s become less portable over time.
If you use cgo, you should have a build tag to make sure that the cgo you’re calling in is protected. Maybe somebody grabs your package and builds it and it fails to build because the build tags don’t work. That may be a good thing, because they may have had a binary that built but didn’t run because a cgo dependency isn’t actually useful on that computer, or even worse, doesn’t work very well and it breaks.
Cgo is not Go.
A lot of people, in the early days of cgo appearing, would write blog posts about how one of their favorite features of Go was how easy it was to connect to C. And I think that’s a trap.
Sometimes you have to. There’s no question. There’s definitely fantastic, efficient, or expansive packages out there that really deserve to be used rather than rewritten. But I’ll confess I’ve actually never used cgo. I’ve never wanted to. I’ve certainly used programs that have it, but I’ve never actually written a cgo invocation myself, except in tests.
Because I think we have a universe where we understand memory, we have safety, we have correctness, we have stability, we have garbage collection… you put cgo in and all those things go out the window. And 90% of the time inside Google when someone says “my program crashes, please help me, my runtime corrupted,” it turns out it’s a cgo or a SWIG problem, and not the Go code that’s causing the problem.
With the unsafe package there are no guarantees.
The other 10% have unsafe.
A lot of people use unsafe and then complain when their code isn’t portable, or changes after a release, or something.
In fact, there’s one guarantee because we’re too afraid to break it, but I’d like to break it. People use the unsafe package to get at things like string headers and slice headers, which means it’s very difficult to consider changing the layout of string headers and slice headers, even though we’d like to. And maybe one day we will, but then we’ll break all those programs that use unsafe to get at them. And tough to them. Don’t use unsafe unless you’re prepared to have your program break one day.
Clear is better than clever.
So far all the proverbs I’ve shown you have been general programming things about Go, and you might say that this one is actually about programming in general: you should always be clear, not clever. But there are languages where cleverness is considered a virtue. The more compact the code is, the tighter it is, the more it uses built-in facilities, the better.
Go doesn’t work that way. Especially when you’re learning Go out of the box, you should be thinking about writing simple, clear code, rather than trying to make the cleverest, densest stuff you can. And that has a lot to do with maintainability, stability, ability for other people to read your code, things like that.
It’s a philosophical point, but it really is part of the Go philosophy.
Reflection is never clear.
Another thing you see on Stack Overflow a lot is people trying to use reflect and wondering why it doesn’t work. It doesn’t work because it’s not for you.
Very, very few people should be playing with reflection. It’s a very powerful but very difficult to use feature. It has only runtime checks. At compile time, there’s very little checking going on. And the code is utterly impenetrable. I’ve probably written about as much reflection code as anyone in Go and I still hate it.
We should encourage beginners to step away from using interface{} and step away from using reflection and use the language proper, because they don’t need to use these things as much as they think they do.
Errors are values.
Beginners struggle with, “Why do I have to type if err != nil all the time?” Well, it’s because you’re not programming. You’re just writing code. They’re not the same thing.
You may know I wrote a blog post about a year ago where I showed a programmer in Japan who was really complaining about this that just wasn’t writing programs. He wasn’t thinking about errors as values and programming them to good effect, so that he could essentially abstract the error handling away and make his program really pretty and clear and easy to understand.
Too often people just write “if err != nil,” and I think beginners to Go who maybe have been influenced by thinking of errors as an exception, or some other control structure like try/catch, they think you just substitute try/catch for “if err != nil”. But you can’t really program with try/catch, because it’s a control structure, it doesn’t work.
But errors are just values! You can write code. You can put them in for loops. You can store them in variables. You can cache them. You can put them in a map. You can send them to your mother. Whatever you want! They’re just values.
Don't just check errors, handle them gracefully.
Don’t just write “if err != nil, return thing,” think about whether you should be doing something with that error. Should you decorate it with more information? Or should you remember it for later and print it when you’ve done some other things?
A big part of all programming for real is how you handle errors. People are too quick to return an error up the tree and forget about it rather than thinking about how an error should really work. A really important part of writing good Go code is getting the error handling right up front - of any program, really - but because errors are just values in Go, it’s easier to program and therefore easier to do it gracefully.
Design the architecture, name the components, document the details.
I’ve tried to find proverbs around these and I ended up writing one to cover all of them.
When you’re writing a big system in Go, you design it as a structure - think about what the components are, what pieces work in parallel - and then think of really good names for the pieces, because those names are going to be what appears on the page in the Go program. If the names are good, then the components will be easy to understand, and the design will be clear inside the structure of your program. If you give good names to things, the programming feels very natural.
But there’s a lot of stuff you have to explain, and so those are the details you explain. But the names should carry a fair bit of the weight of expressing the design that you’ve architected. And the details are just what fills in the gaps - the fine print on the engineering diagram.
Documentation is for users.
A lot of times people write documentation and they say, “This is what this function does.” They don’t think about what the function actually is for. And there’s a really important distinction.
“This function returns this.” Why does it return that? What should it return instead when you use it? Think about the documentation you write - your godoc presentation - think of yourself as the user of that package rather than the writer of it, so that the presentation that godoc gives is something that a user will find helpful.
It’s true for any programming language that the documentation is really about what users see, but because of the automated presentation in Go, like many other languages nowadays - but also because the documentation is so lightweight, it’s not annotated, there’s no HTML or anything - you can just write prose. So think of the user when you write the documentation. Don’t be afraid to explain things so that it makes sense.
Conclusion.
So that’s my list. There’s probably a lot more. I’ve whittled it down a lot to the ones I thought would be easy and pithy. I don’t think of these things as something you guys need to know - I think you know them already - but think about them as ideas you might use to explain. Maybe in a code review, or when you’re teaching someone new about something, or trying to stop an argument on Stack Overflow or whatever.
I’d like to think there’s lots more. Maybe you guys have more you can think of. If you want to mail them to me, maybe we’ll collate and have a bigger list. Maybe this turns into something that the community maintains on the wiki, or maybe when you leave tonight, this’ll be the end of the idea. I don’t know. It’s an exploration.
If you think about the proverbs back to the original idea of Go proverbs from the game, they have to satisfy several criteria to be the idea of a proverb. They have to be really short, they have to be kind of poetic - not all of the ones I wrote today are particularly poetic, but some of them are nice. You want to think of them as a little saying, something that will be memorable.
You also want to be general. Some of them are specific about a particular thing, but it’s the kind of things that almost every program should have an example in somewhere of what that proverb represents. It’s not like - “if your function has three arguments, it should probably only have two” - that’s very, very specific. You want something more general and more silly, in a way.
If possible, they should probably be more about Go than about programming, because there’s a lot of opinions about how to program, but one of the things that Go has done, I think, is create a community around a certain way of programming. And the idea of these proverbs is to try to capture some of those ideas that make programming in Go different and maybe better, I don’t know.
Also, they might be contradictory. Proverbs aren’t always - real proverbs in the real world you can find lots that are exactly the opposite. And that’s okay too, because sometimes one engineering decision is right, sometimes the exact opposite is right.
So if you think of more, or if you have ideas, or if you want me to stop talking about it, let me know. And that’s it. Thank you.