This is my first blog post in a series about designing and developing a distributed key-value cache, but it’s also the only one that won’t go over any code related to the project.

Why?

Well, I don’t even know how to make a “Hello, World” program yet… in my language of choice that is. Since I want to improve my understanding of concurrency, and people online have honeyed words for the language known as Go (and some of it’s interesting features, which I’ll be discussing in this post), I thought it’d be fun to try it out.

But, since this is my first time programming in Go, here I’ll only be discussing my experience ramping up on Go-related concepts I plan to utilize in my distributed kv cache project. In classic programmer fashion, I can conveniently refer to this blog post as the 0th post in the series.

I started my education with this resource: A Tour of Go

  • learned Go packages. Some key notes: packages group related Go code, control visibility, and let code be reused across a project or by other projects.

  • x := 10 creates a new variable and let Go infer its type from the value. In this case, setting the variable x as an integer with value 10.

  • the variable type 'rune' is an alias for int32 and represents a single Unicode character. Very magical, I must use runes somewhere in my cache!

  • And many other interesting syntax quirks I'm not used to, such as how the if and for statements can start with a short statement to execute before the condition.

Most importantly, I learned about goroutines, the main reason I chose Go for my kv cache project. It has to do with concurrency. Normally, concurrency is limited by the number of OS threads. But Go has the concept of goroutines baked in, where you simply add the go keyword before a functional call and it runs concurrently to the rest of the program.

The magic comes from Go's runtime. Instead of creating a new OS thread for every concurrent task, Go schedules lightweight goroutines onto a much smaller pool of OS threads. This process, called multiplexing, allows a single machine to manage hundreds of thousands—or even millions—of concurrent goroutines. Goroutines start with very small stacks that grow and shrink as needed, making them far cheaper than traditional threads.

To oversimplify, it’s like how wizards may have a bunch of clones working at once, but one wizard learned they could get each of their clones to perform the cloning spell to further multiply the amount of concurrent work possible. That powerful magic spell is called Go, and can handle hundreds of thousands to millions of concurrent clones (which, of course, represent Goroutines!). It sounds so powerful that some worlds may call it forbidden magic, but it’s ready for any Go developer to wield. That’s me, soon! And maybe you too, after reading this post.

And sure, parallelism itself is limited by the number of CPU cores and all that, but don’t let that distract you from the magic of Go’s concurrency!

Clones of Go’s mascot

Clones of Go’s mascot!

I also studied related core Go concurrency concepts like channels, buffered channels{aside: these are like async/producer-consumer models since they remove the synchronization of blocking sends/receives until each other is ready}, select statements, mutex, and more. I watched this video for additional concurrency learning. One page of the notes I took:

One page of the notes I took

To get acquainted with the language, I first created a TCP echo server. A small project that taught me a lot. Some concepts I learned and utilized:

  • defer lets you schedule a function call to run when the surrounding function returns. Great for cleanup like unlocking a mutex or closing a channel. I'm a lot more familiar with Python, as an example, as that language has no direct analogue (the closest cousin might be 'finally' in 'try'/'finally' blocks), making this keyword a cool discovery for me.

  • Go concurrency design patterns: the generator pattern (a function that creates and returns a channel which it sends values to using a goroutine running a subfunction).

  • Always check for errors where applicable, log the error and return if err != nil

  • basics of using the net package

Alright, that’s enough prep. Time to actually get started on the project I wanted! Read [[KV1|part 1]] of the series next!