Concurrency in programming simply explained
Table of Contents
I used ton be pretty confused by all the techniques used to achieve concurrency in programming, even the way concurrency should be executed seemed unclear to me. I’ll try to lay down all the things around concurrency in programming and clarify all the terms.
Single thread vs Multi-threading #
Single thread #
Simply put, in a single threaded program, there is only one sequence of information that are executed by the CPU. This thread can be referred as a thread of execution.
If there happened to be a blocking operation, that is to say, an operation which needs to be finished for following operations to start, then the entire programs waits for that operation to be finished.
Multi-threading #
In opposition to the single threaded program described above, a multi-threaded program can run multiple threads of execution in parallel. Hence, avoiding freezing the whole program when a blocking operation is being executed.
Examples #
Let’s explore the differences in Golang.
First, we need a task to do. Let’s create an expensive task that simply simulate a blocking operation, which takes 10 ms. We will run this task synchronously and asynchronously one hundred times:
const numTasks = 100
func expensiveTask() {
time.Sleep(time.Millisecond * 10)
return
}
Now, we create a function representing our single threaded program. Running the expensive task in a for loop, in which the operation is only run after the previous one is ended. We have a single thread of operations:
func singleThreaded() {
for i := 0; i < numTasks; i++ {
expensiveTask()
}
}
Now, for the multi-threaded program, for every expensive operation we want to process, we will spawn a goroutine, which is described as “a lightweight thread managed by the Go runtime”. Goroutines are user-level threads and not kernel-level, but for the following explanation, it is still relevant. Note the waitgroup, which role here is to count the number of goroutines spawned, and prevent the multiThreaded function end before all the goroutines ended their processes:
func multiThreaded() {
var wg sync.WaitGroup
wg.Add(numTasks)
for i := 0; i < numTasks; i++ {
go func() {
expensiveTask()
wg.Done()
}()
}
wg.Wait()
}
And now finally, we will use Go’s benchmark capabilities to run each function multiple times, until it finds the process stable, and it gives us the time it takes to run our processes:
func BenchmarkSingleThreaded(b *testing.B) {
for i := 0; i < b.N; i++ {
singleThreaded()
}
}
func BenchmarkMultiThreaded(b *testing.B) {
for i := 0; i < b.N; i++ {
multiThreaded()
}
}
Everything put together, and with the imports, our main_test.go file looks like this:
package main
import (
"sync"
"testing"
"time"
)
const numTasks = 100
func expensiveTask() {
time.Sleep(time.Millisecond * 10)
return
}
func singleThreaded() {
for i := 0; i < numTasks; i++ {
expensiveTask()
}
}
func multiThreaded() {
var wg sync.WaitGroup
wg.Add(numTasks)
for i := 0; i < numTasks; i++ {
go func() {
expensiveTask()
wg.Done()
}()
}
wg.Wait()
}
func BenchmarkSingleThreaded(b *testing.B) {
for i := 0; i < b.N; i++ {
singleThreaded()
}
}
func BenchmarkMultiThreaded(b *testing.B) {
for i := 0; i < b.N; i++ {
multiThreaded()
}
}
Running the benchmarks with the command go test -bench=. main_test.go , gives me the following results on my machine:
❯ go test -bench=. main_test.go
goos: linux
goarch: amd64
cpu: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
BenchmarkSingleThreaded-8 1 1028413481 ns/op
BenchmarkMultiThreaded-8 100 10508948 ns/op
PASS
ok command-line-arguments 2.093
A little more than 1 second for the single threaded program, which seems logical since 10 ms * 100 = 1 s.
And a little more than 10 ms for the multi-threaded program.
In a case similar to this one, where the processes are all independent and expensive, it seems obvious then to spawn multiple threads for our operations.
But, multi-threading is not free.
What are the costs of multi-threading, and what is the limit of its efficiency ? #
Let’s rerun our previous benchmarks, but instead of having a blocking operation of 10 ms in our expensive, we will make it 10 nanoseconds:
func expensiveTask() {
time.Sleep(time.Nanosecond * 10)
return
}
Here are the results on my machine:
❯ go test -bench=. main_test.go
goos: linux
goarch: amd64
cpu: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
BenchmarkSingleThreaded-8 58623 20821 ns/op
BenchmarkMultiThreaded-8 33874 35286 ns/op
PASS
ok command-line-arguments 2.985s
20 μs for the single threaded program, and 35 μs for the multithreaded one, which is almost two times slower!
It is simply due to the overhead of starting a new thread. It is very lightweight in go, but there is still some cost, and here the overhead of spawning a goroutine is not worth it against the cost of the task.
Let’s also note that more threads means more memory consumed. If an extremely high number of threads are involved in a program, you can run into memory exhaustion.
Finally, to synchronize the multiple threads during tasks that need it, you will implement synchronization mechanisms that will also come with overhead, and will increase the complexity of your application. In Go we will use for examples Mutex or Channels.
When to use multiple threads, and when not to ? #
When I discovered multi-threading, I was asking myself, why not use it in every case possible then ? It seems like free performance improvement.
And it is true that for I/O Bound operations it is hard to renounce to the benefits. But in reality, it is more about when not to use multithreading.
In the benchmarks above we have seen that some overheads have to be taken into account, so if we want to run a huge number of threads, it won’t be a good idea because we will exhaust our memory. If we are running a task that is not so expensive (like the 10 ns blocking operation above), it won’t be worth it.
In a lot of programs, you also simply needs the processes results for the next one to begin. There are actually not that many cases in which spawning multiple-threads is a good idea.
The complexity of multi-threaded program can increase exponentially also. It is way harder to debug and will likely be a headache for the maintainers.
And finally, on a single core machine, the benefits of multi-threading is limited to the impossibility to truly run parallelly our threads. We would still be able to gain benefits with the I/O bound operations, but the added complexity and resource consumption may make it not worth it.
Synchronous vs Asynchronous #
Synchronous #
It simply describes the fact that tasks are executed one after the other, sequentially. When we encounter a blocking operation, the execution of everything is halted until this operation is completed, then only we can move on with the next tasks.
Asynchronous #
In opposition to synchronous, asynchronous describe the fact that tasks don’t wait for any blocking operation to complete. Instead, they all continue executing until finished on their own, then returning to the controller.
Other concepts #
Concurrency #
It is the ability of a system to handle multiple tasks or processes simultaneously. Concurrency defines more precisely the management of the multiple tasks, allowing them to make progress in overlapping time intervals.
Parallelism #
It defines the simultaneous execution of different tasks. Tasks executed in parallel on separate processing units will allow to achieve increased computational throughput.