Golang gRPC In-Depth Tutorial [Create Server & Client]

Getting started with golang gRPC

gRPC stands for Remote Procedure Calls(RPC) and it is a robust open-source RPC framework that is used to build scalable and fast APIs. It uses the client and server architecture. The client and server can communicate with each other transparently as if they were in the same machine and yet they are not. A client application can directly call a method on a server application hosted in a different machine as if the client and server application are in the same machine.

gRPC systems are based on the idea of a service specifying the methods that can be called remotely. These methods have parameters and return types that are defined by the developer as we will see in the coming examples.

On the server side, the server implements this interface and runs a gRPC server that will intern handle requests from the client.

On the client side, the client holds a stub that provides the methods that exist in the server side. The server and client use protocol buffers which are a structured data format which can be serialized and deserialized but different languages

In this article we will discuss the following topics about gRPC

1. gRPC vs REST APIs
2. Protocol buffers
3. gRPC Server application
4. gRPC Client application

 

gRPC vs REST

1. Rules used : gRPC enforces given rules that need to be adhered to in a .proto file. The client and the server have to abide by the rules defined in the .proto file. On the other hand, REST does not enforce rules and guidelines that have been set up to create APIs.
2. HTTP Protocols used : The underlying protocol for gRPC is HTTP2. It allows for client response mode of communication which is used to design web APIs that depend on the HTTP2 protocol. On the other hand, REST uses the request response model  built on top of HTTP 1.1 protocol. Therefore multiple requests made to a REST API will be handled one by one at a time.
3. Data exchange format: gRPC uses protocol buffers for data exchange  over the HTTP2 protocol while REST uses JSON and XML format for data transfer over the HTTP 1.1 protocol.
4. Latency: gRPC relies on the HTTP2 protocol that uses multiplexed streams. This allows for several clients to send multiple requests simultaneously without establishing a new TCP connection for each client request. On the other hand REST uses the HTTP 1.1 per request for TCP handshake , which consumes time hence causing latency issues.
5. Data structure for the payload: gRPC enforces strongly typed messages by default to serialize data payload. This payload is highly compressed , hence lighter and small in size. On the other hand, REST uses JSON or XML to send and receive data which is easy to read but slow compared to gRPC.
6. Browser support: gRPC browser support is limited because it requires a gRPC-web and proxy layer to perform conversions between HTTP1.1 and HTTP2. On the other hand, REST is supported by all browsers available, because the protocol used is HTTP 1.1 which is what is being used by browsers.

 

When to use gRPC

gRPC is an excellent option for working with multi language systems, real time streaming like in IOT application systems which require light-weight message transmission can also be used in mobile applications because they do not need a browser and benefit on small light messages hence preserving processor speed.

 

Prerequisites

1. Go runtime : any three latest major releases. If you haven't already installed Go, you can manually install go.
2. Protocol Buffer compiler, protoc, version 3. Refer protoc-installation for more info.
3. Go plugins for the protocol buffer compiler.

Install plugins by issuing the below commands in the terminal

$ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28
$ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2

Update your path so that the protoc compiler can find plugins.

$ export PATH="$PATH:$(go env GOPATH)/bin"

 

Set up Lab Environment (Application Structure)

In the next section we are going to learn more about protocol buffers as we implement a Go application. In this article, we are going to create a simple todo application with a create todo functionality. The image below shows how the application structure will look like. Please note that we are going to run two applications separately namely server and client. In our application we will assume that these two applications live in different computers and communicate with one another using protocol buffers. That explain the simplified application structure seen in the below image.

Example

Follow the below steps to create the above application structure.

Create a working directory

$ mkdir grpc-todo

Move into the working directory

$ cd grpc-todo/

Create a go module

$ go mod init example.com/grpc-todo

Create server , client and proto folders

$ mkdir server client proto

 

Step-1: Create Protocol buffer code (.proto file)

What are Protocol Buffers

Protocol buffer is an open source mechanism for serializing structured data  and can also be used with other data formats like JSON. Protocol buffers are used to define the structure of data that will be used as well as the services that will be used together with the data. The structure of the data is defined in a file with  a .proto extension. The data in a .proto file is a structured message where each message is a small logical record of information that contains a series of name value pairs called fields.

 

Create proto file

To get up and running, we will first start by defining our application data in our proto file. In your root folder navigate into the proto folder and create a proto file called todo.proto as shown below,

$ cd proto && touch todo.proto.

The above command will move into the proto folder and create a todo.proto file. In the todo.proto file, add the below code.

Example : proto/todo.proto file

syntax="proto3";
 
package proto;
 
option go_package = "example.com/grpc-todo";
 
 
message NewTodo {
   string name = 1;
   string description = 2;
   bool done = 3;
}
 
message Todo {
   string name = 1;
   string description = 2;
   bool done = 3;
   string id = 4;
}
 
service TodoService {
   rpc CreateTodo(NewTodo) returns (Todo) {}
}

Explanation

syntax=”proto3” : defining the syntax of proto that is going to be used. In this example we are using proto3 , which is the latest syntax. If you do not specify the syntax, proto2 syntax will be used by default. So ensure all the time that you are using syntax=”proto3” as the first line of code.

package proto:  is a command that specifies where our proto file is located in our application. In our example, our protocol buffer data will be put inside the proto/todo.proto file.

option go_package = “example.com/grpc-todo” : Specifies the package for our application for good dependency management.

Message definition

message Todo{}andNewTodo{}: Refers to the structured data that we want to use in our application. When using the proto3 syntax, the data that will be used is defined using the keyword message followed by the name of the message. In the message body, we define the properties of the message with fields. A field defines the type of data and the name of data and a unique number. Our NewTodo message for example has three fields name and description of type string and a done field of bool type. The syntax of a defining a field is <data type> <data name> = 1;.

The number that is assigned to each field is used to the field in a binary format and cannot be changed once the message type is in use. For more information refer protocol buffer

Service definition

In our sample todo.proto file, we define two messages , NewTodo{} and Todo{}.

Next we define a service that will make use of the defined data. To define a service use the <b>service</b> keyword followed by the name of the service. In our example, we create a <b>TodoService{}</b>. Use the rpc keyword followed by the name of the method that the service will use. A service body defines the <b>rpc</b> methods that will be called by a client and implemented by a server. The method CreateTodo() takes a <b>NewTodo</b> parameter and returns a <b>Todo</b>. This basically means that the server will expect structured data that has the properties that message NewTodo has. The server will return a Todo response to the client with a new id field added. This is so because the server will be responsible for adding an id for each new todo item created. Please note that proto statements end with semicolons.

 

Compile todo.proto file

After successfully creating a todo.proto code, we need to compile the code in order to create methods that the server implements and the clients calls. The next commands will compile the todo.proto file.

$ protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative proto/todo.proto

After running the above command, the proto folder will have two new files namely todo_grpc.pb.go and todo.pb.go. The todo_grpc.pb.go file contains code for populating, serializing and retrieving NewTodo and Todo message types. On the other hand, todo.pb.go contains client and server code. Take a look into these files and you will notice that the generated code makes our work easier.

 

Step-2: Create golang gRPC Server application

Next we need to create the server application. The server will be responsible for running our gRPC server application and responding to requests from the client application. The code for the server application lives in server/main.go file.Move to the server directory and create a main.go file .

Example

$ cd server && touch main.go

In the above command, we navigate to the server folder of our application and create a main.go file. The main.go file is responsible for starting a  new gRPC server that the client will make requests to.

Example : server/main.go

package main

import (
	"context"
	"log"
	"net"

	pb "example.com/grpc-todo/proto"
	"github.com/google/uuid"
	"google.golang.org/grpc"
)

const (
	// Port for gRPC server to listen to
	PORT = ":50051"
)

type TodoServer struct {
	pb.UnimplementedTodoServiceServer
}

func (s *TodoServer) CreateTodo(ctx context.Context, in *pb.NewTodo) (*pb.Todo, error) {
	log.Printf("Received: %v", in.GetName())
	todo := &pb.Todo{
		Name:        in.GetName(),
		Description: in.GetDescription(),
		Done:        false,
		Id:          uuid.New().String(),
	}

	return todo, nil

}

func main() {
	lis, err := net.Listen("tcp", PORT)

	if err != nil {
		log.Fatalf("failed connection: %v", err)
	}

	s := grpc.NewServer()

	pb.RegisterTodoServiceServer(s, &TodoServer{})

	log.Printf("server listening at %v", lis.Addr())

	if err := s.Serve(lis); err != nil {
		log.Fatalf("failed to server: %v", err)
	}
}

Explanation

In the above server code, we import the proto code from the proto folder using import “example.com/grpc-todo/proto” as pb. pb will give us access to lots of functionality for both server and client code. We also import context, log, time, and grpc packages.

We start off by defining  the port that the server application will be listening on.After defining the port , we define a TodoServer struct that embeds (composes)pb.UnimplementedTodoServiceServer to register it with gRPC server. pb.UnimplementedTodoServiceServer was created by default when we compiled the todo.proto file and it is defined in the tod_grpc.pb.go file in the proto folder.

We then define a receiver function called CreateTodo on the TodoServer struct that implements our TodoServer. The CreateTodo() function is the same function defined in the TodoService in the todo.proto file. The CreateTodo() receiver function takes a context and a *pb.NewTodo and returns a new Todo and an error. The CreateTodo receiver function is responsible for assembling a new todo and adding a Id to the todo and returning the new todo to the caller.

In the main function, we start a new TCP connection that listens on port “:50051” defined earlier using the command lis, err := net.Listen(“tcp”, PORT). After a successful connection, we create  a gRPC server instance using the command s := grpc.NewServer().

Next, we register our TodoServer{} defined earlier with our gRPC server using the command pb.RegisterTodoServiceServer(s, &TodoServer{}). Lastly, we spin the gRPC server using the command s.Server(lis).

 

Step-3: Create golang gRPC Client application

Next thing is to create client side code. The client is responsible for sending create todo requests to the server.

Example client/main.go

package main
 
import (
   "context"
   "log"
   "time"
 
   pb "example.com/grpc-todo/proto"
 
   "google.golang.org/grpc"
)
 
const (
   ADDRESS = "localhost:50051"
)
 
type TodoTask struct {
   Name        string
   Description string
   Done        bool
}
 
func main() {
   conn, err := grpc.Dial(ADDRESS, grpc.WithInsecure(), grpc.WithBlock())
 
   if err != nil {
       log.Fatalf("did not connect : %v", err)
   }
 
   defer conn.Close()
 
   c := pb.NewTodoServiceClient(conn)
 
   ctx, cancel := context.WithTimeout(context.Background(), time.Second)
 
   defer cancel()
 
   todos := []TodoTask{
       {Name: "Code review", Description: "Review new feature code", Done: false},
       {Name: "Make YouTube Video", Description: "Start Go for beginners series", Done: false},
       {Name: "Go to the gym", Description: "Leg day", Done: false},
       {Name: "Buy groceries", Description: "Buy tomatoes, onions, mangos", Done: false},
       {Name: "Meet with mentor", Description: "Discuss blockers in my project", Done: false},
   }
 
   for _, todo := range todos {
       res, err := c.CreateTodo(ctx, &pb.NewTodo{Name: todo.Name, Description: todo.Description, Done: todo.Done})
 
       if err != nil {
           log.Fatalf("could not create user: %v", err)
       }
 
       log.Printf(`
           ID : %s
           Name : %s
           Description : %s
           Done : %v,
       `, res.GetId(), res.GetName(), res.GetDescription(), res.GetDone())
   }
 
}

Explanation

In the client side code, we also import the grpc package together with the protocol buffer code. The client will be serving on port “localhost:50051” while the server will be listening on port “50052”, note the difference. We then define a todo struct called TodoTask that has Name, Description and Done fields of type string, string and bool respectively.

In the main function, we create a connection to gRPC by calling the Dial() function if the connection is successful. Next we bind client connections with proto code using c := pb.NewTodoServiceClient(conn). We then create an array of TodoTask{} and the range over the array. We loop through each task and call CreateTodo() function and pass each todo. For each successful task creation we log the response on the terminal.

 

Step-4: Verifying golang gRPC server and client

Before running the code, we need to get/download all the dependencies we need into our program using the below command.

$ go mod tidy.

Since the server and client are two different applications, they are run differently. The first application to be run is the server followed by the client application. On a separate terminal, but in the same root folder , issue the below commands in order.

Terminal 1 : server application

$ go run server/main.go
2022/11/06 19:21:29 server listening at [::]:50051
2022/11/06 19:21:53 Received: Code review
2022/11/06 19:21:53 Received: Make YouTube Video
2022/11/06 19:21:53 Received: Go to the gym
2022/11/06 19:21:53 Received: Buy groceries
2022/11/06 19:21:53 Received: Meet with mentor

Terminal 2: client application

$ go run client/main.go
2022/11/06 19:21:53
                       ID : 09fa7744-d64b-48e1-adf1-3e962fd39afd
                       Name : Code review
                       Description : Review new feature code
                       Done : false,
 
2022/11/06 19:21:53
                       ID : b3eb84c6-b42e-43be-8555-210dc88e0660
                       Name : Make YouTube Video
                       Description : Start Go for beginners series
                       Done : false,
 
2022/11/06 19:21:53
                       ID : 49df38db-8aa0-4ca7-a15e-b3f47d319800
                       Name : Go to the gym
                       Description : Leg day
                       Done : false,
 
2022/11/06 19:21:53
                       ID : e00243d6-c9aa-4c93-8d22-fdf2dcfc9a77
                       Name : Buy groceries
                       Description : Buy tomatoes, onions, mangos
                       Done : false,
 
2022/11/06 19:21:53
                       ID : 23fd2960-0f2c-4b9d-adba-f8ddd13f78b8
                       Name : Meet with mentor
                       Description : Discuss blockers in my project
                       Done : false,

Explanation

In the above example, we run both the client and server code. The client loops through an array of tasks and sends a request to the server. The server returns back to the client, the newly created tasks and loops the response to the terminal. The server also receives create todo requests from the client, sets up new tasks, and sends back the response to the client.

 

Summary

gRPC is the new kid in the block and challenges other architectures like SOAP and REST. It offers good performance, streaming capabilities, interoperability, code generation from a proto file and speed. Despite all the good things it has, gRPC has limited browser support, has a steep learning curve and uses non-readable readable data formats. Companies like Google, Netflix, Square  IBM, Cisco and dropbox are using gRPC so should you.

 

References

https://grpc.io/docs/languages/go/quickstart/
Quick start | Go - gRPC
The Go language implementation of gRPC. HTTP/2 based RPC
grpc - Go Packages