gRPC is a language-neutral, platform-neutral remote procedure call (RPC) framework and toolset developed at Google. It lets you define a service using Protocol Buffers, a particularly powerful binary serialization toolset and language. It then lets you generate idiomatic client and server stubs from your service definition in a variety of languages.

In this codelab, you'll learn how to build a Node.js service that exposes an API using the gRPC framework. You'll interact with this service using a command line client written in Go that uses the same service description as the Node.js service. Finally, you will write a Node.js command line client for the gRPC service.

What you'll learn

What you'll need - We will install those along the way

How will you use use this tutorial?

Read it through only Read it and complete the exercises

How would you rate your experience with building Node.js apps?

Novice Intermediate Proficient

How would you rate your experience with building Go apps?

Novice Intermediate Proficient

Codelab-at-a-conference setup

The instructor will be sharing with you temporary accounts with existing projects that are already setup so you do not need to worry about enabling billing or any cost associated with running this codelab. Note that all these accounts will be disabled soon after the codelab is over.

Once you have received a temporary username / password to login from the instructor, log into Google Cloud Console: https://console.cloud.google.com/.

Here's what you should see once logged in :

Note the project ID you were assigned ( "codelab-test003" in the screenshot above). It will be referred to later in this codelab as PROJECT_ID.

Google Cloud Shell

While this codelab can be operated from your computer, in this codelab we will be using Google Cloud Shell, a command line environment running in the Cloud.

This Debian-based virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory, and runs on the Google Cloud, greatly enhancing network performance and authentication. This means that all you will need for this codelab is a browser (yes, it works on a Chromebook).

To activate Google Cloud Shell, from the developer console simply click the button on the top right-hand side (it should only take a few moments to provision and connect to the environment):

Then accept the terms of service and click the "Start Cloud Shell" link:

Once connected to the cloud shell, you should see that you are already authenticated and that the project is already set to your PROJECT_ID :

gcloud auth list

Command output

Credentialed accounts:
 - <myaccount>@<mydomain>.com (active)
gcloud config list project

Command output

[core]
project = <PROJECT_ID>

If for some reason the project is not set, simply issue the following command :

gcloud config set project <PROJECT_ID>

Looking for your PROJECT_ID? Check out what ID you used in the setup steps or look it up in the console dashboard:

IMPORTANT: Finally, set the default zone and project configuration:

gcloud config set compute/zone us-central1-f

You can choose a variety of different zones. Learn more in the Regions & Zones documentation.

To run the sample application, the following dependencies are required:

All these dependencies have been pre-installed for you on Google Cloud Shell which we will use for this codelab.

Go to the Developer Console, it should automatically log you in based on your provided login. If you get asked for a username/password, please reuse the credentials we provided to you.

You need to accept the terms of service before you'll be redirected to the Developer Console.

In the upper right corner you'll see the Cloud Shell symbol . Click on it to open your very own Cloud Shell session. All following command line instructions are executed in the Cloud Shell session.

You can open additional Cloud Shell sessions by clicking on "+" in the Cloud Shell sessions tab bar .

So you're ready to get started!

Summary

In this step, you set up your development environment.

Next up

Next, you will download and run the sample application.

Clone the GitHub repository from the command line:

git clone https://github.com/googlecodelabs/cloud-grpc

Sample project layout:

Name

Description

client.go

Command-line client for the server API.

books/books.pb.go

go library for the books gRPC service.

Run the sample application

The sample application folder contains client.go, a command-line client for interacting with the gRPC service that you will create in this codelab.

To run the command-line client, first install the Go grpc package:

$ go get -u google.golang.org/grpc

Now, from the project directory, run the command-line client with no arguments to view the available commands:

$ go run client.go 
client.go is a command-line client for this codelab's gRPC service

Usage:
  client.go list                            List all books
  client.go insert <id> <title> <author>    Insert a book
  client.go get <id>                        Get a book by its ID
  client.go delete <id>                     Delete a book by its ID
  client.go watch                           Watch for inserted books

Try calling one of the available commands:

$ go run client.go list

You will see a list of errors after a few seconds because the node gRPC server does not yet exist!

Let's fix this!

Summary

In this step, you set up and ran the codelab sample application.

Next up

Next, you will implement a Node.js gRPC service that lists books.

In this step you will write the code to implement a Node.js gRPC service that lists books.

gRPC services are defined in .proto files using the protocol buffer language.

The protocol buffer language is used to define services and message types.

Let's start by defining a service for books!

In the project directory cloud-grpc/start, create a new file called books.proto and add the following:

books.proto

syntax = "proto3";

package books;

service BookService {
  rpc List (Empty) returns (Empty) {}
}

message Empty {}

This defines a new service called BookService using the proto3 version of the protocol buffers language. This is the latest version of protocol buffers and is recommended for use with gRPC.

To run this service with node, first install the grpc npm package:

$ npm install grpc

Now, still in our working directory cloud-grpc/start, let's build the Node.js service by starting with a new file called server.js and add the following:

server.js

var grpc = require('grpc');

var booksProto = grpc.load('books.proto');

var server = new grpc.Server();

server.addService(booksProto.books.BookService.service, {});

server.bind('0.0.0.0:50051',
  grpc.ServerCredentials.createInsecure());
console.log('Server running at http://0.0.0.0:50051');
server.start();

This code snippet creates and starts a new gRPC server with the books proto service descriptor and binds to port 50051 with a credentials object.

Open a 2nd Cloud Shell session where we will run the service. Navigate to your working directory and run the server:

$ node server.js

Now run the go gRPC command-line client in your other Cloud Shell Session again to test the server API:

$ go run client.go list

This time, you will see a new error:

Great! This means that the server is running and the go client communicated with it over gRPC!

Let's fix this by implementing a List method!

List books

In the project directory, edit books.proto and update BookService with the following code:

books.proto

syntax = "proto3";

package books;

service BookService {
  rpc List (Empty) returns (BookList) {}
}

message Empty {} 

message Book {
  int32 id = 1;
  string title = 2;
  string author = 3;
}

message BookList {
  repeated Book books = 1;
}

Each service rpc method accepts a (request) message and returns a (response) message.

Making a request to List books requires no parameters so the request message for List has no fields (as defined by the Empty message).

The Book message represents a single book object with id, title, and author fields.

List returns a repeated list of Book messages. repeated specifies that this message can be repeated any number of times.

Now update the Node.js application to respond to calls to the List method of the BookService.

Edit the server.js file and add the following code, replacing the current var server and updating the server.addService with the following:

server.js

// In-memory array of book objects
var books = [ 
  { id: 123, title: 'A Tale of Two Cities', author: 'Charles Dickens' }
];

var server = new grpc.Server();
server.addService(booksProto.books.BookService.service, {
    list: function(call, callback) {
        callback(null, books);
    }
});

To register handlers for gRPC service methods, handler functions are passed into the Server constructor for each method. When the service method is invoked, the handler function is called with a call object representing the request message. To respond to the method, call callback providing an error object (or null) and an object representing the response message. In this case, we return a JavaScript object with fields matching those defined in book.proto for the Book message type.

This implements the List rpc call, returning a Book message.

To test this, stop your running node process in your 2nd Cloud Shell session by pressing CTRL-C and run it again:

$ node server.js

Switch back to your other shell session and run the go gRPC command-line client again:

$ go run client.go list

You should now see this book listed!

Got 1 books.
{
  "books": [
    {
      "id": 123,
      "title": "A Tale of Two Cities",
      "author": "Charles Dickens"
    }
  ]
}

Summary

In this step, you implemented a gRPC service that lists books.

Next up

Next, you will insert new books via gRPC calls.

In this step you will write the code to implement adding new Book objects via the gRPC service.

To begin, edit books.proto and update BookService to the following:

books.proto

service BookService {
  rpc List (Empty) returns (BookList) {}
  // add the following line
  rpc Insert (Book) returns (Empty) {}
}

This defines a new Insert rpc call that takes a Book message as its request and returns an Empty response.

To implement the Insert method in the server, edit server.js and add the insert method to the function map in addService:

server.js

server.addService(booksProto.books.BookService.service, {
    list: function(call, callback) {
        callback(null, books);
    },
    // add the insert method
    insert: function(call, callback) {
        var book = call.request;
        books.push(book);
        callback(null, {});
    }
});

The added insert function implements the Insert rpc call, adding the received Book message to the books array and returning an Empty message.

Handler functions access the request message via call.request. In this case, call.request is a JavaScript object with id, title, and author fields representing a Book message.

To test this, restart the node server and then run the go gRPC command-line client's insert command, passing id, title, and author as arguments:

$ go run client.go insert 2 "The Three Musketeers" "Alexandre Dumas"

You should see an empty response:

Server response:
{}

To verify that the book was inserted, run the list command again to see all books:

$ go run client.go list

You should now see 2 books listed!

Got 2 books.
{
  "books": [
    {
      "id": 123,
      "title": "A Tale of Two Cities",
      "author": "Charles Dickens"
    },
    {
      "id": 2,
      "title": "The Three Musketeers",
      "author": "Alexandre Dumas"
    }
  ]
}

Summary

In this step, you extended the gRPC service to support adding books.

Next up

Next, you will extend the gRPC service further to support getting and deleting individual books.

In this step you will write the code to get and delete Book objects by id via the gRPC service.

To begin, edit books.proto and update BookService with the following:

books.proto

service BookService {
  rpc List (Empty) returns (BookList) {}
  rpc Insert (Book) returns (Empty) {}
  // add the following line
  rpc Get (BookIdRequest) returns (Book) {}
}

// add the message definition below
message BookIdRequest {
  int32 id = 1;
}

This defines a new Get rpc call that takes a BookIdRequest as its request and returns a Book as its response.

A BookIdRequest message type is defined for requests containing only a book's id.

To implement the Get method in the server, edit server.js and add the following get handler function:

server.js

server.addService(booksProto.books.BookService.service, {
    // ...
    // add the following get method
    get: function(call, callback) {
        for (var i = 0; i < books.length; i++)
            if (books[i].id == call.request.id)
                return callback(null, books[i]);
        callback({
            code: grpc.status.NOT_FOUND,
            details: 'Not found'
        });
    }
});

If the books array contains a book with the id requested, the book is returned. If no book is found with the requested id, a NOT_FOUND error is returned.

To test this, restart the node server and then run the go gRPC command-line client's get command, passing id as an argument:

$ go run client.go get 123

You should see the book response!

Server response:
{
  "id": 123,
  "title": "A Tale of Two Cities",
  "author": "Charles Dickens"
}

Now try getting a book that doesn't exist:

$ go run client.go get 404

You should see the error message returned:

Delete books

Now you will write the code to delete a book by id.

Edit books.proto and add the following Delete rpc method:

books.proto

service BookService {
  // ...
  // add the delete method definition
  rpc Delete (BookIdRequest) returns (Empty) {}
}

Now edit server.js and add the following delete handler function:

server.js

server.addService(booksProto.books.BookService.service, {
    // ...
    // add the following delete method
    delete: function(call, callback) {
        for (var i = 0; i < books.length; i++) {
            if (books[i].id == call.request.id) {
                books.splice(i, 1);
                return callback(null, {});
            }
        }
        callback({
            code: grpc.status.NOT_FOUND,
            details: 'Not found'
        });
    }
});

If the books array contains a book with the id requested, the book is removed, otherwise a NOT_FOUND error is returned.

To test this, restart the node server and then run the go gRPC command-line client to delete a book:

$ go run client.go list
Server sent 1 book(s).
{
  "books": [
    {
      "id": 123,
      "title": "A Tale of Two Cities",
      "author": "Charles Dickens"
    }
  ]
}

$ go run client.go delete 123
Server response:
{}

$ go run client.go list
Server sent 0 book(s).
{}

$ go run client.go delete 123
Delete book (123): rpc error: code = 5 desc = "Not found"

Great!

You implemented a fully functioning gRPC service that can list, insert, get, and delete books!

Summary

In this step, you extended the gRPC service to support getting and deleting books.

Next up

Next, you will add the ability for the client to connect to a stream that will receive books when they are inserted.

In this step you will write the code to add a streaming endpoint to the service so the client can establish a stream to the server and listen for added books.

gRPC supports streaming semantics, where either the client or the server (or both) send a stream of messages on a single RPC call. The most general case is Bidirectional Streaming where a single gRPC call establishes a stream where both the client and the server can send a stream of messages to each other.

To begin, edit books.proto and add the following Watch rpc method to BookService:

books.proto

service BookService {
  // ...
  // add the watch method definition
  rpc Watch (Empty) returns (stream Book) {}
}

When the client calls the Watch method, it will establish a stream and server will be able to stream Book messages when books are inserted.

To implement the Watch method in the server, install the events npm package.

$ npm install events

Now edit server.js and add the following events package, bookStream event listener and watch handler function:

server.js

// add the events package
var events = require('events');
// add the bookStream global variable
var bookStream = new events.EventEmitter();

var server = new grpc.Server();
server.addService(booksProto.books.BookService.service, {
    // ...
    // add the following watch method
    watch: function(stream) {
        bookStream.on('new_book', function(book){
            stream.write(book);
        });
    }
});

Handler functions for streaming rpc methods are invoked with a writable stream object.

To stream messages to the client, the stream's write() function is called when an new_book event is emitted.

Edit server.js and update the insert function to emit a new_book event when books are inserted:

server.js

var server = new grpc.Server();
server.addService(booksProto.books.BookService.service, {
    // ...
    insert: function(call, callback) {
        var book = call.request;
        books.push(book);

        // add the following to the insert method
        bookStream.emit('new_book', book);

        callback(null, {});
    },
    // ...
});

To test this, restart the node server and then run the go gRPC command-line client's watch command in a 3rd Cloud Shell Session:

$ go run client.go watch

Now run the go gRPC command-line client's insert command in your main Cloud Shell session to insert a book:

$ go run client.go insert 2 "The Three Musketeers" "Alexandre Dumas"

Check the Cloud Shell session where the client.go watch process is running. It should have printed out the inserted book!

$ go run client.go watch
Server stream data received:
{
  "id": 2,
  "title": "The Three Musketeers",
  "author": "Alexandre Dumas"
}

Press CTRL-C to exit the client.go watch process.

Summary

In this step, you added a streaming gRPC endpoint to the service to stream inserted books to a connected client.

Next up

Next, you will write a command-line client to interact with your gRPC service.

In this step, you will write the code to implement a Node.js command-line client that calls your gRPC service.

The result will be functionally equivalent to the client.go script that you have been using so far in this codelab!

Start by running the gRPC server again if it isn't running already:

$ node server.js

Now create a new file called client.js in the project directory and add the following:

client.js

var grpc = require('grpc');

var booksProto = grpc.load('books.proto');

var client = new booksProto.books.BookService('127.0.0.1:50051', 
  grpc.credentials.createInsecure());

client.list({}, function(error, books) {
  if (error)
    console.log('Error: ', error);
  else
    console.log(books);
});

This requires the grpc node module and loads books.proto (exactly like you did in server.js previously).

The client object for the gRPC service is created by calling the BookService constructor, which is dynamically created from the service definition found in books.proto.

The list() function takes a request message object as a parameter ({} in this case to represent an Empty message), followed by a callback function that will be invoked with an error object (or null) and a response message object (a Book message in this case).

books.proto

service BookService {
  rpc List (Empty) returns (BookList) {}
  rpc Insert (Book) returns (Empty) {}
  rpc Get (BookIdRequest) returns (Book) {}
  rpc Delete (BookIdRequest) returns (Empty) {}
  rpc Watch (Empty) returns (stream Book) {}
}

This means that you can now list(), insert(), get(), delete(), and watch() books!

Now run the command-line client:

$ node client.js list
{ books: 
   [ { id: 123,
       title: 'A Tale of Two Cities',
       author: 'Charles Dickens' } ] }

You should see books listed!

Next, to implement the command-line client, update client.js with functions for list(), insert(), get(), and delete():

client.js

var grpc = require('grpc');

var booksProto = grpc.load('books.proto');

var client = new booksProto.books.BookService('127.0.0.1:50051', 
  grpc.Credentials.createInsecure());

function printResponse(error, response) {
  if (error)
    console.log('Error: ', error);
  else
    console.log(response);
}

function listBooks() {
  client.list({}, function(error, books) {
    printResponse(error, books);
  });
}

function insertBook(id, title, author) {
  var book = { id: parseInt(id), title: title, author: author };
  client.insert(book, function(error, empty) {
    printResponse(error, empty);
  });
}

function getBook(id) {
  client.get({ id: parseInt(id) }, function(error, book) {
    printResponse(error, book);
  });
}

function deleteBook(id) {
  client.delete({ id: parseInt(id) }, function(error, empty) {
    printResponse(error, empty);
  });
}

To implement the watch() function, which receives a stream of Book messages, an event handler is registered:

client.js

// add the following section
function watchBooks() {
  var call = client.watch({});
  call.on('data', function(book) {
    console.log(book);
  });
}

The on('data') function callback will be called with a Book message object whenever a book is inserted.

Finally, add code to parse command-line arguments:

client.js

// add the following section
var processName = process.argv.shift();
var scriptName = process.argv.shift();
var command = process.argv.shift();

if (command == 'list')
  listBooks();
else if (command == 'insert')
  insertBook(process.argv[0], process.argv[1], process.argv[2]);
else if (command == 'get')
  getBook(process.argv[0]);
else if (command == 'delete')
  deleteBook(process.argv[0]);
else if (command == 'watch')
  watchBooks();

Now run the command-line client:

$ node client.js list
{ books: 
   [ { id: 123,
       title: 'A Tale of Two Cities',
       author: 'Charles Dickens' } ] }

All of the commands you previously ran via client.go should now be available via your Node.js gRPC command-line client!

$ node client.js insert 2 "The Three Musketeers" "Alexandre Dumas"
{}

$ node client.js list
{ books: 
   [ { id: 123,
       title: 'A Tale of Two Cities',
       author: 'Charles Dickens' },
     { id: 2,
       title: 'The Three Musketeers',
       author: 'Alexandre Dumas' } ] }

$ node client.js delete 123
{}

$ node client.js list
{ books: 
   [ { id: 2,
       title: 'The Three Musketeers',
       author: 'Alexandre Dumas' } ] }

$ node client.js get 2
{ id: 2,
  title: 'The Three Musketeers',
  author: 'Alexandre Dumas' }

Summary

In this step, you wrote a command-line client in Node.js that interacts with your gRPC service.

What we've covered:

Next Steps:

Give us your feedback