Thanks to its efficiency and support for numerous programming languages, gRPC is a popular choice for microservice integrations and client-server communications. gRPC is a high performance remote procedure call (RPC) framework using HTTP/2 for transport and Protocol Buffers to describe the interface.
To make it easier to use gRPC with your applications, Application Load Balancer (ALB) now supports HTTP/2 end-to-end, enabling you to publish gRPC services alongside non-gRPC services via a single load balancer. You can use Amazon Elastic Compute Cloud (EC2) instances or IP addresses (for example with AWS Fargate) as gRPC targets, with support for gRPC health checks for the target groups. In this way, you can use ALBs to terminate, route and load balance the gRPC traffic between your microservices or between gRPC-enabled clients and services.
ALB provides rich content based routing features to inspect the gRPC calls and route them to appropriate services. More specifically, the ALB provides health checks that can examine the gRPC status code, metrics for gRPC request count, access logs that differentiate gRPC requests, and gRPC specific response headers. Additionally, you can benefit from native features like stickiness, different load balancing algorithms, and TLS termination.
How to Use gRPC with Application Load Balancer
To test this new feature, I start by preparing the gRPC server application. I am using the Python programming language and the route_guide demo included in the grpc repo. I use this application because it quickly introduces some of the many ways a client and a server can interact via gRPC, such as:
- A simple unary remote procedure call (RPC) – the client sends a request to the server and waits for a response, similar to how a function call works.
- Server-side streaming RPC – the client sends a request to the server, gets back a stream of messages, and reads from the stream until there are no more messages.
- Client-side streaming RPC – the client writes a sequence of messages, sends them to the server, and then waits for the server to read them all and return its response.
- Bidirectional streaming RPC – client and server both send messages independently, preserving the order.
First, I prepare a Dockerfile to have the route_guide application run in a container. Technically, this isn’t necessary because I could use a plain EC2 instance, but using containers with gRPC is so common than this makes the example more relevant.
FROM python:3.7
RUN pip install protobuf grpcio
COPY ./grpc/examples/python/route_guide .
CMD python route_guide_server.py
EXPOSE 50051
I build the container image and upload it to Amazon Elastic Container Registry.
aws ecr get-login-password --region eu-north-1 | docker login --username AWS --password-stdin 123412341234.dkr.ecr.eu-north-1.amazonaws.com
docker build -t route-guide .
docker tag route-guide:latest 123412341234.dkr.ecr.eu-north-1.amazonaws.com/route-guide:latest
docker push 123412341234.dkr.ecr.eu-north-1.amazonaws.com/route-guide:latest
In the Amazon ECS console, I create a new cluster using the Networking only template. I name the cluster demo and create a new VPC for this cluster, leaving all other values to their default. The ECS console is using AWS CloudFormation to set up the resources for the cluster. After a few minutes, all resources have been successfully created and the cluster is ready.
To give access to the gRPC traffic, I create a security group to allow inbound TCP traffic on port 50051 from my client laptop. I name this security group gRPC.
Now, I create the load balancer. In the EC2 console, I select Load Balancers on the left, and then Create Load Balancer. In the next step, I create an Application Load Balancer and name it route-guide. I leave the default internet-facing Scheme, because I want to connect to the gRPC service over public internet. In the Listeners section, I select HTTPS as protocol and 50051 for the port. Just below, I select the newly created VPC and the two Availability Zones created by the ECS console.
Next, in the security settings, I choose a certificate I created before, managed by AWS Certificate Manager (ACM).
Then, I select the default security group and the gRPC security group I created above. Each VPC automatically comes with a default security group that gives network access to other resources using the same security group.
In the routing section, I create a new target group named route-guide. Since I am planning to use AWS Fargate to run the gRPC server, I select the IP address target type. The ALB supports both secure and insecure connections for target groups using the gRPC protocol. Here, I use HTTP (because I am using an insecure connection between the load balancer and the containers running the gRPC server) with port 50051. Then, I select the VPC used by the ECS cluster. For Protocol Version, I use gRPC.
In the Advanced health check settings, I can specify which gRPC Success codes to use when checking for a correct response. I leave the default of 12 which means unimplemented.
Code 12 is returned by a gRPC server if a method is not found. It works in this case because I’m using a path that is not implemented by the route_guide application. More generally, checking for code 12 is a quick way to verify that your gRPC server is running correctly. To be more specific in your health checks, you can use a single code, a list, or a range, depending on what you expect from your implementation.
In the next step, I don’t register any target and complete the creation of the target group.
Back in the ECS console, I create a new task definition compatible with the Fargate launch type. I name it route-guideand give it a minimum amount of resources: 0.5 GB of memory and 0.25 CPU units. I add a container definition using the image URI of the container image I uploaded to ECR, and the container port 50051/tcp, the network port exposed in the Dockerfile above.
With the route-guidetask definition selected, I select Create Service in the Actions menu. I use the Fargate launch type, route-guide as service name, and 2 for the number of tasks. In the next step, I add the two subnets in the VPC, and select the default security group to allow the load balancer to reach the tasks. In the Load balancing section, I select Application Load Balancer and the route-guide load balancer.
For the Container to balance, I select route-guide:50051:50051 and Add to load balancer.
Then, I choose the route-guide target group.
At the next step, I select Do not adjust the service’s desired count to not use auto scaling for this demo.
I complete the creation of the ECS service. After a few minutes the two tasks are RUNNING. Looking at the Targets tab of the route-guide target group, I see that the two targets are healthy. The load balancer is now ready to accept traffic.
In the Amazon Route 53 console, I create a DNS A record as an alias for the load balancer. The domain I use is matching the domain name of the certificate I selected when creating the load balancer.
In my client laptop, I edit the route_client.py file in the gRPC repo to use a secure channel when connecting to the load balancer. This is the part of the code that I change:
# create credentials
credentials = grpc.ssl_channel_credentials()
# create secure channel using credentials
with grpc.secure_channel('<ALB-ALIAS-DOMAIN>:50051', credentials) as channel:
stub = route_guide_pb2_grpc.RouteGuideStub(channel)
print("-------------- GetFeature --------------")
guide_get_feature(stub)
print("-------------- ListFeatures --------------")
guide_list_features(stub)
print("-------------- RecordRoute --------------")
guide_record_route(stub)
print("-------------- RouteChat --------------")
guide_route_chat(stub)
Now, I start the client to test the gRPC channel with some workload. In the output, I see that the route_guide application is using unary, server-side streaming, client-side streaming, and bi-directional streaming in the request and the response.
$ python3 route_guide_client.py
-------------- GetFeature --------------
Feature called Berkshire Valley Management Area Trail, Jefferson, NJ, USA at latitude: 409146138
longitude: -746188906
Found no feature at
-------------- ListFeatures --------------
Looking for features between 40, -75 and 42, -73
Feature called Patriots Path, Mendham, NJ 07945, USA at latitude: 407838351
longitude: -746143763
Feature called 101 New Jersey 10, Whippany, NJ 07981, USA at latitude: 408122808
longitude: -743999179
Feature called U.S. 6, Shohola, PA 18458, USA at latitude: 413628156
longitude: -749015468
Feature called 5 Conners Road, Kingston, NY 12401, USA at latitude: 419999544
longitude: -740371136
Feature called Mid Hudson Psychiatric Center, New Hampton, NY 10958, USA at latitude: 414008389
longitude: -743951297
Feature called 287 Flugertown Road, Livingston Manor, NY 12758, USA at latitude: 419611318
longitude: -746524769
Feature called 4001 Tremley Point Road, Linden, NJ 07036, USA at latitude: 406109563
longitude: -742186778
Feature called 352 South Mountain Road, Wallkill, NY 12589, USA at latitude: 416802456
longitude: -742370183
Feature called Bailey Turn Road, Harriman, NY 10926, USA at latitude: 412950425
longitude: -741077389
Feature called 193-199 Wawayanda Road, Hewitt, NJ 07421, USA at latitude: 412144655
longitude: -743949739
Feature called 406-496 Ward Avenue, Pine Bush, NY 12566, USA at latitude: 415736605
longitude: -742847522
Feature called 162 Merrill Road, Highland Mills, NY 10930, USA at latitude: 413843930
longitude: -740501726
Feature called Clinton Road, West Milford, NJ 07480, USA at latitude: 410873075
longitude: -744459023
Feature called 16 Old Brook Lane, Warwick, NY 10990, USA at latitude: 412346009
longitude: -744026814
Feature called 3 Drake Lane, Pennington, NJ 08534, USA at latitude: 402948455
longitude: -747903913
Feature called 6324 8th Avenue, Brooklyn, NY 11220, USA at latitude: 406337092
longitude: -740122226
Feature called 1 Merck Access Road, Whitehouse Station, NJ 08889, USA at latitude: 406421967
longitude: -747727624
Feature called 78-98 Schalck Road, Narrowsburg, NY 12764, USA at latitude: 416318082
longitude: -749677716
Feature called 282 Lakeview Drive Road, Highland Lake, NY 12743, USA at latitude: 415301720
longitude: -748416257
Feature called 330 Evelyn Avenue, Hamilton Township, NJ 08619, USA at latitude: 402647019
longitude: -747071791
Feature called New York State Reference Route 987E, Southfields, NY 10975, USA at latitude: 412567807
longitude: -741058078
Feature called 103-271 Tempaloni Road, Ellenville, NY 12428, USA at latitude: 416855156
longitude: -744420597
Feature called 1300 Airport Road, North Brunswick Township, NJ 08902, USA at latitude: 404663628
longitude: -744820157
Feature called at latitude: 407113723
longitude: -749746483
Feature called at latitude: 402133926
longitude: -743613249
Feature called at latitude: 400273442
longitude: -741220915
Feature called at latitude: 411236786
longitude: -744070769
Feature called 211-225 Plains Road, Augusta, NJ 07822, USA at latitude: 411633782
longitude: -746784970
Feature called at latitude: 415830701
longitude: -742952812
Feature called 165 Pedersen Ridge Road, Milford, PA 18337, USA at latitude: 413447164
longitude: -748712898
Feature called 100-122 Locktown Road, Frenchtown, NJ 08825, USA at latitude: 405047245
longitude: -749800722
Feature called at latitude: 418858923
longitude: -746156790
Feature called 650-652 Willi Hill Road, Swan Lake, NY 12783, USA at latitude: 417951888
longitude: -748484944
Feature called 26 East 3rd Street, New Providence, NJ 07974, USA at latitude: 407033786
longitude: -743977337
Feature called at latitude: 417548014
longitude: -740075041
Feature called at latitude: 410395868
longitude: -744972325
Feature called at latitude: 404615353
longitude: -745129803
Feature called 611 Lawrence Avenue, Westfield, NJ 07090, USA at latitude: 406589790
longitude: -743560121
Feature called 18 Lannis Avenue, New Windsor, NY 12553, USA at latitude: 414653148
longitude: -740477477
Feature called 82-104 Amherst Avenue, Colonia, NJ 07067, USA at latitude: 405957808
longitude: -743255336
Feature called 170 Seven Lakes Drive, Sloatsburg, NY 10974, USA at latitude: 411733589
longitude: -741648093
Feature called 1270 Lakes Road, Monroe, NY 10950, USA at latitude: 412676291
longitude: -742606606
Feature called 509-535 Alphano Road, Great Meadows, NJ 07838, USA at latitude: 409224445
longitude: -748286738
Feature called 652 Garden Street, Elizabeth, NJ 07202, USA at latitude: 406523420
longitude: -742135517
Feature called 349 Sea Spray Court, Neptune City, NJ 07753, USA at latitude: 401827388
longitude: -740294537
Feature called 13-17 Stanley Street, West Milford, NJ 07480, USA at latitude: 410564152
longitude: -743685054
Feature called 47 Industrial Avenue, Teterboro, NJ 07608, USA at latitude: 408472324
longitude: -740726046
Feature called 5 White Oak Lane, Stony Point, NY 10980, USA at latitude: 412452168
longitude: -740214052
Feature called Berkshire Valley Management Area Trail, Jefferson, NJ, USA at latitude: 409146138
longitude: -746188906
Feature called 1007 Jersey Avenue, New Brunswick, NJ 08901, USA at latitude: 404701380
longitude: -744781745
Feature called 6 East Emerald Isle Drive, Lake Hopatcong, NJ 07849, USA at latitude: 409642566
longitude: -746017679
Feature called 1358-1474 New Jersey 57, Port Murray, NJ 07865, USA at latitude: 408031728
longitude: -748645385
Feature called 367 Prospect Road, Chester, NY 10918, USA at latitude: 413700272
longitude: -742135189
Feature called 10 Simon Lake Drive, Atlantic Highlands, NJ 07716, USA at latitude: 404310607
longitude: -740282632
Feature called 11 Ward Street, Mount Arlington, NJ 07856, USA at latitude: 409319800
longitude: -746201391
Feature called 300-398 Jefferson Avenue, Elizabeth, NJ 07201, USA at latitude: 406685311
longitude: -742108603
Feature called 43 Dreher Road, Roscoe, NY 12776, USA at latitude: 419018117
longitude: -749142781
Feature called Swan Street, Pine Island, NY 10969, USA at latitude: 412856162
longitude: -745148837
Feature called 66 Pleasantview Avenue, Monticello, NY 12701, USA at latitude: 416560744
longitude: -746721964
Feature called at latitude: 405314270
longitude: -749836354
Feature called at latitude: 414219548
longitude: -743327440
Feature called 565 Winding Hills Road, Montgomery, NY 12549, USA at latitude: 415534177
longitude: -742900616
Feature called 231 Rocky Run Road, Glen Gardner, NJ 08826, USA at latitude: 406898530
longitude: -749127080
Feature called 100 Mount Pleasant Avenue, Newark, NJ 07104, USA at latitude: 407586880
longitude: -741670168
Feature called 517-521 Huntington Drive, Manchester Township, NJ 08759, USA at latitude: 400106455
longitude: -742870190
Feature called at latitude: 400066188
longitude: -746793294
Feature called 40 Mountain Road, Napanoch, NY 12458, USA at latitude: 418803880
longitude: -744102673
Feature called at latitude: 414204288
longitude: -747895140
Feature called at latitude: 414777405
longitude: -740615601
Feature called 48 North Road, Forestburgh, NY 12777, USA at latitude: 415464475
longitude: -747175374
Feature called at latitude: 404062378
longitude: -746376177
Feature called at latitude: 405688272
longitude: -749285130
Feature called at latitude: 400342070
longitude: -748788996
Feature called at latitude: 401809022
longitude: -744157964
Feature called 9 Thompson Avenue, Leonardo, NJ 07737, USA at latitude: 404226644
longitude: -740517141
Feature called at latitude: 410322033
longitude: -747871659
Feature called at latitude: 407100674
longitude: -747742727
Feature called 213 Bush Road, Stone Ridge, NY 12484, USA at latitude: 418811433
longitude: -741718005
Feature called at latitude: 415034302
longitude: -743850945
Feature called at latitude: 411349992
longitude: -743694161
Feature called 1-17 Bergen Court, New Brunswick, NJ 08901, USA at latitude: 404839914
longitude: -744759616
Feature called 35 Oakland Valley Road, Cuddebackville, NY 12729, USA at latitude: 414638017
longitude: -745957854
Feature called at latitude: 412127800
longitude: -740173578
Feature called at latitude: 401263460
longitude: -747964303
Feature called at latitude: 412843391
longitude: -749086026
Feature called at latitude: 418512773
longitude: -743067823
Feature called 42-102 Main Street, Belford, NJ 07718, USA at latitude: 404318328
longitude: -740835638
Feature called at latitude: 419020746
longitude: -741172328
Feature called at latitude: 404080723
longitude: -746119569
Feature called at latitude: 401012643
longitude: -744035134
Feature called at latitude: 404306372
longitude: -741079661
Feature called at latitude: 403966326
longitude: -748519297
Feature called at latitude: 405002031
longitude: -748407866
Feature called at latitude: 409532885
longitude: -742200683
Feature called at latitude: 416851321
longitude: -742674555
Feature called 3387 Richmond Terrace, Staten Island, NY 10303, USA at latitude: 406411633
longitude: -741722051
Feature called 261 Van Sickle Road, Goshen, NY 10924, USA at latitude: 413069058
longitude: -744597778
Feature called at latitude: 418465462
longitude: -746859398
Feature called at latitude: 411733222
longitude: -744228360
Feature called 3 Hasta Way, Newton, NJ 07860, USA at latitude: 410248224
longitude: -747127767
-------------- RecordRoute --------------
Visiting point latitude: 400066188
longitude: -746793294
Visiting point latitude: 412452168
longitude: -740214052
Visiting point latitude: 401827388
longitude: -740294537
Visiting point latitude: 414777405
longitude: -740615601
Visiting point latitude: 409642566
longitude: -746017679
Visiting point latitude: 406685311
longitude: -742108603
Visiting point latitude: 406523420
longitude: -742135517
Visiting point latitude: 405047245
longitude: -749800722
Visiting point latitude: 418858923
longitude: -746156790
Visiting point latitude: 409532885
longitude: -742200683
Finished trip with 10 points
Passed 10 features
Travelled 863981 meters
It took 0 seconds
-------------- RouteChat --------------
Sending First message at
Sending Second message at longitude: 1
Sending Third message at latitude: 1
Sending Fourth message at
Sending Fifth message at latitude: 1
Received message First message at
Received message Third message at latitude: 1
Well, the output is pretty long, but as I said at the beginning of this post, the route-guide demo is showing many of the different ways a client and a server can interact using gRPC, beyond basic RPC invocations.
Available Now
End-to-end HTTP/2 and gRPC Support is available today for new and existing Application Load Balancers in all regions. You can use this feature via the console, AWS Command Line Interface (CLI), AWS SDKs. We are working to add AWS CloudFormation support soon.
There is no additional cost for using the gRPC protocol with an ALB. For more information, you can see the Elastic Load Balancing pricing page.
With these new features, it’s much easier to use gRPC to integrate your applications, or to improve client/server communication. To learn more, please see the documentation.
— Danilo
