Codesmith NYC Onsite Immersive alum Kenny Nguyen presented gRPSeek, an Open Source Product he's been working on during and after the immersive, at this year’s gRPC conference at the Google Cloud Campus. Kenny discusses gRPSeek, how he prepared for the gRPC conference, and how the gRPC community reacted to the tool.

What were you doing prior to Codesmith? What led you to pursue a career in software engineering?

Before Codesmith, I was working as a bill review analyst for a med-tech company doing data entry work. I was also going to school at CalState Fullerton, pursuing my business degree with accounting and finance as my emphasis. It was very clear that wasn’t what I wanted to do, so I made a pivot and started learning some Python and JavaScript. Even before Codesmith, I knew that the developer and tech space was a lot more welcoming. I’ve solved a lot of technical challenges and blocks by someone else helping me through the problem – on forums, chat rooms, and spaces like that. Whereas, in accounting and finance, I felt like people were gatekeeping knowledge for some reason. In tech, you do things because they make sense, not just because it's the process of how things have been done. I like that kind of forward thinking mindset. It aligns with my values.

When I was in the application and interviewing process for Codesmith, I wanted to do the East Coast Remote Immersive program, but the next open cohort was four or five months from when I was interviewing. I got a call from Codesmith, and they said that the NYC Onsite program, which started in a month, had an open spot. I'm from California, so obviously I didn’t have a place in New York. It was a really big move, but I decided to do it. It was the best decision of my life! I’ve made lifelong friends from the experience.

You recently spoke about the open source tool you built in the Immersive, gRPSeek, at the gRPC conference at Google’s campus! Tell me about gRPSeek.

So, gRPC is a relatively new remote procedure call framework. It's mainly used for micro services and back-end interprocess communication. My team’s Open Source Product (OSP), gRPSeek, is a performance testing tool, specifically for load testing gRPC services. We built this tool with Node.js as our language of choice.

There's different types of streaming in gRPC, such as unary, which is just a simple request response cycle. There's also bidirectional streaming and server streaming. Our load testing tool right now is only doing unary streaming calls. We are working on implementing future streaming calls as part of the load tests that we’ll provide.

Kenny (left) at gRPC conference.

Why did you build a tool based on gRPC, and what challenges does gRPSeek solve for the gRPC community?

One of my OSP teammates, Miri Son, had an engineer friend in the industry. He told us that he uses gRPC at his work, and there's a problem with observability and performance. So, he thought a load testing tool for gRPC would be really useful. After we heard about that, we started researching on forums—Discord, Reddit, chat rooms, and all those places. It was an issue we saw that many people were asking for solutions for. At that time, even k6, which is a big performance testing tool, didn't have many features for gRPC. They only had unary calls. So, it was obvious that there was a demand for a load testing tool in the market.

There aren’t many performance testing tools for gRPC services right now. gRPC’s performance can sometimes fluctuate and drop, so there is a need for people and companies to benchmark and test their services to see what kind of capacity the servers can sustain, and check on the health benefits. Also, if you and your client have a service level agreement, and you're making certain guarantees, these load testing tools can help you meet those promises and guarantees. There are a lot of other benefits to load testing as well, such as making sure your services can scale and identifying any bottlenecks.

What were some of the biggest technical challenges that you encountered while building gRPSeek?

Building the load test engine itself—how do we simulate a number of concurrent users to simulate a performance test on it? Especially with Node.js being single threaded in nature, we could have concurrent calls, but we couldn’t have parallel concurrent calls, which might not matter depending on the use case. But, if you want to simulate multiple users actually hitting your server at the same time, you can't do that with just a single-threaded language. However, in the past couple of years, Node.js has given us the worker thread and cluster module, which lets us spin up multiple clusters that can set up multiple worker threads, essentially enabling it to simulate a number of virtual users, which is how languages like Go and Python and C++ do it. They have these multi-threaded applications that do the load tests for them. So, that's what we used. 

Another problem was getting the metrics. Where are we putting these interceptors? How are we timing our request response accurately? Is the latency being caused by the overhead from our interceptors? That was difficult. 

Kenny presenting gRPSeek at the gRPC conference.

Tell me about presenting gRPSeek at the gRPC conference?

The day before the conference, I came to the Google campus to meet with the gRPC maintainers. They showed me around the campus and the office for the gRPC maintainer team. C++, Go, Python and more all had their own spots. Seeing everybody working on the technology our open source tool was around was super cool! I met Michael, the only Node.js/TypeScript maintainer for gRPC, and Kevin, the manager. 

When I presented gRPSeek, our load testing tool, I talked about the three struggles that we faced in building the tool, and then the three insights that we learned along the way. One of them was how are we going to get this client stub dynamically from the user to generate a stub to invoke the server. The second one was how are we going to get logs, metrics, and traces—the three pillars of observability. The third was building a load test engine itself—how are we achieving concurrency? I also talked about Node.js’s utilization metric compared to CPU usage, because it is a more accurate representation of your system’s hardware limitations. 

After the talk, a couple people asked me questions. Like, is it compatible with Go? I haven't tested that, but I would assume because it's language agnostic that it would be.

Was the gRPC community excited to learn about gRPSeek?

I think they felt gRPSeek could have a lot of potential, and they are definitely interested in seeing what it could be. At the conference, it was clear that there was a need for this benchmarking tool because a lot of people were having performance issues with gRPC. 

What’s next for you after the conference?

Now that we've been to the conference, and we know what the developer community is looking for, I think I have a much clearer sense of direction. We're going to continue iterating on gRPSeek, and see what’s possible!