Rethinking Protocol Buffers with Buf

What is Buf?

A tool to make Protobuf reliable and easy to use for service owners and clients, while keeping it the obvious choice on the technical merits. Our organization should not have to reinvent the wheel to create, maintain, and consume Protobuf APIs efficiently and effectively. It will handle our Protobuf management strategy for us, so we can focus on what matters.

Features

• Automatic file discovery.
• Selectable configuration - 40 lint checkers and 54 breaking checkers
• Selectable error output - file:line:col:message
• Check anything from anywhere - proto files, tar, git, pre-built images or file descriptors.
• Speed - Its internal compiler is super fast (approx. 4x then Protoc)
• Can use buf as a protoc plugin instead of using it as a standalone tool.

Buf CLI

Buf attempts to simplify your Protocol Buffers workflow using the Buf CLI and protoc plugins. The Buf CLI currently provides:

• A linter that enforces good API design choices and structure.
• A breaking change detector that enforces compatibility at the source code level or wire level.
• A generator that invokes your protoc plugins based on a configurable template. A protoc replacement that uses Buf's newly-developed high performance Protobuf compiler.
• A configurable file builder that produces Images, our extension of FileDescriptorSets.

Comparison Between Protobuf and Buf

Layer5 projects currently use protoc as the tool for building their protobuf defintions. The following are some considerations made while determining whether to use Buf.

• Protobuf is not as widely adopted as JSON.
• API Structure
  • No standards enforcement
  • Inconsistency can arise across an organization's Protobuf APIs,
  • Design decisions can be made that can affect your API's future iterability.
• Backward Compatibility
• Stub distribution
• Tooling

Buf aims to solve the above problems and it's long-term goal is to enable schema-driven development: A future where APIs are defined consistently, in a way that service owners and us can depend on

Roadmap to Adopting Buf

In consideration of the use of Buf, we would adopt it in phases, starting with the following ares of integration.

• API Structure Enforcements
  • Linter solves this issue by enforcing standards.
  • Also, we don’t need to use Buf as a standalone tool we can just use linter as plugins.

• Backward Compatibility
  • It will check for different things that can cause breaking change.
  • For example, type change.