AIP-213

Common components

In general, API reviewers encourage API producers to keep their APIs mostly self-contained, except for a relatively small set of common protos which are safe to import (e.g. google.protobuf.Timestamp). This is for good reason: APIs generally need to be able to move forward independently of one another, and mutual dependencies can cause downstream APIs to be forced into taking major version changes or even lead to dependency conflicts.

However, there are also cases where common structures are valuable, especially where a concept is well-known and it is sufficiently clear that it will not change. Common protos serve this use case.

Guidance

The public representation of APIs should be self-contained, meaning that all protos used by the API originate in the same proto package, except for common protos, which may be used freely in any API.

APIs must not define a set of API-specific common protos which live outside of its versioning structure. This prevents independent movement of particular versions and also causes problems for client libraries in many languages that compile the proto messages into classes.

APIs should not directly depend on protos defined in other APIs. Instead, they should copy and paste the applicable messages into their own API. When doing so, APIs should keep the field names and numbers the same.

Existing common protos

The common protos, which public-facing protos for an API may safely import, are as follows:

Note that some common protos may have internal-only fields. APIs should generally only rely on fields which have been released into open source.

Google APIs may also import google.iam.v1.*, which provides the IAM messages used throughout Google.

Note: Many APIs also import protos from other packages for internal-only use (e.g. to apply visibility labels or provide instructions to internal infrastructure). This is acceptable provided that the public protos do not contain such references.

Protobuf types

The google.protobuf package is somewhat special in that it is shipped with protocol buffers itself, rather than with API tooling. (For most API designers, this should be an implementation detail).

This package includes a small library of types useful for representing common programming language constructs:

  • google.protobuf.Duration: Durations, with nanosecond-level precision. The protobuf runtime provides helper functions to convert to and from language-native duration objects where applicable (such as Python’s timedelta).
  • google.protobuf.Struct: JSON-like structures (a dictionary of primitives, lists, and other dictionaries). The protobuf runtime provides helper functions in most languages to convert struct objects to and from JSON.
  • google.protobuf.Timestamp: Timestamps, with nanosecond-level precision. The protobuf runtime provides helper functions in most languages to convert to and from language-native timestamp objects (such as Python’s datetime).

API Types

The google.type package provides a “standard library” of types useful for representing common concepts in APIs. While types are added from time to time and the definitive list is always the code, several types deserve note:

Appendix: Adding to common protos

Occasionally, it may be useful to add protos to these packages or to add to the list of commonly-available protos. In order to do this, the protos must be added to the appropriate location in google3, and then the list above must be amended. If the proto is to be used in an externally-facing API, the applicable protos must also be released into open source.

However, some general guidelines are worth noting for this:

  • Protos should only be promoted to common status if we are certain that they will never change (at all – even in ways that would normally be considered backwards compatible). Common protos are generally not versioned, and it must be the case that we can rely on the proto to be a complete and accurate representation indefinitely.
    • The exception to this is protos describing our infrastructure, which may have rare, backwards-compatible changes.
  • Protos must be applicable to a significant number of APIs for consideration as common protos. It is okay for those APIs to be clustered together (e.g. all in a single PA).
  • There is no good way to “stage” a common proto, because moving references to them is effectively not possible. (In other words, it is infeasible to add a proto to google.geo.type.* and then “graduate” it to google.type.* later.)
  • Adding a common proto requires coordination between several teams, and it may take time between when an addition is approved and when it is available for use.
  • Even after a common proto is added, APIs using local versions must continue to do so until they go to the next major version.

In the event that you believe adding a common proto is appropriate, please open an issue.