25 November 2023

Communicating our APIs: Specifications and Documentation

This blog post is the Part 1 of the “Communication our APIs” series. This part explores communication involved during API development, along with practical tips to enhance it. I highly recommend continuing with Part 2, to understand more about Code First, API First and Consumer First API development methodologies.

APIs, or Application Programming Interfaces enable software systems to communicate with each other. Various API technologies such as REST, GraphQL, WebSockets, SOAP, RPC provide software interfaces and protocols to connect software systems to each other.

Integrating an API involves communication between providers and consumers. The provider implements the API and makes it available for usage by consumers. On the other end, consumers require information about API usage, data formats, failure handling and much more. The APIs might be private, public or partner facing. In the case of a private API the communication is easier as the provider and consumer are developers of the same team or members of the different departments within the same organization. In the case of public or partner facing APIs the communication becomes more challenging, as a provider might serve a large number of consumers (ie: Payment Service Providers, Geo-Location services) to be able to scale. Overall API success depends very much on consumers being able to understand the API, following the instructions and easily integrate.

According to the 2023 State of the API Report published by Postman the top three obstacles for API consumers to use an API seem to be Lack of documentation, Difficulty in discovering APIs and Lack of time. The same study also reveals that providers consider API Usage, Number of active users and Revenue as top three indicators for API success. With these figures, it becomes clear that not only machines, but humans should establish optimal ways of communicating the APIs. Standardizing API specifications, published documentations and automation would not only improve the communication but also speed up the integration between consumers and providers.

This blog post describes the specifications such as OpenAPI and AsyncAPI and the landscape around them, compares contract-first and api-first approaches for implementing them, compiles a handful of tools for auto generating API Reference and Documentation and finally shows how to publish them as a web page on GitHub Pages for making them available to the consumers. The examples given in this blog post are available on Github and can be found on respective sections where they are described.

API Specifications

API Specification is a standardized document that describes the API. It defines the functions and the expected results of an API. Some of the known specifications include: OpenAPI/Swagger Specification for REST APIs; AsyncAPI for message-driven APIs; WSDL for SOAP Web Services; GraphQL Schema for GraphQL; GRPC. With adequate tooling API specifications provide capabilities to generate code, documentation or validate client or server implementations of the API. So they improve the communication between providers and consumers. However they require understanding of the specification language which is rather in machine readable forms.

This section focuses on OpenAPI and AsyncAPI specification, however gives a short introduction for the specifications of some other API technologies.

OpenAPI Specification

OpenAPI Specification is a technology agnostic standard to describe Rest APIs​. Formerly known as Swagger, the specification was renamed to OpenAPI Specification by version 3. The specification can be written as JSON or YAML.

The structure of an OpenAPI Specification looks as follows. The full example is available on GitHub.

openapi: 3.0.3
info:
  title: Recipes API
  ...
servers:
- url: http://localhost:8080

paths:
  /recipes:
    get:
      summary: List all recipes
      ...
      responses:
        ...
        "200":
          description: OK
          content:
            'application/json':
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/Recipe'
        ...

components:
  schemas:
    Recipe:
      type: object
      ...
  • Info: General API information as metadata
  • Servers: Connectivity information about target servers
  • Paths: Paths to the endpoints with their expected request, response and errors. The given example defines a GET /recipes endpoint which would return an HTTP Status 200 with an array of Recipe objects in application/json format.
  • Components: Holds the schemas for the request, response and errors. Defined schemas can be referenced and reused from the paths or other schema objects

More information about the specification format can be found on OpenAPI Specification.

AsyncAPI Specification

AsyncAPI Specification is a technology agnostic standard to describe message-driven APIs​. The specification is brought by the AsyncAPI Initiative and is an adaptation of the OpenAPI specification​. Therefore it has similarities with the OpenAPI Specification, also can be written as JSON or YAML. The specification can describe protocols such as AMQP, HTTP, JMS, Kafka, however it is not limited to any.

The structure of an AsyncAPI Specification looks as follows. The full example is available here.

asyncapi: 2.0.0
info:
  title: Ping Service
  ...

servers:
  localhost:
    url: localhost:9092
    protocol: kafka
    protocolVersion: '1.0.0'

channels:
  ping.topic:
    description: Kafka topic for ping messages
    publish:
      operationId: pingSent
      message:
        $ref : '#/components/messages/Ping'
  pong.topic:
    description: Kafka topic for pong messages
    subscribe:
      operationId: pongReceived
      message:
        $ref : '#/components/messages/Pong'      

components:
  messages:
    Ping:
      name: Ping
      ...
      payload:
        $ref: '#/components/schemas/PingPayload'
    Pong:
      name: Pong
      ...
      payload:
        $ref: '#/components/schemas/PongPayload'

  schemas:
    PingPayload:
      type: object
      ...
    PongPayload:
      type: object       
      ...
  • Info: Metadata information about the API
  • Servers: Connectivity information about servers (i.e. Kafka brokers)
  • Channels: Messages exchange between provider and consumer. The given example showcases a provider publishing a ping message via ping topic and subscribes to the pong message sent back by the consumer via pong topic.
  • Components: Defines the reusable objects such as schemas or messages which could be referenced.

More information about the specification format can be found on AsyncAPI Specification.

Other Specifications

Web Services Description Language

Web Services Description Language (WSDL) is an XML document that describes a Web Service and is mainly used in conjunction with Soap Web Services. It follows basically the Contract First approach and has server and client generation supported by major frameworks. According to the 2023 State of the API Report SOAP has decreased to 26% compared to 34% in 2022. However it remains a popular technology with a long history of Contract First development.

GraphQL Schema

GraphQL is a query language for the APIs which was developed by Facebook in 2012 and became open-source API Technology by 2015. Unlike Rest APIs where the provider defines the response data format, in GraphQL consumers determine the response data they need. A GraphQL API is described with GraphQL Schema and is implemented by the provider. GraphQL allows the schema to be queried by the client via introspection queries, to discover fields and types of the API. Therefore GraphQL APIs are considered self-documenting.

GRPC

gRPC is an open-source high performance Remote Procedure Call (RPC) framework developed by Google. It has a Contract First approach. The provider describes a service contract by using protocol buffers (.proto file) as its Interface Definition Language. Providers and consumers would generate server and client code by using a preferred gRPC plugin.

Documentation Generation

To improve communication even further, API Specifications can be used to generate more human readable forms of documentation such as API reference or API documentation. These would be representations as web pages or markdown. In addition, code samples, try-it-out features, conceptual technical documentation are some more add-ons to improve consumer experience. Ideally they might leverage API Specifications to generate parts of their content.

API Reference

API References are APIs described as web pages. They would not only describe API behavior in a human readable form, but also provide code samples and try-it-out features to improve consumer experience. It is essential to keep the API reference up-to-date, ideally not to update the page manually on each API change. Thankfully there are quite a handful of tools to generate API References automatically from OpenAPI and AsyncAPI specifications.

SwaggerUI

SwaggerUI is one of the most popular tools which can use OpenAPI Specification to render it as an API Reference. Provides API description and try-it-out features.

SwaggerUI can be generated in several ways:

  • IDEs such as IntelliJ or Visual Studio Code provide plugins to preview OpenAPI Specification file as SwaggerUI
  • Swagger Editor
  • Served as a static page with Rest API.

Redoc

An open source tool that produces API Reference documentation from an OpenAPI Specification. A Redocly CLI would generate an HTML page with a single command.

redocly build-docs api/src/main/resources/recipes-api.yaml -o docs/redoc/index.html
  • build-docs: CLI command that builds the provided api definition as HTML file
  • -o: Output html file

The result will look as follows. An example is available on the GitHub repository.

Stoplight Elements

Another open source documentation tool that leverages Open API to render documentation in an HTML page. Elements generates interactive API Reference Documentation with try-it-out features and code samples on different programming languages. The tool provides web components for embedding API Reference into HTML, React, Angular and GatsbyJs.

The following example shows an HTML component usage as an example.

<html lang="en">
<head>
    ...
    <script src="https://unpkg.com/@stoplight/elements/web-components.min.js"></script>
    <link rel="stylesheet" href="https://unpkg.com/@stoplight/elements/styles.min.css">
    ...
</head>
<body>
<elements-api
        apiDescriptionUrl="recipes-api.yaml"
        layout="sidebar"
        router="hash"/>
</body>
</html>
  • head: Contains the JavaScript and CSS provided by Stoplight Elements
  • elements-api: Web component that renders the API Reference Documentation.
    • apiDescriptionUrl: URL of the specification which can be a URL or relative path.
    • layout: sidebar for three-column design, stacked single-column design for embedding API reference to other web sites.
    • More configuration options can be found here.

The page would look as follows, the full example can be found on the GitHub repository.

More API Reference tools

There are a handful of documentation tools which can leverage OpenAPI specification to generate API Reference Documentation. OpenAPI.Tools - Documentation provides a comprehensive and up-to-date list of document generators.

AsyncAPI also can be leveraged to generate API Reference documentation. A similar list of available tools can be found on AsyncAPI Tools. AsyncAPI Generator is one of them and generating an API reference would be as simple as running a asyncapi generate fromTemplate src/main/resources/async-api.yaml @asyncapi/html-template -o docs/ CLI command which would generate the following

Widdershins is another API Reference tool which can generate markdown files from OpenAPI and AsyncAPI specifications. Markdown support brings benefits especially to the providers who provide markdown based Technical Documentation.

API Documentation

API References increase the discoverability. However, In case an API requires additional technical writing or is a piece of a bigger ecosystem more than that is required. On top of API References additional conceptual technical documentation, quick-start guides, use cases, process description or even multiple incorporated APIs can form API Documentations in a wider scope. In some cases, dedicated technical writers might produce the documentation for the consumers. Therefore keeping API Documentation up-to-date is essential.

“Docs as code” is a helpful method to enable collaboration between technical writers and API developers. Using Markup languages like Markdown keeps documentation plain and allows it to be rendered in various formats such as web pages. Auto generated API Reference documents can be embedded into markdown or other web pages and achieve continuous documentation.

API Documentation with Hugo

The following example is a docs-as-code implementation of API Documentation, which combines technical writing, conceptual technical documentation and API Reference together. Hugo Framework is used as an open-source static website generator, capable of rendering markdown content into a static website. Docdock theme is used due to its documentation friendly design.

  • Header: Provides links to the GitHub repo to do inline documentation edits.
  • Left column: Navigates between documentation sections.
    • Introduction and quickstart: Technical writing based on markdown.
    • API Reference (stoplight): API Reference based on Stoplight Elements HTML component to render OpenAPI Specification. Single-column design based on stacked layout for embedding the content. The API reference includes interactive try-it-out and code samples.
    • API Reference (widdershins): Same API Reference generated as markdown based on Widdershins. The reference includes code samples, however no try-it-out feature due to Markdown format.

The full example can be found on the GitHub repository.

Publishing Documentation

We’ve built our docs-as-code based API Documentation. The last bit is serving it to the consumers.

This can be achieved with a CI/CD pipeline that would deploy documentation updates on each API change. This could be a company website, developer portal or whatever platform is available to the consumers. Alternatively version control systems like GitHub or GitLab offer their built-in free web hosting service which could be used for publishing static web pages. This works well in case these version control systems are already used.

GitHub Pages

GitHub provides its free web hosting service GitHub Pages for serving static web pages which integrates well with its GitHub Actions CI/CD service.

Configuring GitHub Pages

GitHub pages can be enabled under Settings > Pages on your repository.

The source can be “Deploy from a branch” which publishes static html files residing on a branch. An alternative option is “GitHub Actions’’, which enables deployment from a Github Actions CI/CD workflow. This option is used for the current demo as a preferred way, due to the fact that it doesn’t require pushing generated static HTML pages into the repository. Instead documentation is generated and published during CI/CD execution.

GitHub Actions workflow

The following workflow definition ensures a two stage pipeline where documentation is generated and published to GitHub Pages. Once the pipeline runs successfully, the published website URL will be <github-account>.github.io/<github-repo>.

name: Publish API Docs with Hugo
***
jobs:
  Generate-API-Docs:
    ***
    steps:
      - name: Install Hugo CLI
        ***
      - name: Deploy API Specification
        run: cp api/src/main/resources/recipes-api.yaml docs/hugo-docdock/static/
      - name: Build with Hugo
        ***
        run: hugo --source docs/hugo-docdock ***
      - name: Upload artifact
        uses: actions/upload-pages-artifact@v1
        with:
          path: 'docs/hugo-docdock/public'
  Publish-API-Docs:
    ***
    steps:
      - name: Setup Pages
        uses: actions/configure-pages@v3
      - name: Deploy to GitHub Pages
        id: deployment
        uses: actions/deploy-pages@v2
  • Generate-API-Docs: Job definition to generate documentation
    • Install Hugo CLI: Ensures Hugo CLI installation
    • Deploy API Specification: Makes OpenAPI available for documentation rendering
    • Build with Hugo: Runs Hugo CLI to generate static web site.
  • Publish-API-Docs: Deploys the generated static web site files into GitHub Pages.

The deployed API Documentation available as a live demo on https://enisspahi.github.io/contract-first-api-example/.

The full example can be found here.

Conclusion

APIs enable machines to communicate with each other. It turns out that API consumers and providers consider human factors (lack of documentation, lack of time, discoverability) as obstacles for a successful API, as per Postman’s State of the API Report 2023. This blog post covers tools and approaches to address these obstacles.

Major API technologies provide standardized specification languages to describe the functions and expected results of an API. OpenAPI/Swagger and AsyncAPI, along with other specification languages, provide capabilities to generate code, documentation or validate contracts.

API Specifications can be leveraged to automatically generate API References, making them more human readable. They are APIs documented as web pages, may provide code samples, try-it-out features and increase the discoverability. However, they have limited scope. If an API requires additional technical writing or is part of a larger ecosystem, API Documentation provides the necessary coverage.

Keeping API Documentation up-to-date via continuous documentation is essential. This can be achieved with the support of CI/CD to deploy documentation on each API change.

This blog post shows a journey starting from API Specification development, compiles a selection of tools to speed up coding and documentation, and finally resulting in a creation of publicly hosted API Documentation.

Communicating our APIs Blog series

  1. Communicating our APIs: Specifications and Documentation
  2. Communicating our APIs: Three methodologies of API Development

Reference & Credits

Enis Spahi

Enis is a Consultant working for OpenValue Munich. He is passionate about Java and its ecosystem, having gathered experience in various domains such as Fintech, Electricity Transmission and Travel. His areas of interest are Software and System Design, API Development, DevOps, Observability and anything that involves communicating Software Architecture. When there is time left he likes hiking, skiing or playing online chess.