API Response Compression for JSON APIs: Gzip vs Brotli, Configuration, and Tuning

Why compress API responses?

APIs routinely return verbose, text-based payloads—JSON, XML, GraphQL—where repeated keys and structural symbols dominate. Response compression exploits this redundancy to reduce bytes on the wire, which typically yields:

• Lower latency on constrained or mobile networks
• Reduced egress costs and CDN bandwidth
• Higher effective throughput under load

For APIs, gzip and Brotli are the most widely supported HTTP content codings. Enabled correctly, they are transparent to clients and safe over HTTPS.

How HTTP content negotiation works

Compression uses standard HTTP content negotiation:

• Clients advertise what they support via the Accept-Encoding request header.
• Servers select an encoding and mark it in the Content-Encoding response header.
• Caches and proxies use Vary: Accept-Encoding to store separate variants per encoding.

Example request/response exchange:

GET /v1/items HTTP/1.1
Host: api.example.com
Accept-Encoding: br, gzip

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Content-Encoding: br
Vary: Accept-Encoding
Content-Length: 4821

Key points:

• If the client does not send Accept-Encoding, the server must return an identity (uncompressed) response.
• The ETag applies to the specific encoded representation; with multiple encodings, ETags will differ per variant unless you use weak ETags.

Gzip vs Brotli at a glance

Both algorithms are lossless and stream-friendly, but they optimize different trade-offs.

• Gzip

  • Pros: ubiquitous support across browsers, CLIs, SDKs; fast to compress at mid levels; very fast to decompress.
  • Cons: larger output than Brotli for textual formats.
  • Levels: 1–9 (6 is a common default). Higher levels shrink more but cost CPU.

• Brotli

  • Pros: typically 10–25% smaller than gzip for JSON, HTML, and CSS; excellent for network-limited clients.
  • Cons: higher CPU cost at high levels; support is broad but not universal in very old clients.
  • Levels: 0–11. Levels 4–6 are a sweet spot for dynamic API responses; 8–11 are best reserved for pre-compressing static assets.

Practical guidance:

• Prefer Brotli for HTTPS API traffic when the client advertises br; fall back to gzip.
• Cap compression level to keep tail latency stable (e.g., gzip 5–6, Brotli 4–6) for dynamic JSON.

When not to compress

Avoid or conditionally skip compression when:

• Payloads are already compressed: images (JPEG/PNG/WebP/AVIF), video, PDF, ZIP, protobuf with gzip framing, etc.
• Very small responses (e.g., under 1 KB). The headers and compression overhead can outweigh benefits.
• Extremely high-CPU endpoints at risk of latency spikes. Consider sampling or lowering levels.
• Streaming over Server-Sent Events or long-polling where timely flushes matter; compression can add buffering unless tuned.

Caching, proxies, and the right headers

• Always set Vary: Accept-Encoding to ensure caches store separate compressed and uncompressed variants.
• If you use strong ETags, generate them after encoding so each representation has a unique ETag. Weak ETags (W/) may simplify.
• Prevent double-compression by ensuring upstream apps and reverse proxies coordinate. If Content-Encoding is already set, proxies should pass it through.

Security considerations

• TLS-level compression is disabled by modern stacks due to CRIME/BREACH. HTTP response compression (gzip/Brotli) is safe for APIs as long as you avoid reflecting secrets in compressible responses alongside attacker-controlled input. If that pattern exists, consider mitigations (separate origins, disable compression on such routes, or randomize response blocks).
• Set reasonable memory and size limits in compressors to avoid excessive resource use.

Enabling compression in common stacks

Nginx (with Brotli module) as reverse proxy

# gzip (builtin)
gzip on;
gzip_comp_level 6;
gzip_min_length 1024;
gzip_vary on;
gzip_types application/json application/javascript text/plain text/css application/xml;

# brotli (requires ngx_brotli module)
brotli on;
brotli_comp_level 5;
brotli_min_length 1024;
brotli_types application/json application/javascript text/plain text/css application/xml;

# Ensure correct caching behavior
add_header Vary Accept-Encoding always;

Notes:

• Use brotli and gzip together; Nginx will choose based on Accept-Encoding.
• Keep min_length at ~1 KB to skip tiny payloads.

Apache HTTP Server (mod_deflate and mod_brotli)

# gzip
AddOutputFilterByType DEFLATE application/json text/plain text/css application/javascript application/xml
DeflateCompressionLevel 6

# brotli
AddOutputFilterByType BROTLI_COMPRESS application/json text/plain text/css application/javascript application/xml
BrotliCompressionQuality 5

# caching variant
Header append Vary Accept-Encoding

Node.js (Express) with gzip and conditional Brotli

Option A: gzip via compression middleware (simple and robust):

import express from 'express';
import compression from 'compression';

const app = express();
app.use(compression({ threshold: 1024, level: 6 })); // gzip by default

app.get('/v1/items', (req, res) => {
  res.json({ ok: true, data: [] });
});

app.listen(3000);

Option B: manual content negotiation with Node’s built-in Brotli for selected routes:

import http from 'http';
import { createBrotliCompress, constants, createGzip } from 'zlib';

http.createServer((req, res) => {
  const accepts = req.headers['accept-encoding'] || '';
  const payload = Buffer.from(JSON.stringify({ ok: true, data: [] }));

  if (accepts.includes('br')) {
    res.setHeader('Content-Type', 'application/json; charset=utf-8');
    res.setHeader('Content-Encoding', 'br');
    res.setHeader('Vary', 'Accept-Encoding');
    const br = createBrotliCompress({ params: { [constants.BROTLI_PARAM_QUALITY]: 5 } });
    br.pipe(res); br.end(payload);
  } else if (accepts.includes('gzip')) {
    res.setHeader('Content-Type', 'application/json; charset=utf-8');
    res.setHeader('Content-Encoding', 'gzip');
    res.setHeader('Vary', 'Accept-Encoding');
    const gz = createGzip({ level: 6 });
    gz.pipe(res); gz.end(payload);
  } else {
    res.setHeader('Content-Type', 'application/json; charset=utf-8');
    res.end(payload);
  }
}).listen(3000);

Go (net/http) with gzip and Brotli

package main

import (
  "net/http"
  "strings"
  "compress/gzip"
  brotli "github.com/andybalholm/brotli"
)

type encWriter struct{ http.ResponseWriter }
func (w encWriter) WriteHeader(code int) { w.Header().Add("Vary", "Accept-Encoding"); w.ResponseWriter.WriteHeader(code) }

func handler(w http.ResponseWriter, r *http.Request) {
  w.Header().Set("Content-Type", "application/json; charset=utf-8")
  ae := r.Header.Get("Accept-Encoding")
  payload := []byte(`{"ok":true,"data":[]}`)

  if strings.Contains(ae, "br") {
    w.Header().Set("Content-Encoding", "br")
    br := brotli.NewWriterLevel(w, 5)
    defer br.Close(); br.Write(payload)
    return
  }
  if strings.Contains(ae, "gzip") {
    w.Header().Set("Content-Encoding", "gzip")
    gz, _ := gzip.NewWriterLevel(w, 6)
    defer gz.Close(); gz.Write(payload)
    return
  }
  w.Write(payload)
}

func main() { http.ListenAndServe(":8080", http.HandlerFunc(handler)) }

.NET (ASP.NET Core) ResponseCompression

// Program.cs
using Microsoft.AspNetCore.ResponseCompression;
using System.IO.Compression;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddResponseCompression(opts =>
{
    opts.EnableForHttps = true; // APIs should be HTTPS-only
    opts.Providers.Add<GzipCompressionProvider>();
    opts.Providers.Add<BrotliCompressionProvider>();
    opts.MimeTypes = new[] {
        "application/json", "text/plain", "application/xml", "application/javascript"
    };
});

builder.Services.Configure<GzipCompressionProviderOptions>(o => o.Level = CompressionLevel.Optimal);
builder.Services.Configure<BrotliCompressionProviderOptions>(o => o.Level = CompressionLevel.Fastest); // good for dynamic JSON

var app = builder.Build();
app.UseResponseCompression();
app.MapGet("/v1/items", () => Results.Json(new { ok = true, data = Array.Empty<object>() }));
app.Run();

Working with CDNs and gateways

• Most CDNs (and API gateways) auto-compress eligible content at the edge when the origin is uncompressed. Verify they respect Vary: Accept-Encoding and that they don’t re-compress already compressed payloads.
• If you pre-compress at the origin, disable edge compression or configure “pass-through” to avoid double work.
• For high-traffic endpoints with stable responses, consider precomputing Brotli at a higher level (e.g., 8–9) and caching that variant.

Measuring impact and choosing levels

Establish a small, repeatable benchmark for realistic payloads.

1. Capture representative responses to files:

curl -s https://api.example.com/v1/items -H 'Accept-Encoding: identity' -o baseline.json

2. Test gzip and Brotli sizes and timings from your stack (or via a proxy):

curl -s -D- https://api.example.com/v1/items -H 'Accept-Encoding: gzip' -o /dev/null | grep -iE 'content-encoding|content-length'
curl -s -D- https://api.example.com/v1/items -H 'Accept-Encoding: br'   -o /dev/null | grep -iE 'content-encoding|content-length'

3. Load test with realistic concurrency to observe tail latency (p95/p99). Track CPU usage on the compression tier and watch for GC pressure in managed runtimes.
4. Tune levels: raise quality until marginal gains flatten or tail latency grows. For dynamic JSON, Brotli 4–6 and gzip 5–6 commonly balance well.

Streaming and flushing

• For event streams or NDJSON, enable periodic flushes so bytes reach clients promptly. In Node, call res.flushHeaders()/res.flush() when supported; in Go, use Flush() on a flusher after writing a block. Note that some compressors buffer data—smaller window sizes or manual flush points may help.

Troubleshooting checklist

• Response is bigger with compression: likely already-compressed type or tiny payload; add a size threshold and restrict MIME types.
• Client got garbled text: double-compressed or missing Content-Encoding. Ensure one compressor runs and headers match the body.
• Cache misses exploded: missing Vary: Accept-Encoding; caches accidentally share variants across clients.
• High CPU at peak: lower compression levels, raise size threshold, offload to CDN, or pre-compress hot responses.
• Mixed results behind proxies: verify proxies don’t strip Accept-Encoding and that hop-by-hop headers are handled correctly.

Recommended defaults (safe starting point)

• Enable both Brotli and gzip.
• Levels: Brotli 5, gzip 6 for dynamic API JSON.
• Threshold: compress only when payload >= 1 KB.
• Types: application/json, text/*, application/javascript, application/xml.
• Headers: always set Vary: Accept-Encoding.
• Measure and iterate using real traffic and p95/p99 latency goals.

Summary

Brotli and gzip provide substantial, low-risk wins for API performance when applied with care. Use content negotiation to prefer Brotli, keep levels moderate for dynamic responses, avoid already-compressed media, and make caches aware with Vary: Accept-Encoding. With a few lines of configuration in your proxy or framework, you can cut bandwidth, speed up slow networks, and keep tail latency in check—all without changing your API contracts.