REST API Soft Delete Patterns: Models, Endpoints, and Production Pitfalls

Overview

Soft delete is a design pattern that marks data as deleted without physically removing it. In REST APIs, soft delete enables undo, auditing, legal holds, and resilient references across services. But it also introduces complexity in reads, uniqueness constraints, search, caching, and data retention. This guide explains practical soft delete patterns for REST APIs, with data models, endpoint semantics, indexing tips, and production pitfalls.

When to use soft delete (and when not to)

Use soft delete when you need:

• Restoration/undo within a grace period.
• Auditability and change history.
• Referential stability for related resources or downstream projections.
• Legal or compliance retention (e.g., holds, e-discovery).

Avoid soft delete when:

• Regulations mandate permanent erasure of data (e.g., strict PII deletion) and retention isn’t allowed.
• Data volume and query patterns make “deleted filters” excessively costly.
• Your domain can safely hard-delete and rebuild derived views.

A common compromise is “soft delete now, purge later”: mark as deleted immediately, then run a scheduled hard-delete after a retention window.

Core data modeling options

Choose one canonical representation and apply it consistently across services.

1. Boolean flag

• Columns: deleted boolean default false
• Pros: compact; simple predicates WHERE NOT deleted
• Cons: no timestamp or actor; limited auditing

2. Deleted-at timestamp (recommended)

• Columns: deleted_at timestamptz NULL, deleted_by uuid/text NULL
• Pros: encodes time and actor; enables TTL, partial indexes, and audit trails
• Cons: needs additional tables/columns for full history

3. Status enum/state machine

• Columns: status enum(‘active’,‘deleted’,‘archived’,‘suspended’, …)
• Pros: models richer lifecycle; avoids multiple flags
• Cons: queries become more verbose; careful transitions required

4. Tombstone row

• Replace the original record with a minimal “tombstone” that keeps identifiers and foreign keys but strips sensitive fields
• Pros: supports privacy scrubbing while keeping referential integrity
• Cons: requires schema discipline and migration effort

5. Separate table

• Move deleted rows to table_name_deleted
• Pros: clean separation; simple reads on the main table
• Cons: complex migrations; cross-table uniqueness and restores are harder

Most teams start with a deleted_at column plus optional deleted_by and audit tables. This balances simplicity and operational power.

Database indexing and constraints

• Default-scope reads: add partial indexes that exclude deleted rows. Example (PostgreSQL):

  CREATE INDEX idx_users_active_email
  ON users (lower(email))
  WHERE deleted_at IS NULL;
  

• Uniqueness across active rows only:

  CREATE UNIQUE INDEX uniq_users_email_active
  ON users (lower(email))
  WHERE deleted_at IS NULL;
  

• For high cardinality filters, pair partial indexes with commonly used predicates (tenant_id, status, deleted_at IS NULL).
• If you scrub PII on delete, update the row and preserve unique constraints by using partial unique indexes on active records.

REST endpoint design

A consistent API contract reduces ambiguity for clients.

Delete

• Endpoint: DELETE /v1/resources/{id}
• Behavior: mark as deleted (set deleted_at, deleted_by)
• Idempotency: repeated DELETE should be safe. Two common choices:
  • Return 204 No Content even if already soft-deleted.
  • Or return 404 Not Found for deleted resources to mimic absence. Choose one and document it.
• Concurrency: require If-Match with an ETag/version to avoid deleting stale versions; return 412 Precondition Failed on mismatch.

Example request:

DELETE /v1/customers/123 HTTP/1.1
If-Match: "v9"
Authorization: Bearer ...

Restore (undelete)

• Endpoint: POST /v1/resources/{id}:restore or PATCH /v1/resources/{id}
• Behavior: clear deleted_at and increment version
• Responses: 200 with the resource body; 409 if restore violates a uniqueness constraint; 404 if the resource cannot be restored (hard-deleted).

Example request:

POST /v1/customers/123:restore HTTP/1.1
If-Match: "v10"

Read semantics

• GET /v1/resources hides deleted by default.
• Opt-in inclusion:
  • Query: GET /v1/resources?include=deleted (or filter[deleted]=true)
  • Single resource: GET /v1/resources/{id}?include=deleted to fetch a soft-deleted record for admins.
• Representation: include deletion metadata when present.

Example representation:

{
  "id": "123",
  "name": "Acme, Inc.",
  "deleted": true,
  "deleted_at": "2026-04-17T15:16:22Z",
  "deleted_by": "user_42",
  "version": 11
}

Listing and filtering

• Default filter: deleted=false (or deleted_at IS NULL)
• Explicit filters:
  • filter[deleted]=true|false
  • filter[deleted_since]=timestamp for purge jobs or admin consoles
• Pagination: make sure counts and next-page cursors exclude deleted by default.

Status codes summary

• 204 No Content on successful soft delete
• 200 OK on successful restore or read including deleted
• 404 Not Found if the API treats soft-deleted as absent and the client didn’t opt-in
• 409 Conflict when restore violates uniqueness or state machine rules
• 412 Precondition Failed for ETag conflicts

Authorization and policy

• Only privileged roles should read deleted resources or restore them.
• Consider separate scopes: resource:delete, resource:read:deleted, resource:restore.
• Multi-tenant systems must scope both delete and restore to the tenant boundary to avoid cross-tenant leaks.

Auditing and compliance

• Record who deleted, when, and why (free-text reason or code).
• For privacy requests, consider “soft delete plus scrub”:
  • On delete, null out or hash PII columns while retaining identifiers.
  • Keep audit trails in a separate immutable store with strict access controls.
• Retention and purge:
  • Policy: e.g., purge 90 days after deleted_at unless on legal hold.
  • Job: scheduled worker that hard-deletes in small batches, emits events, and compacts indexes.

Cascades and referential integrity

Four practical strategies:

• Restrict: block delete if children exist; return 409 with a list of blockers.
• Soft cascade: mark children deleted in the same transaction.
• Orphan keep: allow parent delete; children remain active but become orphaned with nullable FK (rare for core domain data).
• Eventual cascade: emit domain event on delete; downstream services perform their own soft deletes.

Document the chosen policy per relationship. For soft cascades, consider background workers to avoid long transactions.

Caching, search, and messaging

• ETags: deleting changes the representation; update the version/ETag.
• CDN/API cache: differentiate caches when include=deleted is present; vary cache keys on the query.
• Search indexes: either remove documents on delete or mark a deleted flag and filter at query time; prefer removal to avoid accidental exposure.
• Eventing: publish ResourceDeleted and ResourceRestored events with minimal PII and the version. Consumers can manage projections and tombstones.

Concurrency and race conditions

• Use optimistic concurrency (ETag or version column). A soft delete should fail when the client’s version is stale.
• Serialize deletes with updates by updating the same row and version.
• If you soft cascade, ensure child updates also bump versions or emit events in order.

Example schema and queries

Schema with deleted_at and auditing:

CREATE TABLE customers (
  id            uuid PRIMARY KEY,
  tenant_id     uuid NOT NULL,
  name          text NOT NULL,
  email         text NOT NULL,
  deleted_at    timestamptz,
  deleted_by    uuid,
  updated_at    timestamptz NOT NULL DEFAULT now(),
  version       bigint NOT NULL DEFAULT 0
);

-- Only enforce email uniqueness for active customers per tenant
CREATE UNIQUE INDEX uniq_customers_tenant_email_active
ON customers (tenant_id, lower(email))
WHERE deleted_at IS NULL;

-- Speed up default list queries
CREATE INDEX idx_customers_active_tenant_name
ON customers (tenant_id, lower(name))
WHERE deleted_at IS NULL;

Soft delete in a transaction (PostgreSQL):

UPDATE customers
SET deleted_at = now(),
    deleted_by = $actor,
    version    = version + 1,
    updated_at = now()
WHERE id = $id
  AND tenant_id = $tenant
  AND (deleted_at IS NULL)
  AND version = $expected_version;

Restore with conflict handling:

-- First check if another active row already holds the unique email
-- If clear, restore by clearing deleted_at
UPDATE customers
SET deleted_at = NULL,
    version    = version + 1,
    updated_at = now()
WHERE id = $id AND tenant_id = $tenant
RETURNING *;

HTTP workflow examples

Delete, then later include deleted in reads:

DELETE /v1/customers/123 HTTP/1.1
If-Match: "v7"

HTTP/1.1 204 No Content

GET /v1/customers/123 HTTP/1.1
-- Default: hidden if your API treats deleted as 404

HTTP/1.1 404 Not Found

GET /v1/customers/123?include=deleted=true HTTP/1.1
Authorization: Bearer admin

HTTP/1.1 200 OK
{
  "id": "123",
  "deleted": true,
  "deleted_at": "2026-04-17T15:16:22Z",
  "version": 8
}

Restore:

POST /v1/customers/123:restore HTTP/1.1
If-Match: "v8"

HTTP/1.1 200 OK
{
  "id": "123",
  "deleted": false,
  "version": 9
}

Rollout and migrations

A safe, incremental plan:

1. Add deleted_at, deleted_by columns and partial indexes.
2. Backfill: set deleted_at=NULL for all active rows; build new read filters in code behind a flag.
3. Dual read: default to excluding deleted, with an override in admin endpoints.
4. Enable soft deletes in write paths; continue to hard-delete only test data.
5. After validation, remove legacy hard-delete code.
6. Add purge worker and monitoring.

Monitoring and observability

Track and alert on:

• soft_delete_total, restore_total (by resource type and tenant)
• purge_job_runtime_seconds and purge_job_errors_total
• read_ratio_with_include_deleted (should be low outside admin tools)
• cache_hit_ratio for include=deleted queries
• incidents where deleted resources appear in public listings

Log structured metadata on delete/restore (actor, reason, request_id) and include a correlation id in emitted events.

Common pitfalls and how to avoid them

• Leaking deleted data in joins: always scope queries with deleted_at IS NULL for each table.
• Broken uniqueness on restore: enforce partial unique indexes and preflight checks.
• Stale caches: vary cache keys on include flags, invalidate on delete/restore.
• Search index drift: ensure delete events promptly remove documents from search.
• Infinite retention: define and enforce a purge policy; measure backlog.
• Inconsistent semantics across services: document conventions in an ADR and SDKs.

Alternatives and hybrids

• Hard delete with event-sourced audits: keep the history in an append-only log while removing the current state.
• Archive store: move full rows to cold storage with strict access.
• Field-level scrubbing on delete: blank sensitive fields and keep a tombstone; combine with purge later.

Checklist

• Model: deleted_at, deleted_by, version columns and partial indexes
• Endpoints: DELETE, :restore, include=deleted filtering
• Concurrency: ETags/If-Match
• AuthZ: dedicated scopes for read:deleted and restore
• Audit: who/when/why stored immutably
• Purge: scheduled hard delete with retries and metrics
• Docs: precise status code semantics and examples

Conclusion

Soft delete is straightforward in concept but touches every layer of a system: storage, queries, HTTP semantics, caches, search, and compliance. By choosing a single, well-documented model (deleted_at plus metadata), enforcing partial indexes, defining clear endpoint behavior (DELETE and :restore), and operating a disciplined purge pipeline, you gain reversibility and auditability without leaking deleted data—or surprising clients. Treat soft delete as a product decision with operational guardrails, and your REST APIs will remain both safe and predictable at scale.