AshVault.Lookup (AshVault v0.2.0)

Copy Markdown View Source

Deterministic lookup tokens for searchable encrypted fields.

AES-GCM draws a fresh nonce for every write, so the same plaintext encrypts to different bytes every time: WHERE encrypted_email = $1 matches nothing and a unique index on the ciphertext column is meaningless. A searchable field therefore carries a second, deterministic column beside the ciphertext:

plaintext
  ├── AES-GCM → encrypted_email   (randomized, unsearchable)
  └── HMAC    → email_lookup      (deterministic, indexable)

The key is not the DEK, and is not derived from it

The token key comes from AshVault.KeyProvider.lookup_key/1 — a separate, non-rotating, per-scope secret. AshVault.rotate_key!/2 must not change it.

If the token were derived from the rotating data encryption key, rotation would change every future token while every stored token still reflected the old key. Lookups for existing rows would silently stop matching, unique? would stop preventing duplicates and users could not log in — with nothing raised anywhere. That failure passes any test that writes a row and immediately reads it back, which is why test/ash_vault/lookup_rotation_test.exs asserts the provider hands back the same key binary across a rotation rather than inferring it from token equality.

Erasure still erases: AshVault.KeyProvider.destroy/1 destroys the lookup key along with every data key, and the same tombstone gates it afterwards.

Per-field separation comes from HKDF

lookup_key_for_field =
  HKDF-SHA256(ikm:  provider_lookup_key(scope),
              info: "ash_vault:lookup:v1|" <> inspect(resource) <> "|" <> field)

token = HMAC-SHA256(lookup_key_for_field, normalized_plaintext)

Binding the resource and the field means the same plaintext in two columns, or in two resources, produces two unrelated tokens — so a token cannot be replayed across columns any more than a ciphertext can be replayed across its AAD. Binding the scope is what the per-scope provider key already does, and it is what keeps the equality leak inside one tenant.

HKDF is not in OTP, so hkdf/4 implements RFC 5869 extract-and-expand directly on :crypto.mac/4. It is tested against the RFC's own vectors.

Rotating a lookup key is a backfill, not a rotation

There is deliberately no rotate_lookup_key anywhere. Changing the lookup key — like changing normalize: — invalidates every stored token, so it is a data migration: destroy nothing, re-derive, and rewrite the column with mix ash_vault.backfill --lookup.

Summary

Types

A normalization strategy for a searchable field.

Functions

Derive the 32-byte token key for one resource/field from a provider lookup key.

The derived per-field token key for an operation, or an error.

HKDF-SHA256 (RFC 5869): extract-then-expand, in one call.

HKDF-Expand (RFC 5869 §2.3).

HKDF-Extract (RFC 5869 §2.2): HMAC-SHA256(salt, ikm), returning a 32-byte PRK.

The HKDF info string binding a token key to one resource and field.

Apply a field's normalize: strategy to a plaintext value.

Normalize a value with the normalize: strategy configured for a field.

HMAC-SHA256(field_key, normalized_plaintext) — the stored token.

Compute the lookup token for a value, resolving the vault, scope and provider exactly as a write does.

Like token_for/4, but raises the error instead of returning it.

Like token_for/4, for a value that has already been normalized.

Types

normalize()

@type normalize() :: :none | :downcase | :downcase_trim | mfa() | (term() -> term())

A normalization strategy for a searchable field.

Functions

derive_field_key(lookup_key, resource, field)

@spec derive_field_key(binary(), module(), atom()) :: binary()

Derive the 32-byte token key for one resource/field from a provider lookup key.

field_key(resource, field, context)

@spec field_key(module(), atom(), AshVault.Context.t()) ::
  {:ok, binary()} | {:error, Exception.t()}

The derived per-field token key for an operation, or an error.

Exposed so a test can assert the key is stable across a rotation, which is the one property the whole mechanism rests on.

hkdf(ikm, salt, info, length)

@spec hkdf(binary(), binary(), binary(), pos_integer()) :: binary()

HKDF-SHA256 (RFC 5869): extract-then-expand, in one call.

salt may be "", which RFC 5869 §2.2 defines as a string of HashLen zeros.

RFC 5869 Appendix A.1 (SHA-256, with salt and info) round-trips through here; the RFC's three SHA-256 vectors are asserted in test/ash_vault/lookup_test.exs.

hkdf_expand(prk, info, length)

@spec hkdf_expand(binary(), binary(), pos_integer()) :: binary()

HKDF-Expand (RFC 5869 §2.3).

Raises ArgumentError for a length above 255 * HashLen, which the RFC forbids.

hkdf_extract(ikm, salt \\ "")

@spec hkdf_extract(binary(), binary()) :: binary()

HKDF-Extract (RFC 5869 §2.2): HMAC-SHA256(salt, ikm), returning a 32-byte PRK.

An empty salt is replaced by HashLen zero bytes, as the RFC specifies.

info(resource, field)

@spec info(module(), atom()) :: binary()

The HKDF info string binding a token key to one resource and field.

Frozen, human-inspectable, and built by stringification rather than :erlang.term_to_binary/1 — for exactly the reason AshVault.Vault.Runtime.build_aad/2 is: the external term format is not guaranteed stable across OTP releases, and stored tokens have to outlive OTP upgrades.

iex> AshVault.Lookup.info(MyApp.User, :email)
"ash_vault:lookup:v1|MyApp.User|email"

normalize(value, fun)

@spec normalize(term(), normalize()) :: {:ok, binary() | nil} | {:error, term()}

Apply a field's normalize: strategy to a plaintext value.

Returns {:ok, normalized}, or {:error, reason} when the strategy produced something that is not a binary — which is a bug in a custom normalizer, not a user error, and must not be papered over with to_string/1.

nil passes through as {:ok, nil}: a nil plaintext produces no token.

normalize_for(resource, field, value)

@spec normalize_for(module() | Spark.Dsl.t(), atom(), term()) ::
  {:ok, binary() | nil} | {:error, Exception.t()}

Normalize a value with the normalize: strategy configured for a field.

token(field_key, normalized)

@spec token(binary(), binary()) :: binary()

HMAC-SHA256(field_key, normalized_plaintext) — the stored token.

token_for(resource, field, value, context)

@spec token_for(module(), atom(), term(), AshVault.Context.t()) ::
  {:ok, binary() | nil} | {:error, Exception.t()}

Compute the lookup token for a value, resolving the vault, scope and provider exactly as a write does.

Returns {:ok, nil} for a nil plaintext. Returns {:error, exception} for a missing scope, a destroyed scope, an unreachable provider, a provider with no AshVault.KeyProvider.lookup_key/1, or a normalizer that did not return a binary.

Never raises: like AshVault.decrypt_value/5, crypto failures are values here so the write path can add them to a changeset and the read path to a query.

token_for!(resource, field, value, context)

@spec token_for!(module(), atom(), term(), AshVault.Context.t()) :: binary() | nil

Like token_for/4, but raises the error instead of returning it.

This is the form AshVault.Query.filter_by/4 uses: a plain function has no changeset or query to attach an error to, and a lookup that quietly returned no filter would read as "no such user".

token_for_normalized(resource, field, normalized, context)

@spec token_for_normalized(module(), atom(), binary() | nil, AshVault.Context.t()) ::
  {:ok, binary() | nil} | {:error, Exception.t()}

Like token_for/4, for a value that has already been normalized.

The write path normalizes once and encrypts the result, so it must not normalize a second time here: a custom normalize: MFA is not required to be idempotent, and the token has to be the hash of exactly the bytes that were encrypted.