Masking and redaction
A query or output replaces sensitive values. Original records may still exist. Useful for role-specific views, but not proof of anonymisation.
Pseudonymisation vs Anonymisation: What Data Platforms Must Actually Do
A practical engineering comparison of pseudonymisation, anonymisation, masking, hashing and tokenisation, including risks and Python examples.
November 5, 2026
- Pseudonymised data generally remains personal data where additional information can re-identify a person.
- Masking and encryption are not automatically anonymisation.
- Small groups, unusual attributes and linkable identifiers can expose individuals.
The decisive question is whether someone can still be identified
Pseudonymisation replaces direct identifiers while keeping the additional information needed to reconnect them separately protected. When records can still be linked to a person, the data generally remains subject to personal data protections.
Anonymisation requires that the individual cannot be identified using means reasonably likely to be available. For real datasets, the assessment of re-identification risk matters more than the label on an export file.
Why hashing does not automatically anonymise data
Hashing an unchanged email address or customer identifier with a fast hash function often provides inadequate protection. Identical inputs produce identical hashes, limited input spaces are vulnerable to dictionary attacks and stable identifiers can correlate records across systems.
For controlled matching, random tokens or cryptographic methods with protected key management may be more appropriate. If the mapping back to a person remains possible, the result is pseudonymisation.
Four mechanisms with different security properties
Pseudonymisation with tokens or HMAC
Direct identifiers are replaced by substitute values. A protected mapping or secret may permit re-identification; the records generally remain personal data.
Aggregation and generalisation
Individual values are grouped or converted to ranges. Analytical utility may remain, but small groups and unusual attributes need review.
Anonymisation
The aim is to prevent identification given reasonably available information and realistic attacks. It is an assessed property of the result, not a product label.
Threat model: identifying someone without a name
Consider three classes of attack. *Singling out* isolates a person in a dataset. *Linkability* connects records belonging to the same person across sources. *Inference* reveals sensitive attributes from apparently unrelated data. Removing names may leave all three possibilities intact; a rare birth date, a location and an insurance payment can be enough to identify a person.
The recipient's environment matters. An internal team may have access to a CRM while an external research partner may have public demographic data. A service provider might hold logs or archived exports. Therefore the risk assessment must include realistic additional information, the intended users, the purpose and every access path, not just the transformed table.
Hashing is particularly easy to overstate. A limited set of email addresses, customer numbers or pension account formats can allow a dictionary of candidate inputs. Deterministic values also make cross-system linking easier even if the original identifier cannot be read directly.
Example: deterministic reference and random token (Python)
# Demonstration only; not a production tokenisation service
import secrets
import hmac
import hashlib
secret = secrets.token_bytes(32) # Production: KMS/Vault
person_id = "synthetic-member-101"
join_key = hmac.new(
secret, person_id.encode(), hashlib.sha256
).hexdigest()
random_token = secrets.token_urlsafe(24)
print(join_key[:12], random_token)
# Keep mapping data and keys outside the analytics system.What the HMAC sample demonstrates—and what it leaves out
The code derives a deterministic `join_key` with HMAC-SHA-256 and creates a separate random token. The HMAC can help demonstrate secret-dependent matching: identical inputs with the same key yield the same output. A random token has no direct mathematical relationship to the original value if its mapping is stored separately.
These methods have different operational implications. HMAC secrets belong in a key management system or vault, with restricted computation, rotation and audit. Tokenisation requires a protected mapping service, permission checks, resilient storage and deletion procedures. The demo's randomly generated in-memory key is not production key management; a new key would change the output and may break longitudinal matching.
Neither method removes the sensitivity of linked health events, payments or claims. Pseudonymisation protects the direct relationship to identity but does not automatically anonymise an individual's history.
Example: tokenisation with a separately protected mapping
# Illustration only; not a production token vault
import secrets
identity_id = "synthetic-member-101"
token = secrets.token_urlsafe(24)
# The mapping belongs in a separately secured service.
protected_mapping = {token: identity_id}
# Only this minimised record is supplied to analytics.
analytics_record = {
"member_token": token,
"fund_id": "FUND-A",
"accrued_pension_eur": 128500.00,
}
assert "identity_id" not in analytics_record
# Production: access control, encryption, durable storage,
# audit trails, lifecycle management and deletion.Technical data-flow architecture
An operational identity system manages personal master records. A controlled transformation produces a minimised data product for a specific analytical purpose. Mapping tables and keys stay separate; permissions, deletion, auditing and export controls need to work across the entire data flow.
Metadata catalogues such as OpenMetadata can support documentation, but cannot replace real access enforcement. Related engineering: https://valanor.co/en/pii-data-masking and https://valanor.co/en/data-governance.
Healthcare architecture: purpose before datasets
A healthcare data platform may connect administration, clinical systems and identity records. Start by defining the analysis. A capacity report may only need department utilisation and time windows. A longitudinal patient study might legitimately require a stable pseudonymous identifier. Those use cases should not automatically expose the same detail level.
An engineering approach separates source systems, a controlled transformation layer and a protected identity-mapping service. Analysts receive minimised attributes under defined roles, while re-identification is governed by a different approval route. Logs and observability must avoid accidentally recreating the very exposure the architecture was designed to prevent.
Rare diagnoses, small organisations and unusual treatment dates create residual risk. Pseudonymisation is not enough when someone can be inferred from a distinctive combination of fields. Related service context: https://valanor.co/en/healthcare-data-governance.
Insurance and pension-fund architecture
Insurance and pension platforms often have several identifiers for the same person: customer number, policy identifier, claim ID, pension account and warehouse join keys. Keeping all of them unchanged in analytics makes linkage across systems easier. A use-case-specific data contract should spell out allowable identifiers, granularity, purpose and retention.
Portfolio reporting may work with pre-aggregated records. Recurring event analysis may need a pseudonymous key. That requires explicit design for key versions, duplicate records, corrections, mapping security and eventual deletion. A pseudonym is not anonymous simply because a dashboard does not show the name.
See https://valanor.co/en/data-governance-insurance and the ClickHouse pension-fund example: https://valanor.co/en/insights/pii-data-masking-clickhouse-pension-fund.
When synthetic data helps
Synthetic test records can reduce the need to duplicate real identities in development environments. However, generators, training data, distribution properties and potential reconstruction or memorisation still need to be assessed. Synthetic does not automatically mean risk-free.
For regulated healthcare and insurance workloads, assess purpose, legal basis, retention, accountability and remaining re-identification risks separately.
Validation checklist before releasing a dataset
Define purpose and recipient
Record the specific analytical decision, intended users and realistically available auxiliary information.
Inventory identifiers
Include direct and indirect identifiers, timestamps, rare values and combinations of fields—not just obvious PII columns.
Simulate realistic attacks
Test singling out, linkability, differencing and re-identification using plausible external data. Review small cohorts separately.
Measure analytical utility
Check whether generalisation, aggregation or noise changes the conclusions the business needs to draw.
Test lifecycle and access
Exercise role changes, exports, backups, key rotation, deletion requests and incident scenarios.
Document approval and reassessment
Record residual risk, controls, accountable owners and when the dataset must be reviewed again.
Operations: keys, deletion and changes
Pseudonymisation is often treated as a one-time ETL transform. In reality it needs ongoing security operations. Keys should not be embedded in repositories, notebooks or unprotected environment variables. Mapping access must be isolated from analytics. Logs, dead-letter queues, debug exports and backups need their own controls.
Schema changes may introduce new attributes that increase linkability. Rotating a key may break longitudinal analysis unless there is a planned migration; retaining old mappings forever may undermine deletion. Data contracts, regression testing and controlled migrations are part of the mechanism, not optional documentation.
European Commission and EDPB material explain the distinction between personal, pseudonymised and anonymised information: https://commission.europa.eu/law/law-topic/data-protection/information-business-and-organisations/application-gdpr_en and https://www.edpb.europa.eu/topics/ai-and-technology/anonymisation-pseudonymisation_en. Proposed guidance must be distinguished from adopted rules. Legal conclusions require a case-specific assessment.
Frequently asked questions
Are hashed records anonymous?
Not necessarily. Small input spaces, possible reversibility and cross-dataset joins can still allow an individual to be identified.
Is a dynamic masking policy pseudonymisation?
Not automatically. It primarily alters a value shown in a query; protection depends on roles and other access paths.
Is synthetic data always safe?
No. A generator trained on real data can reproduce rare attributes, duplicates or memorised details.
May analysts use pseudonymised data?
Only with an appropriate legal basis, limited purpose and suitable technical and organisational safeguards.
Technical discussion
Secure analytics starts with the data architecture.
We will review your source systems, data sensitivity, roles and a practical way to implement controls.
Aleksandar Basara
Managing Director
aleks@valanor.co
+49 (0) 711 16 44 6 83
Lindenspürstrasse 22, 70176 Stuttgart
