Company Name Normalization & Legal
Suffix Deduplication (SimHash, Python, SQL).
GmbH, S.L., SAS, B.V., Ltd, Sp. z o.o., Sdn Bhd), and 64-bit SimHash Hamming-distance blocking in O(N) time. Central.Enterprises Standard v1.1 applies this Quality Index [D1] pipeline across 8.1M+ companies.
Cross-Border Legal Entity Suffix Normalization Matrix (21 Jurisdictions)
| Jurisdiction | Statutory Legal Suffixes Stripped | Raw Registry String | Normalized Blocking Key |
|---|---|---|---|
| Germany & Austria → | GmbH, UG (haftungsbeschränkt), AG, GmbH & Co. KG, FlexCo, e.U. | Bosch Rexroth GmbH & Co. KG | bosch rexroth |
| Switzerland → | AG, GmbH, Sàrl, SA, Sagl, Genossenschaft | Helvetia Holding AG / SA | helvetia holding |
| Spain → | S.L., S.A., S.L.U., S.L.P., Sociedad Limitada, S.Coop. | Grupo Logística Ibérica S.L.U. | grupo logistica iberica |
| France → | SAS, SASU, SARL, EURL, SA, SCI, SNC | Société Générale de Conseil SASU | societe generale de conseil |
| Netherlands & Belgium → | B.V., N.V., V.O.F., BV / SRL, CV / SC, Stichting | Noordzee Maritiem B.V. | noordzee maritiem |
| United Kingdom & Ireland → | Ltd, Limited, PLC, LLP, DAC, CLG, CIC | Thames Valley Robotics Ltd. | thames valley robotics |
| Poland → | Sp. z o.o., S.A., Sp.k., Sp.j., P.S.A. | Kraków Tech Systems Sp. z o.o. | krakow tech systems |
| Malaysia & Indonesia → | Sdn. Bhd., Berhad, PLT, PT (Perseroan Terbatas), CV | Bintang Nusantara Sdn. Bhd. | bintang nusantara |
| Italy → | S.r.l., S.p.A., S.r.l.s., S.n.c., S.a.s. | Meccanica Emiliana S.r.l. | meccanica emiliana |
| Brazil & Portugal → | Ltda., S.A., SLU, Lda., Unipessoal Lda., SGPS | Atlântico Energia Ltda. | atlantico energia |
| United States & Canada → | Inc., LLC, Corp., Corporation, Co., ULC, LP | Pacific Cloud Analytics, Inc. | pacific cloud analytics |
How Does Python Normalize International Diacritics and Multi-Token Legal Suffixes?
International registry normalization first folds Unicode diacritics into ASCII equivalents (Kraków to krakow, München to muenchen or munchen, Société to societe) before iteratively stripping multi-word trailing designators such as sp z o o, gmbh co kg, and sdn bhd. Combined with deterministic IDs from our Registry Identifiers Pillar and GLEIF LEI mapping, this blocks duplicate vendor records before scoring with Jaro-Winkler or token ratios.
import re, unicodedata
MULTI_SUFFIXES = ("sp z oo", "sp z o o", "gmbh co kg", "gmbh & co kg", "sdn bhd", "unipessoal lda")
SINGLE_SUFFIXES = {
"gmbh", "ag", "ug", "sl", "slu", "slp", "sa", "sas", "sasu", "sarl", "eurl",
"bv", "nv", "vof", "srl", "spa", "srls", "ltd", "limited", "plc", "llp", "dac",
"inc", "incorporated", "corp", "llc", "ltda", "lda", "as", "asa", "uab", "ab", "pt", "cv", "berhad"
}
def normalize_entity_name(raw_name: str) -> str:
txt = unicodedata.normalize("NFKD", raw_name.lower()).encode("ascii", "ignore").decode("ascii")
txt = re.sub(r"[^a-z0-9\s]", " ", txt)
txt = re.sub(r"\s+", " ", txt).strip()
for ms in MULTI_SUFFIXES:
if txt.endswith(" " + ms):
txt = txt[: -len(ms)].strip()
tokens = txt.split()
while len(tokens) > 1 and tokens[-1] in SINGLE_SUFFIXES:
tokens.pop()
return " ".join(tokens)
How Does 64-Bit SimHash + SQL Windowing Scale Across Millions of Companies?
Sixty-four-bit SimHash compresses normalized company character 3-grams and word tokens into four 16-bit bucket keys so PostgreSQL or DuckDB indexes locate candidate pairs at Hamming distance <= 3 in linear time. Once candidates share a 16-bit bucket, exact domain matching and Central.Enterprises Quality Index [D1] thresholds separate subsidiaries from true duplicates. Test these queries directly on 100-row schema previews from Germany, Poland, Netherlands, and France via our CC0 Open Catalog.
-- PostgreSQL: Canonicalize domain & normalized name before window deduplication
WITH cleaned AS (
SELECT
sdc_id,
country_code,
lower(regexp_replace(website, '^https?://(www\.)?|/.*$', '', 'g')) AS root_domain,
trim(regexp_replace(lower(name), '\s+(gmbh|ag|s\.l\.|s\.a\.|sas|sarl|b\.v\.|ltd|inc|llc|sp\.\s*z\s*o\.o\.|sdn\.\s*bhd\.)$', '', 'gi')) AS norm_name,
CentralRating
FROM open_company_core
)
SELECT *
FROM (
SELECT *, ROW_NUMBER() OVER (
PARTITION BY country_code, COALESCE(NULLIF(root_domain, ''), norm_name)
ORDER BY CentralRating DESC, sdc_id ASC
) AS rn
FROM cleaned
) t
WHERE rn = 1;
Eliminating Duplicate Ingestion Downstream in Excel, HubSpot & Salesforce
Downstream CRM workflows remain clean only when ingestion types preserve fixed-width registry IDs as strings and anchor upsert rules on website_domain alongside country_code. Follow our CSV & Excel Leading-Zero Import Guide when loading Switzerland, Spain, Belgium, Indonesia, or Malaysia archives, and inspect pre-deduplicated commercial files in Starter 90D and Pro Annual.
Frequently Asked Questions
Why must legal suffixes be stripped in a loop rather than with a single trailing split?
Many corporate names stack multiple legal or descriptive tokens at the end—such as GmbH & Co. KG in Germany, Sp. z o.o. Sp.k. in Poland, or Sdn. Bhd. in Malaysia. Iterative right-hand token stripping removes multi-word legal markers cleanly without corrupting brand words inside the core entity name.
How does 64-bit SimHash avoid O(N^2) fuzzy matching across millions of company rows?
Pairwise Levenshtein or Jaro-Winkler comparisons over 1,000,000 entities require 500 billion comparisons. SimHash projects normalized n-grams into a fixed 64-bit fingerprint and splits it into four 16-bit bands; by the pigeonhole principle, any pair within Hamming distance <= 3 matches identically in at least one band, reducing complexity to O(N).
What secondary blocking keys eliminate false merges between unrelated holding companies?
Always pair SimHash candidate buckets with normalized domain roots (website_domain) or ISO country_code + canonical registration numbers (SIREN, KvK, CRN, ZEFIX UID, KRS, CNPJ) before merging rows so identically named firms in different jurisdictions stay distinct.
Does Central.Enterprises run SimHash deduplication before publishing CC0 and Pro datasets?
Yes. Every one of our 21 national datasets (8,112,307 legal entities) passes our Quality Index [D1] SimHash deduplication, diacritic normalization, and domain reconciliation pipeline documented in our Verification Methodology.