fix: Improve Korean RRN regex pattern validation

[kyoungbinkim]

• Use negative lookahead/lookbehind instead of word boundaries
• Add gender digit validation ([1-4] for first digit of last 7 digits)

Change Description

1. Replacing word boundaries with negative lookahead/lookbehind

   • Changed from \b to (?<!\d) and (?!\d)
   • Prevents false positives when RRN-like patterns appear within longer numeric sequences
   • Example: 1901201-1234567 (14 digits) will no longer be matched
   • Handles Korean language characteristics: In Korean text, numbers are often followed directly by Korean characters without spaces (e.g., "내 주민번호는 960101-1234567이야")
   • Word boundary \b doesn't work correctly with non-ASCII characters, so negative lookahead/lookbehind ensures proper matching in Korean contexts

2. Adding gender digit validation

   • Changed from \d{7} to [1-4]\d{6}
   • Validates that the first digit of the last 7 digits is between 1-4
   • Enforces Korean RRN specification where this digit indicates gender and century of birth
   • Rejects invalid patterns like 901201-5234567 or 901201-0234567

3. Improving day pattern consistency

   • Normalized [1-2][0-9] to [12]\d for better readability

Before: \b\d{2}(0[1-9]|1[0-2])(0[1-9]|[1-2][0-9]|3[0-1])(-?)\d{7}\b

After: (?<!\d)\d{2}(0[1-9]|1[0-2])(0[1-9]|[12]\d|3[01])(-?)[1-4]\d{6}(?!\d)

Checklist

• [ V] I have reviewed the contribution guidelines
• I have signed the CLA (if required)
• My code includes unit tests
• All unit tests and lint checks pass locally
• My PR contains documentation updates / additions if required

[omri374]

Thanks @kyoungbinkim. Could you please add a few test cases, especially for the cases that fail on the previous pattern and succeed on the new one?

[kyoungbinkim]

Thanks @kyoungbinkim. Could you please add a few test cases, especially for the cases that fail on the previous pattern and succeed on the new one?

@omri374 Thank you for the review!
These are invalid test cases because the gender digit is incorrect. The gender digit must be 1-4, but these examples have gender digits 7 and 0, which are not valid according to the Korean RRN format.

    "000505-7637892 (invalid - gender digit 7)", # invalid RRN: gender digit must be 1-4
    "0005057637892 (invalid - gender digit 7)", # invalid RRN without hyphen: gender digit must be 1-4
    "050912-0000008 (invalid - gender digit 0)", # invalid RRN: gender digit must be 1-4
    "0509120000008 (invalid - gender digit 0)", # invalid RRN without hyphen: gender digit must be 1-4

I've added test cases to verify the new pattern implementation. Please check the test code below:

from presidio_analyzer import AnalyzerEngine, PatternRecognizer, Pattern
from presidio_analyzer.nlp_engine import SpacyNlpEngine, NlpEngineProvider
from presidio_analyzer.predefined_recognizers import KrRrnRecognizer


configuration = {
    "nlp_engine_name": "spacy",
    "models": [{"lang_code": "ko", "model_name": "ko_core_news_sm"}]
}
provider = NlpEngineProvider(nlp_configuration=configuration)
nlp_engine = provider.create_engine()

analyzer = AnalyzerEngine(nlp_engine=nlp_engine, supported_languages=["ko"])
analyzer.registry.add_recognizer(KrRrnRecognizer(supported_language='ko'))

"""
test text containing Korean Resident Registration Numbers

korean RRN format: YYMMDD-GXXXXXX
YYMMDD : birth date
G : gender 
    1 : male (born 1900-1999)
    2 : female (born 1900-1999)
    3 : male (born 2000-2099)
    4 : female (born 2000-2099)
"""
test_texts = [
    "그의 주민등록번호는 960214-1157348입니다.(valid)", # it means "His resident registration number is 960214-1157348."
    "rrn 960325-5123456 (invalid)", # invalid RRN gender digit 5
    "홍길동의 주민등록번호는 850101-7345678 입니다. (invalid)", # it means "Hong Gil-dong's resident registration number is 850101-7345678."
]


"""

before result :

    Text: 그의 주민등록번호는 960214-1157348입니다.(valid)

    Text: rrn 960325-5123456 (invalid)
    Entity: KR_RRN, Start: 4, End: 18, Score: 0.5 text:960325-5123456

    Text: 홍길동의 주민등록번호는 850101-7345678 입니다. (invalid)
    Entity: KR_RRN, Start: 13, End: 27, Score: 0.5 text:850101-7345678


after result :

    Text: 그의 주민등록번호는 960214-1157348입니다.(valid)
    Entity: KR_RRN, Start: 11, End: 25, Score: 0.5 text:960214-1157348

    Text: rrn 960325-5123456 (invalid)

    Text: 홍길동의 주민등록번호는 850101-7345678 입니다. (invalid)
    
"""
for text in test_texts:
    results = analyzer.analyze(text=text, language='ko')
    print(f"\nText: {text}")
    for entity in results:
        print(f"Entity: {entity.entity_type}, Start: {entity.start}, End: {entity.end}, Score: {entity.score} text:{text[entity.start:entity.end]}")

[kyoungbinkim]

@kyoungbinkim please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

• (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.

@microsoft-github-policy-service agree

• (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.

@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree

[SharonHart]

Thanks @kyoungbinkim. Could you please add a few test cases, especially for the cases that fail on the previous pattern and succeed on the new one?

@omri374 Thank you for the review! These are invalid test cases because the gender digit is incorrect. The gender digit must be 1-4, but these examples have gender digits 7 and 0, which are not valid according to the Korean RRN format.

    "000505-7637892 (invalid - gender digit 7)", # invalid RRN: gender digit must be 1-4
    "0005057637892 (invalid - gender digit 7)", # invalid RRN without hyphen: gender digit must be 1-4
    "050912-0000008 (invalid - gender digit 0)", # invalid RRN: gender digit must be 1-4
    "0509120000008 (invalid - gender digit 0)", # invalid RRN without hyphen: gender digit must be 1-4

I've added test cases to verify the new pattern implementation. Please check the test code below:

from presidio_analyzer import AnalyzerEngine, PatternRecognizer, Pattern
from presidio_analyzer.nlp_engine import SpacyNlpEngine, NlpEngineProvider
from presidio_analyzer.predefined_recognizers import KrRrnRecognizer


configuration = {
    "nlp_engine_name": "spacy",
    "models": [{"lang_code": "ko", "model_name": "ko_core_news_sm"}]
}
provider = NlpEngineProvider(nlp_configuration=configuration)
nlp_engine = provider.create_engine()

analyzer = AnalyzerEngine(nlp_engine=nlp_engine, supported_languages=["ko"])
analyzer.registry.add_recognizer(KrRrnRecognizer(supported_language='ko'))

"""
test text containing Korean Resident Registration Numbers

korean RRN format: YYMMDD-GXXXXXX
YYMMDD : birth date
G : gender 
    1 : male (born 1900-1999)
    2 : female (born 1900-1999)
    3 : male (born 2000-2099)
    4 : female (born 2000-2099)
"""
test_texts = [
    "그의 주민등록번호는 960214-1157348입니다.(valid)", # it means "His resident registration number is 960214-1157348."
    "rrn 960325-5123456 (invalid)", # invalid RRN gender digit 5
    "홍길동의 주민등록번호는 850101-7345678 입니다. (invalid)", # it means "Hong Gil-dong's resident registration number is 850101-7345678."
]


"""

before result :

    Text: 그의 주민등록번호는 960214-1157348입니다.(valid)

    Text: rrn 960325-5123456 (invalid)
    Entity: KR_RRN, Start: 4, End: 18, Score: 0.5 text:960325-5123456

    Text: 홍길동의 주민등록번호는 850101-7345678 입니다. (invalid)
    Entity: KR_RRN, Start: 13, End: 27, Score: 0.5 text:850101-7345678


after result :

    Text: 그의 주민등록번호는 960214-1157348입니다.(valid)
    Entity: KR_RRN, Start: 11, End: 25, Score: 0.5 text:960214-1157348

    Text: rrn 960325-5123456 (invalid)

    Text: 홍길동의 주민등록번호는 850101-7345678 입니다. (invalid)
    
"""
for text in test_texts:
    results = analyzer.analyze(text=text, language='ko')
    print(f"\nText: {text}")
    for entity in results:
        print(f"Entity: {entity.entity_type}, Start: {entity.start}, End: {entity.end}, Score: {entity.score} text:{text[entity.start:entity.end]}")

That's great, I would add the invalid cases as well, with a '0' matches assertion

[kyoungbinkim]

@SharonHart There was an error in the test case score calculation, so I fixed it. Thank you.

[SharonHart]

@kyoungbinkim
Can you add the invalid RRNs to the test cases, and make sure they are not detected?

[kyoungbinkim]

@SharonHart Added an invalid test case to detect incorrect input. Thanks!