Tokenization Formats

Tokenization formats define how the original format of data coming in is converted to the format you configure when the token is created. For example, you can specify that the token look like the original data source, or whether to tokenize the whole or partial string.

When you create a tokenization format and select the data domain, you are given configuration options.

Supported Data Domains

Data Domain Description

Data Domain	Description
Credit card number	Decimal number with special restrictions on length. Luhn checksum with limited set of lax character allowed; preserve first and last "n" digits; force invalid.
Decimal	Arbitrary length decimal number; preserve first and last "n" digits; force invalid.
Lax decimal	Arbitrary length decimal number with lax characters allowed; preserve first and last "n" digits; force invalid.
Social security number	Nine digit decimal number with special subcomponent restrictions; a limited number of lax characters (non-decimal digits) are allowed. Preserve serial number component; force invalid.
Email address	Complex domain with separate restrictions for local part (Radix = 83) and domain (Radix = 64) components.
Printable ASCII	Subset of the UTF8 characters between unicode number (int) 32 through 126 (AZa z09, space, double quote) and the following special characters: ! # $ % & '( ) * + , . / : ; < = > ? @ [ \ ] ^ _ ` { } ~. this format preserves first and last "n" characters.
Lax alphanumeric	Uppercase Latin, lowercase Latin, and decimal characters obfuscated but with lax characters allowed; preserve first and last "n" characters.
Lax uppercase/lowercase/decimal - Preserving	Uppercase Latin characters in the cleartext will remain uppercase Latin characters in the obfuscated text. Lowercase Latin characters in the cleartext will remain lowercase Latin in the obfuscated text. Decimal digits in the cleartext will remain decimal digits in the obfuscated text. All other characters will be handled as lax characters. Preserve first and last "n" characters.
Lax uppercase/decimal - Preserving	Uppercase Latin characters in the cleartext will remain uppercase Latin characters in the obfuscated text. Decimal digits in the cleartext will remain decimal digits in the obfuscated text. All other characters will be handled as lax characters. Preserve first and last "n" characters.
Lax lowercase/decimal - Preserving	Lowercase Latin characters in the cleartext remain lowercase Latin in the obfuscated text. Decimal digits in the cleartext remain decimal digits in the obfuscated text. All other characters are handled as lax characters. Preserve first and last "n" characters.
Masking	Replaces characters with a specified masking character, for example `#`. The input is all printable ASCII characters. Printable ASCII characters in the optional Ignore Characters configurable field are ignored. The character specified in the Masking Character configurable option is used to mask character identities.

Credit card number

Decimal number with special restrictions on length. Luhn checksum with limited set of lax character allowed; preserve first and last "n" digits; force invalid.

Decimal

Arbitrary length decimal number; preserve first and last "n" digits; force invalid.

Lax decimal

Arbitrary length decimal number with lax characters allowed; preserve first and last "n" digits; force invalid.

Social security number

Nine digit decimal number with special subcomponent restrictions; a limited number of lax characters (non-decimal digits) are allowed. Preserve serial number component; force invalid.

Email address

Complex domain with separate restrictions for local part (Radix = 83) and domain (Radix = 64) components.

Printable ASCII

Subset of the UTF8 characters between unicode number (int) 32 through 126 (AZa z09, space, double quote) and the following special characters: ! # $ % & '( ) * + , . / : ; < = > ? @ [ \ ] ^ _ ` { } ~. this format preserves first and last "n" characters.

Lax alphanumeric

Uppercase Latin, lowercase Latin, and decimal characters obfuscated but with lax characters allowed; preserve first and last "n" characters.

Lax uppercase/lowercase/decimal - Preserving

Uppercase Latin characters in the cleartext will remain uppercase Latin characters in the obfuscated text. Lowercase Latin characters in the cleartext will remain lowercase Latin in the obfuscated text. Decimal digits in the cleartext will remain decimal digits in the obfuscated text. All other characters will be handled as lax characters. Preserve first and last "n" characters.

Lax uppercase/decimal - Preserving

Uppercase Latin characters in the cleartext will remain uppercase Latin characters in the obfuscated text. Decimal digits in the cleartext will remain decimal digits in the obfuscated text. All other characters will be handled as lax characters. Preserve first and last "n" characters.

Lax lowercase/decimal - Preserving

Lowercase Latin characters in the cleartext remain lowercase Latin in the obfuscated text. Decimal digits in the cleartext remain decimal digits in the obfuscated text. All other characters are handled as lax characters. Preserve first and last "n" characters.

Masking

Replaces characters with a specified masking character, for example #. The input is all printable ASCII characters. Printable ASCII characters in the optional Ignore Characters configurable field are ignored. The character specified in the Masking Character configurable option is used to mask character identities.

Data Domain Configuration Options

Option Description Data Domain

Option	Description	Data Domain
Preserve first characters	Defines the number of characters from the original string to NOT tokenize. Enter a value for the number of beginning digits to preserve. For example, `4`.	Credit card Decimal Lax decimal Lax alphanumeric Printable ASCII Lax uppercase/lowercase/decimal - Preserving Lax uppercase/decimal - Preserving Lax lowercase/decimal - Preserving
Preserve last characters	Enter a value for the number of last digits to preserve. For example, `4`. Using the example values of `4` for first and last respectively, means that the first four digits and last four digits of the original string are preserved, and everything in between will be tokenized.	Credit card number Decimal Lax decimal Lax alphanumeric Printable ASCII Lax uppercase/lowercase/decimal - Preserving Lax uppercase/decimal - Preserving Lax lowercase/decimal - Preserving
Force token characters in illegal range	For example, if you have the CCN `1234567890123456` and you set the Preserve characters option to preserve the last four digits, then force an invalid one digit with an offset of two from left, the generated token would look similar to: `54D3210987653456`. A decimal digit is mapped to the Latin alphabet-character range. You are limited to two illegal characters because you need only one to make the data invalid (such as adding a letter A to a social security number). Adding additional invalid characters results in reducing the number of digits that can be used to randomize the tokenized data.	Credit card number Decimal Social security number
Luhn digit test	This option is for the credit card number data domain only. Use the Luhn checksum formula to validate the digits contained in the original data. The options are: Disabled Generated token must pass Luhn test Generated token must fail Luhn test	Credit card number
Maximum token length	Enter a value for the maximum number of digits for the generated token.	Decimal Lax decimal Email address Lax alphanumeric Printable ASCII Lax uppercase/lowercase/decimal - Preserving Lax uppercase/decimal - Preserving Lax lowercase/decimal - Preserving
Preserve serial number sub-component	This option is for the social security data domain only.	Social security number
Ignore characters	Used by the masking domain to specify which characters are not masked. For example, if Ignore Characters is set to `-` and Masking Character is set to `#`, masking applied to the number `123-45-6789` returns `####-###-####`.	Masking
Masking character	Used by the masking domain to specify the character used to hide sensitive data. For example, the `#` character.	Masking

Preserve first characters

Defines the number of characters from the original string to NOT tokenize. Enter a value for the number of beginning digits to preserve. For example, 4.

Credit card
Decimal
Lax decimal
Lax alphanumeric
Printable ASCII
Lax uppercase/lowercase/decimal - Preserving
Lax uppercase/decimal - Preserving
Lax lowercase/decimal - Preserving

Preserve last characters

Enter a value for the number of last digits to preserve. For example, 4.
Using the example values of 4 for first and last respectively, means that the first four digits and last four digits of the original string are preserved, and everything in between will be tokenized.

Credit card number
Decimal
Lax decimal
Lax alphanumeric
Printable ASCII
Lax uppercase/lowercase/decimal - Preserving
Lax uppercase/decimal - Preserving
Lax lowercase/decimal - Preserving

Force token characters in illegal range

For example, if you have the CCN 1234567890123456 and you set the Preserve characters option to preserve the last four digits, then force an invalid one digit with an offset of two from left, the generated token would look similar to: 54D3210987653456.

A decimal digit is mapped to the Latin alphabet-character range.

You are limited to two illegal characters because you need only one to make the data invalid (such as adding a letter A to a social security number). Adding additional invalid characters results in reducing the number of digits that can be used to randomize the tokenized data.

Credit card number
Decimal
Social security number

Luhn digit test

This option is for the credit card number data domain only. Use the Luhn checksum formula to validate the digits contained in the original data. The options are:

Disabled
Generated token must pass Luhn test
Generated token must fail Luhn test

Credit card number

Maximum token length

Enter a value for the maximum number of digits for the generated token.

Decimal
Lax decimal
Email address
Lax alphanumeric
Printable ASCII
Lax uppercase/lowercase/decimal - Preserving
Lax uppercase/decimal - Preserving
Lax lowercase/decimal - Preserving

Preserve serial number sub-component

This option is for the social security data domain only.

Social security number

Ignore characters

Used by the masking domain to specify which characters are not masked. For example, if Ignore Characters is set to - and Masking Character is set to #, masking applied to the number 123-45-6789 returns ####-###-####.

Masking

Masking character

Used by the masking domain to specify the character used to hide sensitive data. For example, the # character.

Masking