Indian Digital Resident Personal Identity Numbers¶
Introduction¶
The function clean_in_aadhaar()
cleans a column containing Indian digital resident personal identity number (Aadhaar) strings, and standardizes them in a given format. The function validate_in_aadhaar()
validates either a single Aadhaar strings, a column of Aadhaar strings or a DataFrame of Aadhaar strings, returning True
if the value is valid, and False
otherwise.
Aadhaar strings can be converted to the following formats via the output_format
parameter:
compact
: only number strings without any seperators or whitespace, like “234123412346”standard
: Aadhaar strings with proper whitespace in the proper places, like “2341 2341 2346”mask
: mask the first 8 digits as per MeitY guidelines for securing identity information and Sensitive personal data, like “XXXX XXXX 2346”.
Invalid parsing is handled with the errors
parameter:
coerce
(default): invalid parsing will be set to NaNignore
: invalid parsing will return the inputraise
: invalid parsing will raise an exception
The following sections demonstrate the functionality of clean_in_aadhaar()
and validate_in_aadhaar()
.
An example dataset containing Aadhaar strings¶
[1]:
import pandas as pd
import numpy as np
df = pd.DataFrame(
{
"aadhaar": [
"234123412346",
"643343121",
"999 999 999",
"004085616",
"002 724 334",
"hello",
np.nan,
"NULL",
],
"address": [
"123 Pine Ave.",
"main st",
"1234 west main heights 57033",
"apt 1 789 s maple rd manhattan",
"robie house, 789 north main street",
"1111 S Figueroa St, Los Angeles, CA 90015",
"(staples center) 1111 S Figueroa St, Los Angeles",
"hello",
]
}
)
df
[1]:
aadhaar | address | |
---|---|---|
0 | 234123412346 | 123 Pine Ave. |
1 | 643343121 | main st |
2 | 999 999 999 | 1234 west main heights 57033 |
3 | 004085616 | apt 1 789 s maple rd manhattan |
4 | 002 724 334 | robie house, 789 north main street |
5 | hello | 1111 S Figueroa St, Los Angeles, CA 90015 |
6 | NaN | (staples center) 1111 S Figueroa St, Los Angeles |
7 | NULL | hello |
1. Default clean_in_aadhaar
¶
By default, clean_in_aadhaar
will clean aadhaar strings and output them in the standard format with proper separators.
[2]:
from dataprep.clean import clean_in_aadhaar
clean_in_aadhaar(df, column = "aadhaar")
[2]:
aadhaar | address | aadhaar_clean | |
---|---|---|---|
0 | 234123412346 | 123 Pine Ave. | 2341 2341 2346 |
1 | 643343121 | main st | NaN |
2 | 999 999 999 | 1234 west main heights 57033 | NaN |
3 | 004085616 | apt 1 789 s maple rd manhattan | NaN |
4 | 002 724 334 | robie house, 789 north main street | NaN |
5 | hello | 1111 S Figueroa St, Los Angeles, CA 90015 | NaN |
6 | NaN | (staples center) 1111 S Figueroa St, Los Angeles | NaN |
7 | NULL | hello | NaN |
2. Output formats¶
This section demonstrates the output parameter.
standard
(default)¶
[3]:
clean_in_aadhaar(df, column = "aadhaar", output_format="standard")
[3]:
aadhaar | address | aadhaar_clean | |
---|---|---|---|
0 | 234123412346 | 123 Pine Ave. | 2341 2341 2346 |
1 | 643343121 | main st | NaN |
2 | 999 999 999 | 1234 west main heights 57033 | NaN |
3 | 004085616 | apt 1 789 s maple rd manhattan | NaN |
4 | 002 724 334 | robie house, 789 north main street | NaN |
5 | hello | 1111 S Figueroa St, Los Angeles, CA 90015 | NaN |
6 | NaN | (staples center) 1111 S Figueroa St, Los Angeles | NaN |
7 | NULL | hello | NaN |
compact
¶
[4]:
clean_in_aadhaar(df, column = "aadhaar", output_format="compact")
[4]:
aadhaar | address | aadhaar_clean | |
---|---|---|---|
0 | 234123412346 | 123 Pine Ave. | 234123412346 |
1 | 643343121 | main st | NaN |
2 | 999 999 999 | 1234 west main heights 57033 | NaN |
3 | 004085616 | apt 1 789 s maple rd manhattan | NaN |
4 | 002 724 334 | robie house, 789 north main street | NaN |
5 | hello | 1111 S Figueroa St, Los Angeles, CA 90015 | NaN |
6 | NaN | (staples center) 1111 S Figueroa St, Los Angeles | NaN |
7 | NULL | hello | NaN |
mask
¶
[5]:
clean_in_aadhaar(df, column = "aadhaar", output_format="mask")
[5]:
aadhaar | address | aadhaar_clean | |
---|---|---|---|
0 | 234123412346 | 123 Pine Ave. | XXXX XXXX 2346 |
1 | 643343121 | main st | NaN |
2 | 999 999 999 | 1234 west main heights 57033 | NaN |
3 | 004085616 | apt 1 789 s maple rd manhattan | NaN |
4 | 002 724 334 | robie house, 789 north main street | NaN |
5 | hello | 1111 S Figueroa St, Los Angeles, CA 90015 | NaN |
6 | NaN | (staples center) 1111 S Figueroa St, Los Angeles | NaN |
7 | NULL | hello | NaN |
3. inplace
parameter¶
This deletes the given column from the returned DataFrame. A new column containing cleaned Aadhaar strings is added with a title in the format "{original title}_clean"
.
[6]:
clean_in_aadhaar(df, column="aadhaar", inplace=True)
[6]:
aadhaar_clean | address | |
---|---|---|
0 | 2341 2341 2346 | 123 Pine Ave. |
1 | NaN | main st |
2 | NaN | 1234 west main heights 57033 |
3 | NaN | apt 1 789 s maple rd manhattan |
4 | NaN | robie house, 789 north main street |
5 | NaN | 1111 S Figueroa St, Los Angeles, CA 90015 |
6 | NaN | (staples center) 1111 S Figueroa St, Los Angeles |
7 | NaN | hello |
4. errors
parameter¶
coerce
(default)¶
[7]:
clean_in_aadhaar(df, "aadhaar", errors="coerce")
[7]:
aadhaar | address | aadhaar_clean | |
---|---|---|---|
0 | 234123412346 | 123 Pine Ave. | 2341 2341 2346 |
1 | 643343121 | main st | NaN |
2 | 999 999 999 | 1234 west main heights 57033 | NaN |
3 | 004085616 | apt 1 789 s maple rd manhattan | NaN |
4 | 002 724 334 | robie house, 789 north main street | NaN |
5 | hello | 1111 S Figueroa St, Los Angeles, CA 90015 | NaN |
6 | NaN | (staples center) 1111 S Figueroa St, Los Angeles | NaN |
7 | NULL | hello | NaN |
ignore
¶
[8]:
clean_in_aadhaar(df, "aadhaar", errors="ignore")
[8]:
aadhaar | address | aadhaar_clean | |
---|---|---|---|
0 | 234123412346 | 123 Pine Ave. | 2341 2341 2346 |
1 | 643343121 | main st | 643343121 |
2 | 999 999 999 | 1234 west main heights 57033 | 999 999 999 |
3 | 004085616 | apt 1 789 s maple rd manhattan | 004085616 |
4 | 002 724 334 | robie house, 789 north main street | 002 724 334 |
5 | hello | 1111 S Figueroa St, Los Angeles, CA 90015 | hello |
6 | NaN | (staples center) 1111 S Figueroa St, Los Angeles | NaN |
7 | NULL | hello | NaN |
4. validate_in_aadhaar()
¶
validate_in_aadhaar()
returns True
when the input is a valid Aadhaar. Otherwise it returns False
.
The input of validate_in_aadhaar()
can be a string, a Pandas DataSeries, a Dask DataSeries, a Pandas DataFrame and a dask DataFrame.
When the input is a string, a Pandas DataSeries or a Dask DataSeries, user doesn’t need to specify a column name to be validated.
When the input is a Pandas DataFrame or a dask DataFrame, user can both specify or not specify a column name to be validated. If user specify the column name, validate_in_aadhaar()
only returns the validation result for the specified column. If user doesn’t specify the column name, validate_in_aadhaar()
returns the validation result for the whole DataFrame.
[9]:
from dataprep.clean import validate_in_aadhaar
print(validate_in_aadhaar("234123412346"))
print(validate_in_aadhaar("643343121"))
print(validate_in_aadhaar("999 999 999"))
print(validate_in_aadhaar("51824753556"))
print(validate_in_aadhaar("004085616"))
print(validate_in_aadhaar("hello"))
print(validate_in_aadhaar(np.nan))
print(validate_in_aadhaar("NULL"))
True
False
False
False
False
False
False
False
Series¶
[10]:
validate_in_aadhaar(df["aadhaar"])
[10]:
0 True
1 False
2 False
3 False
4 False
5 False
6 False
7 False
Name: aadhaar, dtype: bool
DataFrame + Specify Column¶
[11]:
validate_in_aadhaar(df, column="aadhaar")
[11]:
0 True
1 False
2 False
3 False
4 False
5 False
6 False
7 False
Name: aadhaar, dtype: bool
Only DataFrame¶
[12]:
validate_in_aadhaar(df)
[12]:
aadhaar | address | |
---|---|---|
0 | True | False |
1 | False | False |
2 | False | False |
3 | False | False |
4 | False | False |
5 | False | False |
6 | False | False |
7 | False | False |
[ ]: