Chinese Unified Social Credit Codes

Introduction

The function clean_cn_uscc() cleans a column containing Chinese Unified Social Credit Code (USCC) strings, and standardizes them in a given format. The function validate_cn_uscc() validates either a single USCC strings, a column of USCC strings or a DataFrame of USCC strings, returning True if the value is valid, and False otherwise.

USCC strings can be converted to the following formats via the output_format parameter:

• compact: only number strings without any seperators or whitespace, like “91110000600037341L”

• standard: USCC strings with proper whitespace in the proper places. Note that in the case of USCC, the compact format is the same as the standard one.

Invalid parsing is handled with the errors parameter:

• coerce (default): invalid parsing will be set to NaN

• ignore: invalid parsing will return the input

• raise: invalid parsing will raise an exception

The following sections demonstrate the functionality of clean_cn_uscc() and validate_cn_uscc().

An example dataset containing USCC strings

import pandas as pd
import numpy as np
df = pd.DataFrame(
    {
        "uscc": [
            "9 1 110000 600037341L",
            "A1110000600037341L",
            "51824753556",
            "51 824 753 556",
            "hello",
            np.nan,
            "NULL"
        ],
        "address": [
            "123 Pine Ave.",
            "main st",
            "1234 west main heights 57033",
            "apt 1 789 s maple rd manhattan",
            "robie house, 789 north main street",
            "(staples center) 1111 S Figueroa St, Los Angeles",
            "hello",
        ]
    }
)
df

	uscc	address
0	9 1 110000 600037341L	123 Pine Ave.
1	A1110000600037341L	main st
2	51824753556	1234 west main heights 57033
3	51 824 753 556	apt 1 789 s maple rd manhattan
4	hello	robie house, 789 north main street
5	NaN	(staples center) 1111 S Figueroa St, Los Angeles
6	NULL	hello

1. Default clean_cn_uscc

By default, clean_cn_uscc will clean uscc strings and output them in the standard format with proper separators.

from dataprep.clean import clean_cn_uscc
clean_cn_uscc(df, column = "uscc")

	uscc	address	uscc_clean
0	9 1 110000 600037341L	123 Pine Ave.	91110000600037341L
1	A1110000600037341L	main st	NaN
2	51824753556	1234 west main heights 57033	NaN
3	51 824 753 556	apt 1 789 s maple rd manhattan	NaN
4	hello	robie house, 789 north main street	NaN
5	NaN	(staples center) 1111 S Figueroa St, Los Angeles	NaN
6	NULL	hello	NaN

2. Output formats

This section demonstrates the output parameter.

standard (default)

clean_cn_uscc(df, column = "uscc", output_format="standard")

	uscc	address	uscc_clean
0	9 1 110000 600037341L	123 Pine Ave.	91110000600037341L
1	A1110000600037341L	main st	NaN
2	51824753556	1234 west main heights 57033	NaN
3	51 824 753 556	apt 1 789 s maple rd manhattan	NaN
4	hello	robie house, 789 north main street	NaN
5	NaN	(staples center) 1111 S Figueroa St, Los Angeles	NaN
6	NULL	hello	NaN

compact

clean_cn_uscc(df, column = "uscc", output_format="compact")

	uscc	address	uscc_clean
0	9 1 110000 600037341L	123 Pine Ave.	91110000600037341L
1	A1110000600037341L	main st	NaN
2	51824753556	1234 west main heights 57033	NaN
3	51 824 753 556	apt 1 789 s maple rd manhattan	NaN
4	hello	robie house, 789 north main street	NaN
5	NaN	(staples center) 1111 S Figueroa St, Los Angeles	NaN
6	NULL	hello	NaN

3. inplace parameter

This deletes the given column from the returned DataFrame. A new column containing cleaned USCC strings is added with a title in the format "{original title}_clean".

clean_cn_uscc(df, column="uscc", inplace=True)

	uscc_clean	address
0	91110000600037341L	123 Pine Ave.
1	NaN	main st
2	NaN	1234 west main heights 57033
3	NaN	apt 1 789 s maple rd manhattan
4	NaN	robie house, 789 north main street
5	NaN	(staples center) 1111 S Figueroa St, Los Angeles
6	NaN	hello

4. errors parameter

coerce (default)

clean_cn_uscc(df, "uscc", errors="coerce")

	uscc	address	uscc_clean
0	9 1 110000 600037341L	123 Pine Ave.	91110000600037341L
1	A1110000600037341L	main st	NaN
2	51824753556	1234 west main heights 57033	NaN
3	51 824 753 556	apt 1 789 s maple rd manhattan	NaN
4	hello	robie house, 789 north main street	NaN
5	NaN	(staples center) 1111 S Figueroa St, Los Angeles	NaN
6	NULL	hello	NaN

ignore

clean_cn_uscc(df, "uscc", errors="ignore")

	uscc	address	uscc_clean
0	9 1 110000 600037341L	123 Pine Ave.	91110000600037341L
1	A1110000600037341L	main st	A1110000600037341L
2	51824753556	1234 west main heights 57033	51824753556
3	51 824 753 556	apt 1 789 s maple rd manhattan	51 824 753 556
4	hello	robie house, 789 north main street	hello
5	NaN	(staples center) 1111 S Figueroa St, Los Angeles	NaN
6	NULL	hello	NaN

4. validate_cn_uscc()

validate_cn_uscc() returns True when the input is a valid USCC. Otherwise it returns False.

The input of validate_cn_uscc() can be a string, a Pandas DataSeries, a Dask DataSeries, a Pandas DataFrame and a dask DataFrame.

When the input is a string, a Pandas DataSeries or a Dask DataSeries, user doesn’t need to specify a column name to be validated.

When the input is a Pandas DataFrame or a dask DataFrame, user can both specify or not specify a column name to be validated. If user specify the column name, validate_cn_uscc() only returns the validation result for the specified column. If user doesn’t specify the column name, validate_cn_uscc() returns the validation result for the whole DataFrame.

from dataprep.clean import validate_cn_uscc
print(validate_cn_uscc("9 1 110000 600037341L"))
print(validate_cn_uscc("A1110000600037341L"))
print(validate_cn_uscc("51824753556"))
print(validate_cn_uscc("51 824 753 556"))
print(validate_cn_uscc("hello"))
print(validate_cn_uscc(np.nan))
print(validate_cn_uscc("NULL"))

True
False
False
False
False
False
False

Series

validate_cn_uscc(df["uscc"])

0     True
1    False
2    False
3    False
4    False
5    False
6    False
Name: uscc, dtype: bool

DataFrame + Specify Column

validate_cn_uscc(df, column="uscc")

0     True
1    False
2    False
3    False
4    False
5    False
6    False
Name: uscc, dtype: bool

Only DataFrame

validate_cn_uscc(df)

	uscc	address
0	True	False
1	False	False
2	False	False
3	False	False
4	False	False
5	False	False
6	False	False