Bulgarian VAT Numbers

Introduction

The function clean_bg_vat() cleans a column containing Bulgarian VAT numbers (VAT) strings, and standardizes them in a given format. The function validate_bg_vat() validates either a single VAT strings, a column of VAT strings or a DataFrame of VAT strings, returning True if the value is valid, and False otherwise.

VAT strings can be converted to the following formats via the output_format parameter:

• compact: only number strings without any seperators or whitespace, like “175074752”

• standard: VAT strings with proper whitespace in the proper places. Note that in the case of VAT, the compact format is the same as the standard one.

Invalid parsing is handled with the errors parameter:

• coerce (default): invalid parsing will be set to NaN

• ignore: invalid parsing will return the input

• raise: invalid parsing will raise an exception

The following sections demonstrate the functionality of clean_bg_vat() and validate_bg_vat().

An example dataset containing VAT strings

import pandas as pd
import numpy as np
df = pd.DataFrame(
    {
        "vat": [
            'BG 175 074 752',
            '175074752',
            '175074751',    # invalid check digit
            'BE431150351',  # not Bulgarian VAT
            "002 724 334",  # not VAT
            "hello",
            np.nan,
            "NULL",
        ],
        "address": [
            "123 Pine Ave.",
            "main st",
            "1234 west main heights 57033",
            "apt 1 789 s maple rd manhattan",
            "robie house, 789 north main street",
            "1111 S Figueroa St, Los Angeles, CA 90015",
            "(staples center) 1111 S Figueroa St, Los Angeles",
            "hello",
        ]
    }
)
df

	vat	address
0	BG 175 074 752	123 Pine Ave.
1	175074752	main st
2	175074751	1234 west main heights 57033
3	BE431150351	apt 1 789 s maple rd manhattan
4	002 724 334	robie house, 789 north main street
5	hello	1111 S Figueroa St, Los Angeles, CA 90015
6	NaN	(staples center) 1111 S Figueroa St, Los Angeles
7	NULL	hello

1. Default clean_bg_vat

By default, clean_bg_vat will clean vat strings and output them in the standard format with proper separators.

from dataprep.clean import clean_bg_vat
clean_bg_vat(df, column = "vat")

	vat	address	vat_clean
0	BG 175 074 752	123 Pine Ave.	175074752
1	175074752	main st	175074752
2	175074751	1234 west main heights 57033	NaN
3	BE431150351	apt 1 789 s maple rd manhattan	NaN
4	002 724 334	robie house, 789 north main street	NaN
5	hello	1111 S Figueroa St, Los Angeles, CA 90015	NaN
6	NaN	(staples center) 1111 S Figueroa St, Los Angeles	NaN
7	NULL	hello	NaN

2. Output formats

This section demonstrates the output parameter.

standard (default)

clean_bg_vat(df, column = "vat", output_format="standard")

	vat	address	vat_clean
0	BG 175 074 752	123 Pine Ave.	175074752
1	175074752	main st	175074752
2	175074751	1234 west main heights 57033	NaN
3	BE431150351	apt 1 789 s maple rd manhattan	NaN
4	002 724 334	robie house, 789 north main street	NaN
5	hello	1111 S Figueroa St, Los Angeles, CA 90015	NaN
6	NaN	(staples center) 1111 S Figueroa St, Los Angeles	NaN
7	NULL	hello	NaN

compact

clean_bg_vat(df, column = "vat", output_format="compact")

	vat	address	vat_clean
0	BG 175 074 752	123 Pine Ave.	175074752
1	175074752	main st	175074752
2	175074751	1234 west main heights 57033	NaN
3	BE431150351	apt 1 789 s maple rd manhattan	NaN
4	002 724 334	robie house, 789 north main street	NaN
5	hello	1111 S Figueroa St, Los Angeles, CA 90015	NaN
6	NaN	(staples center) 1111 S Figueroa St, Los Angeles	NaN
7	NULL	hello	NaN

3. inplace parameter

This deletes the given column from the returned DataFrame. A new column containing cleaned VAT strings is added with a title in the format "{original title}_clean".

clean_bg_vat(df, column="vat", inplace=True)

	vat_clean	address
0	175074752	123 Pine Ave.
1	175074752	main st
2	NaN	1234 west main heights 57033
3	NaN	apt 1 789 s maple rd manhattan
4	NaN	robie house, 789 north main street
5	NaN	1111 S Figueroa St, Los Angeles, CA 90015
6	NaN	(staples center) 1111 S Figueroa St, Los Angeles
7	NaN	hello

4. errors parameter

coerce (default)

clean_bg_vat(df, "vat", errors="coerce")

	vat	address	vat_clean
0	BG 175 074 752	123 Pine Ave.	175074752
1	175074752	main st	175074752
2	175074751	1234 west main heights 57033	NaN
3	BE431150351	apt 1 789 s maple rd manhattan	NaN
4	002 724 334	robie house, 789 north main street	NaN
5	hello	1111 S Figueroa St, Los Angeles, CA 90015	NaN
6	NaN	(staples center) 1111 S Figueroa St, Los Angeles	NaN
7	NULL	hello	NaN

ignore

clean_bg_vat(df, "vat", errors="ignore")

	vat	address	vat_clean
0	BG 175 074 752	123 Pine Ave.	175074752
1	175074752	main st	175074752
2	175074751	1234 west main heights 57033	175074751
3	BE431150351	apt 1 789 s maple rd manhattan	BE431150351
4	002 724 334	robie house, 789 north main street	002 724 334
5	hello	1111 S Figueroa St, Los Angeles, CA 90015	hello
6	NaN	(staples center) 1111 S Figueroa St, Los Angeles	NaN
7	NULL	hello	NaN

4. validate_bg_vat()

validate_bg_vat() returns True when the input is a valid VAT. Otherwise it returns False.

The input of validate_bg_vat() can be a string, a Pandas DataSeries, a Dask DataSeries, a Pandas DataFrame and a dask DataFrame.

When the input is a string, a Pandas DataSeries or a Dask DataSeries, user doesn’t need to specify a column name to be validated.

When the input is a Pandas DataFrame or a dask DataFrame, user can both specify or not specify a column name to be validated. If user specify the column name, validate_bg_vat() only returns the validation result for the specified column. If user doesn’t specify the column name, validate_bg_vat() returns the validation result for the whole DataFrame.

from dataprep.clean import validate_bg_vat
print(validate_bg_vat("BG 175 074 752"))
print(validate_bg_vat("175074752"))
print(validate_bg_vat('175074751'))
print(validate_bg_vat('BE431150351'))
print(validate_bg_vat("004085616"))
print(validate_bg_vat("hello"))
print(validate_bg_vat(np.nan))
print(validate_bg_vat("NULL"))

True
True
False
False
False
False
False
False

Series

validate_bg_vat(df["vat"])

0     True
1     True
2    False
3    False
4    False
5    False
6    False
7    False
Name: vat, dtype: bool

DataFrame + Specify Column

validate_bg_vat(df, column="vat")

0     True
1     True
2    False
3    False
4    False
5    False
6    False
7    False
Name: vat, dtype: bool

Only DataFrame

validate_bg_vat(df)

	vat	address
0	True	False
1	True	False
2	False	False
3	False	False
4	False	False
5	False	False
6	False	False
7	False	False