Connector supports the most used authorization methods in Web APIs:
API Key
Bearer Token
Query parameter
Request header parameter
OAuth 2.0 “Client Credentials” and “Authorization Code” grants.
Client Credentials grant
Authorization Code grant
Let’s review them in detail:
Depending on the Web API specification, you must send your API Key as a bearer token, a parameter in the URL (query parameter), or a parameter in the request header. Let’s review these cases in detail:
If the Web API supports the Bearer Token authorization type, follow the next steps:
You have to specify in your configuration file that the desired authentication method to be used is “Bearer token”. The following property should be included below the “method” property in your configuration file:
"authorization": { "type": "Bearer" },
For example, you can review our Yelp configuration file for the “businesses” endpoint, which implements the “Bearer token” authorization method.
Use the connect function with the “yelp” string and your Yelp access token, both specified as parameters. This action allows you to create a Connector to the Yelp Web API. Next, through the query function, you can retrieve data from this endpoint. The parameter “businesses” indicates you want to query the Yelp “businesses” endpoint with the search term “sushi” and location “Vancouver”:
import asyncio from dataprep.connector import connect yelp_connector = connect("yelp", _auth={"access_token":"<Your Yelp access token>"}) df = await yelp_connector.query("businesses", term="sushi", location="Vancouver")
You have to specify in your configuration file that you will send your API Key as a parameter in the request URL (“type”: “QueryParam”). The following property should be included below the method property in your configuration file:
"authorization": { "type": "QueryParam", "keyParam": "<API Key parameter name>" },
You must replace the string with the parameter’s exact name to be used to send the API Key to the remote endpoint. This parameter name is defined by the Web API. Review the Web API documentation to identify the exact name.
For example, the Finnhub API - IPO Calendar endpoint, names this parameter as token:
"authorization": { "type": "QueryParam", "keyParam": "token" },
To query information from this Finnhub endpoint, you can use both connect and query functions:
import asyncio from dataprep.connector import connect finnhub_connector = connect("finnhub", _auth={"access_token":"<Your Finnhub API Key>"}) df = await finnhub_connector.query("ipo_calendar",from_="2020-01-01", to="2020-04-30")
Internally, Connector will take your API Key and send it as the token parameter’s value in the request URL.
You have to specify in your configuration file that you will send your API Key as a parameter in the request header (“type”: “Header”). The following property should be included below the method property in your configuration file:
"authorization": { "type": "Header", "keyName": "<API Key parameter name>" },
For example, the Twitch.tv API - channels endpoint, names this parameter as Client-ID:
"authorization": { "type": "Header", "keyName": "Client-ID", ... },
To query information from this Twitch endpoint, you can use both connect and query functions:
from dataprep.connector import connect twitch_connector = connect("twitch", _auth={"access_token":"<Your Twitch API Key>"}) df = await twitch_connector.query("channels",q="a_seagull")
Internally, Connector will take your API Key and send it as the Client-ID parameter’s value in the request header.
Connector supports the authorization scheme OAuth 2.0 - “Client Credentials” and “Authorization Code” grants. Let’s review the details:
In your configuration file, you have to specify that you’ll use the OAuth 2.0 authorization type - Client Credentials grant. The following property should be included below the method property in your configuration file:
"authorization": { "type": "OAuth2", "grantType": "ClientCredentials", "tokenServerUrl": "<OAuth 2.0 token URL>" },
You must replace the <OAuth 2.0 token URL” string with the OAuth 2.0 token URL defined by the Web API. Review the Web API documentation to identify the exact URL.
For example, see the Twitter API - Tweets endpoint, configuration below:
"authorization": { "type": "OAuth2", "grantType": "ClientCredentials", "tokenServerUrl": "https://api.twitter.com/oauth2/token" },
Before executing any query, you must get your Web API Client ID and Client Secret information. Once obtained, you can pass these values as parameters in the connect function and then execute the query method to retrieve the desired data:
import asyncio from dataprep.connector import connect twitter_connector = connect("twitter", _auth={"client_id":twitter_client_id, "client_secret":twitter_client_secret}) df = await twitter_connector.query("tweets", q="data science")
Internally, Connector will take both Client ID and Client Secret values and execute the OAuth 2.0 - Client Credentials grant process on your behalf.
In your configuration file, you have to specify that you’ll use the OAuth 2.0 authorization type - Authorization Code grant. The following property should be included below the method property in your configuration file:
"authorization": { "type": "OAuth2", "grantType": "AuthorizationCode", "tokenServerUrl": "<OAuth 2.0 token URL>" },
For example, see the Twitter API - Tweets endpoint - configuration below:
"authorization": { "type": "OAuth2", "grantType": "AuthorizationCode", "tokenServerUrl": "https://api.twitter.com/oauth2/token" },
Internally, Connector will take both Client ID and Client Secret values and execute the OAuth 2.0 - Authorization Code grant process on your behalf.