API Guidance

Introduction

This page provides guidance on how to code the technical stuff so that you can access the information without having to go to the site. This is a brief overview to APIs with some examples of SQL querying and python scripting.

Probably worth noting, to fully utilise the API, some general programming skills and a basic idea of SQL are helpful but not essential.

Example API calls

The base URL for the calls is: https://connecteddata.westernpower.co.uk/api/3/action/

Here are some examples of using the API to get data from the portal:

End Point Example Description
group_list https://connecteddata.westernpower.co.uk/api/3/action/group_list Displays list of Data Groups
package_list https://connecteddata.westernpower.co.uk/api/3/action/package_list Displays list of datasets
tag_list https://connecteddata.westernpower.co.uk/api/3/action/tag_list Displays a list of tags
package_show?id={dataset_id} https://connecteddata.westernpower.co.uk/api/3/action/package_show?id=55621879-bd56-48d8-8179-36daa38ede99 Displays datasets matching a criteria
tag_show?id={tag_name} https://connecteddata.westernpower.co.uk/api/3/action/tag_show?id=Impedance Displays tags matching a criteria
group_show?id={group_name} https://connecteddata.westernpower.co.uk/api/3/action/group_show?id=demand Displays groups matching a criteria
resource_show?id={resource_id} https://connecteddata.westernpower.co.uk/api/3/action/resource_show?id=7f4e46db-0dcb-449e-8842-f5f585370e4a Displays resources matching provided id
package_search?q={query} https://connecteddata.westernpower.co.uk/api/3/action/package_search?q=West Displays datasets matching provided id
resource_search?query={name:{query}} https://connecteddata.westernpower.co.uk/api/3/action/resource_search?query=name:West Midlands Displays resources that the name contains the query provided
datastore_search?resource_id={resource_id}&limit=10 https://connecteddata.westernpower.co.uk/api/3/action/datastore_search?resource_id=7f4e46db-0dcb-449e-8842-f5f585370e4a&limit=10 Displays search data in a tabular file

The above just scratches the surface. There are hundreds of calls for getting data from the portal. These are documented here docs.ckan.org/en/latest/api/index.html. Scrolling through the API documentation will give you a better idea of how to extract what you need.

SQL Querying

You can also query the data via SQL. For example if only wanted column ‘PRIMARY’ from the ‘Embedded Capacity Register’. The cURL would look something like this:

However, the call requires the resource_id rather than the name. Not a problem though, you can find the resource_id by clicking on the data set and scraping the last section of the cURL, like below.

Alternatively, there is an API call to get the resource_ids for a given package.

Python 3.8 Example

This example gets all the packages, finds the embedded capacity register package and pulls the names and ids of the resources.

Simple Python Example using WPD CKAN API

#import the necessary libraries
import urllib3
import json
import pprint

#Allows for arbitrary requests while transparently keeping track of necessary connection pools for you.
http = urllib3.PoolManager()

#Get the data
response = http.request('GET','https://connecteddata.westernpower.co.uk/api/3/action/package_list')

# Use the json module to load CKAN's response into a dictionary.
response_dict = json.loads(response.data)

# Get the list of packages
print(response_dict['result'])
['constraint-management-zone-cmz-polygons', 'constraint-management-zone-cmz-postcode', 'distribution-substation-location-easting-northings', 'electric-vehicle-capacity-map', 'embedded-capacity-register', 'flexdgrid-fault-level-monitor-data', 'generation-capacity-register', 'green-recovery-map', 'live-data', 'primary-substation-location-easting-northings', 'time-to-connect', 'time-to-quote', 'transformer-detail-for-the-south-west-licence-area', 'western-power-distribution-south-west-grid-supply-point-demand', 'wpd-network-capacity-map']

# Get the list if resources for each package

for package in response_dict['result']:
    #loops until it finds the embedded capacity register
    if package == 'embedded-capacity-register':
        #gets the resources
        response = http.request('GET','https://connecteddata.westernpower.co.uk/api/3/action/package_show',{'id': package})
        #converts to json
        response_dict = json.loads(response.data)
        #loop through the response to get the resource id and name
        for resource in (response_dict['result']['resources']):
            print('resource_name=',resource['name'],'\nresource_id=',resource['id'])
resource_name= Embedded Capacity Register - December 2020 
resource_id= 7f4e46db-0dcb-449e-8842-f5f585370e4a
resource_name= Embedded Capacity Register - February 2021 
resource_id= 50609de1-9484-41e5-82e6-dfa5b2287759
resource_name= Embedded Capacity Register January 2021 
resource_id= d4270bb9-59e4-403d-ad8f-1c382c10963f                    
                

cURLs

"Integrate this dataset using cURL" section at the bottom of dataset pages.

You can get the metadata and a list of resources of a dataset by the following

Datapackage

Dataset id

Dataset Id

Reource id

Resource Id

You can download the resource file by

Resources