data.qld.gov.au API customisation
Last updated: 18th January 2022
This guide contains information on the data.qld.gov.au API customisations. This guide should be used in conjunction with the official CKAN API user guide.
Contents
Creating a Dataset with Resources
Creating a Dataset with Resources
Dataset fields:
| Field | Open Data Portal Field | Mandatory | Type | Example Value | Notes |
|---|---|---|---|---|---|
| ‘title’ | Title | Yes | String | ‘Example Dataset’ | This is the title of the dataset. |
| ‘name’ | URL | Yes | String | ‘example-dataset’ | This value will form the ending of the URL. Spaces should be replaced with hyphens, e.g. www.data.qld.gov.au/dataset/example-dataset |
| ‘notes’ | Description | Yes | String | ‘Any text you want to describe the dataset.’ | Free text field which will be the description of the dataset. |
| ‘tags’ | Tags | No | List of Dictionaries | [{'name': 'tagone'}, {‘name’: ‘tagtwo’}] | One or more tags can be added by specifying the name/s of the tag/s within a dictionary. You can create your own tag/s or choose an existing tag from the list here https://www.data.qld.gov.au/api/action/tag_list?all_fields=TRUE |
| ‘license_id’ | License | Yes | String | 'cc-by-4' | A list of available licences and the respective IDs are available here. https://www.data.qld.gov.au/api/action/license_list |
| ‘owner_org’ | Organisation | Yes | String | ‘example-organisation’ | One organisation needs to be specified by ID or NAME. Only one can be specified, and the publisher must have access to publish under this organisation. A list of organisations and their IDs and NAMEs is here https://www.data.qld.gov.au/api/action/organization_list?all_fields=true |
| ‘private’ | Visibility | Yes | Boolean | ‘TRUE’ or ‘FALSE’ | If TRUE is selected, this dataset will be published as a private dataset. If false is selected, it will be public. |
| ‘url’ | Source | No | String | ‘www.exampledata.com.au’ | A source URL for the dataset. |
| ‘version’ | Version | Yes | String | ‘1.0’ | A version number for the dataset. |
| ‘author_email’ | Yes | String | ‘example@example.qld.gov.au’ | The email address of the individual or team responsible for this data. | |
| ‘security_classification’ | Security Classification | Yes | String | 'PUBLIC' or ‘UNCLASSIFIED’ | The Queensland Government security classification of the data. |
| 'data_driven_application' | Used in data driven application | Yes | Boolean | ‘YES’ or ‘NO’ | Records whether applications or visualisations are known to depend upon this dataset. Selecting YES prevents the deletion of the dataset. Selecting NO enables deletion. |
| 'update_frequency' | Update Frequency | Yes | String | ‘non-regular’ ‘quarterly’ ‘monthly’ ‘annually’ ‘near-realtime’ ‘not-updated’ ‘weekly’ ‘half-yearly’ ‘daily’ ‘hourly’ ‘fortnightly’ | The update frequency of the dataset. |
| 'next_update_due' | Next update due | No | Date | ‘YYYY-MM-DD’ | Only used by the system for datasets with an update frequency of annually, quarterly, monthly or half-yearly. Where this value is not supplied, it is automatically calculated based on the update_frequency value. |
| 'de_identified_data' | Contains de-identified data | Yes | Boolean | ‘YES’ or ‘NO’ | Records whether the dataset includes data which is derived from data which features personally identifiable information. |
| ‘groups’ | Groups | No | List of Dictionaries | [{'id': 'group_id'}, {'name': 'group_name'}] | The dataset can be added to one or more groups by ID or NAME. These values are here. https://www.data.qld.gov.au/api/action/group_list?all_fields=True |
| ‘resources’ | Resources | No | List of Dictionaries | [{‘name’:’xxxx’…}] | Refer to resource information below for resource metadata fields and values. |
Resource fields:
| Field | Open Data Portal Field | Mandatory | Type | Example Value | Notes |
|---|---|---|---|---|---|
‘url’ | URL | String | Conditional | Only required for resources where resource is a linked URL. For uploaded resources please see here: https://docs.ckan.org/en/latest/maintaining/filestore.html | |
‘name’ | Name | Yes | String | ‘Example Resource’ | The name of the resource. |
‘format’ | Format | Yes | String | 'csv' | The file type of resource you are adding. A list of acceptable file formats is available here. |
‘description’ | Description | Yes | String | ‘Any text you want to describe the resource.’ | Free text field which will be the description of the resource. |
‘size’ | Size | Conditional | Integer | ‘1000’ | A whole number value indicating the size of the resource in bytes. This is mandatory for linked resources but not required for uploaded resources. |
‘schema’ | Data Schema | No | List of Dictionaries | {"fields": [{"format": "%d/%m/%Y", "type": "date", "name": "example_name", "title": "Example Name"}] | A table schema can be entered for the data. Refer to https://github.com/frictionlessdata/ckanext-validation#data-schema for more information. |
‘validation_options’ | Validation Options | No | List of Dictionaries | {"skip_rows": ["4"]} | Validation options can be entered to validate data that may not match the schema. Refer to https://github.com/frictionlessdata/ckanext-validation#validation-options for more information. |
'resource_visibility' | Re-identification risk governance acknowledgement/Resource visibility | YES | String | ‘Resource visible and re-identification risk governance acknowledgement not required’ or ‘Resource NOT visible/Pending acknowledgement or ‘Appropriate steps have been taken to minimise personal information re-identification risk prior to publishing’ | An acknowledgement is required based on whether the dataset contains de-identified data. Setting the resource to NOT visible will hide the resource. |
If updating a dataset then the dataset name or ID are required in the dataset dictionary:
| Field | Mandatory | Type | Example Value | Notes |
|---|---|---|---|---|
‘id’ | Yes | String | ‘ID’ or ‘example_dataset’ | You can view a dataset's ID or details by appending the name to the end of https://www.data.qld.gov.au/api/action/package_show?id= e.g. https://www.data.qld.gov.au/api/action/package_show?id=example-dataset |
When adding a resource to an existing dataset, the package_id field is required in the dictionary:
| Field | Mandatory | Type | Example Value | Notes |
|---|---|---|---|---|
‘package_id’ | Yes | String | ‘package_id’ | The package ID is the “id” field within a dataset. You can view a dataset's ID or details by appending the name to the end of https://www.data.qld.gov.au/api/action/package_show?id= e.g. https://www.data.qld.gov.au/api/action/package_show?id=example-dataset |
When updating a resource, the resource id is required within the resource dictionary:
| Field | Mandatory | Type | Example Value | Notes |
|---|---|---|---|---|
‘id’ | Yes | String | [{‘id’: ‘id’…]} | The resource ID is identified by the last part of the resource page URL e.g. https://www.data.qld.gov.au/dataset/example-dataset/resource/ (resource ID here) You can view a resource ID or details by appending the dataset name to the end of https://www.data.qld.gov.au/api/action/package_show?id= e.g. https://www.data.qld.gov.au/api/action/package_show?id=example-dataset |
ckan.logic.action.get
ckan.logic.action.get.package_search
From December 2021, a user with Member, Editor, Admin or Sysadmin access can provide their API token as authentication to access additional field values where applicable.
Additional field values include:
- data_last_updated
- next_update_due
- de_identified_data
- nature_of_change
- resource_visibility
Please refer to the open data portal publishing guide for more information about these fields.
ckan.logic.action.get.package_show
From December 2021, a user with Member, Editor, Admin or Sysadmin access can provide their API token as authentication to access additional field values where applicable.
Additional field values include:
- data_last_updated
- next_update_due
- de_identified_data
- nature_of_change
- resource_visibility
Please refer to the open data portal publishing guide for more information about these fields.
ckan.logic.action.create
ckan.logic.action.create.package_create
From December 2021, when creating a dataset by API, the following additional values must be included in the dictionary:
- de_identified_data - "YES" or "NO" is required.
- next_update_due - This field value enables automated checks of data currency for time-series data for datasets with update frequencies of Monthly, Quarterly, Half-yearly or Annually. It is NOT mandatory. For relevant datasets, the value will be automatically calculated if not included. A user can manually specify a value in the format "YYYY-MM-DD".
Example:
dictionary = {'de_identified_data': 'YES', 'next_update_due': '2026-02-17'}
Please refer to the open data portal publishing guide for more information about these fields.
If creating resources in the same request, additional fields will need to be entered for each resource. Please see resource_create below for information on these.
ckan.logic.action.create.resource_create
From December 2021, when creating a resource by API,the following additional values must be included in the dictionary:
- resource_visibility -
- If the dataset de-identified_data field has been set to "NO", the value for resource_visibility must be set to either "Resource visible and re-identification risk governance acknowledgement not required" or "Resource NOT visible/Pending acknowledgement"
- If the dataset de-identified_data field has been set to "YES", the value for resource_visibility must be set to either “Appropriate steps have been taken to minimise personal information re-identification risk prior to publishing” or "Resource NOT visible/Pending acknowledgement"
Example:
dictionary = {'resource_visibility': 'Appropriate steps have been taken to minimise personal information re-identification risk prior to publishing'}
Please refer to the open data portal publishing guide for more information about these fields.
ckan.logic.action.update
ckan.logic.action.update.package_update
From December 2021, when updating a dataset by API,the following additional values must be included in the dictionary:
- de_identified_data - "YES" or "NO" is required.
- next_update_due - This field value enables automated checks of data currency for time-series data for datasets with update frequencies of Monthly, Quarterly, Half-yearly or Annually. It is NOT mandatory. For relevant datasets, the value will be automatically calculated if not included. A user can manually specify a value in the format "YYYY-MM-DD".
Example:
dictionary = {'de_identified_data': 'YES', 'next_update_due': '2026-02-17'}
Please refer to the open data portal publishing guide for more information about these fields.
This request type overwrites resources of the same "name" but not for "id". Different field values are required for resource creation and updating. Please see resource_create above or resource_update below for relevant information on each of these.
ckan.logic.action.update.resource_update
From December 2021, when updating a resource by API, the following additional values must be included in the dictionary:
- nature_of_change - Either "edit-resource-with-no-new-data" or “add-new-time-series"
- resource_visibility -
- If the dataset de-identified_data field has been set to "NO", the value for resource_visibility must be set to either "Resource visible and re-identification risk governance acknowledgement not required" or "Resource NOT visible/Pending acknowledgement"
- If the dataset de-identified_data field has been set to "YES" the value for resource_visibility must be set to either “Appropriate steps have been taken to minimise personal information re-identification risk prior to publishing” or "Resource NOT visible/Pending acknowledgement"
Example:
dictionary = {'nature_of_change': 'add-new-time-series', 'resource_visibility': 'Appropriate steps have been taken to minimise personal information re-identification risk prior to publishing'}
Please refer to the open data portal publishing guide for more information about these fields.
ckan.logic.action.patch
ckan.logic.action.patch.package_patch
From December 2021, the package_patch API functionality has not changed, however, the following additional metadata fields can now be updated if necessary:
- de_identified_data - "YES" or "NO"
- next_update_due - This field value enables automated checks of data currency for time-series data for datasets with update frequencies of Monthly, Quarterly, Half-yearly or Annually. It is NOT mandatory. For relevant datasets, the value will be automatically calculated if not included. A user can manually specify a value in the format "YYYY-MM-DD".
Example:
dictionary = {'de_identified_data': 'YES', 'next_update_due': '2026-02-17'}
ckan.logic.action.patch.resource_patch
From December 2021, when updating a resource by API, the following additional values must be included in the dictionary:
- nature_of_change - Either "edit-resource-with-no-new-data" or “add-new-time-series"
- resource_visibility -
- If the dataset de-identified_data field has been set to "NO" the value for resource_visibility must be set to either "Resource visible and re-identification risk governance acknowledgement not required" or "Resource NOT visible/Pending acknowledgement"
- If the dataset de-identified_data field has been set to "YES" the value for resource_visibility must be set to either “Appropriate steps have been taken to minimise personal information re-identification risk prior to publishing” or "Resource NOT visible/Pending acknowledgement"
Example:
dictionary = {'nature_of_change': 'add-new-time-series', 'resource_visibility': 'Appropriate steps have been taken to minimise personal information re-identification risk prior to publishing'}
The nature_of_change and resource_visibility fields require a value every time a user updates a resource. Because of this, these fields are also mandatory when updating a resource by resource_patch.
Please refer to the open data portal publishing guide for more information about these fields.
ckan.logic.action.delete
ckan.logic.action.delete.package_delete
From December 2021, when deleting a dataset, it is now mandatory for a user to specify a reason for the deletion and this must be added to the API call.
- deletion_reason - Enter a brief reason or reasons for deleting this dataset. The entered text must be between 10 and 250 characters.
Example:
dictionary = {'deletion_reason': 'This is being deleted example text'}
Please refer to the open data portal publishing guide for more information about these fields.