Getting Technical #1: IT Updates Across the Global LEI System & Golden Copy File Download API
In this new blog series, Christoph Schneider, Head of IT Development and Operations at GLEIF, explains the latest technical updates happening across GLEIF’s global IT infrastructure as the IT team strives to make LEI data more readily available for its users. This first installment provides an overview of the latest updates to the Golden Copy file download API and its importance for achieving scalability.
Author: Christoph Schneider
Date: 2022-09-30
Views:
At the heart of GLEIF’s operation is a commitment to providing transparent and accessible legal entity identification and data services. Key to this is a robust, global IT infrastructure that ensures all data consumers in the Global LEI System can easily integrate, access, and utilize LEI data across multiple platforms and systems.
GLEIF’s IT department works tirelessly to ensure that its global infrastructure offers the best user experience possible, constantly updating and refining as the Global LEI System grows. The latest update in this process regards the Global Copy File Download API, which will be updated as of 10 October. The Golden Copy Files, and delta files, are updated by GLEIF three times per day and ensure easy access to the latest information on new and updated LEIs. The connecting Download-API is critical for many development teams who rely on automatic data updates to provide accurate, timely data to their organization. The latest change to the Golden Copy Files Download-API will help users to achieve better scalability and reliability while overcoming performance bottlenecks related to file downloads being served directly by the API.
Who will this update affect?
This update will result in a redirect to the direct file path instead of a direct download from the API endpoint. Teams using Golden Copy Files Download-API URLs that result in a direct file download (e.g., https://goldencopy.gleif.org/api/v2/golden-copies/publishes/lei2/20220601-0000.csv) will need to update configurations in line with this change. If you are using the API in this manner, please flag this update to your technical team to ensure that the redirection would work as expected.
Please note that the below domains are affected due to this upcoming change:
The Golden Copy Files Download-API provides several convenient API endpoints for downloading the Golden Copy (delta) files directly based on a publish date pattern or the special “latest” keyword. For further information, please refer to the GLEIF Golden Copy and Delta Files Specification and User Manual, a link can be found at the end of this article.
Currently, invocation of a download API endpoint will result in direct file download (200 HTTP response code). A curl-based example is shown below:
Curl https://goldencopy.gleif.org/api/v2/golden-copies/publishes/lei2/20220601-0000.csv
# HTTP/1.1 200 OK
And after the suggested date (2022-10-10), all API requests will be redirected to the direct file download path (302 HTTP response code with a Location header). A curl-based example is shown below:
curl -L https://goldencopy.gleif.org/api/v2/golden-copies/publishes/lei2/20220601-0000.csv
#HTTP/1.1 302 Found
# Location: https://goldencopy.gleif.org/storage/golden-copy-files/2022/06/01/6/20220601-0000-gleif-goldencopy-lei2-golden-copy.csv.zip
# ...
# HTTP/1.1 200 OK
In the above curl-based example, the only required change is the addition of the location flag (-L or --location) to ensure that the new redirected location is followed. Such behavior may be default or may need to be configured or handled differently based on the HTTP client used. All data consumers should ensure their respective HTTP clients/applications properly follow and handle the redirects, as shown above.
How can relevant teams configure the update?
Prepare and test your system before the change is rolled out. The HTTP 302 redirect status code will be introduced, and the existing systems must be updated to support the new behavior in a forward-compatible way through any of the following methods:
(Recommended) Ensure your HTTP client/application follows and handles redirects properly. This behavior may be default or may need to be configured or handled differently based on the HTTP client used.
Manually update your system to additionally understand the HTTP 302 responses and inspect the Location header to follow the direct file download URL.
Is there a temporary test environment available?
GLEIF has provided a temporary test environment for data consumers to check if their systems will work with the new behavior and to ensure that redirects to the direct file download are followed. This will ease the transition to the new redirect behavior.
Please find below the temporary test environment examples that showcase the new redirect behavior:
Please note that this test environment only supports the publish date 19990101-0000, but the change will apply to all patterns, including the special “latest” keyword (e.g., …/publishes/lei2/latest.csv). Furthermore, the test environment is an example snapshot of a subset of the Golden Copy data for demonstration purposes only. It is not intended for actual use, and the environment will be decommissioned after the new Golden Copy API behavior is rolled out.
GLEIF encourages engagement with data consumers across the Global LEI System. If you have a question regarding the latest update or would like to discuss ways in which GLEIF can support your team harness the power of the LEI, please contact info@gleif.org.
To keep up to date with all the latest technical developments from GLEIF’s IT team, click here to subscribe to the technical notifications newsletter.
Key terms:
API - API stands for Application Programming Interface. An API is a set of definitions and protocols that allow computer programs to communicate with each other via the world wide web.
Endpoint - An endpoint is a location receiving web requests. An API is a collection of endpoints.
API Request - An API call, or an API request, is a message sent to an endpoint (the server) asking an API to provide a service or information.
API Response - The response data received from an API to provide a service or information.
HTTP response status codes - This is a list of Hypertext Transfer Protocol (HTTP) response status codes. Status codes are issued by a server in response to an API request made to an endpoint, the server. Each status code has a specific meaning.
Golden Copy Files - Files including LEIs and related reference data uploaded by the LEI issuers to GLEIF.
Delta Files - The delta files identify only newly issued LEIs and/or revisions to an LEI’s reference data historically reported in a Golden Copy File published earlier (eight hours earlier, 24 hours earlier, seven days earlier, and the largest being 31 days earlier).
GLEIF Services - A set of services provided to both the public and our partners to ensure operational integrity across the Global LEI System.
If you would like to comment on a blog post, please identify yourself with your first and last name. Your name will appear next to your comment. Email addresses will not be published. Please note that by accessing or contributing to the discussion board you agree to abide by the terms of the GLEIF Blogging Policy, so please read them carefully.
Christoph Schneider is Head of IT Development and Operations at the Global Legal Entity Identifier Foundation (GLEIF). In June 2017, Christoph joined the International Organization for Standardization (ISO) as co-leader of the Technical Committee 68 FinTech Technical Advisory Group (ISO TC 68 FinTech TAG) workstream dealing with digital identity. He has extensive experience in developing and implementing solutions in financial technology. Christoph holds a Master of Science degree in Business Information Systems from Technische Universität Darmstadt.
