By the same authors

Restmule: Enabling resilient clients for remote APIs

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Full text download(s)

Published copy (DOI)

Author(s)

Department/unit(s)

Publication details

Title of host publicationProceedings - 2018 ACM/IEEE 15th International Conference on Mining Software Repositories, MSR 2018
DatePublished - 28 May 2018
Pages537-541
Number of pages5
PublisherIEEE Computer Society Press
Original languageEnglish
ISBN (Print)9781450357166

Abstract

Mining data from remote repositories, such as GitHub and StackExchange, involves the execution of requests that can easily reach the limitations imposed by the respective APIs to shield their services from overload and abuse. Therefore, data mining clients are left alone to deal with such protective service policies which usually involves an extensive amount of manual implementation effort. In this work we present RestMule, a framework for handling various service policies, such as limited number of requests within a period of time and multi-page responses, by generating resilient clients that are able to handle request rate limits, network failures, response caching, and paging in a graceful and transparent manner. As a result, RestMule clients generated from OpenAPI specifications (i.e. standardized REST API descriptors), are suitable for intensive data-fetching scenarios. We evaluate our framework by reproducing an existing repository mining use case and comparing the results produced by employing a popular hand-written client and a RestMule client.

Bibliographical note

© 2018 Copyright held by the owner/author(s). Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details

    Research areas

  • HTTP API clients, openAPI specification, resilience

Discover related content

Find related publications, people, projects, datasets and more using interactive charts.

View graph of relations