Restmule: Enabling resilient clients for remote APIs

Beatriz A. Sanchez; Konstantinos Barmpis; Patrick Neubauer; Richard F. Paige; Dimitrios S. Kolovos

doi:10.1145/3196398.3196405

Restmule: Enabling resilient clients for remote APIs

Beatriz A. Sanchez, Konstantinos Barmpis, Patrick Neubauer, Richard F. Paige, Dimitrios S. Kolovos

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Mining data from remote repositories, such as GitHub and StackExchange, involves the execution of requests that can easily reach the limitations imposed by the respective APIs to shield their services from overload and abuse. Therefore, data mining clients are left alone to deal with such protective service policies which usually involves an extensive amount of manual implementation effort. In this work we present RestMule, a framework for handling various service policies, such as limited number of requests within a period of time and multi-page responses, by generating resilient clients that are able to handle request rate limits, network failures, response caching, and paging in a graceful and transparent manner. As a result, RestMule clients generated from OpenAPI specifications (i.e. standardized REST API descriptors), are suitable for intensive data-fetching scenarios. We evaluate our framework by reproducing an existing repository mining use case and comparing the results produced by employing a popular hand-written client and a RestMule client.

Original language	English
Title of host publication	Proceedings - 2018 ACM/IEEE 15th International Conference on Mining Software Repositories, MSR 2018
Publisher	IEEE Computer Society
Pages	537-541
Number of pages	5
ISBN (Print)	9781450357166
DOIs	https://doi.org/10.1145/3196398.3196405
Publication status	Published - 28 May 2018
Event	15th ACM/IEEE International Conference on Mining Software Repositories, MSR 2018, co-located with the 40th International Conference on Software Engineering, ICSE 2018 - Gothenburg, Sweden Duration: 28 May 2018 → 29 May 2018

Conference

Conference	15th ACM/IEEE International Conference on Mining Software Repositories, MSR 2018, co-located with the 40th International Conference on Software Engineering, ICSE 2018
Country/Territory	Sweden
City	Gothenburg
Period	28/05/18 → 29/05/18

Bibliographical note

Keywords

HTTP API clients
openAPI specification
resilience

Access to Document

10.1145/3196398.3196405

p537-sanchez
© 2018 Copyright held by the owner/author(s). Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details
Final published version, 716 KB

Cite this

@inproceedings{dcea4b8645d94146b3d10e6f87abd064,

title = "Restmule: Enabling resilient clients for remote APIs",

abstract = "Mining data from remote repositories, such as GitHub and StackExchange, involves the execution of requests that can easily reach the limitations imposed by the respective APIs to shield their services from overload and abuse. Therefore, data mining clients are left alone to deal with such protective service policies which usually involves an extensive amount of manual implementation effort. In this work we present RestMule, a framework for handling various service policies, such as limited number of requests within a period of time and multi-page responses, by generating resilient clients that are able to handle request rate limits, network failures, response caching, and paging in a graceful and transparent manner. As a result, RestMule clients generated from OpenAPI specifications (i.e. standardized REST API descriptors), are suitable for intensive data-fetching scenarios. We evaluate our framework by reproducing an existing repository mining use case and comparing the results produced by employing a popular hand-written client and a RestMule client.",

keywords = "HTTP API clients, openAPI specification, resilience",

author = "Sanchez, {Beatriz A.} and Konstantinos Barmpis and Patrick Neubauer and Paige, {Richard F.} and Kolovos, {Dimitrios S.}",

note = "{\textcopyright} 2018 Copyright held by the owner/author(s). Uploaded in accordance with the publisher{\textquoteright}s self-archiving policy. Further copying may not be permitted; contact the publisher for details; 15th ACM/IEEE International Conference on Mining Software Repositories, MSR 2018, co-located with the 40th International Conference on Software Engineering, ICSE 2018 ; Conference date: 28-05-2018 Through 29-05-2018",

year = "2018",

month = may,

day = "28",

doi = "10.1145/3196398.3196405",

language = "English",

isbn = "9781450357166",

pages = "537--541",

booktitle = "Proceedings - 2018 ACM/IEEE 15th International Conference on Mining Software Repositories, MSR 2018",

publisher = "IEEE Computer Society",

address = "United States",

}

Sanchez, BA, Barmpis, K , Neubauer, P , Paige, RF & Kolovos, DS 2018, Restmule: Enabling resilient clients for remote APIs. in Proceedings - 2018 ACM/IEEE 15th International Conference on Mining Software Repositories, MSR 2018. IEEE Computer Society, pp. 537-541, 15th ACM/IEEE International Conference on Mining Software Repositories, MSR 2018, co-located with the 40th International Conference on Software Engineering, ICSE 2018, Gothenburg, Sweden, 28/05/18. https://doi.org/10.1145/3196398.3196405

TY - GEN

T1 - Restmule

T2 - 15th ACM/IEEE International Conference on Mining Software Repositories, MSR 2018, co-located with the 40th International Conference on Software Engineering, ICSE 2018

AU - Sanchez, Beatriz A.

AU - Barmpis, Konstantinos

AU - Neubauer, Patrick

AU - Paige, Richard F.

AU - Kolovos, Dimitrios S.

PY - 2018/5/28

Y1 - 2018/5/28

N2 - Mining data from remote repositories, such as GitHub and StackExchange, involves the execution of requests that can easily reach the limitations imposed by the respective APIs to shield their services from overload and abuse. Therefore, data mining clients are left alone to deal with such protective service policies which usually involves an extensive amount of manual implementation effort. In this work we present RestMule, a framework for handling various service policies, such as limited number of requests within a period of time and multi-page responses, by generating resilient clients that are able to handle request rate limits, network failures, response caching, and paging in a graceful and transparent manner. As a result, RestMule clients generated from OpenAPI specifications (i.e. standardized REST API descriptors), are suitable for intensive data-fetching scenarios. We evaluate our framework by reproducing an existing repository mining use case and comparing the results produced by employing a popular hand-written client and a RestMule client.

AB - Mining data from remote repositories, such as GitHub and StackExchange, involves the execution of requests that can easily reach the limitations imposed by the respective APIs to shield their services from overload and abuse. Therefore, data mining clients are left alone to deal with such protective service policies which usually involves an extensive amount of manual implementation effort. In this work we present RestMule, a framework for handling various service policies, such as limited number of requests within a period of time and multi-page responses, by generating resilient clients that are able to handle request rate limits, network failures, response caching, and paging in a graceful and transparent manner. As a result, RestMule clients generated from OpenAPI specifications (i.e. standardized REST API descriptors), are suitable for intensive data-fetching scenarios. We evaluate our framework by reproducing an existing repository mining use case and comparing the results produced by employing a popular hand-written client and a RestMule client.

KW - HTTP API clients

KW - openAPI specification

KW - resilience

UR - http://www.scopus.com/inward/record.url?scp=85051676109&partnerID=8YFLogxK

U2 - 10.1145/3196398.3196405

DO - 10.1145/3196398.3196405

M3 - Conference contribution

AN - SCOPUS:85051676109

SN - 9781450357166

SP - 537

EP - 541

BT - Proceedings - 2018 ACM/IEEE 15th International Conference on Mining Software Repositories, MSR 2018

PB - IEEE Computer Society

Y2 - 28 May 2018 through 29 May 2018

ER -

Restmule: Enabling resilient clients for remote APIs

Abstract

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Cite this