Requesting Crypto Prices from the Coinmarketcap API using Python

You can do various things with cryptocurrency price data, such as creating forecasting models, illustrating historical prices, or performing chart analysis. But first, you need to get hold of the data. Nowadays, several APIs provide access to cryptocurrency price data. One of the most important and trusted sources is Coinmarketcap.com. The website offers multiple API endpoints that provide access to historical price data for various cryptocurrencies like Bitcoin, Ethereum, etc. In addition, Coinmarketap offers meta information about coins and crypto exchanges, including listing and halving dates, circulating supply, or the current market cap. This tutorial gives an overview of the coinmarketcap API and shows how to use the API with Python. We will request historical price data and store them in a local SQLite database using the Pewee SQL framework.

The rest of this article is a Python hands-on tutorial: First, we lay the groundwork for working with the Coinmarketcap API by signing up for a free API key. Subsequently, we store this in a YAML file and use it to query bitcoin quotes from the Coinmarketcap API endpoint. Our goal is to store the data with Peewee in an SQLite DB. We will first define a schema to store the data. Finally, we will learn how to query the obtained rates from our local database and use them later, for example, to train a price prediction model.

Also: Streaming Tweets and Images via the Twitter API in Python

The Coinmarketcap API

CoinMarketCap is one of the world’s most referenced crypto asset price tracking websites. The website provides high-quality and accurate information to private and business users so they can draw their own informed conclusions. In April 2020, CoinMarketCap was acquired by Binance – the largest cryptocurrency exchange in the world.

Generally, all relevant coins are listed on Coinmarketcap – however, some are scam coins and coins whose further development was abandoned long ago. The CoinMarketCap API is a set of powerful RESTful JSON endpoints. According to the website, these APIs are designed to meet the mission-critical needs of application developers, data scientists, and enterprise platforms.

Let’s look at some of the available REST endpoints.

Endpoints

Coinmarketcap offers several API endpoints for various crypto-related data, including prices, exchanges, and other information.

Below is a selection of API endpoints:

A big advantage of the Coinmarketcap API is that the data goes way back. For example, the historical price data for Bitcoin ranges back to 2013.

Acquiring an API Key from the Developer Portal

All requests to the coinmarketcap API require authentication with an API key. Authentication can be achieved in two ways:

• Send the API key in the custom header of the HTTP request with the attribute ‘X-CMC_PRO_API_KEY’ (Prefered).
• Provide the key in the URL path.

Before requesting data from the coinmarketcap API, you must acquire your API key by registering on the coinmarketcap API developer portal. You will choose from one of several pricing plans during the registration process.

The Free basic plan provides access to most data and is sufficient for this tutorial. However, it has severe limitations, such as a limited number of requests you can make daily and monthly. After the registration, you can log in to your account and display the API key in the “Overview” section.

Each call to the Coinmarketcap API consumes API credits. The number of calls and data we can retrieve via API is limited to 100 credits for the free plan. You can see how many of your daily and monthly credits are still available on the overview page.

Storing the API Key

Obtaining Bitcoin Prices via the Coinmarketcap API

In this tutorial, we demonstrate how to use Python to send periodic requests to the Coinmarketcap API to obtain cryptocurrency prices for the top-ranked digital asset by market capitalization. Since each request to the Coinmarketcap API consumes credits, we utilize a local SQLite database to store the price data. This allows us to retrieve the data without the need for additional API requests and preserves credits for future use.

To efficiently manage the SQLite database, we employ the PeeWee framework, which offers a robust and user-friendly interface for interacting with SQL databases in Python. This approach enables users to easily create, modify, and query data within the SQLite database.

Overall, this tutorial provides a valuable resource for anyone looking to automate the collection and storage of cryptocurrency price data using the Coinmarketcap API and SQLite database with the Pewee framework.

The code is available on the GitHub repository.

Prerequisites

Before beginning the coding part, please ensure you have set up your Python 3 environment and required packages. If you don’t have a Python environment available yet, you can follow the steps in this article to set up the Anaconda Python environment.

Also, make sure you install the required packages. In this tutorial, we will be working with the following standard packages: 

• pandas
• NumPy
• matplotlib
• Seaborn

In addition, we will use Peewee, a lightweight Object-Relational-Mapper (ORM) framework that lets us interact with SQLite in an object-oriented and more convenient way.

You can install packages using console commands:

• pip install <package name>
• conda install <package name> (if you are using the anaconda packet manager)

Step #1 Create a Relational Model in SQLite

We begin by defining a relational data model in SQlite using Pewee. We will then use this model to persist the crypto price data from the API and make them available for later reuse. Our model will contain two tables:

• Run: A table where we will store an ID and a timestamp makes identifying past runs easier. We start querying data from the API, we will add a new runid to this table.
• The second table is called price and contains the price quotes for a predetermined interval. We will store the price quotes in this table and reference the runid via a foreign key.

Running the code below creates empty tables.

from peewee import *
from datetime import date, timedelta, datetime
import numpy as np 
import pandas as pd
import seaborn as sns
import time
import json
import yaml
from requests import Request, Session
from requests.exceptions import ConnectionError, Timeout, TooManyRedirects
from pandas.plotting import register_matplotlib_converters

import matplotlib.dates as mdates
import matplotlib.pyplot as plt

# persist information
db = SqliteDatabase('assets.db')

class BaseModel(Model):
    class Meta:
        database = db

class Run(BaseModel): # This model uses the "assets.db" database.
    timestamp = DateTimeField(unique=True)
    symbol = CharField()
    id = AutoField()

class Price(BaseModel): # This model uses the "assets.db" database.
    datetime = DateTimeField(unique=True)
    volume_24 = FloatField()
    price = FloatField()
    runid = ForeignKeyField(Run, backref='prices')
         
# By default, Peewee includes an IF NOT EXISTS clause when creating tables. 
def create_tables():
    with db:
        db.create_tables([Price])
        db.create_tables([Run])

def drop_tables():
    with db:
        db.drop_tables([Price])
        db.drop_tables([Run])
        
create_tables()

#drop_tables()    

If you want to drop the tables again, you can call our custom drop_tables function.

Step #2 Streaming Price Quotes

Now that we have created the relational data model, we can define a request to the Coinmarketcap API and initiate the stream of price data. For this purpose, we will call the coinmarketcap REST endpoint “cryptocurrency/listings/latest.” This endpoint returns a JSON response that contains the latest price data for a specific number of cryptocurrencies based on their market capitalization rank. The response contains the following information for each cryptocurrency included:

• Price value
• The timestamp
• The trading volume
• The cost of the request in credits

Before querying the API, we need to define basic API parameters:

• Conversion_Currency: The currency in which the API will return the prices.
• Start: The position in the market cap ranking from which the API will include cryptocurrencies in the result.
• Limit: The number of cryptocurrencies for which the API will return prices based on the start position in the current market cap ranking.

We will select USD as “conversion_currency,” set the limit, and start to “one.” As a result, the API will return only the price of Bitcoin. We use a query interval of 10 seconds. You can change the interval, but be aware that you will deplete your free credits quickly if you request data by the second. We will therefore limit the requests to 200. You can increase the timeframe to retrieve price quotes for more extended periods.

We will store and load our API key from a YAML file. Be sure to replace the api_key_path in the code below with the path where you placed the api_config_coinmarketcap.yml file. Once you have made this adjustment, you can start the API requests by running the code below.

symbol='BTCUSD'
query_interval = 10 # in seconds
query_number = 100 # the number of queries after which the API stops

# load the API key from our local file
with open(r'API Keys/api_config_coinmarketcap.yml') as file:
    apikey = yaml.load(file, Loader=yaml.FullLoader)['api_key']

# Define some essential API parameters
# Coinmarketcap API for latest market ticker quotes and averages for cryptocurrencies and exchanges.
# https://coinmarketcap.com/api/documentation/v1/#operation/getV1CryptocurrencyListingsLatest
url = 'https://pro-api.coinmarketcap.com/v1/cryptocurrency/listings/latest'
parameters = {
  'start':'1',
  'limit':'1',
  'convert':'USD'
}
headers = {
  'Accepts': 'application/json',
  'X-CMC_PRO_API_KEY': apikey,
}

session = Session()
session.headers.update(headers)

# create a new run and save it to our local SQLite DB
run_timestamp = datetime.now()
run = Run(symbol=symbol, timestamp = run_timestamp)
run.save()
current_run_id = run.id 
print(f'run_id: {current_run_id} - timestamp: {run_timestamp} - interval: {query_interval} - number of queries: {query_number}')

# query the coinmarketcap API every x seconds
for s in range(0, query_number):
    try:
        response = session.get(url, params=parameters)
        data = json.loads(response.text)
        #print(data)
        
        # response - quote
        data_quote = data['data'][0]['quote']['USD']
        price = np.round(data_quote['price'], 1) # the price
        volume_24 = np.round(data_quote['volume_24h'], 1) # the volume in the last 24 hours
        
        # response - status
        data_status = data['status']
        api_timestamp = data_status['timestamp'] # the timestamp of the pricepoint
        api_credits = data_status['credit_count'] # the number of credits spent on the last request
        
        # create a new pricepoint and save it to the SQLite db
        new_pricepoint = Price(datetime=api_timestamp, price=price, volume_24=volume_24, runid=current_run_id)
        id = new_pricepoint.save()

        # display what we have just saved
        print(f'request number: {s} - added {price} at {api_timestamp} - 24 hour volume: {volume_24} - credits used: {api_credits}')      

    except (ConnectionError, Timeout, TooManyRedirects) as e:
        print(e)
    time.sleep(query_interval)
print('finished')

run_id: 2 - timestamp: 2021-05-03 22:03:01.172363 - interval: 5 - number of queries: 200
request number: 0 - added 57487.7 at 2021-05-03T20:03:02.587Z - 24 hour volume: 48962167377.0 - credits used: 1
request number: 1 - added 57487.7 at 2021-05-03T20:03:08.059Z - 24 hour volume: 48962167377.0 - credits used: 1
request number: 2 - added 57487.7 at 2021-05-03T20:03:13.256Z - 24 hour volume: 48962167377.0 - credits used: 1
request number: 3 - added 57453.4 at 2021-05-03T20:03:18.470Z - 24 hour volume: 48948337250.5 - credits used: 1

Step #3 Querying the Data from Our SQL Table

We now have the price data persisted in our local SQLite DB. From there, we can query the data using the peewee “select” SQL command. For example, we will query the run table and retrieve the number of requests made to the coinmarketcap API.

query = Run.select().where(Run.id==current_run_id)
run_overview = pd.DataFrame(list(query.dicts()))
run_overview

Next, select the price quotes for this specific run-id and display them in a line plot.

query = Price.select().where(Price.runid==current_run_id)
df_prices = pd.DataFrame(list(query.dicts()))
print(df_prices)

register_matplotlib_converters()
fig, ax = plt.subplots(figsize=(10, 8))
plt.title('BTC prices from the coinmarketcap API', fontsize=14)
datetimes = pd.to_datetime(df_prices['datetime'])

sns.lineplot(data=df_prices, x=datetimes, y="price")
#ax.set_xlim([df_prices['datetime'].min(),df_prices['datetime'].max()])
#ax.xaxis.set_major_locator(mdates.MinuteLocator())
plt.xticks(rotation=75)

And voila, the line plot shows the last data received from the API. We now have stored this data in our SQLite database.

Summary

This article gave a quick overview of the Coinmarketcap API – one of the most referenced APIs in the crypto space. You have learned to request historical price data from a Coinmarketcap API endpoint with Python. We wrote a short Python script that queries one of several REST endpoints (cryptocurrency/listings/latest) in regular intervals. To ensure that we can later reuse the requested data, we stored the response from the API in an SQLite database using the Pewee framework.

I am always happy to receive feedback from my readers. So if the tutorial helped you, or if you have any comments, please write them in the comments.

Sources and Further Reading

The links above to Amazon are affiliate links. By buying through these links, you support the Relataly.com blog and help to cover the hosting costs. Using the links does not affect the price.

If you are not sure yet, if the coinmarketcap API is for you, consider these other ways of obtaining cryptocurrency price data:

• Working with the Coinbase API
• Using the Yahoo Finance API to obtain Cryptocurrency Price Data
• Streaming Crypto Prices via the Gate.io API

For more Python tutorials on stock-market prediction, check out the following articles:

• Stock Market Prediction using Multivariate Time Series and Recurrent Neural Networks in Python
• Stock-Market prediction using Neural Networks for Multi-Output Regression in Python
• Stock Market Prediction using Univariate Time Series Models based on Recurrent Neural Networks with Python
• Stock Market Prediction – Adjusting Time Series Prediction Intervals in Python
• Feature Engineering for Multivariate Time Series Prediction with Python
• Building a Twitter Bot for Crypto Trading Signals using Python and Gate.io