Programmatically fetch multiple APIs in parallel using async and await in JavaScript

When I was building ethfolio, I had to figure out how to retrieve the token information for multiple Ethereum addresses. To get this information, you have to query an API per address that you want to retrieve.

Ideally, all of these calls would happen Asynchronously and in parallel, to give the  best and fastest experience to the user. However, how exactly to do this is not so obvious (to me at least).

Let’s assume that you have an array of URLs that you want to fetch at the same time. You can use an asynchronous function like this to easily retrieve all the data in the best way possible:

async function getAllUrls(urls) {
    try {
        var data = await Promise.all(
            urls.map(
                url =>
                    fetch(url).then(
                        (response) => response.json()
                    )));

        return (data)

    } catch (error) {
        console.log(error)

        throw (error)
    }
}

This function will return an array of ‘responses’ which you can then use for the rest of your program. Calling the function is simple too!

var responses = await getAllUrls(urls)

Ultimately the trick here is that a Promise gets executed as soon as it gets created. The map function will create all the fetch promises and they will immediately start to execute. Then, we wait for all the promises to complete using the Promise.all function.

I tried other ways to construct this kind of function, but none were quite as simple or effective as this. I hope you find this helpful 🙂

ethfolio: A client side app to show your Ethereum token distribution

A common question that I have for others investing in cryptocurrencies is: “What coins are you invested in?”

It is surprisingly hard to distinguish fear, uncertainty, doubt (FUD), fear of missing out (FOMO), and solid informed advice apart from one another. But as the saying goes:

Put your money where your mouth is.

This is the goal of ethfolio:

  • Allow users to easily import the coins they currently are holding
  • Allow users to share this token distribution
  • And do so safely without needing to expose their Ethereum addresses or total value of their holdings

This actually seemed very straightforward, so I decided to tackle the project. Rather than go through all the work to actually query the blockchain as I have been learning to do in my other posts, I decided to just use an API which consolidates all this information.

Ethplorer.io was perfect for this. They have a /getAddressInfo/ api endpoint which provides details about the tokens at an Ethereum address, and even details about those tokens like the price in USD.

This was perfect! So I begin to envision the user story:

  1. User goes to ethfolio, types in one or more Ethereum addresses
  2. User presses a “submit” button
  3. In the background we query the Ethplorer API to get all the tokens across all the addresses, and their current price in USD
  4. We then add up total amount of each token the user has across their wallets
  5. Then, we use the price information, to calculate the relative USD price of each token balance
  6. Then we calculate the total USD price across all balances, to find the percentage a particular token represents in the overall portfolio
  7. Display the results

Following this process, the user can then share details about their token distribution without ever needing to share details about their specific addresses (which is important for anonymity) and without needing to share details about the total balance they hold (which could be sensitive to the user.)

With these steps in mind, I spent a day and created this pretty quickly:

http://shawntabrizi.com/ethfolio/

You may notice that there are some options to “Store total USD value” and “Store all addresses”. Lets talk about what I tried to do with “Save & Share”.

I wanted this to be a completely client side application. This means I don’t have to do any server stuff,  the user can better trust the application (because they can see all the code), and calling the API would come from each user, so I wouldn’t get throttled if the site takes off. My question then was: “How can I create a database for a client side application?”

This means that writing and reading from the database needs to be in such a way that it won’t compromise the data being stored. My first thought was to try and use Google Forms/Sheets. Google Forms is a method for publicly entering data, which can get stored into a Google Sheet which can be publicly viewed. Using Google’s authorization layer, I could make it so that users could not edit the sheet, only add data to it, and read from it.

I got the part of the app to write to Google working. It doesn’t look pretty because the app gives a CORS error, but Google seems to accept the data anyway.

Ignore the error

Cause data gets saved anyway…

However, I was not able to ever retrieve this data! Even though this sheet is public, it seems like to call any of Google’s APIs you must be authenticated, at least with an API key.

I created an API key, but trying to use it I get this error:

"message": "API Key not found. Please pass a valid API key."

Every API key is associated with a “Google App Project”, and I needed to specifically enable the Google Sheets API for the project where I was creating the API key.

Now that this call works, I just need to build an interface to read from the sheet, and re-display the data! That will be coming up next!