Let's GO

    C# is a great language. Each new version takes the best solutions from other languages and keeps it very modern. However writing C# all the time both at work and at home feels boring sometimes. I like learning new languages as this provides good exercise. I've decided to write a simple go program. And because I didn't have a better idea I took an app that I had already written in .NET Core and created it again in Go. The app should download history of average daily cryptocurrency value from exchange Binance. I want it to get all crypto pairs from beginning of time until now. I’ve started with simplest thing ever:

package main

const hostUrl = "https://binance.com"
const apiResourceUri = "/api/v1/klines"
const apiGetSymbolUri = "/api/v1/exchangeInfo"

const downloadLocation = "data" // here I’ll save data

So far so good Opening url http://binance.com/api/v1/exchangeInfo will get us among other things list of all symbols that this exchange handles. Let’s see part of response:

{
    […]
    "exchangeFilters": [],
    "symbols": [
        {
            "symbol": "ETHBTC",
            "status": "TRADING",
            "baseAsset": "ETH",
            "baseAssetPrecision": 8,
            "quoteAsset": "BTC",
            "quotePrecision": 8,
            "orderTypes": [
                "LIMIT",
                "LIMIT_MAKER",
                "MARKET",
                "STOP_LOSS_LIMIT",
                "TAKE_PROFIT_LIMIT"
            ],
            "icebergAllowed": true,
            "isSpotTradingAllowed": true,
            "isMarginTradingAllowed": true,
            "filters": […]
        },
        {
            "symbol": "LTCBTC",
            "status": "TRADING",
            "baseAsset": "LTC",
            "baseAssetPrecision": 8,
            "quoteAsset": "BTC",
            "quotePrecision": 8,
            "orderTypes": [
                "LIMIT",
                "LIMIT_MAKER",
                "MARKET",
                "STOP_LOSS_LIMIT",
                "TAKE_PROFIT_LIMIT"
            ],
            "icebergAllowed": true,
            "isSpotTradingAllowed": true,
            "isMarginTradingAllowed": false,
            "filters": […]
        },
        {
            "symbol": "BNBBTC",
            "status": "TRADING",
            "baseAsset": "BNB"

and so on. I need only the „symbols” array and only „symbol” inside it. My model to deserialize the response to would look like this:

type Symbol struct {
    Symbol string
}

type GetSymbolsResponse struct {
    Symbols []Symbol
}

My method that returns all symbol looks like this:

//Gets all available symbols from binance
func GetSymbols() []string {
    resp, err := http.Get(hostUrl + apiGetSymbolUri)

    if err != nil {
        panic(err)          // if there is an error, just stop and write about it
    }
               //  this is like a "finally" block in c#, it will be executed on exiting function
defer resp.Body.Close()
    response := new(GetSymbolsResponse)
    json.NewDecoder(resp.Body).Decode(response)
    fmt.Println(len(response.Symbols))
    result := make([]string, 0)
    for _, val := range response.Symbols {
        result = append(result, val.Symbol)
    }
    return result
}

In order for this code to work, I need to add imports (the method above uses http, fmt and json). I put imports right after my package main:

import (
    "encoding/json"
    "fmt"
    "net/http"
)

Now when I have all symbols I’m interested in, I want to download all historical data for them. I will download it a month at a time and save response in my data folder. Outline of my program will be something like:

func loadHistoricalData() {

    checkDownloadLocation() // create data folder if doesn’t exist

    fmt.Println("Getting symbols")
    r := GetSymbols() 
    count := len(r)

    syncChannel := make(chan bool, 3) // concurrent jobs at most
    jobsCount := 0
    fmt.Println("Found %v symbols", count)

    for _, b := range r {
        for year := time.Now().Year(); year > 2010; year-- { // get all data since 2010
            for month := 12; month >= 1; month-- {
                jobsCount++
                go downloadMonthOfData(b, year, month, syncChannel)
            }
        }
    }

    for i := 0; i < jobsCount; i++ {
        syncChannel <- code="" done="" fmt.println="" func="" ll="" loadhistoricaldata="" main="" true="">

Channels look strange but my goal is to limit number of concurrent jobs that are being run. In order to do so I use syncChannel. What it does is pause thread that tries to put fourth item to the channel (because I have set it's capacity to 3) until there is a place (which means some other thread has consumed one of the 3 items in channel) and also makes thread which tries to take item from empty channel to wait until something shows up there. I know how many jobs total number is so I'm trying to put that many items in syncChannel (pausing if there are already 3 items not picked up by a worker thread). Function downloadMonthOfData looks like this:

func downloadMonthOfData(symbol string, year, month int, c chan bool) {
signal := <-c // wait until signal to start

 if !signal {
  return
 }
 start := time.Date(year, time.Month(month), 1, 0, 0, 0, 0, time.UTC)
 end := time.Date(year, time.Month(month+1), 1, 0, 0, 0, 0, time.UTC)

 if start.After(time.Now()) {
  return
 }

 fileName := fmt.Sprintf("%s_%d_%02d.json", symbol, year, month)

 fmt.Println("Downloading: " + fileName)

 pathToFile := path.Join(downloadLocation, fileName)

 if _, err := os.Stat(pathToFile); os.IsNotExist(err) {
  r, e := http.Get(createGetSymbolUrl(symbol, start.Unix()*1000, end.Unix()*1000))
  if e != nil {
   panic(e)
  }
  defer r.Body.Close()
  out, _ := os.Create(pathToFile)
  defer out.Close()
  io.Copy(out, r.Body)
 }
}

That's it. Whole program is available here: https://github.com/jaropawlak/getBinanceStats. I needed to add some more imports to make it work so not every line of code is in this article but all is there in github ;). This is my humble start in go. It wasn't hard and all questions/doubts I had were resolved after a moment of googling it. Very pleasant experience I must say. For sure this is not my last date with go. If you have any doubts to try it yourself, I highly recommend it.