User Guide

Custom Data Source

The FMZ Quant Trading Platform's backtesting system supports custom data sources. The backtesting system uses the GET method to request custom URLs (publicly accessible addresses) to obtain external data sources for backtesting. The additional request parameters are as follows:

Parameter	Meaning	Description
symbol	Symbol name	Spot market data example: `BTC_USDT`, Futures market data example: `BTC_USDT.swap`, Perpetual futures funding rate data example: `BTC_USDT.funding`, Perpetual futures price index data example: `BTC_USDT.index`
eid	Exchange	For example: OKX, Futures_OKX
round	Data precision	When true, indicates that the data returned by the custom data source defines specific precision. The request sent by the FMZ Quant Trading Platform backtesting system to the custom data source is fixed as: `round=true`
period	K-line data period (milliseconds)	For example: `60000` represents a 1-minute period
depth	Order book depth levels	1-20
trades	Whether tick data is required	True (1) / False (0)
from	Start time	Unix timestamp
to	End time	Unix timestamp
detail	Request detailed information of the symbol	When true, indicates that it needs to be provided by the custom data source. The request sent by the FMZ Quant Trading Platform backtesting system to the custom data source is fixed as: `detail=true`
custom	--	This parameter can be ignored

When the data source of spot exchange or futures exchange objects is set to custom data source (feeder), examples of requests sent by the backtesting system to the custom data source service:


            url
            
            
http://customserver:9090/data?custom=0&depth=20&detail=true&eid=Bitget&from=1351641600&period=86400000&round=true&symbol=BTC_USDT&to=1611244800&trades=1
http://customserver:9090/data?custom=0&depth=20&detail=true&eid=Futures_OKX&from=1351641600&period=86400000&round=true&symbol=BTC_USDT.swap&to=1611244800&trades=1

Data Format

The returned format must be one of the following two formats (automatically recognized by the system):

Simulated-level Tick, here is a JSON data example:


            json
            
            
{
    "detail": {
        "eid": "Binance",
        "symbol": "BTC_USDT",
        "alias": "BTCUSDT",
        "baseCurrency": "BTC",
        "quoteCurrency": "USDT",
        "marginCurrency": "USDT",
        "basePrecision": 5,
        "quotePrecision": 2,
        "minQty": 0.00001,
        "maxQty": 9000,
        "minNotional": 5,
        "maxNotional": 9000000,
        "priceTick": 0.01,
        "volumeTick": 0.00001,
        "marginLevel": 10
    },
    "schema":["time", "open", "high", "low", "close", "vol"],
    "data":[
        [1564315200000, 9531300, 9531300, 9497060, 9497060, 787],
        [1564316100000, 9495160, 9495160, 9474260, 9489460, 338]
    ]
}

Live-level Tick, here is a JSON data example:
Tick-level backtest data (includes order book depth information, depth format is an array of [price, quantity]. Can include multiple levels of depth, asks sorted in ascending order by price, bids sorted in descending order by price).


            json
            
            
{
    "detail": {
        "eid": "Binance",
        "symbol": "BTC_USDT",
        "alias": "BTCUSDT",
        "baseCurrency": "BTC",
        "quoteCurrency": "USDT",
        "marginCurrency": "USDT",
        "basePrecision": 5,
        "quotePrecision": 2,
        "minQty": 0.00001,
        "maxQty": 9000,
        "minNotional": 5,
        "maxNotional": 9000000,
        "priceTick": 0.01,
        "volumeTick": 0.00001,
        "marginLevel": 10
    },
    "schema":["time", "asks", "bids", "trades", "close", "vol"],
    "data":[
        [1564315200000, [[9531300, 10]], [[9531300, 10]], [[1564315200000, 0, 9531300, 10]], 9497060, 787],
        [1564316100000, [[9531300, 10]], [[9531300, 10]], [[1564316100000, 0, 9531300, 10]], 9497060, 787]
    ]
}

Field	Description
detail	Detailed information of the requested instrument, including quote currency name, base currency name, precision, minimum order quantity, etc.
schema	Specifies the column attributes in the data array, case-sensitive. Limited to time, open, high, low, close, vol, asks, bids, trades
data	Data recorded according to the column structure set by schema

detail field

Field	Description
eid	Exchange ID, note that spot and futures of the same exchange use different eids
symbol	Trading instrument code
alias	The corresponding symbol of the current trading instrument code on the exchange
baseCurrency	Base currency
quoteCurrency	Quote currency
marginCurrency	Margin currency
basePrecision	Base currency precision
quotePrecision	Quote currency precision
minQty	Minimum order quantity
maxQty	Maximum order quantity
minNotional	Minimum order amount
maxNotional	Maximum order amount
priceTick	Minimum price tick size
volumeTick	Minimum volume tick size
marginLevel	Futures leverage multiplier
contractType	For perpetual contracts set to: `swap`, the backtest system will continue to send funding rate and price index requests

Special column attributes asks, bids, trades description:

Field	Description	Remarks
asks / bids	[[price, quantity], ...]	For example, data in the `Live-level Tick` data example: `[[9531300, 10]]`
trades	[[time, direction(0:buy,1:sell), price, quantity], ...]	For example, data in the `Live-level Tick` data example: `[[1564315200000, 0, 9531300, 10]]`

When backtesting perpetual contracts on futures exchanges, custom data sources also need to provide additional funding rate data and price index data. Only when the requested market data is returned and the detail field in the return structure contains the "contractType": "swap" key-value pair, will the backtest system continue to send funding rate requests.
After the backtest system receives the funding rate data, it will continue to send price index data requests.

Funding rate data structure is as follows:


            json
            
            
{
    "detail": {
        "eid": "Futures_Binance",
        "symbol": "BTC_USDT.funding",
        "alias": "BTC_USDT.funding",
        "baseCurrency": "BTC",
        "quoteCurrency": "USDT",
        "marginCurrency": "",
        "basePrecision": 8,
        "quotePrecision": 8,
        "minQty": 1,
        "maxQty": 10000,
        "minNotional": 1,
        "maxNotional": 100000000,
        "priceTick": 1e-8,
        "volumeTick": 1e-8,
        "marginLevel": 10
    },
    "schema": [
        "time",
        "open",
        "high",
        "low",
        "close",
        "vol"
    ],
    "data": [
        [
            1584921600000,
            -16795,
            -16795,
            -16795,
            -16795,
            0
        ],
        [
            1584950400000,
            -16294,
            -16294,
            -16294,
            -16294,
            0
        ]
        // ...
    ]
}

Adjacent period interval is 8 hours
For example, Binance funding rate updates every 8 hours, why is the funding rate data -16795?
This is because, like K-line data, to avoid floating-point precision loss during network transmission, data is represented as integers; funding rate data can also be negative.

Example of funding rate data request sent by the backtest system:


            url
            
            
http://customserver:9090/data?custom=0&depth=20&detail=true&eid=Futures_Binance&from=1351641600&period=86400000&round=true&symbol=BTC_USDT.funding&to=1611244800&trades=0

Price index data structure is as follows:


            json
            
            
{
    "detail": {
        "eid": "Futures_Binance",
        "symbol": "BTC_USDT.index",
        "alias": "BTCUSDT",
        "baseCurrency": "BTC",
        "quoteCurrency": "USDT",
        "contractType": "index",
        "marginCurrency": "USDT",
        "basePrecision": 3,
        "quotePrecision": 1,
        "minQty": 0.001,
        "maxQty": 1000,
        "minNotional": 0,
        "maxNotional": 1.7976931348623157e+308,
        "priceTick": 0.1,
        "volumeTick": 0.001,
        "marginLevel": 10,
        "volumeMultiple": 1
    },
    "schema": [
        "time",
        "open",
        "high",
        "low",
        "close",
        "vol"
    ],
    "data": [
        [1584921600000, 58172, 59167, 56902, 58962, 0],
        [1584922500000, 58975, 59428, 58581, 59154, 0],
        // ...
    ]
}

Example of price index data request sent by the backtest system:


            url
            
            
http://customserver:9090/data?custom=0&depth=20&detail=true&eid=Futures_Binance&from=1351641600&period=86400000&round=true&symbol=BTC_USDT.index&to=1611244800&trades=0

Custom Data Source Example

Specify the data source address, for example: http://120.24.2.20:9090/data. The custom data source service program is written in Golang:


            golang
            
            
package main

import (
    "fmt"
    "net/http"
    "encoding/json"
)

func Handle (w http.ResponseWriter, r *http.Request) {
    // e.g. set on backtest DataSourse: http://xxx.xx.x.xx:9090/data

    // request: GET http://xxx.xx.x.xx:9090/data?custom=0&depth=20&detail=true&eid=OKX&from=1584921600&period=86400000&round=true&symbol=BTC_USDT&to=1611244800&trades=1
    //              http://xxx.xx.x.xx:9090/data?custom=0&depth=20&detail=true&eid=Futures_Binance&from=1599958800&period=3600000&round=true&symbol=BTC_USDT.swap&to=1611244800&trades=0
    fmt.Println("request:", r)

    // response
    defer func() {
        // response data
        /* e.g. data
        {
            "detail": {
                "eid": "Binance",
                "symbol": "BTC_USDT",
                "alias": "BTCUSDT",
                "baseCurrency": "BTC",
                "quoteCurrency": "USDT",
                "marginCurrency": "USDT",
                "basePrecision": 5,
                "quotePrecision": 2,
                "minQty": 0.00001,
                "maxQty": 9000,
                "minNotional": 5,
                "maxNotional": 9000000,
                "priceTick": 0.01,
                "volumeTick": 0.00001,
                "marginLevel": 10
            },
            "schema": [
                "time",
                "open",
                "high",
                "low",
                "close",
                "vol"
            ],
            "data": [
                [1610755200000, 3673743, 3795000, 3535780, 3599498, 8634843151],
                [1610841600000, 3599498, 3685250, 3385000, 3582861, 8015772738],
                [1610928000000, 3582499, 3746983, 3480000, 3663127, 7069811875],
                [1611014400000, 3662246, 3785000, 3584406, 3589149, 7961130777],
                [1611100800000, 3590194, 3641531, 3340000, 3546823, 8936842292],
                [1611187200000, 3546823, 3560000, 3007100, 3085013, 13500407666],
                [1611273600000, 3085199, 3382653, 2885000, 3294517, 14297168405],
                [1611360000000, 3295000, 3345600, 3139016, 3207800, 6459528768],
                [1611446400000, 3207800, 3307100, 3090000, 3225990, 5797803797],
                [1611532800000, 3225945, 3487500, 3191000, 3225420, 8849922692]
            ]
        }
        */

        // /* Simulation-level Tick
        ret := map[string]interface{}{
            "detail": map[string]interface{}{
                "eid": "Binance",
                "symbol": "BTC_USDT",
                "alias": "BTCUSDT",
                "baseCurrency": "BTC",
                "quoteCurrency": "USDT",
                "marginCurrency": "USDT",
                "basePrecision": 5,
                "quotePrecision": 2,
                "minQty": 0.00001,
                "maxQty": 9000,
                "minNotional": 5,
                "maxNotional": 9000000,
                "priceTick": 0.01,
                "volumeTick": 0.00001,
                "marginLevel": 10,
            },
            "schema": []string{"time","open","high","low","close","vol"},
            "data": []interface{}{
                []int64{1610755200000, 3673743, 3795000, 3535780, 3599498, 8634843151},  // 1610755200000 : 2021-01-16 08:00:00
                []int64{1610841600000, 3599498, 3685250, 3385000, 3582861, 8015772738},  // 1610841600000 : 2021-01-17 08:00:00
                []int64{1610928000000, 3582499, 3746983, 3480000, 3663127, 7069811875},
                []int64{1611014400000, 3662246, 3785000, 3584406, 3589149, 7961130777},
                []int64{1611100800000, 3590194, 3641531, 3340000, 3546823, 8936842292},
                []int64{1611187200000, 3546823, 3560000, 3007100, 3085013, 13500407666},
                []int64{1611273600000, 3085199, 3382653, 2885000, 3294517, 14297168405},
                []int64{1611360000000, 3295000, 3345600, 3139016, 3207800, 6459528768},
                []int64{1611446400000, 3207800, 3307100, 3090000, 3225990, 5797803797},
                []int64{1611532800000, 3225945, 3487500, 3191000, 3225420, 8849922692},
            },
        }
        // */

        /* Live trading-level Tick
        ret := map[string]interface{}{
            "detail": map[string]interface{}{
                "eid": "Binance",
                "symbol": "BTC_USDT",
                "alias": "BTCUSDT",
                "baseCurrency": "BTC",
                "quoteCurrency": "USDT",
                "marginCurrency": "USDT",
                "basePrecision": 5,
                "quotePrecision": 2,
                "minQty": 0.00001,
                "maxQty": 9000,
                "minNotional": 5,
                "maxNotional": 9000000,
                "priceTick": 0.01,
                "volumeTick": 0.00001,
                "marginLevel": 10,
            },
            "schema": []string{"time", "asks", "bids", "trades", "close", "vol"},
            "data": []interface{}{
                []interface{}{1610755200000, []interface{}{[]int64{9531300, 10}}, []interface{}{[]int64{9531300, 10}}, []interface{}{[]int64{1610755200000, 0, 9531300, 10}}, 9497060, 787},
                []interface{}{1610841600000, []interface{}{[]int64{9531300, 15}}, []interface{}{[]int64{9531300, 15}}, []interface{}{[]int64{1610841600000, 0, 9531300, 11}}, 9497061, 789},
            },
        }
        */

        b, _ := json.Marshal(ret)
        w.Write(b)
    }()
}

func main () {
    fmt.Println("listen http://localhost:9090")
    http.HandleFunc("/data", Handle)
    http.ListenAndServe(":9090", nil)
}

Test strategy, JavaScript example:


            javascript
            
            
/*backtest
start: 2021-01-16 08:00:00
end: 2021-01-22 00:00:00
period: 1d
basePeriod: 1d
exchanges: [{"eid":"OKX","currency":"BTC_USDT","feeder":"http://120.24.2.20:9090/data"}]
args: [["number",2]]
*/

function main() {
    var ticker = exchange.GetTicker()
    var records = exchange.GetRecords()
    Log(exchange.GetName(), exchange.GetCurrency())
    Log(ticker)
    Log(records)
}