Back testing and parameters optimization

A great opportunity to quickly and safely check how changing parameters affects the result

Concept

Modified Martingale strategy responds to market signals, constantly adjusting to changing conditions. The system itself is passive - it only responds to an incoming data stream. The reaction depends on:

1. Specified limits of parameters change
2. System status at a given time
3. Incoming data from exchange:
   • market data stream
   • user data stream
4. Time scale, some process have fixed or parametric time delay

For quality simulation and obtaining a repeatable result, all of the above matters.

The following solutions have been implemented for these items:

1. This is the subject of research and optimization, with identical parameters, the strategy gives the same result
2. Data collection and training sessions start from a zero state. The ability to start from any saved state is being studied
3. Exchange data emulator
   • market data - during the TC (Trade & Collect) session, the required set of data streams is stored
   • user data - developed an exchange emulator that reliably reproduces basic functions, such as placing and executing orders and updating the accounting balance
4. Playback of the data stream is accelerated xN times, the "local time generator" is synchronized with this stream, all internal processes of the system are synchronously accelerated

Limitations

• Only the main functions of the exchange are emulated, except for the above ones, time delays when placing orders are taken into account. However, this is a simple model that does not implement partial order fulfillment, market liquidity, slippage, the impact of own orders on the order book, etc. It should be taken into account that market conditions are changing in real time and super productive parameters in the past period may not be so effective in the future.

• Use Trade & Collect mode on product system with caution. The accumulation of streaming data requires additional memory. There is a safety mechanism - if the amount of free memory in the system becomes less than 30%, data from the memory is uploaded to the file and the memory occupied for this process is freed.

Collect mode

New parameter group added to configuration file:

# Backtest mode parameters
ex.MODE = 'TC'  # 'T' - Trade, 'TC' - Trade and Collect, 'S' - Simulate
ex.SAVE_DS = True  # Save session result data (ticker, orders) for compare

Use ex.MODE = 'TC' for collect WS stream data for later back-testing. You can use this mode for both - "paper" and real trading. As result, after stop strategy new file structure was created:

Where:

• back_test/binance_ETHBUSD/raw - saved WS data for later back-testing
• back_test/binance_ETHBUSD/snapshot - snapshot of the current session, containing the order history, separately buy/sell and ticker
• back_test/binance_ETHBUSD/cli_7_ETHBUSD.py - copy of session parameters
• back_test/binance_ETHBUSD_mmdd-HH:MM.zip - archive with unloaded portion of session data

The data is uploaded to the archive by an event that occurs earlier:

• stop strategy by command
• recording time exceeded, set by parameter SAVE_PERIOD = 1 * 60 * 60 # sec, timetable for save data portion. Default value 1h.
• free memory in the system has become less than 30%

Simulate mode

Take saved back_test/binance_ETHBUSD/cli_7_ETHBUSD.py, set ex.MODE = 'S' and start it.

For the first time, do not change the trading parameters, so you can make sure that the result is reliably obtained. For comparison you can use visual method

The trading simulation will start without further confirmation. A real connection to the exchange will be used to obtain initial data to initialize the strategy. In this mode, real orders are not sent to the exchange.

After ending of data set, common result wold be displayed:

Backtest candles *** 15m *** timeSeries ended
Backtest candles *** 1h *** timeSeries ended
Backtest candles *** 1m *** timeSeries ended
20:02:09.606230 Backtest *** ticker *** timeSeries ended
Original time: 0:08:56.241000, test time: 0:00:01.817368, x = 295.06
Session data saved to: /home/ubuntu/.MartinBinance/back_test/binance_ETHBUSD_0609-20:02:41
Session profit: 0E-7, free: 0.0183900, total: 0.01839
20:02:09.615905 Got signal for exit

And session data snapshot can be found at the specified location.

By changing the trading parameters, you can get a different result. It's not very productive, but it's quite entertaining. For efficient parameter matching, use Parameters optimization solution.

Visual comparison of session shots

Run martin_binance/backtest/VCoSEL.py. Select path for sessions snapshot for compare:

• snapshot of the original session back_test/binance_ETHBUSD/snapshot/
• snapshot of testing session back_test/binance_ETHBUSD_mmdd-HH:MM/

View result at http://127.0.0.1:8050/

Parameters optimization