Cryptocurrency Orderbook Analysis Tool

Ivan E. Perez
March 8, 2021
v1.0.2

Contents
1 Introduction 2

2 crobat Features 3
2.1 Orderbook updating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

3 Getting Started and Installation 4

3.1 CoPrA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

4 Time-series Data Structure Layout 5

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
4.2 Understanding The Raw Order Book Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
4.2.1 Order Book Snapshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
4.2.2 Signed Order Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.3 Final notes on Orderbook interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

5 Usage 9

6 Modules 11
6.1 recorder full . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
6.1.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
6.1.2 class L2 Update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
6.2 LOB funcs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
6.2.1 Description: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
6.2.2 class history() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
6.3 history funcs.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

7 Credits 39
7.1 Development Lead . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
7.2 Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

8 License 40

9 Appendix 41

10 To do list 41

1
1 INTRODUCTION 2

1 Introduction
This project is an extension of my thesis, A Study of CUSUM Statistics on Bitcoin Transactions, where
I was tasked with implementing CUSUM statistic processes to identify price actions periods in bitcoin
markets. After developing a tool for market orders, the natural extension was to find relationships from
activities in the limit order book. I started developing this tool to record instances of the limit order
book in order to record Limit Order insertions (LO), cancellations (CO), and Market Orders (MO).
As the project grew I wanted to make a tool that could be used by academics looking to apply and
develop market microstructure models in live markets. As a result, the styles in which the limit order
book and order book events are recorded are being developed in accordance to the conventions presented
in recent market microstructure papers correspond to the following papers:

1. Huang W., Lehalle C.A. and Rosenbaum M. - Simulating and analyzing order book data:The
queue-reactive model[1]
2. Cont R., Stoikov S. and Talreja R. - A stochastic model for order book dynamics[5]
3. Cont R., Kukanov A. and Stoikov S. - The price impact of order book events[4]

4. Cartea. A, Jaimungal S. and Wang Y. - Spoofing and Price Manipulation in Order Driven
Markets[2]
5. Silyantev, E. - Order flow analysis of cryptocurrency markets[6]1

1 This paper shows a working model implementing Order Flow Imbalance (OFI) and Trade Flow Imbalance(TFI) to

BTC-USD trades was done by Ed Silyantev. He developed a tool to assess OFI and TFI of XBT-USD pair.
2 CROBAT FEATURES 3

2 crobat Features
2.1 Orderbook updating
ˆ Order book recording module that maintains a time-series of:

1. Limit order insertions, cancellations and market orders,

2. volumes at nth best limits, and
3. prices at nth best limits.

ˆ Outputs these time-series objects as .xlsx files.

3 GETTING STARTED AND INSTALLATION 4

3 Getting Started and Installation

3.1 CoPrA
3.2 Installation
This project is a work in progress. I suggest you fork the project, and clone to your local environment.
The file csv out.py located in crobat/crobat/test/ will contain a basic testing environment for a set
length.
Note: crobat is not an optimized order book implementation and may not be I/O bound. This
suboptimality is due to frequent scanning of all prices for insertion forces the complexity to be greater
than ∼ O(n) for n being the length of the snapshot being iterated over. You may find that depending
on the popularity of the asset and the computational power of your PC, you may run into errors arising
from the computer not being able to keep up with the market (especially BTC-USD). I would suggest
experimenting with an unpopular pair, (e.g., XRP-USD), or a crypto-crypto pair (e.g., XRP-BTC), and
timing your queries outside of NYSE, and London Stock Exchange trading hours as they tend to have
less activity.
however if you want an easy installation:
“‘pip3 install crobat“‘
4 TIME-SERIES DATA STRUCTURE LAYOUT 5

4 Time-series Data Structure Layout

4.1 Introduction
Since this is an order book recorder my use until now has been to record the order book. However there
are accessors in the LOB funcs.py file, under in the history class. In the /test folder there is a small
use case called csv out test.py wiith the associated jupyter notebook pending.
1. For now we only have the full order book and we call thatrecorder full.py.
2. We change the settings variable in the CSV out test.py file that has arguments for:

Parameter Function Arg Type/Format Description

Recording Duration duration int recording time in seconds
Position Range position range int ordinal distance from the best bid(ask)
Currency Pair currency pair str The currency pair from list of supported pairs
2

3. When you are ready, you can start the build. When it finishes you should get a message Connection
Closed from CoPrA. And the files for the limit order book for each side should be created with a
timestamp in the form YYYY-MM-DDTHH:MM:SS.ffffff. Currently it will output everything given in
the table below, but I will update the function to only output select things for people.
Filename side Time Series Description
L2 orderbook events ask+timestamp ask Order book events on the ask side
L2 orderbook events bid+timestamp bid Order book events on the bid side
L2 orderbook events signed+timestamp both Order book events on both sides,
- sign for bid, +sign for ask
L2 orderbook ask volm+timestamp ask Volume snapshots of order book on the ask side
L2 orderbook bid volm+timestamp bid Volume snapshots of order book on the bid side
L2 orderbook signed volm+timestamp both Volume snapshots of the signed order book,
- sign for bid, + sign for ask
L2 orderbook ask volm+timestamp ask Price snapshots of order book on the ask side
L2 orderbook bid volm+timestamp bid Price snapshots of order book on the bid side
L2 orderbook signed volm+timestamp both Price snapshots of the signed order book,
- sign for bid, +sign for ask

4.2 Understanding The Raw Order Book Data

The Coinbase exchange operates using the double auction model, the Coinbase Pro API, and by extension
the CoPrA API makes it relatively easy to get still images of an instance of the orderbook as snapshots
and it sends updates in real time of the volume at a particular price level as l2 update messages. If you
would like to know more O’Hara’s book[3] does a great job introducing the double auction model for the
purposes of defining the types of orders, and how they record events and make sense of them.

4.2.1 Order Book Snapshots

Private data providers (e.g., NASDAQ - BookViewer,LOBSTER provide reconstructions of order books.
Coinbase updates the levels on the instance of a update message from the exchange of the order book.
The format of the order book snapshot at time t is a 2 × n array for n price levels, where the first element
is the price-level, and the second element is the volume at that price-level. The time series of order book
snapshots is also an array with dimensions 2 × m, for m recorded timestamps, where the first element is
the timestamp,t m, and the second element contains the entire 2 × n snapshot at tm .

2 List of currency pairs supported by Coinbase

4 TIME-SERIES DATA STRUCTURE LAYOUT 6

Event recording are a time series of MO, LO, CO’s as afforded from the l2 update messages which
are used to update the price, volume pair size at each price level. The format of the Event recorder is as
follows:

Item Description format

Timestamp Timestamp of when the event occurred YYYY-MM-DDTHH:MM:SS.ffffff
order type MO, LO, CO str ∈ {‘market’,‘limit’, ‘cancellation’}
price level price of event occurrence in quote currency float64
event size size of event in base currency float64
position signed position (– for bids, + for asks) int
mid price (best-ask + best-bid)/2 float64
spread best-ask + best-bid float64

4.2.2 Signed Order Book

4.2.2.1 Signed Order Book Snapshot Prices The signed orderbook takes a different approach
to position labelling so please keep that in mind. (note: I should shift the position index to start at 1,
for singe side order book snapshot time series). The signed orderbook snapshot is generated in a similar
fashion with a volume, and price at each position. However, it uses the convention established in [3] for
the signed order book. where positions on the bid are negative, with negative volume (XTC). I’ll show
the default setting that displays the 5 best bids and asks on each side.

Item Description/Format
Timestamp YYYY-MM-DDTHH:MM:SS.ffffff
−n price quote at the nth best bid (i.e., worst bid)
−n + 1 aggregate XTC limit buys at the second to worst bid
... ...
−2 aggregate XTC being bid for at the 2nd best bid
−1 aggregate XTC limit being bid for at the best bed bid
1 aggregate XTC offered at the best ask
2 aggregate XTC offered at the 2nd best ask
... ...
n−1 aggregate XTC offered at the second worst ask
n aggregate XTC offered at the worst ask

Similar to the single side implementation, there is an associated price quote (e.g., USD per XTC)
snapshot generated at each timepoint. The default format is given below:

Item Description/Format
Timestamp YYYY-MM-DDTHH:MM:SS.ffffff
−n price quote at the nth best bid (i.e., worst bid)
−n + 1 price quote at the second to worst bid
... ...
−2 price quote at the 2nd best bid
−1 price quote at the best bed bid
1 price quote at the best ask
2 price quote at the 2nd best ask
... ...
n−1 price quote at the second worst ask
n price quote at the worst ask

4.2.2.2 Signed Events

Signed event recordings follow the convention from The Price impact of Orderbook events, where positive
order flow is due to MO’s on the buy side, CO on the sell side, and LO on the buy side. Conversely,
negative order flow is due to MO’s on the sell side, CO on the buy side, and LO on the buy side. The
format is similar to the single side order book events time-series, but the order volume is signed based
on the aforementioned construction.
4 TIME-SERIES DATA STRUCTURE LAYOUT 7

Item Description format

Timestamp Timestamp of when the event occurred YYYY-MM-DDTHH:MM:SS.ffffff
order type MO, LO, CO str ∈ {‘market’,‘limit’, ‘cancellation’}
price level price of event occurrence in quote currency float64
event size size of event in base currency float64
position signed position (– for bids, + for asks) int
side bid/ask side where the event occurs str ∈ {‘buy’,‘sell’}
mid price (best-ask + best-bid)/2 float64
spread best-ask + best-bid float64
4 TIME-SERIES DATA STRUCTURE LAYOUT 8

4.3 Final notes on Orderbook interpretation

write about how the program object sees the orderbook with an example of code.

Start
 5 USAGE 9

5 Usage
Preface: crobat is used with an open websocket connection to the Coinbase exchange. It uses CoPrA
to parse JSON objects, and it records time series arrays of order book snapshots and events.

We demonstrate where crobat is used by beginning with CoPrA’s heartbeat.py example:

1 import asyncio
2
3 from copra . websocket import Channel , Client
4
5 loop = asyncio . get_ event_lo op ()
6
7 ws = Client ( loop , Channel ( ’ heartbeat ’ , ’BTC - USD ’) )
8
9 try :
10 loop . run_forever ()
11 except K e y b o a r d I n t e r r u p t :
12 loop . r u n _ u n t i l _ c o m p l e t e ( ws . close () )
13 loop . close ()

If we were to execute the code, we would run methods in the instance of the class Client based
on the arrival of messages on the heartbeat channel for the pair BTC-USD. crobat primarily inherits
methods from this class and adds logic for managing incoming messages. Below we introduce our own
version of this loop found in CSV out.py.
1 import recorder_full as rec
2 import asyncio , time
3 from datetime import datetime
4 import copra . rest
5 from copra . websocket import Channel , Client
6
7 class input_args ( object ) :
8 def __init__ ( self , currency_pair = ’ETH - USD ’ , positi on_rang e =5 , r e c o r d i n g _ d u r a t i o n =5) :
9 self . currency_pair = currency_pair
10 self . pos ition_r ange = p osition _range
11 self . r e c o r d i n g _ d u r a t i o n = r e c o r d i n g _ d u r a t i o n
12
13 def main () :
14 settings_1 = input_args ()
15
16 loop = asyncio . get_e vent_lo op ()
17
18 channel1 = Channel ( ’ level2 ’ , settings_1 . currency_pair )
19 channel2 = Channel ( ’ ticker ’ , settings_1 . currency_pair )
20
21 ws_1 = rec . L2_Update ( loop , channel1 , settings_1 )
22 ws_1 . subscribe ( channel2 )
23
24 timestart = datetime . utcnow ()
25
26 try :
27 loop . run_forever ()
28 except K e y b o a r d I n t e r r u p t :
29 loop . r u n _ u n t i l _ c o m p l e t e ( ws_1 . close () )
30 loop . close ()
31
32 if __name__ == ’ __main__ ’:
33 main ()

In this modified event loop we have the context where the bulk of crobat sits, the class l2 update.
we can the loop object that is passed to ws 1, the instance of the class L2 Update, receives msg objects
through the async methods inherited from Client. Here we give an example of what l2 Update looks
like:
 5 USAGE 10

1 import asyncio , time

2 from datetime import datetime
3 import copra . rest
4 from copra . websocket import Channel , Client
5
6 class L2_Update ( Client ) :
7 def __init__ ( self , loop , channel , input_args ) :
8 self . time_now = datetime . utcnow () # initial start time
9 self . po sition_r ange = input_args . posit ion_rang e
10 self . r e c or d i n g _ d u r a t i o n = input_args . r e c o r d i n g _ d u r a t i o n
11 self . snap_received = False
12 super () . __init__ ( loop , channel )
13
14 def on_open ( self ) :
15 print ( " Let ’s count the L2 messages ! " , self . time_now )
16 super () . on_open () # inheriting things from the parent class who really knows
17
18 def on_message ( self , msg ) :
19 if msg [ ’ type ’] in [ ’ snapshot ’ ]:
20 print ( " received the snapshot " )
21 time = self . time_now
22 self . snap_received = True
23 if msg [ ’ type ’] in [ ’ ticker ’ ]:
24 print ( " received ticker message " )
25 if self . snap_received :
26 # Do Something Here
27 else :
28 print ( " market order arrived but no snapshot received yet " )
29
30 if msg [ ’ type ’] in [ ’ l2update ’ ]:
31 print ( " received an l2update message " )
32 else :
33 print ( " unknown message " )
34
35 if ( datetime . utcnow () - self . time_now ) . total_seconds () > self . r e c o r d i n g _ d u r a t i o n :
36 self . loop . create_task ( self . close () )
37
38 def on_close ( self , was_clean , code , reason ) :
39 print ( " Connection to server is closed " )
40 print ( was_clean )
41 print ( code )
42 print ( reason )
43
44 def main () :
45 pass
46
47 if __name__ == ’ __main__ ’:
48 main ()

Here we can see that we can create an instance of this class, and on its initialization we create arrays
that can hold our changes and order book states. The logic tree for when each type of message arrives
can dictate what kinds of records, and changes to the snapshot are made the arrival of a message. In
this current version, the history module contains methods to both update the order book and record
changes, but in future versions the processes should be separated as other may have better ideas as to
what to do with order book changes. In the following section we will introduce the modules and their
associated functions and classes.
6 MODULES 11

6 Modules
Introduction: corbat has one main module, recorder full typically imported as rec. Its dependencies
are:
1. asyncio, time for asynchronous methods,
2. datetime, to work with datetime objects,
3. copra.websocket, to connect to coinbase,
4. pandas, to store out time series arrays,
5. LOB funcs.py, to manage messages and create time series arrays,
6. history funcs.py, for some helper functions, and
7. gc and numpy.
The modules, and their dependencies are listed in the tree diagram below.

CSV out test.py

recorder full.py
asyncio, time
dateime
copra.websocket
pandas(note can move this later)
LOB funcs.py
pandas
copy
bisect
numpy
history funcs.py
history funcs.py
pandas
bisect
gc
numpy
datetime
copra.rest
copra.websocket

6.1 recorder full

6.1.1 Description
The recorder full module contains the class L2 Update. L2 Update is responsible for:
ˆ initializing instances of the order book history class, history.
ˆ interpreting the snapshot, ticker and l2update messages that come from the websocket.
ˆ calling the appropriate functions and classes to carry out the orderly update to the limit order
book, and order book history arrays.

6.1.2 class L2 Update

Description: The class that handles messages incoming from the websocket connection. Its methods
are:
ˆ init (self, loop, channel, input args)
ˆ on open(self)
ˆ on message(self, msg)
ˆ on close(self, was clean, code, reason)
init (self, loop, channel, input args) begins by inheriting attributes loop, and channel,
from Client, and it
1. initializes the class history imported from LOB funcs and
2. passes on the settings from the class input args and
3. uses functions on message(self, msg),
4. on close(self, was clean, code, reason) to manage incoming messages.
6 MODULES 12

6.1.2.1 function init (self, loop, channel, input args)

Description: Initializes the class, using the attributes loop, channel from Client and attributes
position range, recording duration from the class input args. It also creates an instance of the
class history.
parameters: loop: object
Comes from CoPrA
channel: object
comes from CoPrA
input args: class
passes on arguments for recording duration, and position range
returns: None
Example: None
6 MODULES 13

6.1.2.2 function on open(self)

Description: From class Client uses method on open() to set things up. See ACoPrA on open() for
more information.
restated from CoPrA:
on open is called as soon as the initial WebSocket opening handshake is complete. The connection is
open, but the client is not yet subscribed.
If you override this method it is important that you still call it from your subclass’ on open method,
since the parent method sends the initial subscription request to the WebSocket server. Somewhere in
your on open method you should have super().on open().
In addition to sending the subscription request, this method also logs that the connection was opened.
parameters: None: None
returns: None: None
Example: None
 6 MODULES 14

6.1.2.3 function on message(self, msg),

Description: After matching msg[’type’] to ’snapshot’,’ticker’, ’l2update’ do one of three
actions,
msg[‘type’] == action
‘snapshot’ initialize the limit order book using the initialize snap events
‘ticker’ parse a market order,
‘l2update’ parse a limit order insertion or cancellation to the limit order book.
restated from CoPrA:
on messageon is called every time a message is received. message is a dict representing the message. Its
content will depend on the type of message, the channels subscribed to, etc. Please read Coinbase Pro’s
WebSocket API documentation to learn about these message formats.
Note that with the exception of errors, every other message triggers this method including things like
subscription confirmations. Your code should be prepared to handle unexpected messages.
This default method just prints the message received. If you override this method, there is no need to
call the parent method from your subclass’ method.
parameters: msg: dict
returns: None: None
Example:
1 # Let our module be imported and our channels be assigned as follows :
2 import recorder_full as rec
3 channel1 = Channel ( ’ level2 ’ , ’BTC - USD ’)
4 channel2 = Channel ( ’ ticker ’ , ’BTC - USD ’)
5
6
7 # Let the instance of the class L2_Update be ws_1
8 ws_1 = rec . L2_Update ( loop , channel1 , settings_1 )
9 ws_1 . subscribe ( channel2 )
10
11 # Let the message received from the websocket be :
12 msg = {
13 " type " : " l2update " ,
14 " product_id " : " BTC - USD " ,
15 " time " : " 2019 -08 -14 T20 :42:27.265 Z " ,
16 " changes " : [
17 [
18 " buy " ,
19 " 10 101.8000 0000 " ,
20 " 0.162567 "
21 ]
22 ]
23 }
24
25 # let the current state of the orderbook be defined as follows :
26 ws_1 . o r d e r b o o k _ i n s t a n c e . snapshot_bid = [
27 [10101.00 , 5.23] , [10101.50 , 1.11] , [10101.80 , 0.5]
28 ]
29
30 ws_1 . o r d e r b o o k _ i n s t a n c e . sn a ps ho t_ s ig ne d = [
31 [10101.00 , -5.23] , [10101.50 , -1.11] , [10101.80 , -0.5] ,
32 [10101.90 , 0.4] , [10102.00 , 1.3] , [10102.10 , 5.00]
33 ]
34
35 ws_1 . o r d e r b o o k _ i n s t a n c e . bid_events = []
36 ws_1 . o r d e r b o o k _ i n s t a n c e . signed_events = []
37 # Suppose on loop you this message was passed to on_message ( msg )
38 on_message ) ( msg )
39
40 # the new values for the the snapshots , and events would be :
41 >> print ( ws_1 . o r d e r b o o k _ i n s t a n c e . snapshot_bid )
42 > >[[10101.00 , 5.23] , [10101.50 , 1.11] , [10101.80 , 0.162567]]
43
44 >> print ( ws_1 . o r d e r b o o k _ i n s t a n c e . sn ap sh o t_ si gn e d )
45 > >[[10101.00 , -5.23] , [10101.50 , -1.11] , [10101.80 , -0.162567] , [10101.90 , 0.4] ,
[10102.00 , 1.3] , [10102.10 , 5.00]]
46
47 >> print ( ws_1 . o r d e r b o o k _ i n s t a n c e . bid_events )
48 > >[[2019 -08 -14 T20 :42:27.265 Z , 10101.80 , cancellation , 0.337433 , 1 , 10101.85 , 0.10]]
49
50 >> print ( ws_1 . o r d e r b o o k _ i n s t a n c e . signed_events )
51 > >[[2019 -08 -14 T20 :42:27.265 Z , 10101.80 , cancellation , 0.337433 , -1 , 10101.85 , 0.10]]
 6 MODULES 15

6.1.2.4 function on close(self, was clean, code, reason)

description: The function called when the close() task is executed. Begins the post processing of
accumulated time-series using the functions:

ˆ hf.convert array to list dict

ˆ hf.convert array to list dict sob

ˆ hf.pd excel save

and gc.collect() to clear memory after storing .xlsx files of the processed data.
restated from CoPrA:
on close is called whenever the connection between the client and server is closed. was clean is a
boolean indicating whether or not the connection was cleanly closed. code, an integer, and reason, a
string, are sent by the end that initiated closing the connection.
If the client did not initiate this closure and client.auto reconnect is set to True, the client will attempt
to reconnect to the server and resubscribe to the channels it was subscribed to when the connection was
closed. This method also logs the closure.
If your subclass overrides this method, it is important that the subclass method calls the parent
method if you want to preserve the auto reconnect functionality. This can be done by including
super().on close(was clean, code, reason) in your subclass method.
parameters: was clean: bool
code: int or None
reason: str or None
returns: None: None
52 # Needs an Example
Listing 1: Python example
6 MODULES 16

6.2 LOB funcs

6.2.1 Description:
Contains the class history and the methods, UpdateSnapshot Bid Seq and UpdateSnapshot Ask Seq
that outline how to update snapshots.

6.2.2 class history()

Description: Contains attributes for the time series array of the limit order book, the current snapshot
of the limit order book and the time-series of the events in the limit order book. We detail the role
of each attribute in the table below. The class contains methods that operate on these attributes,
in order to update the order book and record changes. The functions UpdateSnapshot bid Seq and
UpdateSnapshot ask Seqfound outside of these methods, dictate the sequence in which the methods
contained in history are executed.

6.2.2.1 function init (self)

Description: Initializes the lists and variables that will be operated on by other class methods. Each
side, bid, ask and the combined signed has a list demarked by history, that will be populated by
timestamped states of the limit order book. The lists prefixed by snapshot are populated by the state
of limit order book demarked by the suffix bid, ask, and signed. The variables order type, position
and event size denote the type, position and size of the event based on the change in the order book
as a result of an l2 update message. The token variable determines whether the change in the order
book is recorded in the events list.
parameters: None: None
returns: None: None
Example: None
 6 MODULES 17

6.2.2.2 function initialize snap events(self, msg, time)

Description: Method that parses the msg payload if msg[‘type’] == snapshot . Adds the first entry
to the collection of history variables. Uses np.round and hf.min dec to calculate the smallest amount
of the quote currency is needed to make the ticks 0.01 units of the base currency.3
function dependency:
ˆ np.round

ˆ hf.min dec
parameters: msg: dict
time datetime object
returns: None: None
Example:
53 # let the instance of our L2_Update class be defined as follows :
54 channel1 = Channel ( ’ level2 ’ , ’BTC - USD ’)
55 ws_1 = L2_Update ( loop , channel1 , settings_1 )
56
57 # from our __init__ method , we initialized the arrays in history :
58 >> print ( vars ( ws_1 . hist ) )
59 > >{ bid_history :[] , ask_history :[] , s igned_h istory :[] , snpashot_bid :[] , snapshot_ask :[] ,
s na ps ho t _s ig ne d :[] , bid_events :[] , ask_events :[] , signed_events :[] , order_type : None ,
token : False , position :0 , event_size :0}
60
61 # after receiving the snapshot from our event loop we will run i n i t i a l i z e _ s n a p _ e v e n t s
62 # to populate the variables created in __init__ .
63
64 # Let our message be
65 ws_1 . time_now = " 2019 -08 -14 T20 :42:27.265 Z "
66 msg = {
67 " type " : " snapshot " ,
68 " product_id " : " BTC - USD " ,
69 " bids " : [[ " 10101.10 " , " 0.450541 " ] ,[ " 10101.20 " , " 0.44100 " ] , [ " 10101.55 " , " 0.013400 " ]] ,
70 " asks " : [[ " 10102.55 " , " 0.577535 " ] ,[ " 10102.58 " , " 0.63219 " ] , [ " 10102.60 " , " 0.803200 " ]]
71 }
72
73 # note there is not a timestamp for the snapshot that we receive from coinbase so it is
passed from the datetime . utcnow () from the init method of the L2_Update class .
74
75 ws_1 . hist . i n i t i a l i z e _ s n a p e v e n t s ( msg , ws_1 . time_now )
76
77 >> print ( vars ( ws_1 . hist ) )
78 > >{ bid_history :[[2019 -08 -14 T20 :42:27.265 Z , [[10101.10 , 0.450541] , [10101.20 , 0.44100] ,
[10101.55 , 0.013400]]]] ,
79 ask_history :[[2019 -08 -14 T20 :42:27.265 Z , [[10102.55 , 0.577535] , [10102.58 , 0.63219] ,
[10102.60 , 0.803200]]]] ,
80 sign ed_histo ry :[2019 -08 -14 T20 :42:27.265 Z , [[[10101.10 , -0.450541] , [10101.20 ,
-0.44100] , [10101.55 , -0.013400] , [10102.55 , 0.577535] , [10102.58 , 0.63219] ,
[10102.60 , 0.803200]]]] ,
81 snapshot_bid :[[10101.10 , 0.450541] ,[10101.20 , 0.44100] , [10101.55 , 0.013400]] ,
82 snapshot_ask :[[10102.55 , 0.577535] ,[10102.58 , 0.63219] , [10102.60 , 0.803200]] ,
83 s na ps ho t _s ig ne d :[[10101.10 , -0.450541] ,[10101.20 , -0.44100] , [10101.55 , -0.013400] ,
[10102.55 , 0.577535] ,[10102.58 , 0.63219] , [10102.60 , 0.803200]] ,
84 bid_events :[] , ask_events :[] , signed_events :[] , order_type : None , token : False , position
:0 , event_size :0}

3 For example if BTC-USD = 10101 USD/BTC, then min dec would return the number of BTC needed for 0.01 USD in

BTC and np.round truncates the volume returns to the decimal places returned by hf.min dec. Doing so resolves the issue
with artifacts from floating point arithmetic.
 6 MODULES 18

6.2.2.3 add market order message

Description: Parses a message object, aggregates for duplicate timestamps, and appends them to the
events lists.
parameters: message: array
events array
returns: events: array
Example:
1 # let our events be take the form
2 # some_event = [ time , order_type , float ( msg [ ’ price ’]) , size , position , mid_price , spread
].
3 # for 4 events we have
4
5 bid_events = [
6 [2019 -08 -14 T20 :42:27.265 Z , " market " , 10101.80 , 0.123030 , 0 , 10101.85 , 0.10] ,
7 [2019 -08 -14 T20 :42:27.560 Z , " insertion " , 10100.00 , 0.123030 , 2 , 10101.85 , 0.10] ,
8 [2019 -08 -14 T20 :42:28.123 Z , " insertion " , 10100.00 , 0.256000 , 2 , 10101.85 , 0.10] ,
9 [2019 -08 -14 T20 :42:27.560 Z , " cancellation " , 10100.00 , 0.256000 , 2 , 10101.85 , 0.10]
10 ]
11
12 # we receive a ’ ticker ’ message from our event loop
13
14 msg = {
15 " type " : " ticker " ,
16 " trade_id " : 20153558 ,
17 " sequence " : 3262786978 ,
18 " time " : " 2019 -08 -14 T20 :42:27.966 Z " ,
19 " product_id " : " BTC - USD " ,
20 " price " : " 10101.8000000 " ,
21 " side " : " sell " , // Taker side
22 " last_size " : " 0.03000000 " ,
23 " best_bid " : " 10101.8000000 " ,
24 " best_ask " : " 10101.9000000 " ,
25 }
26
27 # on message conventiently converts the message into an array that can be used by
add_market_order_message
28
29 message = [2019 -08 -14 T20 :42:27.966 Z , " market " , 10101.80 , 0.030000 , 0 , 10101.85 , 0.10]
30
31 bid_events = a d d _ m a r k e t _ o r d e r _ m e s s a g e ( message , bid_events )
32
33 # we would see that the event would be appended to the list of bid events
34 >> print ( bid_events )
35 > >[[2019 -08 -14 T20 :42:27.265 Z , " market " , 10101.80 , 0.123030 , 0 , 10101.85 , 0.10] ,
36 [2019 -08 -14 T20 :42:27.560 Z , " insertion " , 10100.00 , 0.123030 , 2 , 10101.85 , 0.10] ,
37 [2019 -08 -14 T20 :42:28.123 Z , " insertion " , 10100.00 , 0.256000 , 2 , 10101.85 , 0.10] ,
38 [2019 -08 -14 T20 :42:27.560 Z , " cancellation " , 10100.00 , 0.256000 , 2 , 10101.85 , 0.10] ,
39 [2019 -08 -14 T20 :42:27.966 Z , " market " , 10101.80 , 0.030000 , 0 , 10101.85 , 0.10]] # new msg !

Example 2: When two ticker messages arrive at the same time

1 # starting from our last entry , lets examine what would happen if another ticker message
containing a market order with an identical timestamp were to be received from the
event loop .
2 bid_events = [
3 [2019 -08 -14 T20 :42:27.265 Z , " market " , 10101.80 , 0.123030 , 0 , 10101.85 , 0.10] ,
4 [2019 -08 -14 T20 :42:27.560 Z , " insertion " , 10100.00 , 0.123030 , 2 , 10101.85 , 0.10] ,
5 [2019 -08 -14 T20 :42:28.123 Z , " insertion " , 10100.00 , 0.256000 , 2 , 10101.85 , 0.10] ,
6 [2019 -08 -14 T20 :42:27.560 Z , " cancellation " , 10100.00 , 0.256000 , 2 , 10101.85 , 0.10] ,
7 [2019 -08 -14 T20 :42:27.966 Z , " market " , 10101.80 , 0.030000 , 0 , 10101.85 , 0.10]
8 ]
9
10 # we receive a new message
11 msg = {
12 " type " : " ticker " ,
13 " trade_id " : 20153559 ,
14 " sequence " : 3262786979 ,
15 " time " : " 2019 -08 -14 T20 :42:27.966 Z " ,
16 " product_id " : " BTC - USD " ,
17 " price " : " 10101.8000000 " ,
18 " side " : " sell " , // Taker side
19 " last_size " : " 0.15000000 " ,
20 " best_bid " : " 10101.8000000 " ,
21 " best_ask " : " 10101.9000000 " ,
22 }
23
 6 MODULES 19

24 # converting with on_message

25 message = [2019 -08 -14 T20 :42:27.966 Z , " market " , 10101.80 , 0.150000 , 0 , 10101.85 , 0.10]
26
27 # running bid events again
28 bid_events = a d d _ m a r k e t _ o r d e r _ m e s s a g e ( message , bid_events )
29
30 # we would observe a concatenated form
31 >> print ( bid_events )
32 > >[[2019 -08 -14 T20 :42:27.265 Z , " market " , 10101.80 , 0.123030 , 0 , 10101.85 , 0.10] ,
33 [2019 -08 -14 T20 :42:27.560 Z , " insertion " , 10100.00 , 0.123030 , 2 , 10101.85 , 0.10] ,
34 [2019 -08 -14 T20 :42:28.123 Z , " insertion " , 10100.00 , 0.256000 , 2 , 10101.85 , 0.10] ,
35 [2019 -08 -14 T20 :42:27.560 Z , " cancellation " , 10100.00 , 0.256000 , 2 , 10101.85 , 0.10] ,
36 [2019 -08 -14 T20 :42:27.966 Z , " market " , 10101.80 , 0.180000 , 0 , 10101.85 , 0.10]]
 6 MODULES 20

6.2.2.4 function remove price level(snap array, level depth, match index)
Description: Checks the level depth of an l2 update message, removes the existing price level from
the snapshot array if the level depth is 0. Counts as a change to the limit order book therefore self.token
is set to True.
parameters: snap array array
level depth float64
match index int
returns: snap array: array
Example:
1 # let our snapshot_bid be :
2 snapshot_bid = [[10101.80 , 0.013400] ,[10101.20 , 0.44100] , [10101.10 , 0.450541]]
3
4 # we receive a ’ l2update ’ message from our event loop
5
6 msg = {
7 " type " : " l2update " ,
8 " product_id " : " BTC - USD " ,
9 " time " : " 2019 -08 -14 T20 :42:27.265 Z " ,
10 " changes " : [
11 [
12 " buy " ,
13 " 10 101.8000 0000 " ,
14 " 0.0 "
15 ]
16 ]
17 }
18
19 # on_message () conventiently parses the message to create relevant local variables
20 # time , changes , side , price_level , level_depth , pre_level_depth , hist . token
21 >> print ( time , side , price_level , level_depth , prelevel_depth , hist . token )
22 > >2019 -08 -14 T20 :42:27.265 Z " buy " 10101. 80000000 0.0 0 False
23
24 # in on_message () , the variable p r i c e _ m a t c h _ i n d e x is assigned by finding the index
25 # with the matching price , in the snapshot . In this case :
26 p r i c e _ m a t c h _ i n d e x = [0]
27
28 snapshot_bid = r e m o v e _ p r i c e _ l e v e l ( snapshot_bid , level_depth , p r i c e _ m a t c h _ i n d e x )
29
30 >> print ( snapshot_bid , hist . token )
31 > >[[10101.20 , 0.44100] , [10101.10 , 0.450541]] True
 6 MODULES 21

6.2.2.5 function update level depth(snap array, level depth, match index, pre level depth)

Descirption: updates the existing price level to a new size. Sets self.order type to insertion if
the new size is larger than the old size, or cancellation if the new size is smaller than the old size. sets
the position to the self.position variable to the index where the event occurred.
parameters: snap array: list
The snapshot array that will be modified.
level depth: float64
The new level depth that will be checked against the old level
depth in the snapshot array.
match index: list
single item list that contains the index where the price in the
snapshot array matches that of the l2update message.
returns: snap array : list
The modified snapshot array.
pre level depth: float64
An updated pre level depth4
Example:
1 # let our snapshot_bid be :
2 snapshot_bid = [[10101.80 , 0.013400] ,[10101.20 , 0.44100] , [10101.10 , 0.450541]]
3
4 # we receive a ’ l2update ’ message from our event loop
5
6 msg = {
7 " type " : " l2update " ,
8 " product_id " : " BTC - USD " ,
9 " time " : " 2019 -08 -14 T20 :42:27.265 Z " ,
10 " changes " : [
11 [
12 " buy " ,
13 " 10 101.8000 0000 " ,
14 " 0.500000 "
15 ]
16 ]
17 }
18
19 # on_message () conventiently parses the message to create relevant local variables
20 # time , changes , side , price_level , level_depth , pre_level_depth , hist . token
21 >> print ( time , side , price_level , level_depth , prelevel_depth , hist . token )
22 > >2019 -08 -14 T20 :42:27.265 Z " buy " 10101. 80000000 0.500000 0 False
23
24 # in on_message () , the variable p r i c e _ m a t c h _ i n d e x is assigned by finding the index
25 # with the matching price , in the snapshot . In this case :
26 p r i c e _ m a t c h _ i n d e x = [0]
27
28 snapshot_bid , pr e _l ev el _ de pt h = u p d a t e d a t e _ p r i c e _ l e v e l ( snapshot_bid , level_depth ,
price_match_index , pr e _l ev el _ de pt h )
29
30 >> print ( price )
31
32 >> print ( snapshot_bid , hist . token )
33 > >[[10101.20 , 0.44100] , [10101.10 , 0.450541]] True
 6 MODULES 22

6.2.2.6 function update price index buy(self, level depth, price level, pre level depth)

Description: If there is a new price level introduced, this function determines its location and appropriately
inserts it into the snapshot bid array. It will also set the position, and token depending on where the
change occurs.
parameters: level depth: float64
level depth received from the l2update message
price level: float64
The price level being introduced
pre level depth: float64
some pre level depth that is reset to 0, idk this feature
returns: snapshot bid list
The modified snapshot bid.
bid range: list
The modified range of bids available in the snapshot.
pre level depth float64
again the pre level depth that idk why i still have this.
latent changes: token boolean
bool that informs whether a significant change has occurred in the order book.
order type
Example: None
1 # let our snapshot_bid be :
2 snapshot_bid = [[10101.80 , 0.013400] ,[10101.20 , 0.44100] , [10101.10 , 0.450541]]
3
4 # we receive a ’ l2update ’ message from our event loop
5
6 msg = {
7 " type " : " l2update " ,
8 " product_id " : " BTC - USD " ,
9 " time " : " 2019 -08 -14 T20 :42:27.265 Z " ,
10 " changes " : [
11 [
12 " buy " ,
13 " 10 101.9000 0000 " ,
14 " 0.500000 "
15 ]
16 ]
17 }
18
19 # on_message () conventiently parses the message to create relevant local variables
20 # time , changes , side , price_level , level_depth , pre_level_depth , hist . token
21 >> print ( time , side , price_level , level_depth , prelevel_depth , hist . token )
22 > >2019 -08 -14 T20 :42:27.265 Z " buy " 10101. 90000000 0.500000 0 False
23
24 # in on_message () , the variable p r i c e _ m a t c h _ i n d e x is assigned by finding the index
25 # with the matching price , in the snapshot . In this case :
26 p r i c e _ m a t c h _ i n d e x = []
27
28 snapshot_bid , pr e _l ev el _ de pt h = u p d a t e _ p r i c e _ i n d e x _ b u y ( level_depth , price_level ,
p re _l ev e l_ de pt h )
29
30 >> print ( snapshot_bid , hist . token )
31 >[[10101.90 , 0.500000] , [10101.80 , 0.013400] ,[10101.20 , 0.44100] , [10101.10 ,
0.450541]] True
 6 MODULES 23

6.2.2.7 function update price index sell(self, level depth, price level, pre level depth)

Description: If there is a new price level introduced, this function determines its location and appropriately
inserts it into the snapshot ask array. It will also set the position, and token depending on where the
change occurs.
parameters: level depth: float64
level depth received from the l2update message
price level: float64
The price level being introduced
pre level depth: float64
some pre level depth that is reset to 0, idk this feature
returns: snapshot ask: list
The modified snapshot ask.
bid range: list
The modified range of bids available in the snapshot.
pre level depth float64
again the pre level depth that idk why i still have this.
latent changes: token: boolean
bool that informs whether a significant change has occurred in the order book.
order type: string
type of order in [‘market’, ‘insertion’, ‘cancelation’]
Example:
1 # let our snapshot_bid be :
2 snapshot_ask = [[10101.90 , 0.013400] ,[10102.00 , 0.52100] , [10102.11 , 0.89041]]
3
4 # we receive a ’ l2update ’ message from our event loop
5
6 msg = {
7 " type " : " l2update " ,
8 " product_id " : " BTC - USD " ,
9 " time " : " 2019 -08 -14 T20 :42:27.265 Z " ,
10 " changes " : [
11 [
12 " sell " ,
13 " 10 101.9500 0000 " ,
14 " 0.500000 "
15 ]
16 ]
17 }
18
19 # on_message () conventiently parses the message to create relevant local variables
20 # time , changes , side , price_level , level_depth , pre_level_depth , hist . token
21 >> print ( time , side , price_level , level_depth , prelevel_depth , hist . token )
22 > >2019 -08 -14 T20 :42:27.265 Z " buy " 10101. 90000000 0.500000 0 False
23
24 # in on_message () , the variable p r i c e _ m a t c h _ i n d e x is assigned by finding the index
25 # with the matching price , in the snapshot . In this case :
26 p r i c e _ m a t c h _ i n d e x = []
27
28 snapshot_ask , pr e _l ev el _ de pt h = u p d a t e _ p r i c e _ i n d e x _ s e l l ( level_depth , price_level ,
p re _l ev e l_ de pt h )
29
30 >> print ( snapshot_bid , hist . token )
31 > [[10101.90 , 0.013400] ,[10101.95 , 0.500000] ,[10102.00 , 0.52100] , [10102.11 , 0.89041]]
True
 6 MODULES 24

6.2.2.8 function update snapshot bid(self)

Description: Updates the variable bid range. I’m not sure why I need to do this but it does keep
update snapshot bid/ask from making mistakes when doing bisect/insert.
parameters: None: None
None
returns: snapshot bid: list
The snapshot on the bid side.
bid range: list
ordered range of prices for the bid side
latent changes: None None
None
Example: None
1 x = _ _m y_ fu n ct io n_ _ (1.234 , True , 3)
2 >> print ( x )
3 > > -1.23
4
5 x = _ _m y_ fu n ct io n_ _ (1.234 , False , 3)
6 >> print ( x )
7 > >1.23
 6 MODULES 25

6.2.2.9 function update snapshot ask(self)

Description: Updates the variable ask range. I’m not sure why I need to do this but it does keep
update snapshot bid/ask from making mistakes when doing bisect/insert.
parameters: None: None
None
returns: snapshot ask: list
The snapshot on the ask side.
ask range: list
ordered range of prices for the ask side
latent changes: None None
None
Example: None
1 ?????????
 6 MODULES 26

6.2.2.10 function trim coordinator(self, position, bound)

Description: Sets the tokenvariable to False if the change to the orderbook was outside of thte position
bound.
parameters: position: int
position where the order book was updated.
bound: int
the largest position where the function can trigger
returns: token boolean
The boolean argument that determines whether the change in the order book is recorded.
latent changes: None None.
Example: None
1 ???????
 6 MODULES 27

6.2.2.11 function append snapshot bid(self, time, price level)

Description: Appends the current form of the snapshot to bid history and computes the event and
appends the event record to bid events.
parameters: : float64
some input that will be passed to the function
param 2: boolean
a conditional that will be checked by the function
param 3: int
some int that is needed for the function
returns: trans number float64
The number with the correct significant digits and sign.
latent changes: trans number float64
The number with the correct significant digits and sign.
Example: None
1 x = _ _m y_ fu n ct io n_ _ (1.234 , True , 3)
2 >> print ( x )
3 > > -1.23
4
5 x = _ _m y_ fu n ct io n_ _ (1.234 , False , 3)
6 >> print ( x )
7 > >1.23
 6 MODULES 28

6.2.2.12 function append snapshot ask(self, time, price level)

Description: Appends the current form of the snapshot to ask history and computes the event and
appends the event record to ask events.
parameters: time: datetime object
timestamp that when the message was received
price level: float64
price level where the change occurred
returns: snapshot ask list
The snapshot on the ask side.
ask range list
the range of prices in snapshot ask.
latent changes: None: None
Example: None
1 x = _ _m y_ fu n ct io n_ _ (1.234 , True , 3)
2 >> print ( x )
3 > > -1.23
4
5 x = _ _m y_ fu n ct io n_ _ (1.234 , False , 3)
6 >> print ( x )
7 > >1.23
 6 MODULES 29

6.2.2.13 function append signed book(self, time, price level)

Description: Appends the current form of the snapshot to ask history and computes the event and
appends the event record to ask events.
parameters: time: datetime object
timestamp that when the message was received
price level: float64
price level where the change occurred
returns: snapshot ask list
The snapshot on the ask side.
ask range list
the range of prices in snapshot ask.
latent changes: None: None
Example: None
1 x = _ _m y_ fu n ct io n_ _ (1.234 , True , 3)
2 >> print ( x )
3 > > -1.23
4
5 x = _ _m y_ fu n ct io n_ _ (1.234 , False , 3)
6 >> print ( x )
7 > >1.23
 6 MODULES 30

6.2.2.14 function chk mkt can overlap(self, events, order type)

Description: passes the order type either market order or canceled order. If it was a market order
checks for a recent canceled order of the same size and delectes the canceled order. likewise, if the
ordertype passedwas a canceled order it looks for a recent market order and deletes the canceled order.
The reason this is done this way is because the l2update channel only updates queue sizes while the
ticker channel only announces market order arrivals.
parameters: events: list
current event list to be modified
order type: string
type of order observed [’market’,’cancellation’]
returns: events list
modified event list
latent changes: None: None
Example: None
1 x = _ _m y_ fu n ct io n_ _ (1.234 , True , 3)
2 >> print ( x )
3 > > -1.23
4
5 x = _ _m y_ fu n ct io n_ _ (1.234 , False , 3)
6 >> print ( x )
7 > >1.23
 6 MODULES 31

Accessors: The following functions, prefaced with last are the currently named accessors. They cover
the last order of a named type, the last state of the order book, and the market depth for a the last
given state of the order book. If I come up with other relevant accessors or refine how each function
works their changes will likely be noted in the versions, or at least a git entry.

6.2.2.15 function last inserted order(self, side=‘‘signed’’)

Description: Returns the latest limit order insertion from the bid, ask, or signed events. Defaults to
signed.
parameters: side: string
market side to query
returns: out(please rename) list
The last inserted event.
latent changes: None: None
Example: None
1 x = _ _m y_ fu n ct io n_ _ (1.234 , True , 3)
2 >> print ( x )
3 > > -1.23
4
5 x = _ _m y_ fu n ct io n_ _ (1.234 , False , 3)
6 >> print ( x )
7 > >1.23
 6 MODULES 32

6.3 history funcs.py

Description:Helper functions that operate in recorder full.py and LOBfuncs.py. It contains simple
methods that are agnostic to side (e.g., bid/ask/signed) for the most part.

6.3.0.1 function check order(snapshot, side)

Description: Checks that the snapshot is sorted correctly. Used in debugging when snapshots do not
look well ordered. Only checks bid and ask side, because signed snapshot is a concatenated form of a
bid and ask snapshot. pass placeholder can be changed to a print statement.
parameters: snapshot: list
snpshot to check
side: string
side bid, ask
returns: snapshot list

latent changes: None: None

Example: None
1 # let our snapshot_bid be :
2 snapshot_bid = [[10101.80 , 0.013400] ,[10101.20 , 0.44100] , [10101.10 , 0.450541]]
3
4 # I will say that if the snapshot is not ordered I can ask the function to return a
boolean , True if ordered , else False
5 >>x = check order ( snapshot_bid , " bid " )
6 >> print ( x )
 6 MODULES 33

6.3.0.2 function conver array to list dict(history, pos limt=5)

Description: converts the time-series of order book states into a list of dictionaries for a single side.
parameters: history: list
time series of order book states
pos limit: int, default := 5
ordinal distance from the best bid/ask that is worth saving.
returns: volm list: list
list of dictionaries for volume sizes and their respective position.
price list: list
list of dictionaries for prices and their respective position.
latent changes: None: None
Example: None
1 # let our snapshot_bid be :
2 bid_history = [
3 [2019 -08 -14 T20 :42:27.265 Z ,[[10101.80 , 0.013400] ,[10101.20 , 0.44100] , [10101.10 ,
0.450541] , [10100.55 , 5.24501] , [10099.00 , 10.24511] , [10090.11 , 24.21395]]] ,
4 [2019 -08 -14 T20 :42:27.500 Z ,[[10101.20 , 0.44100] , [10101.10 , 0.450541] , [10100.55 ,
5.24501] , [10099.00 , 10.24511] , [10090.11 , 24.21395] ,[10090.05]] ] ,
5 [2019 -08 -14 T20 :42:27.963 Z ,[[10101.20 , 0.55200] , [10101.10 , 0.450541] , [10100.55 ,
5.24501] , [10099.00 , 10.24511] , [10090.11 , 24.21395]]]
6 ]
7
8 ordinal_volumes , o rdinal_p rices = c o n v e r t _ a r r a y _ l i s t _ t o _ d i c t ( snapshot )
9
10 >> print ( o r di na l_ v ol um es )
11 > >[{ " time " :2019 -08 -14 T20 :42:27.265 Z , " 1 " :0.013400 , " 2 " :0.44100 , " 3 " :0.450541 , " 4 "
:5.24501 , " 5 " :10.24511} ,
12 { " time " :2019 -08 -14 T20 :42:27.500 Z , " 1 " :0.44100 , " 2 " :0.450541 , " 3 " :5.24501 , " 4 "
:10.24511 , " 5 " :24.21395} ,
13 { " time " :2019 -08 -14 T20 :42:27.963 Z , " 1 " :0.52200 , " 2 " :0.450541 , " 3 " :5.24501 , " 4 "
:10.24511 , " 5 " :2421395}]
14
15 >> print ( ordin al_pric es )
16 > >[{ " time " :2019 -08 -14 T20 :42:27.265 Z , " 1 " :10101.80 , " 2 " :10101.20 , " 3 " :10101.10 , " 4 "
:10100.55 , " 5 " :10099.00} ,
17 { " time " :2019 -08 -14 T20 :42:27.500 Z , " 1 " :10101.20 , " 2 " :10101.10 , " 3 " :10100.55 , " 4 "
:10099.00 , " 5 " :10090.11} ,
18 { " time " :2019 -08 -14 T20 :42:27.963 Z , " 1 " :10101.20 , " 2 " :10101.10 , " 3 " :10100.55 , " 4 "
:10099.00 , " 5 " :10090.11}]
 6 MODULES 34

6.3.0.3 function conver array to list dict sob(history, events, pos limt=5)
Description: converts the time-series of order book states into a list of dictionaries for the signed order
book. This has some nuance because the signed book format has negative volumes, and the ordinal scale
is prefaced by a negative sign. Works similarly to conver array to list dict but uses the mid-price
from events to find where to separate the bid and ask sides.
parameters: history: list
time series of order book states
events: list
The events list where the function will extract the mid price from.
pos limit: int, default := 5
Ordinal distance from the best bid/ask that is worth saving.
returns: volm list: list
list of dictionaries for volume sizes and their respective position.
price list: list
list of dictionaries for prices and their respective position.
latent changes: None: None
Example: None
1 # let our snapshot_bid be :
2 bid_history = [
3 [2019 -08 -14 T20 :42:27.265 Z ,[[10101.80 , 0.013400] ,[10101.20 , 0.44100] , [10101.10 ,
0.450541] , [10100.55 , 5.24501] , [10099.00 , 10.24511] , [10090.11 , 24.21395]]] ,
4 [2019 -08 -14 T20 :42:27.500 Z ,[[10101.20 , 0.44100] , [10101.10 , 0.450541] , [10100.55 ,
5.24501] , [10099.00 , 10.24511] , [10090.11 , 24.21395] ,[10090.05]] ] ,
5 [2019 -08 -14 T20 :42:27.963 Z ,[[10101.20 , 0.55200] , [10101.10 , 0.450541] , [10100.55 ,
5.24501] , [10099.00 , 10.24511] , [10090.11 , 24.21395]]]
6 ]
7
8 ordinal_volumes , o rdinal_p rices = c o n v e r t _ a r r a y _ l i s t _ t o _ d i c t ( snapshot )
9
10 >> print ( o r di na l_ v ol um es )
11 > >[{ " time " :2019 -08 -14 T20 :42:27.265 Z , " 1 " :0.013400 , " 2 " :0.44100 , " 3 " :0.450541 , " 4 "
:5.24501 , " 5 " :10.24511} ,
12 { " time " :2019 -08 -14 T20 :42:27.500 Z , " 1 " :0.44100 , " 2 " :0.450541 , " 3 " :5.24501 , " 4 "
:10.24511 , " 5 " :24.21395} ,
13 { " time " :2019 -08 -14 T20 :42:27.963 Z , " 1 " :0.52200 , " 2 " :0.450541 , " 3 " :5.24501 , " 4 "
:10.24511 , " 5 " :2421395}]
14
15 >> print ( ordin al_pric es )
16 > >[{ " time " :2019 -08 -14 T20 :42:27.265 Z , " 1 " :10101.80 , " 2 " :10101.20 , " 3 " :10101.10 , " 4 "
:10100.55 , " 5 " :10099.00} ,
17 { " time " :2019 -08 -14 T20 :42:27.500 Z , " 1 " :10101.20 , " 2 " :10101.10 , " 3 " :10100.55 , " 4 "
:10099.00 , " 5 " :10090.11} ,
18 { " time " :2019 -08 -14 T20 :42:27.963 Z , " 1 " :10101.20 , " 2 " :10101.10 , " 3 " :10100.55 , " 4 "
:10099.00 , " 5 " :10090.11}]
 6 MODULES 35

6.3.0.4 function pd excel save(title, hist obj dict)

Description: Convert the list of dictionary objects into a pandas dataframe object. Excises the first
entry, because it is usually empty, saves the dataframe object as a .xlsx file with the title given as an
argument.
parameters: title: string
Title of the output .xlsx file
hist obj dict: list
The list of dictionaries to be converted to a pandas dataframe object.
returns: None: None
latent changes: None: None
Example: None
1 o rd in al _ vo lu me s = [{ " time " :2019 -08 -14 T20 :42:27.265 Z , " 1 " :0.013400 , " 2 " :0.44100 , " 3 "
:0.450541 , " 4 " :5.24501 , " 5 " :10.24511} ,
2 { " time " :2019 -08 -14 T20 :42:27.500 Z , " 1 " :0.44100 , " 2 " :0.450541 , " 3 " :5.24501 , " 4 " :10.24511 ,
" 5 " :24.21395} ,
3 { " time " :2019 -08 -14 T20 :42:27.963 Z , " 1 " :0.52200 , " 2 " :0.450541 , " 3 " :5.24501 , " 4 " :10.24511 ,
" 5 " :2421395}]
4
5 volm_title = " L 2 _ o r d e r b o o k _ v o l m _ b i d " + str ( bid_events [ -1][ ’ time ’ ]) + " . xlsx "
6
7 >> pd_excel_save ( volm_title , o rd in al _ vo lu me s )

Below a simple table generated from L2 orderbook volm bid2019-08-14T20:42:27.963Z.xlsx is shown.

time 1 2 3 4 5
2019-08-14T20:42:27.265 0.013400 0.44100 0.450541 5.24501 10.24511
2019-08-14T20:42:27.500 0.44100 0.450541 5.24501 10.24511 24.21395
2019-08-14T20:42:27.963 0.52200 0.450541 5.24501 10.24511 24.21395
 6 MODULES 36

6.3.0.5 function set sign(event size, side, order type)

Description: Sets the multiplicative sign,either 1 or -1 depending on the order type and event side.
parameters: event size: float64
Size of the event, artifact from when I returned the signed event
side: string
Event side either ”bid” or ”ask”
order type: str
the type of order either ‘‘insertion’’, ‘‘cancellation’’ or ‘‘market’’.
returns: sign int
The correct sign given the event’s order type and side.
latent changes: None: None
Example: None
1 x = _ _m y_ fu n ct io n_ _ (1.234 , True , 3)
2 >> print ( x )
3 > > -1.23
4
5 x = _ _m y_ fu n ct io n_ _ (1.234 , False , 3)
6 >> print ( x )
7 > >1.23
 6 MODULES 37

6.3.0.6 function set signed positiion(position, side)

Description: Simple conditional that sets the position for the bid side.... I’m tired...
parameters: param 1: float64
some input that will be passed to the function
param 2: boolean
a conditional that will be checked by the function
param 3: int
some int that is needed for the function
returns: trans number float64
The number with the correct significant digits and sign.
latent changes: trans number float64
The number with the correct significant digits and sign.
Example: None
1 x = _ _m y_ fu n ct io n_ _ (1.234 , True , 3)
2 >> print ( x )
3 > > -1.23
4
5 x = _ _m y_ fu n ct io n_ _ (1.234 , False , 3)
6 >> print ( x )
7 > >1.23
 6 MODULES 38

6.3.0.7 function get min dec(min currency denom, min asset value)
Description: determines the decimal place for the smallest amount of base currency needed for the
minimal denomination of the float currency.
Example: Suppose you have an exchange rate of 10010.99U SD/BT C, (i.e., you need 10010.99U SD
to buy 1BT C) then the amount of BTC in 1 cent or 0.01 USD which is the smallest denomination
of the USD is 0.000000999BT C. We see that this is 10−7 , get min dec would return 7, to pass onto
numpy.around.
parameters: min currency denom : float64
the minimum currency denomination (e.g., 0.01 USD)
min asset value: float646
the smallest current valuation of the asset.
returns: min dec out int
The number of decimal places out for the smallest movement of the asset for a minimal d
latent changes: None: None
Example: None
1 ??????
7 CREDITS 39

7 Credits
7.1 Development Lead
ˆ Ivan E. Perez - perez.ivan.e@gmail.com

7.2 Contributors
ˆ BTC-DAVE - jewmansown69@gmail.com
8 LICENSE 40

8 License
GNU GPLv3 License
Copyright (c) 2020, Ivan E. Perez
This program is free software: you can redistribute it and/or modify it under the terms of the GNU
General Public License as published by the Free Software Foundation, either version 3 of the License,
or (at your option) any later version. This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not,
see https://www.gnu.org/licenses/.
 9 APPENDIX 41

9 Appendix
10 To do list
1. daily goals

(a) write 2 whole function definitions with description, examples, i/o,

(b) help format or rewrite one unique section.

10.0.0.1 function my function (self, param 1, param 2=False, param 3)

Description: Put your description here. In the next table we will state the parameters, on the left, with
the object type on the right. just below it on the next line, we will give a brief description of the input
parameter. After that we will use the returns portion in the exact same way. The final section is an
example that uses lstlisting package to write snippets of python code for us. The filled out template
is then included in the class .tex file and the class .tex file is included in the master manual .tex file.
For example this function will convert param 1 as a float to the nearest significant digits as defined
by param 3 and will multiply by -1 if param 2:=True.
parameters: param 1: float64
some input that will be passed to the function
param 2: boolean
a conditional that will be checked by the function
param 3: int
some int that is needed for the function
returns: trans number float64
The number with the correct significant digits and sign.
latent changes: trans number float64
The number with the correct significant digits and sign.
Example: None
1 x = _ _m y_ fu n ct io n_ _ (1.234 , True , 3)
2 >> print ( x )
3 > > -1.23
4
5 x = _ _m y_ fu n ct io n_ _ (1.234 , False , 3)
6 >> print ( x )
7 > >1.23
8

variable name type

definition/description
snapshot bid: list
The modified snapshot bid.
bid range: list
The modified range of bids available in the snapshot.
pre level depth: float64
again the pre level depth that idk why i still have this.
 10 TO DO LIST 42

10.0.0.2 function my function (self, param 1, param 2=False, param 3)

Description: Put your description here. In the next table we will state the parameters, on the left, with
the object type on the right. just below it on the next line, we will give a brief description of the input
parameter. After that we will use the returns portion in the exact same way. The final section is an
example that uses lstlisting package to write snippets of python code for us. The filled out template
is then included in the class .tex file and the class .tex file is included in the master manual .tex file.
For example this function will convert param 1 as a float to the nearest significant digits as defined
by param 3 and will multiply by -1 if param 2:=True.
parameters: param 1: float64
some input that will be passed to the function
param 2: boolean
a conditional that will be checked by the function
param 3: int
some int that is needed for the function
returns: trans number float64
The number with the correct significant digits and sign.
latent changes: trans number float64
The number with the correct significant digits and sign.
Example: None
1 x = _ _m y_ fu n ct io n_ _ (1.234 , True , 3)
2 >> print ( x )
3 > > -1.23
4
5 x = _ _m y_ fu n ct io n_ _ (1.234 , False , 3)
6 >> print ( x )
7 > >1.23
8
REFERENCES 43

References
[1] Weibing Huang, Charles-Albert Lehalle, and Mathieu Rosenbaum. Simulating and analyzing order
book data: The queue-reactive model. Journal of the American Statistical Association, 110(509):107–
122, 2015.

[2] Álvaro Cartea, Sebastian Jaimungal, and Yixuan Wang. Spoofing and price manipulation in order-
driven markets. Applied Mathematical Finance, 27(1-2):67–98, 2020.
[3] Maureen O’Hara. Market Microstructure Theory. John Wiley & Sons Inc., 1998.
[4] Arseniy Kukanov Rama Cont and Sasha Stoikov. The price impact of order book events. SSRN,
pages 1–32, 2012.

[5] Sasha Stoikov Rama Cont and Rishi Talreja. A stochasic model for order book dynamics. SSRN,
pages 1–33, 2009.
[6] Edward Silyantev. Order flow analysis of crpytocurrency markets. Digital Finance, pages 191–218,
2019.