Libsodium not bundled in Windows wheels #1148
Description
I was dissatisfied with the speed of zeromq curve encryption, until I noticed that it appears to vary by an order of magnitude across different ways of installing pyzmq. This script:
from __future__ import print_function
import sys
import os
import time
import zmq
import zmq.auth.thread
context = zmq.Context()
auth = zmq.auth.thread.ThreadAuthenticator(context)
auth.start()
auth.configure_curve(domain='*', location=zmq.auth.CURVE_ALLOW_ANY)
del auth # Ensure auth thread doesn't keep the interpreter open
server_publickey, server_secretkey = zmq.curve_keypair()
client_publickey, client_secretkey = zmq.curve_keypair()
server = context.socket(zmq.PULL)
server.curve_publickey = server_publickey
server.curve_secretkey = server_secretkey
server.curve_server = True
port = server.bind_to_random_port('tcp://127.0.0.1')
client = context.socket(zmq.PUSH)
client.curve_secretkey = client_secretkey
client.curve_publickey = client_publickey
client.curve_serverkey = server_publickey
client.connect('tcp://127.0.0.1:%d' % port)
print('Python Version: %s' % ' | '.join(sys.version.splitlines()))
print('Platform: %s' % sys.platform)
print('pyzmq version: %s'%zmq.__version__)
print('zmq version: %s'%zmq.zmq_version())
plaintext = os.urandom(100*1024**2)
start_time = time.time()
client.send(plaintext)
assert server.recv() == plaintext
print("time to encrypt and send 100MB:", time.time() - start_time, 's')Returns the following for all the different Pythons I have installed, where I have anaconda2 and anaconda3, which came with pyzmq, system python 2 and 3, for which I installed pyzmq via apt, and two virtualenvs that I made using system Python 2 and 3, and installed pyzmq using pip install pyzmq --install-option="--zmq=bundled"
$ /usr/bin/python2.7 test.py
Python Version: 2.7.14 (default, Sep 23 2017, 22:06:14) | [GCC 7.2.0]
Platform: linux2
pyzmq version: 16.0.2
zmq version: 4.2.1
time to encrypt and send 100MB: 0.500539064407 s
$ /usr/bin/python3.6 test.py
Python Version: 3.6.3 (default, Oct 3 2017, 21:45:48) | [GCC 7.2.0]
Platform: linux
pyzmq version: 16.0.2
zmq version: 4.2.1
time to encrypt and send 100MB: 0.49707913398742676 s
$ ~/anaconda2/bin/python test.py
Python Version: 2.7.13 |Anaconda, Inc.| (default, Sep 30 2017, 18:12:43) | [GCC 7.2.0]
Platform: linux2
pyzmq version: 17.0.0
zmq version: 4.1.6
~/anaconda3/bin/python test.py
time to encrypt and send 100MB: 6.23277807236 s
$ ~/anaconda3/bin/python test.py
Python Version: 3.6.3 |Anaconda custom (64-bit)| (default, Nov 3 2017, 19:19:16) | [GCC 7.2.0]
Platform: linux
pyzmq version: 16.0.3
zmq version: 4.2.2
time to encrypt and send 100MB: 2.81573748588562 s
$ ~/tmp/venvs/zmqtestpy2/bin/python test.py
Python Version: 2.7.14 (default, Sep 23 2017, 22:06:14) | [GCC 7.2.0]
Platform: linux2
pyzmq version: 17.0.0
zmq version: 4.1.6
time to encrypt and send 100MB: 6.02314901352 s
$ ~/tmp/venvs/zmqtestpy3/bin/python test.py
Python Version: 3.6.3 (default, Oct 3 2017, 21:45:48) | [GCC 7.2.0]
Platform: linux
pyzmq version: 17.0.0
zmq version: 4.1.6
time to encrypt and send 100MB: 6.559395790100098 s
As you can see, the speed of encryption varies a lot, and in particular, is slowest with the fresh virtualenvs with the pip install and bundled zmq, and is fastest with the system versions.
At first it seems like zeromq 4.2 is what makes it fast - until you see that anaconda3 with zmq 4.2.2 is still way slower than the system ones. So it seems it's probably something about configuration rather than the specific versions of zmq in use.
So it appears maybe the bundled zeromq is a a bit slow with the encryption, and it would be better if the default install resulted in fast crypto. My users are mostly using anaconda, so if whatever ends up in anaconda could be as fast as the versions in the Ubuntu repos, that would be great!