Skip to content

Commit 4a5eb51

Browse files
author
Tom Clark
committed
Added python version
1 parent 3cf800f commit 4a5eb51

File tree

1 file changed

+104
-3
lines changed

1 file changed

+104
-3
lines changed

pages/retrieving-storm-data-from-nimbus.md

Lines changed: 104 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -74,13 +74,16 @@ require 'thrift'
7474
require 'nimbus'
7575

7676
begin
77-
transport = Thrift::FramedTransport.new(Thrift::Socket.new('localhost', 6627))
78-
protocol = Thrift::BinaryProtocol.new(transport)
79-
client = Nimbus::Client.new(protocol)
77+
socket = Thrift::Socket.new('localhost', 6627)
78+
transport = Thrift::FramedTransport.new(socket)
79+
protocol = Thrift::BinaryProtocol.new(transport)
80+
client = Nimbus::Client.new(protocol)
8081

8182
transport.open
83+
8284
summary = client.getClusterInfo
8385
puts summary.inspect
86+
8487
transport.close
8588

8689
rescue Thrift::Exception => tx
@@ -95,5 +98,103 @@ ruby test.rb
9598
{% endhighlight %}
9699

97100
Now you can take that data and write it to where you'd like. Enjoy!
101+
</div>
102+
103+
<div
104+
markdown="1"
105+
class="tutorial"
106+
data-github-author="Whitespace"
107+
data-license="http://creativecommons.org/licenses/by/3.0/"
108+
data-facets='{"Operating System": "OS X 10.7", "Language": "Python", "Storm Version": "0.7.0", "Thrift Version": "0.7.0", "Package Management": "PyPI"}'>
98109

110+
## Assumptions
111+
112+
We assume you'll store your data in `~/code`.
113+
114+
## Setup
115+
116+
Let's make a location to store our project:
117+
118+
{% highlight sh %}
119+
mkdir -p ~/code/nimbus-thrift-demo
120+
{% endhighlight %}
121+
122+
## Download the Storm Source Code
123+
124+
In order to connect to nimbus, we need to get the `storm.thrift` file from the storm source, and use that to generate the thrift bindings in python:
125+
126+
{% highlight sh %}
127+
cd ~/code
128+
wget https://github.com/downloads/nathanmarz/storm/storm-0.7.0.zip
129+
unzip storm-0.7.0.zip
130+
{% endhighlight %}
131+
132+
## Install Thrift 0.7.0 and the Thrift Python Package
133+
134+
Now let's install the same version of thrift that storm 0.7.0 uses:
135+
136+
{% highlight sh %}
137+
cd /usr/local
138+
brew update
139+
brew versions thrift
140+
git checkout 141ddb6 /usr/local/Library/Formula/thrift.rb
141+
brew install thrift
142+
{% endhighlight %}
143+
144+
And the thrift python package:
145+
146+
{% highlight sh %}
147+
pip install thrift==0.7.0
148+
{% endhighlight %}
149+
150+
## Generate the Python Bindings
151+
152+
{% highlight sh %}
153+
cd ~/code/nimbus-thrift-demo
154+
cp ~/code/storm-0.7.0/src/storm.thrift .
155+
thrift --gen py storm.thrift
156+
{% endhighlight %}
157+
158+
## Connect to Nimbus
159+
160+
Now that we have the python bindings in place, we can create a sample app to connect to our nimbus cluster. Copy the following to `test.py`, replacing `localhost` with the hostname of your nimbus instance:
161+
162+
{% highlight python %}
163+
import sys
164+
165+
sys.path.append('gen-py')
166+
167+
from thrift import Thrift
168+
from thrift.transport import TSocket
169+
from thrift.transport import TTransport
170+
from thrift.protocol import TBinaryProtocol
171+
172+
from storm import Nimbus
173+
from storm.ttypes import *
174+
from storm.constants import *
175+
176+
try:
177+
socket = TSocket.TSocket('localhost', 6627)
178+
transport = TTransport.TFramedTransport(socket)
179+
protocol = TBinaryProtocol.TBinaryProtocol(transport)
180+
client = Nimbus.Client(protocol)
181+
182+
transport.open()
183+
184+
summary = client.getClusterInfo()
185+
print summary
186+
187+
transport.close()
188+
189+
except Thrift.TException, tx:
190+
print "%s" % (tx.message)
191+
{% endhighlight %}
192+
193+
Now we can run our test app:
194+
195+
{% highlight sh %}
196+
python test.py
197+
{% endhighlight %}
198+
199+
Now you can take that data and write it to where you'd like. Enjoy!
99200
</div>

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy