Skip to content

Commit 6a3f438

Browse files
anchitjanchitj
authored
Add linux ARM64 wheels block in Semaphore (confluentinc#1496)
1 parent 1eade53 commit 6a3f438

File tree

6 files changed

+43
-12
lines changed

6 files changed

+43
-12
lines changed

‎.semaphore/semaphore.yml

Lines changed: 20 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,6 @@ global_job_config:
99
value: v2.0.2
1010
prologue:
1111
commands:
12-
- export HOME=$WORKSPACE
13-
- mkdir $WORKSPACE/confluent-kafka-python
14-
- cd $WORKSPACE/confluent-kafka-python
1512
- checkout
1613
blocks:
1714
- name: "Wheels: OSX x64"
@@ -54,7 +51,7 @@ blocks:
5451
- PIP_INSTALL_OPTIONS="--user" tools/wheels/build-wheels.sh "${LIBRDKAFKA_VERSION#v}" wheelhouse
5552
- tar -czf wheelhouse-macOS-${ARCH}.tgz wheelhouse
5653
- artifact push workflow wheelhouse-macOS-${ARCH}.tgz
57-
54+
5855
- name: Source package verification with Python 3 (OSX x64) +docs
5956
dependencies: []
6057
task:
@@ -83,3 +80,22 @@ blocks:
8380
# install confluent-kafka
8481
- python setup.py build && python setup.py install
8582
- make docs
83+
- name: "Wheels: Linux arm64"
84+
run:
85+
when: "tag =~ '.*'"
86+
dependencies: []
87+
task:
88+
agent:
89+
machine:
90+
type: s1-prod-ubuntu20-04-arm64-1
91+
env_vars:
92+
- name: OS_NAME
93+
value: linux
94+
- name: ARCH
95+
value: arm64
96+
jobs:
97+
- name: Build
98+
commands:
99+
- ./tools/build-manylinux.sh "${LIBRDKAFKA_VERSION#v}"
100+
- tar -czf wheelhouse-linux-${ARCH}.tgz wheelhouse
101+
- artifact push workflow wheelhouse-linux-${ARCH}.tgz

‎CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44

55
- Added `set_sasl_credentials`. This new method (on the Producer, Consumer, and AdminClient) allows modifying the stored
66
SASL PLAIN/SCRAM credentials that will be used for subsequent (new) connections to a broker (#1511).
7+
- Wheels for Linux / arm64 (#1496).
78

89

910
## v2.0.2

‎src/confluent_kafka/src/confluent_kafka.c

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2758,7 +2758,11 @@ static struct PyModuleDef cimpl_moduledef = {
27582758
static PyObject *_init_cimpl (void) {
27592759
PyObject *m;
27602760

2761+
/* PyEval_InitThreads became deprecated in Python 3.9 and will be removed in Python 3.11.
2762+
* Prior to Python 3.7, this call was required to initialize the GIL. */
2763+
#if PY_VERSION_HEX < 0x03090000
27612764
PyEval_InitThreads();
2765+
#endif
27622766

27632767
if (PyType_Ready(&KafkaErrorType) < 0)
27642768
return NULL;

‎tools/bootstrap-librdkafka.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ mkdir -p "$BUILDDIR/librdkafka"
2929
pushd "$BUILDDIR/librdkafka"
3030

3131
test -f configure ||
32-
curl -q -L "https://github.com/edenhill/librdkafka/archive/${VERSION}.tar.gz" | \
32+
curl -q -L "https://github.com/confluentinc/librdkafka/archive/refs/tags/${VERSION}.tar.gz" | \
3333
tar -xz --strip-components=1 -f -
3434

3535
./configure --clean

‎tools/build-manylinux.sh

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,13 @@ if [[ ! -f /.dockerenv ]]; then
3333
exit 1
3434
fi
3535

36-
docker run -t -v $(pwd):/io quay.io/pypa/manylinux2010_x86_64:latest /io/tools/build-manylinux.sh "$LIBRDKAFKA_VERSION"
36+
if [[ $ARCH == arm64* ]]; then
37+
docker_image=quay.io/pypa/manylinux_2_28_aarch64:latest
38+
else
39+
docker_image=quay.io/pypa/manylinux_2_28_x86_64:latest
40+
fi
41+
42+
docker run -t -v $(pwd):/io $docker_image /io/tools/build-manylinux.sh "v${LIBRDKAFKA_VERSION}"
3743

3844
exit $?
3945
fi
@@ -44,14 +50,14 @@ fi
4450
#
4551

4652
echo "# Installing basic system dependencies"
47-
yum install -y zlib-devel gcc-c++
53+
yum install -y zlib-devel gcc-c++ python3 curl-devel perl-IPC-Cmd perl-Pod-Html
4854

4955
echo "# Building librdkafka ${LIBRDKAFKA_VERSION}"
5056
$(dirname $0)/bootstrap-librdkafka.sh --require-ssl ${LIBRDKAFKA_VERSION} /usr
5157

5258
# Compile wheels
5359
echo "# Compile"
54-
for PYBIN in /opt/python/*/bin; do
60+
for PYBIN in /opt/python/cp*/bin; do
5561
echo "## Compiling $PYBIN"
5662
CFLAGS="-Werror -Wno-strict-aliasing -Wno-parentheses" \
5763
"${PYBIN}/pip" wheel /io/ -w unrepaired-wheelhouse/
@@ -73,13 +79,13 @@ done
7379

7480
# Install packages and test
7581
echo "# Installing wheels"
76-
for PYBIN in /opt/python/*/bin/; do
82+
for PYBIN in /opt/python/cp*/bin/; do
7783
echo "## Installing $PYBIN"
7884
"${PYBIN}/pip" install confluent_kafka -f /io/wheelhouse
7985
"${PYBIN}/python" -c 'import confluent_kafka; print(confluent_kafka.libversion())'
86+
"${PYBIN}/pip" install -r /io/tests/requirements.txt
87+
"${PYBIN}/pytest" /io/tests/test_Producer.py
8088
echo "## Uninstalling $PYBIN"
8189
"${PYBIN}/pip" uninstall -y confluent_kafka
8290
done
8391

84-
85-

‎tools/wheels/install-librdkafka.sh

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,11 @@ if [[ $OSTYPE == linux* ]]; then
2929
# Linux
3030

3131
# Copy the librdkafka build with least dependencies to librdkafka.so.1
32-
cp -v runtimes/linux-$ARCH/native/{centos6-librdkafka.so,librdkafka.so.1}
32+
if [[ $ARCH == arm64* ]]; then
33+
cp -v runtimes/linux-$ARCH/native/{librdkafka.so,librdkafka.so.1}
34+
else
35+
cp -v runtimes/linux-$ARCH/native/{centos6-librdkafka.so,librdkafka.so.1}
36+
fi
3337
ldd runtimes/linux-$ARCH/native/librdkafka.so.1
3438

3539
elif [[ $OSTYPE == darwin* ]]; then

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy