Skip to content

Commit 536bdcb

Browse files
Add lib-builder docker image (espressif#155)
* Test docker Image * Fix Dockerfile * Fix file permissions * Fix bugs and add run script * Update tools/docker/Dockerfile * Add textual to docker image * Reorder * Fix * Test * Fix * Add UI checks * Fix docker image * Fix exec in docker * Move argument * Add workflow * Add checks to entrypoint * Improvements to Dockerfile * Test example run scripts * Fix powershell script * Change parameter name * Add comment * Add Readme * Fix comparison * Remove cache * Fix comment * Improve permission settings * Add warning * Update readme * Fix repositories --------- Co-authored-by: Me No Dev <me-no-dev@users.noreply.github.com>
1 parent 0c37f25 commit 536bdcb

File tree

8 files changed

+388
-3
lines changed

8 files changed

+388
-3
lines changed

.github/workflows/docker.yml

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
name: Build and push Docker image
2+
3+
on:
4+
push:
5+
branches:
6+
- 'master'
7+
- 'release/*'
8+
tags:
9+
- 'v*.*'
10+
11+
env:
12+
# Build the image for amd64 and arm64
13+
BUILD_PLATFORMS: linux/amd64,linux/arm64
14+
DOCKERHUB_REPO: ${{ github.repository_owner }}/esp32-arduino-lib-builder
15+
16+
jobs:
17+
docker:
18+
# Disable the job in forks
19+
if: ${{ github.repository_owner == 'espressif' }}
20+
21+
runs-on: ubuntu-latest
22+
steps:
23+
# Depending on the branch/tag, set CLONE_BRANCH_OR_TAG variable (used in the Dockerfile
24+
# as a build arg) and TAG_NAME (used when tagging the image).
25+
#
26+
# The following 3 steps cover the alternatives (tag, release branch, master branch):
27+
- name: Set variables (tags)
28+
if: ${{ github.ref_type == 'tag' }}
29+
run: |
30+
echo "CLONE_BRANCH_OR_TAG=$GITHUB_REF_NAME" >> $GITHUB_ENV
31+
echo "TAG_NAME=$GITHUB_REF_NAME" >> $GITHUB_ENV
32+
- name: Set variables (release branches)
33+
if: ${{ github.ref_type == 'branch' && startsWith(github.ref_name, 'release/') }}
34+
run: |
35+
echo "CLONE_BRANCH_OR_TAG=$GITHUB_REF_NAME" >> $GITHUB_ENV
36+
echo "TAG_NAME=release-${GITHUB_REF_NAME##release/}" >> $GITHUB_ENV
37+
- name: Set variables (main branch)
38+
if: ${{ github.ref_type == 'branch' && github.ref_name == 'master' }}
39+
run: |
40+
echo "CLONE_BRANCH_OR_TAG=master" >> $GITHUB_ENV
41+
echo "TAG_NAME=latest" >> $GITHUB_ENV
42+
43+
# Display the variables set above, just in case.
44+
- name: Check variables
45+
run: |
46+
echo "CLONE_BRANCH_OR_TAG: $CLONE_BRANCH_OR_TAG"
47+
echo "CHECKOUT_REF: $CHECKOUT_REF"
48+
echo "TAG_NAME: $TAG_NAME"
49+
50+
# The following steps are the standard boilerplate from
51+
# https://github.com/marketplace/actions/build-and-push-docker-images
52+
- name: Checkout
53+
uses: actions/checkout@v4
54+
- name: Login to Docker Hub
55+
uses: docker/login-action@v3
56+
with:
57+
username: ${{ secrets.DOCKERHUB_USERNAME }}
58+
password: ${{ secrets.DOCKERHUB_TOKEN }}
59+
- name: Set up QEMU for multiarch builds
60+
uses: docker/setup-qemu-action@v3
61+
- name: Set up Docker Buildx
62+
uses: docker/setup-buildx-action@v3
63+
- name: Build and push
64+
uses: docker/build-push-action@v5
65+
with:
66+
context: tools/docker
67+
push: true
68+
tags: ${{ env.DOCKERHUB_REPO }}:${{ env.TAG_NAME }}
69+
platforms: ${{ env.BUILD_PLATFORMS }}
70+
build-args: |
71+
LIBBUILDER_CLONE_URL=${{ github.server_url }}/${{ github.repository }}.git
72+
LIBBUILDER_CLONE_BRANCH_OR_TAG=${{ env.CLONE_BRANCH_OR_TAG }}
73+
74+
- name: Update Docker Hub repository description (master branch)
75+
if: ${{ github.ref_type == 'branch' && github.ref_name == 'master' }}
76+
uses: peter-evans/dockerhub-description@v4
77+
with:
78+
username: ${{ secrets.DOCKERHUB_USERNAME }}
79+
password: ${{ secrets.DOCKERHUB_TOKEN }}
80+
repository: ${{ env.DOCKERHUB_REPO }}
81+
readme-filepath: ./tools/docker/README.md

tools/config_editor/app.py

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,6 @@ def compose(self) -> ComposeResult:
8686

8787
def on_mount(self) -> None:
8888
# Event handler called when the app is mounted for the first time
89-
self.title = "Configurator"
9089
self.sub_title = "Main Menu"
9190
print("Main screen mounted.")
9291

@@ -104,6 +103,7 @@ class ConfigEditorApp(App):
104103
# Options to be set by the command line arguments
105104
setting_target = ""
106105
setting_arduino_path = ""
106+
setting_output_permissions = ""
107107
setting_arduino_branch = ""
108108
setting_idf_branch = ""
109109
setting_idf_commit = ""
@@ -131,6 +131,7 @@ def on_mount(self) -> None:
131131
print("IDF Branch: " + str(self.setting_idf_branch))
132132
print("IDF Commit: " + str(self.setting_idf_commit))
133133
print("IDF Debug Level: " + str(self.setting_debug_level))
134+
self.title = "Configurator"
134135
self.push_screen("main")
135136

136137
def arduino_default_path():
@@ -199,6 +200,13 @@ def main() -> None:
199200
required=False,
200201
help="Path to arduino-esp32 directory. Default: " + arduino_default_path())
201202

203+
parser.add_argument("--output-permissions",
204+
metavar="<uid:gid>",
205+
type=str,
206+
default="",
207+
required=False,
208+
help=argparse.SUPPRESS) # Hidden option. It is only supposed to be used by the docker container
209+
202210
parser.add_argument("-A", "--arduino-branch",
203211
metavar="<arduino branch>",
204212
type=str,
@@ -256,14 +264,18 @@ def main() -> None:
256264
elif args.arduino_path == arduino_default_path():
257265
print("Warning: Default Arduino path not found. Disabling copy to Arduino.")
258266
app.setting_enable_copy = False
267+
elif args.arduino_path == "/arduino-esp32": # Docker mount point
268+
print("Warning: Docker mount point not found. Disabling copy to Arduino.")
269+
app.setting_enable_copy = False
259270
else:
260-
print("Invalid path to Arduino core: " + os.path.abspath(args.arduino_path))
271+
print("Error: Invalid path to Arduino core: " + os.path.abspath(args.arduino_path))
261272
exit(1)
262273
else:
263274
app.setting_enable_copy = False
264275

265276
# Set the other options
266277
app.setting_arduino_path = os.path.abspath(args.arduino_path)
278+
app.setting_output_permissions = args.output_permissions
267279
app.setting_arduino_branch = args.arduino_branch
268280
app.setting_idf_branch = args.idf_branch
269281
app.setting_idf_commit = args.idf_commit

tools/config_editor/compile.py

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import sys
22
import subprocess
33
import os
4+
import re
45

56
from rich.console import RenderableType
67

@@ -60,6 +61,11 @@ def print_success(self, message: str) -> None:
6061
self.button_widget.add_class("-success")
6162
#print("Success: " + message) # For debugging
6263

64+
def print_warning(self, message: str) -> None:
65+
# Print warning message to the RichLog widget
66+
self.log_widget.write("[b bright_yellow]" + message)
67+
#print("Warning: " + message) # For debugging
68+
6369
def print_info(self, message: str) -> None:
6470
# Print info message to the RichLog widget
6571
self.log_widget.write("[b bright_cyan]" + message)
@@ -72,7 +78,12 @@ def compile_libs(self) -> None:
7278

7379
label = self.query_one("#compile-title", Static)
7480
self.child_process = None
75-
if self.app.setting_target == ",".join(self.app.supported_targets):
81+
82+
if not self.app.setting_target:
83+
self.print_error("No target selected")
84+
label.update("No target selected")
85+
return
86+
elif self.app.setting_target == ",".join(self.app.supported_targets):
7687
target = "all targets"
7788
else:
7889
target = self.app.setting_target.replace(",", ", ").upper()
@@ -133,6 +144,30 @@ def compile_libs(self) -> None:
133144
print("Error reading child process errors: " + str(e))
134145
label.update("Compilation failed for " + target)
135146
else:
147+
if self.app.setting_output_permissions:
148+
regex = r"^[1-9][0-9]*:[1-9][0-9]*$" # Regex to match the uid:gid format. Note that 0:0 (root) is not allowed
149+
if re.match(regex, self.app.setting_output_permissions):
150+
print_info("Setting permissions to: " + self.app.setting_output_permissions)
151+
chown_process = None
152+
try:
153+
chown_process = subprocess.run(["chown", "-R", self.app.setting_output_permissions, self.app.setting_arduino_path])
154+
chown_process.wait()
155+
except Exception as e:
156+
print("Error changing permissions: " + str(e))
157+
158+
if chown_process and chown_process.returncode != 0:
159+
self.print_error("Error changing permissions")
160+
self.print_error("Please change the ownership of generated files manually")
161+
else:
162+
self.print_success("Permissions changed successfully")
163+
elif self.app.setting_output_permissions == "0:0":
164+
self.print_warning("Permissions settings are set to root (0:0)")
165+
self.print_warning("Please change the ownership of generated files manually")
166+
self.print_warning("If you are compiling for Windows, you may ignore this warning")
167+
else:
168+
self.print_error("Invalid permissions format: " + self.app.setting_output_permissions)
169+
self.print_error("Please change the ownership of generated files manually")
170+
136171
self.print_success("Compilation successful for " + target)
137172
label.update("Compilation successful for " + target)
138173

@@ -161,3 +196,8 @@ def compose(self) -> ComposeResult:
161196
self.button_widget = Button("Back", id="compile-back-button")
162197
yield self.button_widget
163198
yield Footer()
199+
200+
def on_mount(self) -> None:
201+
# Event handler called when the screen is mounted
202+
print("Compile screen mounted")
203+
self.sub_title = "Compilation"

tools/docker/Dockerfile

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# To Do: Check if it is worth to use espressif/idf as base image (image size will be much bigger)
2+
FROM ubuntu:22.04
3+
4+
# switch to root, let the entrypoint drop back to host user
5+
USER root
6+
SHELL ["/bin/bash", "-c"]
7+
8+
ARG DEBIAN_FRONTEND=noninteractive
9+
10+
RUN : \
11+
&& apt-get update \
12+
&& apt-get install -y --no-install-recommends \
13+
bison \
14+
ccache \
15+
cmake \
16+
curl \
17+
flex \
18+
git \
19+
gperf \
20+
jq \
21+
libncurses-dev \
22+
libssl-dev \
23+
libusb-1.0 \
24+
ninja-build \
25+
patch \
26+
python3 \
27+
python3-click \
28+
python3-cryptography \
29+
python3-future \
30+
python3-pip \
31+
python3-pyelftools \
32+
python3-pyparsing \
33+
python3-serial \
34+
python3-setuptools \
35+
python3-venv \
36+
wget \
37+
&& pip3 install --upgrade pip textual-dev \
38+
&& apt-get autoremove -y \
39+
&& rm -rf /var/lib/apt/lists/* \
40+
&& :
41+
42+
# To build the image for a branch or a tag of the lib-builder, pass --build-arg LIBBUILDER_CLONE_BRANCH_OR_TAG=name.
43+
# To build the image with a specific commit ID of lib-builder, pass --build-arg LIBBUILDER_CHECKOUT_REF=commit-id.
44+
# It is possibe to combine both, e.g.:
45+
# LIBBUILDER_CLONE_BRANCH_OR_TAG=release/vX.Y
46+
# LIBBUILDER_CHECKOUT_REF=<some commit on release/vX.Y branch>.
47+
# Use LIBBUILDER_CLONE_SHALLOW=1 to peform shallow clone (i.e. --depth=1 --shallow-submodules)
48+
# Use LIBBUILDER_CLONE_SHALLOW_DEPTH=X to define the depth if LIBBUILDER_CLONE_SHALLOW is used (i.e. --depth=X)
49+
50+
ARG LIBBUILDER_CLONE_URL=https://github.com/espressif/esp32-arduino-lib-builder
51+
ARG LIBBUILDER_CLONE_BRANCH_OR_TAG=master
52+
ARG LIBBUILDER_CHECKOUT_REF=
53+
ARG LIBBUILDER_CLONE_SHALLOW=
54+
ARG LIBBUILDER_CLONE_SHALLOW_DEPTH=1
55+
56+
ENV LIBBUILDER_PATH=/opt/esp/lib-builder
57+
# Ccache is installed, enable it by default
58+
ENV IDF_CCACHE_ENABLE=1
59+
60+
RUN echo LIBBUILDER_CHECKOUT_REF=$LIBBUILDER_CHECKOUT_REF LIBBUILDER_CLONE_BRANCH_OR_TAG=$LIBBUILDER_CLONE_BRANCH_OR_TAG && \
61+
git clone --recursive \
62+
${LIBBUILDER_CLONE_SHALLOW:+--depth=${LIBBUILDER_CLONE_SHALLOW_DEPTH} --shallow-submodules} \
63+
${LIBBUILDER_CLONE_BRANCH_OR_TAG:+-b $LIBBUILDER_CLONE_BRANCH_OR_TAG} \
64+
$LIBBUILDER_CLONE_URL $LIBBUILDER_PATH && \
65+
git config --system --add safe.directory $LIBBUILDER_PATH && \
66+
if [ -n "$LIBBUILDER_CHECKOUT_REF" ]; then \
67+
cd $LIBBUILDER_PATH && \
68+
if [ -n "$LIBBUILDER_CLONE_SHALLOW" ]; then \
69+
git fetch origin --depth=${LIBBUILDER_CLONE_SHALLOW_DEPTH} --recurse-submodules ${LIBBUILDER_CHECKOUT_REF}; \
70+
fi && \
71+
git checkout $LIBBUILDER_CHECKOUT_REF && \
72+
git submodule update --init --recursive; \
73+
fi
74+
75+
COPY entrypoint.sh $LIBBUILDER_PATH/entrypoint.sh
76+
77+
WORKDIR /opt/esp/lib-builder
78+
ENTRYPOINT [ "/opt/esp/lib-builder/entrypoint.sh" ]
79+
CMD [ "python3", "tools/config_editor/app.py" ]

tools/docker/README.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
<!-- This is a brief version of the Arduino Core for ESP32 documentation (add specific link)
2+
intended to be displayed on the Docker Hub page: https://hub.docker.com/r/espressif/esp32-arduino-lib-builder.
3+
When changing this page, please keep the documentation in sync.
4+
(Keep the differences between Markdown and restructuredText in mind.)
5+
-->
6+
7+
# ESP-IDF Docker Image
8+
9+
This is a Docker image for the [ESP32 Arduino Lib Builder](https://github.com/espressif/esp32-arduino-lib-builder). It is intended for building the static libraries of ESP-IDF components for use in Arduino projects.
10+
11+
This image contains a copy of the esp32-arduino-lib-builder repository and already include or will obtain all the required tools and dependencies to build the Arduino static libraries.
12+
13+
Currently supported architectures are:
14+
- `amd64`
15+
- `arm64`
16+
17+
## Tags
18+
19+
Multiple tags of this image are maintained:
20+
21+
- `latest`: tracks `master` branch of esp32-arduino-lib-builder
22+
- `release-vX.Y`: tracks `release/vX.Y` branch of esp32-arduino-lib-builder
23+
24+
## Basic Usage
25+
26+
```bash
27+
docker run --rm -it -e "TERM=xterm-256color" -v <path to arduino-esp32>:/arduino-esp32 espressif/esp32-arduino-lib-builder:latest
28+
```
29+
30+
The above command explained:
31+
32+
- `docker run`: Runs a command in a new container.
33+
- `--rm`: Optional. Automatically removes the container when it exits. Remove this flag if you plan to use the container multiple times.
34+
- `-i`: Runs the container in interactive mode.
35+
- `-t`: Allocates a pseudo-TTY.
36+
- `-e "TERM=xterm-256color"`: Optional. Sets the terminal type to `xterm-256color` to display colors correctly.
37+
- `-v <path to arduino-esp32>:/arduino-esp32`: Optional. Mounts the Arduino Core for ESP32 repository at `/arduino-esp32` inside the container. Replace `<path to arduino-esp32>` with the path to the repository on the host machine. If not provided, the container will not copy the compiled libraries to the host machine.
38+
- `espressif/esp32-arduino-lib-builder:latest`: The Docker image to use.
39+
40+
After running the above command, you will be inside the container and can build the libraries using the user interface.
41+
42+
By default the docker container will run the user interface script. If you want to run a specific command, you can pass it as an argument to the docker run command. For example, to run a terminal inside the container, you can run:
43+
44+
```bash
45+
docker run -it espressif/esp32-arduino-lib-builder:latest /bin/bash
46+
```
47+
48+
## Documentation
49+
50+
<!-- Pending: Add link to the full documentation. -->
51+
52+
For more information about this image and the detailed usage instructions, please refer to the Arduino Core for ESP32 documentation.

tools/docker/entrypoint.sh

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
#!/usr/bin/env bash
2+
set -e
3+
4+
# LIBBUILDER_GIT_SAFE_DIR has the same format as system PATH environment variable.
5+
# All path specified in LIBBUILDER_GIT_SAFE_DIR will be added to user's
6+
# global git config as safe.directory paths. For more information
7+
# see git-config manual page.
8+
if [ -n "${LIBBUILDER_GIT_SAFE_DIR+x}" ]
9+
then
10+
echo "Adding following directories into git's safe.directory"
11+
echo "$LIBBUILDER_GIT_SAFE_DIR" | tr ':' '\n' | while read -r dir
12+
do
13+
git config --global --add safe.directory "$dir"
14+
echo " $dir"
15+
done
16+
fi
17+
18+
# Check if the mount point /arduino-esp32 exists
19+
if [ -d "/arduino-esp32" ] && [[ "$@" == "python3 tools/config_editor/app.py"* ]]; then
20+
# Running UI with mount point detected, adding -c and --output-permissions arguments
21+
echo "Output folder permissions: `stat -c "%u:%g" /arduino-esp32`"
22+
exec "$@" -c /arduino-esp32 --output-permissions `stat -c "%u:%g" /arduino-esp32`
23+
else
24+
# Running UI without mount point detected or running another command
25+
exec "$@"
26+
fi

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy