Skip to content

Fixes: Removal of test folders #102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 26, 2016
Merged

Fixes: Removal of test folders #102

merged 1 commit into from
Apr 26, 2016

Conversation

sukrit007
Copy link
Contributor

@sukrit007 sukrit007 commented Apr 19, 2016 •
edited by yosifkit
Loading

Problem
Find command as defined in Dockerfile was not deleting the test folders.

docker run --rm  -it python:3.5.1-alpine  /bin/sh -c  'du -d 1 -h /usr/local/lib/python*/test'
20.0K   /usr/local/lib/python3.5/test/xmltestdata
112.0K  /usr/local/lib/python3.5/test/test_import
.....................................................................
18.4M   /usr/local/lib/python3.5/test

As a result it contributed a significant chunk to alpine docker images.

Fix:
Execute exec for each action group.

Note: Placing it in a single group does not guarantee the order and might cause issue when pyc files are present in test folder. Deleting independently for both conditions does not cause this issue.

Image Size
Before Fix (alpine : 3.5.1): 89.74 MB
After Fix (alpine: 3.5.1): 72.07 MB

@sukrit007 sukrit007 force-pushed the patch_remove-test-directories branch from b0ae780 to bb92d66 Compare April 19, 2016 09:27
@tianon
Copy link
Member

tianon commented Apr 20, 2016

Couldn't this also be solved with another set of ( without duplicating the -exec portion?

ie:

    && find /usr/local \
        \( \
            \( -type d -a \( -name test -o -name tests \) \) \
            -o \
            \( -type f -a \( -name '*.pyc' -o -name '*.pyo' \) \) \
        \) -exec rm -rf '{}' + \
    && rm -rf /usr/src/python

@sukrit007
Copy link
Contributor Author

sukrit007 commented Apr 21, 2016 •
edited
Loading

@tianon That is how I started initially , but since test folder also has *.pyc files , find command command errors out (at-least in alpine linux, not tried others).

Here is the output of failed docker build :

Installing collected packages: pip
Successfully installed pip-8.1.1
+ find /usr/local ( ( -type d -a -name test -o -name tests ) -o ( -type f -a -name *.pyc -o -name *.pyo ) ) -exec rm -rf {} +
find: /usr/local/lib/python3.5/ctypes/test/__pycache__/test_funcptr.cpython-35.pyc: No such file or directory
find: /usr/local/lib/python3.5/ctypes/test/__pycache__/test_find.cpython-35.opt-1.pyc: No such file or directory
....................
....................
find: /usr/local/lib/python3.5/test/test_stat.py: No such file or directory
find: /usr/local/lib/python3.5/test/test_descrtut.py: No such file or directory
The command '/bin/sh -c set -ex     && apk add --no-cache --virtual .fetch-deps curl gnupg  && curl -fSL "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz" -o python.tar.xz   && curl -fSL "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz.asc" -o python.tar.xz.asc   && export GNUPGHOME="$(mktemp -d)"  && gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$GPG_KEY"    && gpg --batch --verify python.tar.xz.asc python.tar.xz     && rm -r "$GNUPGHOME" python.tar.xz.asc     && mkdir -p /usr/src    && tar -xJC /usr/src -f python.tar.xz   && mv "/usr/src/Python-$PYTHON_VERSION" /usr/src/python     && rm python.tar.xz     && apk del .fetch-deps      && apk add --no-cache --virtual .build-deps         bzip2-dev       gcc         libc-dev        linux-headers       make        ncurses-dev         openssl-dev         pax-utils       readline-dev        sqlite-dev      zlib-dev    && cd /usr/src/python   && ./configure --enable-shared --enable-unicode=ucs4    && make -j$(getconf _NPROCESSORS_ONLN)  && make install     && pip3 install --no-cache-dir --upgrade --ignore-installed pip==$PYTHON_PIP_VERSION    && find /usr/local      \(         \( -type d -a -name test -o -name tests \)           -o          \( -type f -a -name '*.pyc' -o -name '*.pyo' \)         \) -exec rm -rf '{}' +  && runDeps="$(      scanelf --needed --nobanner --recursive /usr/local          | awk '{ gsub(/,/, "\nso:", $2); print "so:" $2 }'          | sort -u           | xargs -r apk info --installed             | sort -u   )"  && apk add --virtual .python-rundeps $runDeps   && apk del .build-deps  && rm -rf /usr/src/python ~/.cache' returned a non-zero code: 1

Here is the dockerfile for the same:
https://github.com/sukrit007/python/blob/patch_single-exec/3.5/alpine/Dockerfile#L46-L51

@tianon
Copy link
Member

tianon commented Apr 25, 2016

oh 😞

LGTM, thanks! 👍

@yosifkit
Copy link
Member

We can guarantee order with -depth 😄 as long as we add the extra parenthesis.

@sukrit007
Copy link
Contributor Author

@yosifkit You rock. I did not knew find supported depth first traversal.

find /usr/local -depth \
        \( \
                    \( -type d -a -name test -o -name tests \) \
            -o \
            \( -type f -a -name '*.pyc' -o -name '*.pyo' \) \
        \) -exec rm -rf '{}' +

I will reset back to master and apply this change instead.

@sukrit007 sukrit007 force-pushed the patch_remove-test-directories branch from 33747f3 to 22af00a Compare April 26, 2016 03:26
sukrit007 added a commit to sukrit007/python that referenced this pull request Apr 26, 2016
@sukrit007
Fixes docker-library#102 (tests folder removal)
22af00a
sukrit007 added a commit to sukrit007/python that referenced this pull request Apr 26, 2016
@sukrit007
Fixes docker-library#102 (tests folder removal)
78de7d7
@sukrit007 sukrit007 force-pushed the patch_remove-test-directories branch from 78de7d7 to 3232863 Compare April 26, 2016 03:38
@sukrit007
Copy link
Contributor Author

@yosifkit @tianon Fixed.

@sukrit007 sukrit007 mentioned this pull request Apr 26, 2016
Closed
@sukrit007
Copy link
Contributor Author

Travis failed but looks like it should pass if build is retriggered.

@tianon
Copy link
Member

tianon commented Apr 26, 2016

Nice! LGTM 👍

@yosifkit
Copy link
Member

LGTM

@yosifkit yosifkit merged commit 85da414 into docker-library:master Apr 26, 2016
tianon added a commit to infosiftr/stackbrew that referenced this pull request Apr 26, 2016
@tianon
Update docker-library images
a293dbd 
- `cassandra`: 2.2.6 and 2.1.14
- `elasticsearch`: 2.3.2 (docker-library/elasticsearch#101)
- `hello-world`: update userguide link to avoid redirect (docker-library/hello-world#12)
- `java`: 7u101-2.6.6-1~deb8u1 (https://lists.debian.org/debian-security-announce/2016/msg00134.html)
- `logstash`: 2.3.2 (docker-library/logstash#46)
- `mariadb`: fix double install of `mysql-common` (MariaDB/mariadb-docker#56)
- `python`: fix removal of test folders (docker-library/python#102)
- `rocket.chat`: 0.28.0
- `ruby`: 2.3.1 and 2.2.5
- `tomcat`: add Tomcat Native Libraries (docker-library/tomcat#28)
- `wordpress`: 4.5.1
@tianon tianon mentioned this pull request Apr 26, 2016
Merged
RichardScothern pushed a commit to RichardScothern/official-images that referenced this pull request Jun 14, 2016
@tianon @rscothern
Update docker-library images
30f6763 
- `cassandra`: 2.2.6 and 2.1.14
- `elasticsearch`: 2.3.2 (docker-library/elasticsearch#101)
- `hello-world`: update userguide link to avoid redirect (docker-library/hello-world#12)
- `java`: 7u101-2.6.6-1~deb8u1 (https://lists.debian.org/debian-security-announce/2016/msg00134.html)
- `logstash`: 2.3.2 (docker-library/logstash#46)
- `mariadb`: fix double install of `mysql-common` (MariaDB/mariadb-docker#56)
- `python`: fix removal of test folders (docker-library/python#102)
- `rocket.chat`: 0.28.0
- `ruby`: 2.3.1 and 2.2.5
- `tomcat`: add Tomcat Native Libraries (docker-library/tomcat#28)
- `wordpress`: 4.5.1
VenusPR pushed a commit to VenusPR/Python_Richard that referenced this pull request Mar 9, 2023
@devsyncer
Fixes docker-library/python#102 (tests folder removal)
97c6fa3
ProActiveSpirit pushed a commit to ProActiveSpirit/python that referenced this pull request Mar 1, 2024
Fixes docker-library/python#102 (tests folder removal)
068852e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@sukrit007 @tianon @yosifkit
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy