size of executable using pyinstaller and numpy

Question

I wrote a short program main.py using numpy and Qt:

from PyQt5 import QtWidgets
import numpy as np
import sys

if __name__ == '__main__':
    app = QtWidgets.QApplication(sys.argv)
    w = QtWidgets.QTextEdit('hello')
    w.show()
    sys.exit(app.exec_())

When I use pyinstaller --windowed main.py the resulting dist folder has a size of 390MB.

If I remove import numpy as np the resulting dist folder is only 70MB.

Is there any way to reduce the size while still using numpy? Maybe by only including the dlls I really use?

Basically even the most simple functionalities in numpy are based on some BLAS library (and some more complex: LAPACK). These are huge and some (MKL) even bigger than others (OpenBLAS). — sascha
– sascha, Commented Dec 14, 2017 at 2:57
So no chance to decrease the executable size and keeping it standalone? — Jonas
– Jonas, Commented Dec 14, 2017 at 9:38
I have the same question, did you find any workarounds? - besides factoring out Numpy altogether in the codebase... — balletpiraat
– balletpiraat, Commented Oct 10, 2018 at 10:04
See link. It turns out that the size of the executable depends mostly on the type of library that is used for numpy, as sachsa explained. Some people manage to get a <100mb executable by making sure OpenBLAS is used instead of e.g. MKL. — balletpiraat
– balletpiraat, Commented Oct 11, 2018 at 9:55
related: stackoverflow.com/a/67954011, stackoverflow.com/q/62262398 — djvg
– djvg, Commented Feb 14, 2022 at 10:01

Jonas · Accepted Answer · 2018-10-19 20:46:15Z

7

Thanks to the user balletpiraat I found a solution.

Install numpy with: conda install -c conda-forge numpy

To test this I created two anaconda environments. Environment "normalnumpy" was created with:

conda create -n normalnumpy python=3.7
activate normalnumpy
conda install numpy
pip install pyinstaller

resulting in:

altgraph                  0.16.1                    <pip>
blas                      1.0                         mkl
certifi                   2018.10.15               py37_0
future                    0.16.0                    <pip>
icc_rt                    2017.0.4             h97af966_0
intel-openmp              2019.0                      118
macholib                  1.11                      <pip>
mkl                       2019.0                      118
mkl_fft                   1.0.6            py37hdbbee80_0
mkl_random                1.0.1            py37h77b88f5_1
numpy                     1.15.2           py37ha559c80_0
numpy-base                1.15.2           py37h8128ebf_0
pefile                    2018.8.8                  <pip>
pip                       10.0.1                   py37_0
PyInstaller               3.4                       <pip>
python                    3.7.0                hea74fb7_0
pywin32-ctypes            0.2.0                     <pip>
setuptools                40.4.3                   py37_0
vc                        14.1                 h0510ff6_4
vs2015_runtime            14.15.26706          h3a45250_0
wheel                     0.32.1                   py37_0
wincertstore              0.2                      py37_0

Environment "extranumpy" was created with:

conda create -n extranumpy python=3.7
activate extranumpy
conda install -c conda-forge numpy
pip install pyinstaller

resulting in:

altgraph                  0.16.1                    <pip>
blas                      1.1                    openblas    conda-forge
certifi                   2018.10.15            py37_1000    conda-forge
future                    0.16.0                    <pip>
libflang                  5.0.0             vc14_20180208  [vc14]  conda-forge
llvm-meta                 5.0.0                         0    conda-forge
macholib                  1.11                      <pip>
numpy                     1.15.2          py37_blas_openblash8d851b4_1  [blas_openblas]  conda-forge
openblas                  0.2.20                   vc14_8  [vc14]  conda-forge
openmp                    5.0.0                    vc14_1  [vc14]  conda-forge
pefile                    2018.8.8                  <pip>
pip                       10.0.1                   py37_0
PyInstaller               3.4                       <pip>
python                    3.7.0                hea74fb7_0
pywin32-ctypes            0.2.0                     <pip>
setuptools                40.4.3                   py37_0
vc                        14                            0    conda-forge
vs2015_runtime            14.15.26706          h3a45250_0
wheel                     0.32.1                   py37_0
wincertstore              0.2                      py37_0

I tested both environments with "main.py":

import numpy as np
if __name__ == '__main__':
    test = np.array([42])
    print(test)

and

pyinstaller --onefile main.py

the "normalnumpy" executable is 228MB, the "extranumpy" executable is 10MB.

answered Oct 19, 2018 at 20:46

Jonas

1,8774 gold badges23 silver badges37 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Miffy Over a year ago

Will the exe file be stored in Dist directory like normal?

Miffy Over a year ago

I have created (base) environment and kind of stuck. How do I get out?

Jonas Over a year ago

@Miffy There are many tutorials how to create an environment. If you did that you have to activate it and run pyinstaller like normal. The files will be in a dist folder also like running it from your "normal" python installation. All commands and code are posted in my answer. I suggest you try the example and continue from there.

Jonas Over a year ago

@Miffy You should ask a separate question about that, its hard to resolve without the error message and in the comments.

Steven C. Howell Over a year ago

I had to add the constraint blas=*=openblas to prevent mkl from being installed. Found this recommendation here.

|

user9045698 · Accepted Answer · 2017-12-14 02:54:04Z

0

Have you tried excluding modules? This is a common problem. Also you might want to check out:

Reducing size of pyinstaller exe

Worth mention, are you using Anaconda? A fresh oracle box python system and less of a huge file.

I believe in there docs somewhere you can adjust where the directories are found.

https://pythonhosted.org/PyInstaller/spec-files.html

answered Dec 14, 2017 at 2:54

user9045698

2 Comments

Jonas Over a year ago

If I exclude modules the executable is no longer standalone, can I exclude the parts of eg numpy that I dont use?

user9045698 Over a year ago

@Jonas You can exclude modules sure, but you cannot exclude parts of numpy. Maybe in theory you could.. no that'd break numpy. According to the post I linked, UPX and virtual environment and exclusion are the general methods to reduce size.

Collectives™ on Stack Overflow

size of executable using pyinstaller and numpy

2 Answers 2

6 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related