Nuitka #9

oleksis · 2022-12-14T10:38:36Z

oleksis
Dec 14, 2022
Maintainer

I try the new version Nuitka 1.2.7 on one this project. Here the Docs with Nuitka

This time i try with Visual Studio 2022 Preview

The command and the output below

➜ python -m nuitka --onefile --windows-icon-from-ico=./assets/picta-dl.ico  --windows-company-name=Picta-DL --windows-product-name=picta-dl --windows-product-version=2021.1.25.0 --windows-file-description="Picta video downloader" picta_dl/__main__.py
Nuitka-Options:INFO: Used command line options: --onefile --windows-icon-from-ico=./assets/picta-dl.ico --windows-company-name=Picta-DL --windows-product-name=picta-dl --windows-product-version=2021.1.25.0 --windows-file-description="Picta video downloader" picta_dl/__main__.py
Nuitka:WARNING: Using very slow fallback for ordered sets, please install 'ordered-set' PyPI package for best Python compile time performance.
Nuitka:INFO: Starting Python compilation with Nuitka '1.2.7' on Python '3.10' commercial grade 'not installed'.
Nuitka-Plugins:INFO: multiprocessing: Injecting pre-module load code for module 'multiprocessing':
Nuitka-Plugins:INFO: multiprocessing:     Monkey patching "multiprocessing" load environment.
Nuitka-Plugins:INFO: multiprocessing: Injecting post-module load code for module 'multiprocessing':
Nuitka-Plugins:INFO: multiprocessing:     Monkey patching "multiprocessing" for compiled methods.
Nuitka:INFO: Completed Python level compilation and optimization.                              
Nuitka:INFO: Generating source code for C backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.                      
Nuitka:INFO: Running C compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: cl (cl 14.3).
Nuitka-Scons:INFO: Backend linking program with 157 files (no progress information available).
Nuitka-Scons:INFO: Running 'link @~\\AppData\\Local\\Temp\\tmppx8_79z7.lnk' took 713.71 seconds
Nuitka-Scons:INFO: Compiled 157 C files using clcache with 0 cache hits and 157 cache misses.
Nuitka-Postprocessing:INFO: Adding 1 icon(s) from icon file './assets/picta-dl.ico'.
Nuitka-Plugins:INFO: data-files: Included data file 'Cryptodome\Util\.\.keep_dir.txt' due to empty dir needed for 'Cryptodome.Util._raw_api'.
Nuitka-Plugins:INFO: data-files: Included data file 'certifi\cacert.pem' due to package data for 'certifi'.
Detecting used DLLs: 0.0%|                         | 0/38, __main__.exeNuitka will make use of Dependency Walker (https://dependencywalker.com) tool
to analyze the dependencies of Python extension modules.

Is it OK to download and put it in '~\AppData\Local\Nuitka\Nuitka\Cache\downloads\depends\x86_64'.

No installer needed, cached, one time question.

Proceed and download? [Yes]/No
Yes
Nuitka:INFO: Downloading 'https://dependencywalker.com/depends22_x64.zip'.
Nuitka:INFO: Extracting to '~\AppData\Local\Nuitka\Nuitka\Cache\downloads\depends\x86_64\depends.exe'
Nuitka-Postprocessing:INFO: Creating single file from dist folder, this may take a while.           
Nuitka-Onefile:INFO: Running bootstrap binary compilation via Scons.
Nuitka-Onefile:WARNING: Onefile mode cannot compress without 'zstandard' module installed.
Nuitka-Scons:INFO: Onefile C compiler: cl (cl 14.3).
Nuitka-Scons:INFO: Onefile linking program with 1 files (no progress information available).
Nuitka-Scons:INFO: Compiled 1 C files using clcache with 0 cache hits and 1 cache misses.
Nuitka-Onefile:INFO: Keeping onefile build directory '__main__.onefile-build'.
Nuitka-Postprocessing:INFO: Adding 1 icon(s) from icon file './assets/picta-dl.ico'.
Nuitka:INFO: Keeping dist folder '__main__.dist' for inspection, no need to use it.                   
Nuitka:INFO: Keeping build directory '__main__.build'.
Nuitka:INFO: Successfully created '__main__.exe'.

I still tinking why the bundle of Nuitka is more big than PyInstaller option

PyInstaller	Nuitka
9.53602886199951 MB	34.9740676879883 MB

Some comments or advices ?

kayhayen · 2022-12-14T16:22:03Z

kayhayen
Dec 14, 2022

I need to make the compilation report include DLL sizes. Maybe Nuitka is more dumb about what it picks to include. Can you compare the DLLs included in their onefile and nuitka standalone options? I noticed that the delveplugin duplicated some things on Windows.

As an aside, main.exe looks wrong. I suspect you wanted to use --python-flag=-m mayhaps, then Nuitka would know it's an executable package.

0 replies

oleksis · 2022-12-14T23:05:03Z

oleksis
Dec 14, 2022
Maintainer Author

PyInstaller no UPX and create a one-folder bundle containing an executable

pyinstaller -w -D --noupx --icon=assets/picta-dl.ico --exclude-module=test --name=picta-dl picta_dl/__main__.py

1331 INFO: PyInstaller: 5.6.2
1331 INFO: Python: 3.10.2
1399 INFO: Platform: Windows-10-10.0.25252-SP0
1403 INFO: wrote ~\PYTHON\picta-dl\picta-dl.spec
1755 INFO: UPX is available.
1797 INFO: Extending PYTHONPATH with paths
['~\\PYTHON\\picta-dl']
4468 INFO: checking Analysis
4605 INFO: Building because inputs changed
4605 INFO: Initializing module dependency graph...
4624 INFO: Caching module graph hooks...
4804 INFO: Analyzing base_library.zip ...
13265 INFO: Loading module hook 'hook-heapq.py' from '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\PyInstaller\\hooks'...
14218 INFO: Loading module hook 'hook-encodings.py' from '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\PyInstaller\\hooks'...
24867 INFO: Loading module hook 'hook-pickle.py' from '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\PyInstaller\\hooks'...
30630 INFO: Loading module hook 'hook-sysconfig.py' from '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\PyInstaller\\hooks'...
31068 INFO: Loading module hook 'hook-importlib_metadata.py' from '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\PyInstaller\\hooks'...
31584 INFO: Loading module hook 'hook-platform.py' from '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\PyInstaller\\hooks'...
39898 INFO: Caching module dependency graph...
40784 INFO: running Analysis Analysis-00.toc
40862 INFO: Adding Microsoft.Windows.Common-Controls to dependent assemblies of final executable
  required by ~\AppData\Local\Programs\Python\Python310\python.exe
42019 INFO: Analyzing ~\PYTHON\picta-dl\picta_dl\__main__.py
43261 INFO: Loading module hook 'hook-xml.py' from '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\PyInstaller\\hooks'...
43558 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\PyInstaller\\hooks'...
46144 INFO: Loading module hook 'hook-multiprocessing.util.py' from '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\PyInstaller\\hooks'...
50563 INFO: Loading module hook 'hook-certifi.py' from '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
51031 INFO: Loading module hook 'hook-Cryptodome.py' from '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
60186 INFO: Loading module hook 'hook-sqlite3.py' from '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\PyInstaller\\hooks'...
64858 INFO: Loading module hook 'hook-websockets.py' from '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
76299 INFO: Processing module hooks...
78713 INFO: Looking for ctypes DLLs
78963 INFO: Analyzing run-time hooks ...
78999 INFO: Including run-time hook '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\PyInstaller\\hooks\\rthooks\\pyi_rth_inspect.py'
79071 INFO: Including run-time hook '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\PyInstaller\\hooks\\rthooks\\pyi_rth_subprocess.py'
79101 INFO: Including run-time hook '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\PyInstaller\\hooks\\rthooks\\pyi_rth_pkgutil.py'
79133 INFO: Including run-time hook '~\\PYTHON\\picta-dl\\venv\\lib\\site-packages\\PyInstaller\\hooks\\rthooks\\pyi_rth_multiprocessing.py'
79265 INFO: Looking for dynamic libraries
4153 INFO: Extra DLL search directories (AddDllDirectory): []
4153 INFO: Extra DLL search directories (PATH): ['~\\PYTHON\\picta-dl\\venv\\Scripts', 'C:\\Program Files\\PowerShell\\7', 'C:\\Program Files\\Common Files\\Oracle\\Java\\javapath', 'C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath', 'C:\\Program Files (x86)\\Microsoft SDKs\\Azure\\CLI2\\wbin', 'C:\\Windows\\System32', 'C:\\Windows', 'C:\\Windows\\System32\\wbem', 'C:\\ProgramData\\Oracle\\Java\\javapath', 'C:\\Program Files (x86)\\Intel\\iCLS Client', 'C:\\Program Files\\Intel\\iCLS Client', 'C:\\Windows\\System32\\WindowsPowerShell\\v1.0', 'C:\\Program Files\\Intel\\WiFi\\bin', 'C:\\Program Files\\Common Files\\Intel\\WirelessCommon', 'C:\\Program Files\\Intel\\Intel(R) Management Engine Components\\DAL', 'C:\\Program Files (x86)\\Intel\\Intel(R) Management Engine Components\\DAL', 'C:\\Program Files\\Intel\\Intel(R) Management Engine Components\\IPT', 'C:\\Program Files (x86)\\Intel\\Intel(R) Management Engine Components\\IPT', 'C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\', 'C:\\Windows\\System32\\OpenSSH\\', 'C:\\Program Files\\Intel\\WiFi\\bin\\', 'C:\\Program Files\\Common Files\\Intel\\WirelessCommon\\', 'C:\\Qt\\5.15.2\\msvc2019_64\\bin\\', 'C:\\Qt\\vcpkg\\', 'C:\\Qt\\vcpkg\\installed\\x64-windows\\tools\\protobuf', 'C:\\Qt\\vcpkg\\installed\\x64-windows\\tools\\openssl', 'C:\\Qt\\Tools\\CMake_64\\bin\\', 'C:\\Program Files\\Microsoft VS Code\\bin', 'C:\\Program Files (x86)\\GnuWin32\\bin', '~\\bin', 'C:\\Program Files\\PostgreSQL\\14\\bin', 'C:\\Program Files\\Vim\\vim82', 'C:\\Program Files\\dotnet\\', 'C:\\Program Files (x86)\\GnuPG\\bin', 'C:\\Program Files\\PuTTY\\', 'C:\\Program Files\\nodejs\\', 'C:\\WINDOWS\\system32', 'C:\\WINDOWS', 'C:\\WINDOWS\\System32\\Wbem', 'C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\', 'C:\\WINDOWS\\System32\\OpenSSH\\', 'C:\\xampp\\php', 'C:\\xampp\\mysql\\bin', 'C:\\WINDOWS\\system32', 'C:\\WINDOWS', 'C:\\WINDOWS\\System32\\Wbem', 'C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\', 'C:\\WINDOWS\\System32\\OpenSSH\\', 'C:\\Program Files\\Microsoft\\Azure Functions Core Tools\\', 'C:\\Program Files\\apache-maven-3.8.6\\bin', 'C:\\Program Files\\heroku\\client\\bin', '~\\.vscode-server-launcher\\bin', 'C:\\Program Files\\PowerShell\\7\\', 'C:\\Program Files\\GitHub CLI\\', 'C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit\\', 'C:\\Program Files\\Git\\cmd', 'C:\\Program Files\\Docker\\Docker\\resources\\bin', '~\\AppData\\Local\\Programs\\Python\\Python310\\Scripts\\', '~\\AppData\\Local\\Programs\\Python\\Python310\\', '~\\AppData\\Roaming\\.dotnet', '~\\AppData\\Local\\Microsoft\\WindowsApps', 'C:\\Program Files\\Python39', '~\\AppData\\Local\\GitHubDesktop\\bin', '~\\.dotnet\\tools', 'C:\\Program Files\\heroku\\bin', 'C:\\Program Files (x86)\\Nmap', '~\\AppData\\Roaming\\npm', '~\\AppData\\Local\\Programs\\Rancher Desktop\\resources\\resources\\win32\\bin', '~\\AppData\\Local\\Programs\\Rancher Desktop\\resources\\resources\\linux\\bin', '~\\AppData\\Local\\Programs\\oh-my-posh\\bin', 'c:\\dapr', '~\\.dapr\\bin', '~\\AppData\\Roaming\\Code\\User\\globalStorage\\ms-vscode-remote.remote-containers\\cli-bin', '~\\AppData\\Local\\Microsoft\\WindowsApps', '~\\AppData\\Local\\Programs\\Microsoft VS Code Insiders\\bin', '~\\.dotnet\\tools']
87819 INFO: Looking for eggs
87819 INFO: Using Python library ~\AppData\Local\Programs\Python\Python310\python310.dll
87819 INFO: Found binding redirects:
[]
87859 INFO: Warnings written to ~\PYTHON\picta-dl\build\picta-dl\warn-picta-dl.txt
88221 INFO: Graph cross-reference written to ~\PYTHON\picta-dl\build\picta-dl\xref-picta-dl.html
88578 INFO: checking PYZ
88670 INFO: Building because name changed
88671 INFO: Building PYZ (ZlibArchive) ~\PYTHON\picta-dl\build\picta-dl\PYZ-00.pyz
91647 INFO: Building PYZ (ZlibArchive) ~\PYTHON\picta-dl\build\picta-dl\PYZ-00.pyz completed successfully.
91782 INFO: checking PKG
91871 INFO: Building because name changed
91872 INFO: Building PKG (CArchive) picta-dl.pkg
92200 INFO: Building PKG (CArchive) picta-dl.pkg completed successfully.
92221 INFO: Bootloader ~\PYTHON\picta-dl\venv\lib\site-packages\PyInstaller\bootloader\Windows-64bit\runw.exe
92223 INFO: checking EXE
92332 INFO: Rebuilding EXE-00.toc because picta-dl.exe missing
92347 INFO: Building EXE from EXE-00.toc
92397 INFO: Copying bootloader EXE to ~\PYTHON\picta-dl\build\picta-dl\picta-dl.exe.notanexecutable
93215 INFO: Copying icon to EXE
93235 INFO: Copying icons from ['~\\PYTHON\\picta-dl\\assets\\picta-dl.ico']
93695 INFO: Writing RT_GROUP_ICON 0 resource with 20 bytes
93696 INFO: Writing RT_ICON 1 resource with 37000 bytes
93709 INFO: Copying 0 resources to EXE
93709 INFO: Embedding manifest in EXE
93712 INFO: Updating manifest in ~\PYTHON\picta-dl\build\picta-dl\picta-dl.exe.notanexecutable
94038 INFO: Updating resource type 24 name 1 language 0
94073 INFO: Appending PKG archive to EXE
94107 INFO: Fixing EXE headers
96584 INFO: Building EXE from EXE-00.toc completed successfully.
96603 INFO: checking COLLECT
96606 INFO: Building COLLECT because COLLECT-00.toc is non existent
96611 INFO: Building COLLECT COLLECT-00.toc
109679 INFO: Building COLLECT COLLECT-00.toc completed successfully.

Nuitka without `zstandard`

See this comments

Reproducibe builds with diffoscope

output-Nuitka-vs-pyinstaller.html V1

pip install diffoscope

diffoscope --html output.html --exclude '__pycache__' ./__main__.dist ./dist/picta-dl

Directory size

Nuitka	pyinstaller
34.9 MB	21.2 MB

0 replies

oleksis · 2022-12-15T03:28:25Z

oleksis
Dec 15, 2022
Maintainer Author

With compression

pip install zstandard

python -m nuitka --onefile --python-flag=-m --windows-icon-from-ico=./assets/picta-dl.ico  --windows-company-name=Picta-DL --windows-product-name=picta-dl --windows-product-version=2021.1.25.0 --windows-file-description="Picta video downloader" picta_dl

Nuitka-Options:INFO: Used command line options: --onefile --python-flag=-m --windows-icon-from-ico=./assets/picta-dl.ico --windows-company-name=Picta-DL --windows-product-name=picta-dl --windows-product-version=2021.1.25.0 --windows-file-description="Picta video downloader" picta_dl
Nuitka:WARNING: Using very slow fallback for ordered sets, please install 'ordered-set' PyPI package for best Python compile time
Nuitka:WARNING: performance.
Nuitka:INFO: Starting Python compilation with Nuitka '1.2.7' on Python '3.10' commercial grade 'not installed'.
Nuitka-Plugins:INFO: multiprocessing: Injecting pre-module load code for module 'multiprocessing':
Nuitka-Plugins:INFO: multiprocessing:     Monkey patching "multiprocessing" load environment.
Nuitka-Plugins:INFO: multiprocessing: Injecting post-module load code for module 'multiprocessing':
Nuitka-Plugins:INFO: multiprocessing:     Monkey patching "multiprocessing" for compiled methods.
Nuitka:INFO: Completed Python level compilation and optimization.
Nuitka:INFO: Generating source code for C backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.
Nuitka:INFO: Running C compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: cl (cl 14.3).
Nuitka-Scons:INFO: Backend linking program with 157 files (no progress information available).
Nuitka-Scons:INFO: Running 'link @~\\AppData\\Local\\Temp\\tmpeulgw5l4.lnk' took 755.53 seconds
Nuitka-Scons:INFO: Compiled 157 C files using clcache with 39 cache hits and 118 cache misses.
Nuitka-Postprocessing:INFO: Adding 1 icon(s) from icon file './assets/picta-dl.ico'.
Nuitka-Plugins:INFO: data-files: Included data file 'Cryptodome\Util\.\.keep_dir.txt' due to empty dir needed for 'Cryptodome.Util._raw_api'.
Nuitka-Plugins:INFO: data-files: Included data file 'certifi\cacert.pem' due to package data for 'certifi'.
Nuitka-Postprocessing:INFO: Creating single file from dist folder, this may take a while.
Nuitka-Onefile:INFO: Running bootstrap binary compilation via Scons.
Nuitka-Scons:INFO: Onefile C compiler: cl (cl 14.3).
Nuitka-Scons:INFO: Onefile linking program with 1 files (no progress information available).
Nuitka-Scons:INFO: Compiled 1 C files using clcache with 1 cache hits and 0 cache misses.
Nuitka-Onefile:INFO: Keeping onefile build directory 'picta_dl.onefile-build'.
Nuitka-Postprocessing:INFO: Adding 1 icon(s) from icon file './assets/picta-dl.ico'.
Nuitka-Onefile:INFO: Using compression for onefile payload.
Nuitka-Onefile:INFO: Onefile payload compression ratio (31.44%) size 36485048 to 11472098.
Nuitka:INFO: Keeping dist folder 'picta_dl.dist' for inspection, no need to use it.
Nuitka:INFO: Keeping build directory 'picta_dl.build'.
Nuitka:INFO: Successfully created 'picta_dl.exe'.

Reproducibe builds with diffoscope

output-Nuitka-vs-pyinstaller.html V2

diffoscope --html output.html --exclude '__pycache__' ./picta_dl.dist ./dist/picta-dl

Directory size

Nuitka	pyinstaller
34.9 MB	21.2 MB

Binary size

Binary	Build	Size
picta_dl.exe	Nuitka	11.1 MB
picta-dl.exe	pyinstaller	9.53 MB

0 replies

kayhayen · 2022-12-15T05:33:27Z

kayhayen
Dec 15, 2022

That is reasonable. Still makes sense to look at files that only Nuitka includes, but I understand you did that. Compiled via C the code is larger than Python bytecode, of course, so it was never going to be smaller if both are handling everything else optimally.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Nuitka #9

Uh oh!

{{title}}

Uh oh!

Replies: 4 comments

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Nuitka #9

Uh oh!

oleksis Dec 14, 2022 Maintainer

Replies: 4 comments

Uh oh!

kayhayen Dec 14, 2022

Uh oh!

Uh oh!

oleksis Dec 14, 2022 Maintainer Author

PyInstaller no UPX and create a one-folder bundle containing an executable

Nuitka without zstandard

Reproducibe builds with diffoscope

Directory size

Uh oh!

oleksis Dec 15, 2022 Maintainer Author

With compression

Reproducibe builds with diffoscope

Directory size

Binary size

Uh oh!

kayhayen Dec 15, 2022

oleksis
Dec 14, 2022
Maintainer

kayhayen
Dec 14, 2022

oleksis
Dec 14, 2022
Maintainer Author

Nuitka without `zstandard`

oleksis
Dec 15, 2022
Maintainer Author

kayhayen
Dec 15, 2022