Error in sqlparse setup command: use_2to3 is invalid.

[mko-x]

Hi wechange-eg @SaschaN !

I'm trying to setup cosinnus locally by following option B of the Readme.

I keep getting errors for sqlparse like e.g. for cosinnus-poll:

running egg_info
creating cosinnus_poll.egg-info
writing cosinnus_poll.egg-info/PKG-INFO
writing dependency_links to cosinnus_poll.egg-info/dependency_links.txt
writing requirements to cosinnus_poll.egg-info/requires.txt
writing top-level names to cosinnus_poll.egg-info/top_level.txt
writing manifest file 'cosinnus_poll.egg-info/SOURCES.txt'
reading manifest file 'cosinnus_poll.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*' under directory 'cosinnus_event/locale'
warning: no files found matching '*' under directory 'cosinnus_event/migrations'
warning: no files found matching '*' under directory 'cosinnus_event/static'
warning: no files found matching '*' under directory 'cosinnus_event/templates'
adding license file 'LICENSE'
adding license file 'AUTHORS'
writing manifest file 'cosinnus_poll.egg-info/SOURCES.txt'
running build_ext
Creating /Users/muka/Documents/repos/wechangeenv/lib/python3.9/site-packages/cosinnus-poll.egg-link (link to .)
Adding cosinnus-poll 0.1.dev20190707122435 to easy-install.pth file

Installed /Users/muka/Documents/repos/cosinnus-devops/cosinnus-poll
Processing dependencies for cosinnus-poll==0.1.dev20190707122435
Searching for sqlparse==0.1.10
Reading https://pypi.org/simple/sqlparse/
Downloading https://files.pythonhosted.org/packages/53/4f/5874394af1d54a8e7bd5563815c5439cb11b87e033019653b875f1a16c10/sqlparse-0.1.10.tar.gz#sha256=d159e104fbf1710bba2e4d410fe0371dcbff80457d4752a0bd414d88ddbdd476
Best match: sqlparse 0.1.10
Processing sqlparse-0.1.10.tar.gz
Writing /var/folders/gw/gf_0ld_54432v3d2zrddw_gw0000gn/T/easy_install-1jt8kzi4/sqlparse-0.1.10/setup.cfg
Running sqlparse-0.1.10/setup.py -q bdist_egg --dist-dir /var/folders/gw/gf_0ld_54432v3d2zrddw_gw0000gn/T/easy_install-1jt8kzi4/sqlparse-0.1.10/egg-dist-tmp-eczn7zf2
error: Setup script exited with error in sqlparse setup command: use_2to3 is invalid.

If I try to run the local setup the error reoccurs:

Collecting sqlparse==0.1.10
  Downloading sqlparse-0.1.10.tar.gz (53 kB)
     |████████████████████████████████| 53 kB 2.5 MB/s             
  Preparing metadata (setup.py) ... error
ERROR: Command errored out with exit status 1:
   command: /Users/muka/Documents/repos/wechangeenv/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/gw/gf_0ld_54432v3d2zrddw_gw0000gn/T/pip-install-ysc_vam3/sqlparse_544a043442d24c31bfb7c09f23b394cd/setup.py'"'"'; __file__='"'"'/private/var/folders/gw/gf_0ld_54432v3d2zrddw_gw0000gn/T/pip-install-ysc_vam3/sqlparse_544a043442d24c31bfb7c09f23b394cd/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/gw/gf_0ld_54432v3d2zrddw_gw0000gn/T/pip-pip-egg-info-nonn844i
       cwd: /private/var/folders/gw/gf_0ld_54432v3d2zrddw_gw0000gn/T/pip-install-ysc_vam3/sqlparse_544a043442d24c31bfb7c09f23b394cd/
  Complete output (1 lines):
  error in sqlparse setup command: use_2to3 is invalid.
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/53/4f/5874394af1d54a8e7bd5563815c5439cb11b87e033019653b875f1a16c10/sqlparse-0.1.10.tar.gz#sha256=d159e104fbf1710bba2e4d410fe0371dcbff80457d4752a0bd414d88ddbdd476 (from https://pypi.org/simple/sqlparse/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement sqlparse==0.1.10 (from versions: 0.1.0, 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5, 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.1.10, 0.1.11, 0.1.12, 0.1.13, 0.1.14, 0.1.15, 0.1.16, 0.1.17, 0.1.18, 0.1.19, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.3.0, 0.3.1, 0.4.0, 0.4.1, 0.4.2)
ERROR: No matching distribution found for sqlparse==0.1.10

Do you have any recommendation for handling that? Seems like some issue with the setup tools version but I'm not sure.

[saschasommer]

Hi @mko-x i just pushed an update to the local_setup.sh script that should avoid the issue you were having.

Can you try to complete the process and let me know if this helped you?

[mko-x]

Hey @SaschaN !
Thank you very much for the reply and the update. local_setup.sh now runs just fine.

Now I see a similar issue if I run the next command in the list at option B:

pip install -r ./cosinnus-devops/requirements_local.txt

Collecting six==1.11.0
  Using cached six-1.11.0-py2.py3-none-any.whl (10 kB) ### worked
Collecting sqlparse==0.1.10
  Using cached sqlparse-0.1.10.tar.gz (53 kB)
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/muka/Documents/repos/wechangeenv/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/gw/gf_0ld_54432v3d2zrddw_gw0000gn/T/pip-install-qwyyg1hj/sqlparse_dff025b31e3441db8bd46e265450d208/setup.py'"'"'; __file__='"'"'/private/var/folders/gw/gf_0ld_54432v3d2zrddw_gw0000gn/T/pip-install-qwyyg1hj/sqlparse_dff025b31e3441db8bd46e265450d208/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/gw/gf_0ld_54432v3d2zrddw_gw0000gn/T/pip-pip-egg-info-eou_avdn
       cwd: /private/var/folders/gw/gf_0ld_54432v3d2zrddw_gw0000gn/T/pip-install-qwyyg1hj/sqlparse_dff025b31e3441db8bd46e265450d208/
  Complete output (1 lines):
  error in sqlparse setup command: use_2to3 is invalid.
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/53/4f/5874394af1d54a8e7bd5563815c5439cb11b87e033019653b875f1a16c10/sqlparse-0.1.10.tar.gz#sha256=d159e104fbf1710bba2e4d410fe0371dcbff80457d4752a0bd414d88ddbdd476 (from https://pypi.org/simple/sqlparse/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement sqlparse==0.1.10 (from versions: 0.1.0, 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5, 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.1.10, 0.1.11, 0.1.12, 0.1.13, 0.1.14, 0.1.15, 0.1.16, 0.1.17, 0.1.18, 0.1.19, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.3.0, 0.3.1, 0.4.0, 0.4.1, 0.4.2)
ERROR: No matching distribution found for sqlparse==0.1.10

Maybe you have another fix for that issue?

[saschasommer]

Hi @mko-x,

I realized from your log that the submodules were mis-configured and didn't track the main branch of the cosinnus repositories as they should. This resulted in old requirements being used for pip install.

I fixed the submodules, so could you please pull the most recent committ from cosinnus-devops and update/re-initialze all of your cosinnus-submodules, and try again? Unless there is a specific incompatibility for your machine with a specific pip-package version, you shouldn't get any errors during the install process now!

Let me know if this helped you.
Best regards,
Sascha

[mko-x]

Hey @SaschaN,

thank you very much again for your reply.

I pulled the most recent commit from cosinnus-devopsand updated all the submodules within.

As I ran pip install -r ./cosinnus-devops/requirements_local.txtI received an error according to Celery dependency of dataclassesof version 0.8 are necessary but they're only provided up to 0.7.

I digged into it with verbose flag and found the dependency in Celery for running with Python <=3.6 & <3.7 - is that true?
I was running with CPython 3.9.9 (brewed) on macOS 11.6.1.

Then as I setup Python 3.6.8 on my machine I found the same sqlparse dependency error. Maybe I crashed my Python envs, so I will now try to set it up clean via pyenv again. Do you have a recommended Python version?

A lot of thanks and warm regards,
muka

[saschasommer]

@mko-x I think python 3.8.* should work the best, maybe best give it a shot with that. I don't think we've tried this build process on 3.9.* yet.

[mko-x]

Hey @SaschaN

If I use 3.8.6 I get the following error:

cosinnus-devops on  master [?] via weenv via 🐍 3.8.6 
➜ pyenv exec pip install -r ./requirements_local.txt                                    
Collecting Django<2.2,>=2.1.5
  Using cached Django-2.1.15-py3-none-any.whl (7.3 MB)
Collecting MarkupSafe==1.1
  Using cached MarkupSafe-1.1.0.tar.gz (18 kB)
Collecting Pillow==6.2.0
  Using cached Pillow-6.2.0.tar.gz (37.4 MB)
Collecting Celery==4.2.0
  Using cached celery-4.2.0-py2.py3-none-any.whl (401 kB)
ERROR: Could not find a version that satisfies the requirement dataclasses==0.8 (from -r ././cosinnus-core/requirements.txt (line 7)) (from versions: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6)
ERROR: No matching distribution found for dataclasses==0.8 (from -r ././cosinnus-core/requirements.txt (line 7))
WARNING: You are using pip version 20.2.1; however, version 21.3.1 is available.
You should consider upgrading via the '/Users/muka/.pyenv/versions/3.8.6/bin/python3.8 -m pip install --upgrade pip' command.

If using 3.8.12 it is the same:

cosinnus-devops on  master [?] via weenv via 🐍 3.8.12 
➜ pyenv exec pip install -r ./requirements_local.txt
...
Collecting MarkupSafe==1.1
  Using cached MarkupSafe-1.1.0.tar.gz (18 kB)
Collecting Pillow==6.2.0
  Using cached Pillow-6.2.0.tar.gz (37.4 MB)
Collecting Celery==4.2.0
  Using cached celery-4.2.0-py2.py3-none-any.whl (401 kB)
ERROR: Could not find a version that satisfies the requirement dataclasses==0.8 (from versions: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6)
ERROR: No matching distribution found for dataclasses==0.8
WARNING: You are using pip version 21.1.1; however, version 21.3.1 is available.
You should consider upgrading via the '/Users/muka/.pyenv/versions/3.8.12/bin/python3.8 -m pip install --upgrade pip' command.

As I read in the docs dataclasses 0.8 is a backport of dataclasses available from Python 3.7 and above. See https://pypi.org/project/dataclasses/#description

I would really like to contribute but it seems hard to do so. How is this possible to work at your machine if using Python 3.8.x with 0.8 dataclasses? The developer states at the github readme "As of version 0.4, this code no longer works with Python 3.7. For 3.7, use the built-in dataclasses module." I am confused XD

[saschasommer]

@mko-x I've updated cosinnus-core's requirements.txt. This was a change that hadn't made it into the 'main' branch yet.

Please update your cosinnus-core submodule and try again! Thank you for your feedback and patience!

[mko-x]

Hey @SaschaN !

Once again, great thanks to you.

Now I have a dependency issue with wagtail and beuatifulsoup in the log:

ERROR: Cannot install -r ././cosinnus-core/requirements.txt (line 73) and beautifulsoup4==4.8.1 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested beautifulsoup4==4.8.1
    wagtail 2.3 depends on beautifulsoup4<4.6.1 and >=4.5.1

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
WARNING: You are using pip version 21.1.1; however, version 21.3.1 is available.
You should consider upgrading via the '/Users/muka/.pyenv/versions/3.8.12/bin/python3.8 -m pip install --upgrade pip' command.

Do you have a hint for wagtail?

I commented the line 73 (i.e. wagtail) in cosinnus-core/requirements.txt so it starts compilation, shows some warnings but fails with a linker error for (I guess) ssl lib at psycopg compilation.

clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -I/usr/local/opt/zlib/include -I/usr/local/opt/bzip2/include -DPSYCOPG_VERSION=2.8.3 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=140001 -DHAVE_LO64=1 -I/Users/muka/.pyenv/versions/3.8.12/include/python3.8 -I. -I/usr/local/include -I/usr/local/include/postgresql/server -c psycopg/typecast.c -o build/temp.macosx-11.6-x86_64-3.8/psycopg/typecast.o
    clang -bundle -undefined dynamic_lookup -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/Users/muka/.pyenv/versions/3.8.12/lib -L/usr/local/opt/zlib/lib -L/usr/local/opt/bzip2/lib -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/Users/muka/.pyenv/versions/3.8.12/lib -L/usr/local/opt/zlib/lib -L/usr/local/opt/bzip2/lib -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib -L/usr/local/opt/zlib/lib -L/usr/local/opt/bzip2/lib -I/usr/local/opt/zlib/include -I/usr/local/opt/bzip2/include build/temp.macosx-11.6-x86_64-3.8/psycopg/psycopgmodule.o build/temp.macosx-11.6-x86_64-3.8/psycopg/green.o build/temp.macosx-11.6-x86_64-3.8/psycopg/pqpath.o build/temp.macosx-11.6-x86_64-3.8/psycopg/utils.o build/temp.macosx-11.6-x86_64-3.8/psycopg/bytes_format.o build/temp.macosx-11.6-x86_64-3.8/psycopg/libpq_support.o build/temp.macosx-11.6-x86_64-3.8/psycopg/win32_support.o build/temp.macosx-11.6-x86_64-3.8/psycopg/solaris_support.o build/temp.macosx-11.6-x86_64-3.8/psycopg/connection_int.o build/temp.macosx-11.6-x86_64-3.8/psycopg/connection_type.o build/temp.macosx-11.6-x86_64-3.8/psycopg/cursor_int.o build/temp.macosx-11.6-x86_64-3.8/psycopg/cursor_type.o build/temp.macosx-11.6-x86_64-3.8/psycopg/column_type.o build/temp.macosx-11.6-x86_64-3.8/psycopg/replication_connection_type.o build/temp.macosx-11.6-x86_64-3.8/psycopg/replication_cursor_type.o build/temp.macosx-11.6-x86_64-3.8/psycopg/replication_message_type.o build/temp.macosx-11.6-x86_64-3.8/psycopg/diagnostics_type.o build/temp.macosx-11.6-x86_64-3.8/psycopg/error_type.o build/temp.macosx-11.6-x86_64-3.8/psycopg/conninfo_type.o build/temp.macosx-11.6-x86_64-3.8/psycopg/lobject_int.o build/temp.macosx-11.6-x86_64-3.8/psycopg/lobject_type.o build/temp.macosx-11.6-x86_64-3.8/psycopg/notify_type.o build/temp.macosx-11.6-x86_64-3.8/psycopg/xid_type.o build/temp.macosx-11.6-x86_64-3.8/psycopg/adapter_asis.o build/temp.macosx-11.6-x86_64-3.8/psycopg/adapter_binary.o build/temp.macosx-11.6-x86_64-3.8/psycopg/adapter_datetime.o build/temp.macosx-11.6-x86_64-3.8/psycopg/adapter_list.o build/temp.macosx-11.6-x86_64-3.8/psycopg/adapter_pboolean.o build/temp.macosx-11.6-x86_64-3.8/psycopg/adapter_pdecimal.o build/temp.macosx-11.6-x86_64-3.8/psycopg/adapter_pint.o build/temp.macosx-11.6-x86_64-3.8/psycopg/adapter_pfloat.o build/temp.macosx-11.6-x86_64-3.8/psycopg/adapter_qstring.o build/temp.macosx-11.6-x86_64-3.8/psycopg/microprotocols.o build/temp.macosx-11.6-x86_64-3.8/psycopg/microprotocols_proto.o build/temp.macosx-11.6-x86_64-3.8/psycopg/typecast.o -L/usr/local/lib -lpq -lssl -lcrypto -o build/lib.macosx-11.6-x86_64-3.8/psycopg2/_psycopg.cpython-38-darwin.so
    ld: library not found for -lssl
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    error: command 'clang' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/muka/.pyenv/versions/3.8.12/bin/python3.8 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/gw/gf_0ld_54432v3d2zrddw_gw0000gn/T/pip-install-muz4sduf/psycopg2_a88b1dc7c0674c2f99538ffbdc023831/setup.py'"'"'; __file__='"'"'/private/var/folders/gw/gf_0ld_54432v3d2zrddw_gw0000gn/T/pip-install-muz4sduf/psycopg2_a88b1dc7c0674c2f99538ffbdc023831/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/gw/gf_0ld_54432v3d2zrddw_gw0000gn/T/pip-record-_aiht661/install-record.txt --single-version-externally-managed --compile --install-headers /Users/muka/.pyenv/versions/3.8.12/include/python3.8/psycopg2 Check the logs for full command output.
WARNING: You are using pip version 21.1.1; however, version 21.3.1 is available.
You should consider upgrading via the '/Users/muka/.pyenv/versions/3.8.12/bin/python3.8 -m pip install --upgrade pip' command.

Seems like the ssl lib path is missing, maybe in the LDFLAGS- what do you think?