Setup failure

[adeprez]

Hello,

I would like to try jRDF2Vec for an open source project leveraging RDF embeddings. To make somes tests I wanted to use it through command-line interface. It fails when I check the installation using java -jar jrdf2vec-1.3-SNAPSHOT.jar -checkInstallation.

Python and Java are both installed with versions matching the docs, and I tried on both Windows and Mac computers with similar errors.

jRDF2Vec version : 1.3
OS : Mac & Windows
Java version : tried with 11 & 1.8.0_291
Python version : 3.12.1

1. pkg_resources is missing

Installing the setuptools dependency fixed it

2. It looks like the requirements_file is absent from the request headers.

Python server logs :

2023-12-13 12:28:41,822 INFO:Received argument: 1808
2023-12-13 12:28:41,823 INFO:Starting server using port 1808
2023-12-13 12:28:41,826 INFO:�[31m�[1mWARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.�[0m
 * Running on http://127.0.0.1:1808
2023-12-13 12:28:41,826 INFO:�[33mPress CTRL+C to quit�[0m
2023-12-13 12:28:50,649 INFO:127.0.0.1 - - [13/Dec/2023 12:28:50] "GET /melt_ml.html HTTP/1.1" 200 -
2023-12-13 12:28:50,660 INFO:received requirements file path: None
2023-12-13 12:28:50,661 ERROR:Exception on /check-requirements [GET]
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/flask/app.py", line 1455, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/flask/app.py", line 869, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/flask/app.py", line 867, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/flask/app.py", line 852, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adeprez/python-server/python_server.py", line 54, in check_requirements
    with pathlib.Path(requirements_file).open() as requirements_txt:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/pathlib.py", line 1162, in __init__
    super().__init__(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/pathlib.py", line 373, in __init__
    raise TypeError(
TypeError: argument should be a str or an os.PathLike object where __fspath__ returns a str, not 'NoneType'
2023-12-13 12:28:50,663 INFO:127.0.0.1 - - [13/Dec/2023 12:28:50] "�[35m�[1mGET /check-requirements HTTP/1.1�[0m" 500 -

3. The check fails with error 500

Command output :

Using server port: 1808
13 déc. 2023 13:06:14 DEBUG [main] (RequestAddCookies.java:123) - CookieSpec selected: default
13 déc. 2023 13:06:14 DEBUG [main] (RequestAuthCache.java:77) - Auth cache not set in the context
13 déc. 2023 13:06:14 DEBUG [main] (PoolingHttpClientConnectionManager.java:267) - Connection request: [route: {}->http://127.0.0.1:1808][total available: 0; route allocated: 0 of 2; total allocated: 0 of 20]
13 déc. 2023 13:06:14 DEBUG [main] (PoolingHttpClientConnectionManager.java:312) - Connection leased: [id: 0][route: {}->http://127.0.0.1:1808][total available: 0; route allocated: 1 of 2; total allocated: 1 of 20]
13 déc. 2023 13:06:14 DEBUG [main] (MainClientExec.java:234) - Opening connection {}->http://127.0.0.1:1808
13 déc. 2023 13:06:14 DEBUG [main] (DefaultHttpClientConnectionOperator.java:139) - Connecting to /127.0.0.1:1808
13 déc. 2023 13:06:14 DEBUG [main] (LoggingManagedHttpClientConnection.java:96) - http-outgoing-0: Shutdown connection
13 déc. 2023 13:06:14 DEBUG [main] (ConnectionHolder.java:129) - Connection discarded
13 déc. 2023 13:06:14 DEBUG [main] (PoolingHttpClientConnectionManager.java:351) - Connection released: [id: 0][route: {}->http://127.0.0.1:1808][total available: 0; route allocated: 0 of 2; total allocated: 0 of 20]
13 déc. 2023 13:06:14  INFO [main] (Gensim.java:691) - Server is not yet running. Waiting 10 seconds. Trial 1 / 10
/Users/adeprez/python-server/python_server.py:9: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  import pkg_resources
 * Serving Flask app 'python_server'
 * Debug mode: off
13 déc. 2023 13:06:24 DEBUG [main] (PoolingHttpClientConnectionManager.java:411) - Connection manager is shutting down
13 déc. 2023 13:06:24 DEBUG [main] (PoolingHttpClientConnectionManager.java:434) - Connection manager shut down
13 déc. 2023 13:06:24 DEBUG [main] (RequestAddCookies.java:123) - CookieSpec selected: default
13 déc. 2023 13:06:24 DEBUG [main] (RequestAuthCache.java:77) - Auth cache not set in the context
13 déc. 2023 13:06:24 DEBUG [main] (PoolingHttpClientConnectionManager.java:267) - Connection request: [route: {}->http://127.0.0.1:1808][total available: 0; route allocated: 0 of 2; total allocated: 0 of 20]
13 déc. 2023 13:06:24 DEBUG [main] (PoolingHttpClientConnectionManager.java:312) - Connection leased: [id: 1][route: {}->http://127.0.0.1:1808][total available: 0; route allocated: 1 of 2; total allocated: 1 of 20]
13 déc. 2023 13:06:24 DEBUG [main] (MainClientExec.java:234) - Opening connection {}->http://127.0.0.1:1808
13 déc. 2023 13:06:24 DEBUG [main] (DefaultHttpClientConnectionOperator.java:139) - Connecting to /127.0.0.1:1808
13 déc. 2023 13:06:24 DEBUG [main] (DefaultHttpClientConnectionOperator.java:146) - Connection established 127.0.0.1:49537<->127.0.0.1:1808
13 déc. 2023 13:06:24 DEBUG [main] (MainClientExec.java:255) - Executing request GET /melt_ml.html HTTP/1.1
13 déc. 2023 13:06:24 DEBUG [main] (MainClientExec.java:260) - Target auth state: UNCHALLENGED
13 déc. 2023 13:06:24 DEBUG [main] (MainClientExec.java:266) - Proxy auth state: UNCHALLENGED
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:133) - http-outgoing-1 >> GET /melt_ml.html HTTP/1.1
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:136) - http-outgoing-1 >> Host: 127.0.0.1:1808
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:136) - http-outgoing-1 >> Connection: Keep-Alive
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:136) - http-outgoing-1 >> User-Agent: Apache-HttpClient/4.5.13 (Java/1.8.0_291)
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:136) - http-outgoing-1 >> Accept-Encoding: gzip,deflate
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-1 >> "GET /melt_ml.html HTTP/1.1[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-1 >> "Host: 127.0.0.1:1808[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-1 >> "Connection: Keep-Alive[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-1 >> "User-Agent: Apache-HttpClient/4.5.13 (Java/1.8.0_291)[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-1 >> "Accept-Encoding: gzip,deflate[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-1 >> "[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-1 << "HTTP/1.1 200 OK[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-1 << "Server: Werkzeug/3.0.1 Python/3.12.1[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-1 << "Date: Wed, 13 Dec 2023 12:06:24 GMT[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-1 << "Content-Type: text/html; charset=utf-8[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-1 << "Content-Length: 49[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-1 << "Connection: close[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-1 << "[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:87) - http-outgoing-1 << "MELT ML Server running. Ready to accept requests."
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:122) - http-outgoing-1 << HTTP/1.1 200 OK
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:125) - http-outgoing-1 << Server: Werkzeug/3.0.1 Python/3.12.1
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:125) - http-outgoing-1 << Date: Wed, 13 Dec 2023 12:06:24 GMT
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:125) - http-outgoing-1 << Content-Type: text/html; charset=utf-8
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:125) - http-outgoing-1 << Content-Length: 49
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:125) - http-outgoing-1 << Connection: close
13 déc. 2023 13:06:24  INFO [main] (Gensim.java:687) - Server is running.
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:79) - http-outgoing-1: Close connection
13 déc. 2023 13:06:24 DEBUG [main] (ConnectionHolder.java:104) - Connection discarded
13 déc. 2023 13:06:24 DEBUG [main] (PoolingHttpClientConnectionManager.java:351) - Connection released: [id: 1][route: {}->http://127.0.0.1:1808][total available: 0; route allocated: 0 of 2; total allocated: 0 of 20]
13 déc. 2023 13:06:24  INFO [main] (Gensim.java:721) - Server started.
Server URL: http://127.0.0.1:1808
Server port: 1808
13 déc. 2023 13:06:24 DEBUG [main] (RequestAddCookies.java:123) - CookieSpec selected: default
13 déc. 2023 13:06:24 DEBUG [main] (RequestAuthCache.java:77) - Auth cache not set in the context
13 déc. 2023 13:06:24 DEBUG [main] (PoolingHttpClientConnectionManager.java:267) - Connection request: [route: {}->http://127.0.0.1:1808][total available: 0; route allocated: 0 of 2; total allocated: 0 of 20]
13 déc. 2023 13:06:24 DEBUG [main] (PoolingHttpClientConnectionManager.java:312) - Connection leased: [id: 2][route: {}->http://127.0.0.1:1808][total available: 0; route allocated: 1 of 2; total allocated: 1 of 20]
13 déc. 2023 13:06:24 DEBUG [main] (MainClientExec.java:234) - Opening connection {}->http://127.0.0.1:1808
13 déc. 2023 13:06:24 DEBUG [main] (DefaultHttpClientConnectionOperator.java:139) - Connecting to /127.0.0.1:1808
13 déc. 2023 13:06:24 DEBUG [main] (DefaultHttpClientConnectionOperator.java:146) - Connection established 127.0.0.1:49538<->127.0.0.1:1808
13 déc. 2023 13:06:24 DEBUG [main] (MainClientExec.java:255) - Executing request GET /check-requirements HTTP/1.1
13 déc. 2023 13:06:24 DEBUG [main] (MainClientExec.java:260) - Target auth state: UNCHALLENGED
13 déc. 2023 13:06:24 DEBUG [main] (MainClientExec.java:266) - Proxy auth state: UNCHALLENGED
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:133) - http-outgoing-2 >> GET /check-requirements HTTP/1.1
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:136) - http-outgoing-2 >> requirements_file: /Users/adeprez/./python-server/requirements.txt
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:136) - http-outgoing-2 >> Host: 127.0.0.1:1808
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:136) - http-outgoing-2 >> Connection: Keep-Alive
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:136) - http-outgoing-2 >> User-Agent: Apache-HttpClient/4.5.13 (Java/1.8.0_291)
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:136) - http-outgoing-2 >> Accept-Encoding: gzip,deflate
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 >> "GET /check-requirements HTTP/1.1[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 >> "requirements_file: /Users/adeprez/./python-server/requirements.txt[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 >> "Host: 127.0.0.1:1808[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 >> "Connection: Keep-Alive[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 >> "User-Agent: Apache-HttpClient/4.5.13 (Java/1.8.0_291)[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 >> "Accept-Encoding: gzip,deflate[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 >> "[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 << "HTTP/1.1 500 INTERNAL SERVER ERROR[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 << "Server: Werkzeug/3.0.1 Python/3.12.1[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 << "Date: Wed, 13 Dec 2023 12:06:24 GMT[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 << "Content-Type: text/html; charset=utf-8[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 << "Content-Length: 265[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 << "Connection: close[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 << "[\r][\n]"
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:122) - http-outgoing-2 << HTTP/1.1 500 INTERNAL SERVER ERROR
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:125) - http-outgoing-2 << Server: Werkzeug/3.0.1 Python/3.12.1
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:125) - http-outgoing-2 << Date: Wed, 13 Dec 2023 12:06:24 GMT
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:125) - http-outgoing-2 << Content-Type: text/html; charset=utf-8
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:125) - http-outgoing-2 << Content-Length: 265
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:125) - http-outgoing-2 << Connection: close
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 << "<!doctype html>[\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 << "<html lang=en>[\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 << "<title>500 Internal Server Error</title>[\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 << "<h1>Internal Server Error</h1>[\n]"
13 déc. 2023 13:06:24 DEBUG [main] (Wire.java:73) - http-outgoing-2 << "<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>[\n]"
13 déc. 2023 13:06:24 DEBUG [main] (LoggingManagedHttpClientConnection.java:79) - http-outgoing-2: Close connection
13 déc. 2023 13:06:24 DEBUG [main] (ConnectionHolder.java:104) - Connection discarded
13 déc. 2023 13:06:24 DEBUG [main] (PoolingHttpClientConnectionManager.java:351) - Connection released: [id: 2][route: {}->http://127.0.0.1:1808][total available: 0; route allocated: 0 of 2; total allocated: 0 of 20]
<!doctype html>
<html lang=en>
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

Installation is not ok! [❌]
Is Python 3 installed? Please check the log for missing dependencies.
13 déc. 2023 13:06:24  INFO [Thread-0] (Gensim.java:715) - JVM shutdown detected - close python server if still open.
13 déc. 2023 13:06:24 DEBUG [Thread-0] (PoolingHttpClientConnectionManager.java:411) - Connection manager is shutting down
13 déc. 2023 13:06:24 DEBUG [Thread-0] (PoolingHttpClientConnectionManager.java:434) - Connection manager shut down
13 déc. 2023 13:06:24  INFO [Thread-0] (Gensim.java:717) - Shutdown completed.

Do you have any idea of what could be missing?

Thank you for your support and sorry for the long post,
Alexis

[Weiweihere]

Hi Alexis,

I want to ask have you solved this problem? I also met this issue.

Weiqing

[adeprez]

Hi Weiqing,

I couldn't make it work. I'm looking at other implementations advised by http://rdf2vec.org/#implementations

[AmosDinh]

Tracing down the issue:
pallets/werkzeug#2693
"The following commit in Werkzeug changed the Werkzeug developer server to drop any header fields with underscores:"
problem is requirements_file has the underscore.

To "solve" the issue:
pip install Flask==2.2.0
pip install Werkzeug==2.2.0

note that this is no fix. the actual fix should be removing the underscores in the request header names because "_" and "-" are both mapped to "_" by standard in request headers. To avoid ambiguity, variables with underscores are ignored by many frameworks entirely.
See:
https://stackoverflow.com/a/22856867