Can't apply all migrations when starting in docker

[michaelarnauts]

I used the example docker-compose.yml file, and I get the following error during startup of the pypi container:

pypi_server_1  | [I 191115 13:03:59 __init__:69] Database initialized as 'pypi-server'. Checking migrations...
pypi_server_1  | [I 191115 13:03:59 actions:19] Applying migration: "create_tables.create_users"
pypi_server_1  | [I 191115 13:03:59 actions:19] Applying migration: "create_default_user.create_default_user"
pypi_server_1  | [I 191115 13:03:59 actions:19] Applying migration: "create_package_tables.create_package"
pypi_server_1  | [I 191115 13:03:59 actions:19] Applying migration: "create_package_tables.create_package_version"
pypi_server_1  | [I 191115 13:03:59 actions:19] Applying migration: "create_package_tables.create_package_file"
pypi_server_1  | [I 191115 13:03:59 actions:19] Applying migration: "packagefile_add_url_and_fetched.add_url_fileld"
pypi_server_1  | [I 191115 13:03:59 actions:19] Applying migration: "packagefile_add_url_and_fetched.add_fetched_fileld"
pypi_server_1  | [I 191115 13:03:59 actions:19] Applying migration: "packagefile_basename_unique.add_uniquie_basename_index"
db_1           | 2019-11-15 13:03:59.813 UTC [70] ERROR:  relation "packagefile_basename" already exists
db_1           | 2019-11-15 13:03:59.813 UTC [70] STATEMENT:  CREATE UNIQUE INDEX "packagefile_basename" ON "packagefile" ("basename")
pypi_server_1  | [E 191115 13:03:59 actions:26] Migration failed.
pypi_server_1  | [E 191115 13:03:59 actions:27] relation "packagefile_basename" already exists
pypi_server_1  |     Traceback (most recent call last):
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/pypi_server/db/migrator/actions.py", line 20, in migrate_db
pypi_server_1  |         migration(migrator, DB)
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/pypi_server/db/migrator/migrations/005_packagefile_basename_unique.py", line 11, in add_uniquie_basename_index
pypi_server_1  |         migrator.add_index('packagefile', ('basename',), True)
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/playhouse/migrate.py", line 717, in migrate
pypi_server_1  |         operation.run()
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/playhouse/migrate.py", line 144, in run
pypi_server_1  |         getattr(self.migrator, self.method)(*self.args, **kwargs))
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/playhouse/migrate.py", line 133, in _handle_result
pypi_server_1  |         self.execute(result)
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/playhouse/migrate.py", line 129, in execute
pypi_server_1  |         self.migrator.database.execute_sql(sql, params)
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/peewee.py", line 3830, in execute_sql
pypi_server_1  |         cursor.execute(sql, params or ())
pypi_server_1  |     psycopg2.errors.DuplicateTable: relation "packagefile_basename" already exists
pypi_server_1  |     
pypi_server_1  | [C 191115 13:03:59 server:204] Exception on main loop:
pypi_server_1  | [E 191115 13:03:59 server:205] relation "packagefile_basename" already exists
pypi_server_1  |     Traceback (most recent call last):
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/pypi_server/server.py", line 158, in run
pypi_server_1  |         io_loop.run_sync(lambda: init_db(options.database))
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/tornado/ioloop.py", line 458, in run_sync
pypi_server_1  |         return future_cell[0].result()
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/tornado/concurrent.py", line 238, in result
pypi_server_1  |         raise_exc_info(self._exc_info)
pypi_server_1  |       File "<string>", line 4, in raise_exc_info
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/tornado/ioloop.py", line 436, in run
pypi_server_1  |         result = func()
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/pypi_server/server.py", line 158, in <lambda>
pypi_server_1  |         io_loop.run_sync(lambda: init_db(options.database))
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/pypi_server/db/__init__.py", line 91, in init_db
pypi_server_1  |         migrate_db(DB, Migrations, migrator)
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/pypi_server/db/migrator/actions.py", line 20, in migrate_db
pypi_server_1  |         migration(migrator, DB)
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/pypi_server/db/migrator/migrations/005_packagefile_basename_unique.py", line 11, in add_uniquie_basename_index
pypi_server_1  |         migrator.add_index('packagefile', ('basename',), True)
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/playhouse/migrate.py", line 717, in migrate
pypi_server_1  |         operation.run()
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/playhouse/migrate.py", line 144, in run
pypi_server_1  |         getattr(self.migrator, self.method)(*self.args, **kwargs))
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/playhouse/migrate.py", line 133, in _handle_result
pypi_server_1  |         self.execute(result)
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/playhouse/migrate.py", line 129, in execute
pypi_server_1  |         self.migrator.database.execute_sql(sql, params)
pypi_server_1  |       File "/usr/share/python/app/lib/python3.7/site-packages/peewee.py", line 3830, in execute_sql
pypi_server_1  |         cursor.execute(sql, params or ())
pypi_server_1  |     psycopg2.errors.DuplicateTable: relation "packagefile_basename" already exists
pypi_server_1  |     

[HarryMWinters]

Same. Bummer though. This would have fulfilled 120% of my use case.

[la-sanchez]

Same here.

[MoLt1eS]

Having the same issue

[jaredhanson11]

+1, I ran into the same issue. I switched to https://github.com/pypiserver/pypiserver. Doesn't have a nice gui and user nice management, but it does the job effectively enough.

[ruohola]

Still happening.

[vlastocom]

Just verified when researching #54 , this is still the case for the postgres configuration.

[ruohola]

Unfortunately this issue made the whole package unusable, so I just used https://github.com/pypiserver/pypiserver.

[fr-Pursuit]

There is a simple way to circumvent this problem. It's not a propre fix, but it makes this package usable: launch the program a first time, and let it crash. Afterwards, manually add an entry into the "migrations" table to prevent the faulty migration from being applied:

INSERT INTO migrations (id, name, ts) VALUES (8, 'packagefile_basename_unique.add_uniquie_basename_index', CURRENT_TIMESTAMP);

Edit: Thanks to @ruohola for his suggestion about the usage of CURRENT_TIMESTAMP !

[ruohola]

There is a simple way to circumvent this problem. It's not a propre fix, but it makes this package usable: launch the program a first time, and let it crash. Afterwards, manually add an entry into the "migrations" table to prevent the faulty migration from being applied:

INSERT INTO migrations (id, name, ts) VALUES (8, 'packagefile_basename_unique.add_uniquie_basename_index', 'TIMESTAMP');

Replace TIMESTAMP with a valid timestamp (you can simply copy/paste one of the timestamps you get with the request SELECT ts from migrations)

You could just use CURRENT_TIMESTAMP when inserting.

[Truongphikt]

There is a simple way to circumvent this problem. It's not a propre fix, but it makes this package usable: launch the program a first time, and let it crash. Afterwards, manually add an entry into the "migrations" table to prevent the faulty migration from being applied:

INSERT INTO migrations (id, name, ts) VALUES (8, 'packagefile_basename_unique.add_uniquie_basename_index', CURRENT_TIMESTAMP);

Edit: Thanks to @ruohola for his suggestion about the usage of CURRENT_TIMESTAMP !

Could you please provide more detail to modify "migrations" table ? Thanks.