Docker Compose Terrareg with poetry has failures

[markdjones82]

I am running into issues locally since downloading the latest version running with docker compose up. When MIGRATE database is true I get this error:
Command not found: alembic

• '[' True == True ']'

• poetry run alembic upgrade head

Command not found: alembic

With migrate disabled;

• poetry run python ./terrareg.py

Traceback (most recent call last):

File "/app/./terrareg.py", line 4, in

from terrareg.server import Server

File "/app/terrareg/server/init.py", line 5, in

from flask import Flask, session, redirect, request

ModuleNotFoundError: No module named 'flask'

[MatthewJohn]

Created gitlab issue: https://gitlab.dockstudios.co.uk/pub/terrareg/-/issues/568
gitlab-issue-id:568

[MatthewJohn]

Hey @markdjones82 ,

As a part of sanity on my end, the "pr" branch is running: https://terrareg-main.gitlab-pr.dockstudios.co.uk/, which uses the docker image, runs DB migration and imports a couple of things.
The CI/CD step that starts the built container to the point of homepage does also work.
Let me try and replicate locally

Matt

[MatthewJohn]

Hey @markdjones82

I just ran it locally:

➜  terrareg git:(aries043-fix-readme-typo) ✗ docker run -ti --rm -p 5000:5000 \
    -e PUBLIC_URL=http://localhost:5000 \
    -e MIGRATE_DATABASE=True \
    -e SECRET_KEY=$SECRET_KEY \
    -e ADMIN_AUTHENTICATION_TOKEN=MySuperSecretPassword \
    ghcr.io/matthewjohn/terrareg:latest
Trying to pull ghcr.io/matthewjohn/terrareg:latest...
Getting image source signatures
Copying blob sha256:a3416d46f27bf775d3b9c2babd746761382e470b5e7373197175db9d5c41e89b
Copying blob sha256:1454c05bd781d690480f0b6a71f84dd53b7c5207e227ec3e10826a9d2b622394
Copying blob sha256:254e724d77862dc53abbd3bf0e27f9d2f64293909cdd3d0aad6a8fe5a6680659
Copying blob sha256:89eaef72e1bef833b0d2bf0539a123a50a45195a59ee528462f1a2ec7c5c7615
Copying blob sha256:7b75476b4a2f9f3973d87593a1970b3122f5af1e2fa1e47f274d76ff205ba129
Copying blob sha256:f58dd0d235bf9af9f0a8d9364df02e3f288494539cb673d2ed699f65a5d1d081
Copying blob sha256:c253f26e4fd60fa06031e642719f1006891b08ac71eced745fd0e89a946c557d
Copying blob sha256:5817c27c2db7109d2af198e5f854f77a36dfdab169059c6881846e278ccd4155
Copying blob sha256:fd53d8966ad97c75b818ca3eebca206097764881a764e93729aa6b35a1efae8e
Copying blob sha256:fa342e77d6a06ebd3b61c4092d1a7c36a4e77b12f2dd67d0a351ed1fdfc7f605
Copying blob sha256:a281b45085195e6f66b208913ed3c3d23bafa6a023dd4c7271b73ec745370c22
Copying blob sha256:133ac56ae5b029eb8816fe11f633e91a52e3b644c5bb9273a9afffa376c4a2da
Copying blob sha256:2bef679e7bb42248349fb51877e288ee92f289735146e596e564cb6615f91fa3
Copying blob sha256:7190aa7a597a2b654cef621d91cb9150d78732bc0ac6b754f44d69fa35ee490a
Copying blob sha256:2fb437ae2c24e39920d8e14e867d322bac6e0489f42b22a79898bf672e6cb64f
Copying blob sha256:fb7b924ac82222dfc0e5cdbd8cbec62b8361daa6333b0c6c95ecff3cd6b266c9
Copying blob sha256:d4f8943e5a367bfec58426b0d80a3f4ff1e3508903b1ea840a3cc26f1b72c5f4
Copying blob sha256:1ac20bbe52d3a21ffda06a2f38206135f4966a5b1304f68a70e3a370c3fb6b20
Copying blob sha256:4f01b616eeffcbe7c6ff33dc348c6f6b3fc1e2ee37443300e3955b5cd3bedf05
Copying blob sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
Copying blob sha256:2a5e255dc829a9b776a35325986240801a1671e9ad8d1d74084d8b6140dbf42b
Copying blob sha256:94cf5b5ded4b5351b9094e8223b12c129cfa23aad7710a326bd870c40ee47019
Copying blob sha256:755b36b20ff225e817c0f6d59799c39a05bd2399836a986e9fae7ac142e59782
Copying blob sha256:a19f9758f96d8cb604aa01342551e9988b5ef10b40c297b16a1cbba861264d85
Copying blob sha256:43b1bf10eaee01680df500b6e0f1570392c3f6dbd702709624de3944371f46dd
Copying blob sha256:c8cc78fcac44343ac751ab8cfed43f75f8b6234b9e50c09ef62f3c4ab3379640
Copying blob sha256:830a8bb08e81bf633793f0e8fab66303c3443dcf2abf4475d9ce810399a8b927
Copying blob sha256:6501e86483d914c31ca1cb1860eb2600d614a8ece0ffca9ef1e28cae2d464304
Copying blob sha256:ff1b48063a303f215d7b9987e56065931e155c69f3911eaff4ec2f7b085867d5
Copying blob sha256:1010b7609d9b10825cc9e72694c21498489e6dfb76bcda390a072272e5c29cd7
Copying blob sha256:a4391ce6b1edb503a6097b9d6e5c9e2e1c544677481f2e6e4500dbca8f8add9c
Copying blob sha256:419284a43ec45b48aecf11d7e6dc8d54ecf4a620ebb49ad54bc1817d27b4d183
Copying blob sha256:d059d25ace55ccd6d548512a8058a5c28f3279720e7be6ca94aae0022083e3b4
Copying blob sha256:37da073db8212d851971a74b5e8fd1f8e615422fe482fe57f653d3ac7f90118c
Copying blob sha256:2b4510f5c70aa1499e38235138ac1bb940025435863c067391ff9f4788d66b4d
Copying blob sha256:040f5e6013d94af159355100e733addc9b9ee8a57d12252379b3e5ad15d37d0e
Copying config sha256:f9acf3f2b22b4604f47be7d994abe9969ae4095523087b52afcc42bcaaeda80d
Writing manifest to image destination
+ '[' True == True ']'
+ poetry run alembic upgrade head
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> aef5947a7e1d, empty message
INFO  [alembic.runtime.migration] Running upgrade aef5947a7e1d -> 0e3de802e5e0, Convert blobs to medium blobs
INFO  [alembic.runtime.migration] Running upgrade 0e3de802e5e0 -> 95f56a77a6e6, Add beta flag to module version
INFO  [alembic.runtime.migration] Running upgrade 95f56a77a6e6 -> b0f952e4a027, Add table for example file contents
INFO  [alembic.runtime.migration] Running upgrade b0f952e4a027 -> ef71db86c2a1, Add internal flag column to module_version table and foreign key to module provider to point to latest version
INFO  [alembic.runtime.migration] Running upgrade ef71db86c2a1 -> 0776813feaeb, Update analytics to perform no action when parent module version is deleted
INFO  [alembic.runtime.migration] Running upgrade 0776813feaeb -> eea5e27cac2b, Add names to all foreign key constaints
/usr/local/lib/python3.12/contextlib.py:144: SAWarning: WARNING: SQL-parsed foreign key constraint '('latest_version_id', 'module_version', 'id')' could not be located in PRAGMA foreign_keys for table module_provider
  next(self.gen)
/usr/local/lib/python3.12/contextlib.py:144: SAWarning: WARNING: SQL-parsed foreign key constraint '('git_provider_id', 'git_provider', 'id')' could not be located in PRAGMA foreign_keys for table module_provider
  next(self.gen)
/usr/local/lib/python3.12/contextlib.py:144: SAWarning: WARNING: SQL-parsed foreign key constraint '('module_provider_id', 'module_provider', 'id')' could not be located in PRAGMA foreign_keys for table module_version
  next(self.gen)
INFO  [alembic.runtime.migration] Running upgrade eea5e27cac2b -> ee82678fcda1, Remove foreign key constraint from analytics parent module version ID column
/usr/local/lib/python3.12/contextlib.py:144: SAWarning: WARNING: SQL-parsed foreign key constraint '('parent_module_version', 'module_version', 'id')' could not be located in PRAGMA foreign_keys for table analytics
  next(self.gen)
INFO  [alembic.runtime.migration] Running upgrade ee82678fcda1 -> a36ffbb6580e, Add index to parent_module_version column of analytics table
INFO  [alembic.runtime.migration] Running upgrade a36ffbb6580e -> 47e45e505e22, Move common submodule/moduleversion columns to new module_details table
/usr/local/lib/python3.12/contextlib.py:144: SAWarning: WARNING: SQL-parsed foreign key constraint '('parent_module_version', 'module_version', 'id')' could not be located in PRAGMA foreign_keys for table submodule
  next(self.gen)
INFO  [alembic.runtime.migration] Running upgrade 47e45e505e22 -> d499042fad3b, Add module details column for holding tfsec output
INFO  [alembic.runtime.migration] Running upgrade d499042fad3b -> 437ccafb9882, Add session table to hold active sessions
INFO  [alembic.runtime.migration] Running upgrade 437ccafb9882 -> 51d3707d6334, Add git_path to module provider table
INFO  [alembic.runtime.migration] Running upgrade 51d3707d6334 -> 6416ffbf606d, Add Infracost output column to module_details table
INFO  [alembic.runtime.migration] Running upgrade 6416ffbf606d -> acd5e83c690f, Add table for module version files
INFO  [alembic.runtime.migration] Running upgrade acd5e83c690f -> d72c5339c21b, Add table for namespaces and replace namespace field in module provider with foreign key
INFO  [alembic.runtime.migration] Running upgrade d72c5339c21b -> 02c137632f24, Add tables for user groups and user group namespace permissions
INFO  [alembic.runtime.migration] Running upgrade 02c137632f24 -> 161e0f33603c, Add audit table
INFO  [alembic.runtime.migration] Running upgrade 161e0f33603c -> 8b594ed19f9d, Add columns for graph data
INFO  [alembic.runtime.migration] Running upgrade 8b594ed19f9d -> 71d311c23025, Add module version column for extraction version
INFO  [alembic.runtime.migration] Running upgrade 71d311c23025 -> 210586684f86, Add namespace display_name
INFO  [alembic.runtime.migration] Running upgrade 210586684f86 -> db7a6e0d0b9d, Add namespace redirect table and audit enum values for NAMESPACE_MODIFY_NAME and NAMESPACE_MODIFY_DISPLAY_NAME
INFO  [alembic.runtime.migration] Running upgrade db7a6e0d0b9d -> ff6378a8ab9d, Add module provider redirect table
INFO  [alembic.runtime.migration] Running upgrade ff6378a8ab9d -> 1f1ebdc845a0, Add audit events for module provider name/namespace modification
INFO  [alembic.runtime.migration] Running upgrade 1f1ebdc845a0 -> 2e9fe1292b2b, Add audit events for namespace deletion
INFO  [alembic.runtime.migration] Running upgrade 2e9fe1292b2b -> 81609b772646, Add module_details columns for terraform_version and terraform_modules
INFO  [alembic.runtime.migration] Running upgrade 81609b772646 -> 7b6cac0d522f, Add columns to analytics to record provided namespace name, module name and provider
INFO  [alembic.runtime.migration] Running upgrade 7b6cac0d522f -> 0832273b5d04, Add audit type for module provider redirect deletion
INFO  [alembic.runtime.migration] Running upgrade 0832273b5d04 -> fb6a94791a14, Add tables for Terraform IDP sessions
INFO  [alembic.runtime.migration] Running upgrade fb6a94791a14 -> ee14b27baeb1, Add tables for provider sources, providers and tables for supporting data for provider versions
INFO  [alembic.runtime.migration] Running upgrade ee14b27baeb1 -> 2bc9677e93d1, Add git ref column to module version table
INFO  [alembic.runtime.migration] Running upgrade 2bc9677e93d1 -> 9dbcb4240c55, Add git_path_template column to git provider
INFO  [alembic.runtime.migration] Running upgrade 9dbcb4240c55 -> f9a80ea383cc, Add archive_git_path column to module provider
INFO  [alembic.runtime.migration] Running upgrade f9a80ea383cc -> 6dd8adb5e1e3, postgres-fixes
+ '[' '' == True ']'
+ '[' '' '!=' '' ']'
+ poetry run python ./terrareg.py
 * Serving Flask app 'terrareg.server'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:5000
 * Running on http://10.88.0.34:5000
Press CTRL+C to quit

Which seems to be working fine.
Are you overriding PATH environment variables or anything else that stands out whilst running the container?

Matt

[markdjones82]

I am using docker compose up -d to build so maybe that has something to do with it. Let me try just running the terrareg in docker and see what happens.

[MatthewJohn]

I'll also try docker-compose to see if it makes a difference on my side :)

[markdjones82]

I just did a plain docker build with the docker file and ran it and that is working fine there too. So, it appears to be something with the docker-compose setup/yml.

I'm going to see if everything works fine in our Jenkins setup in our dev environment.

[markdjones82]

My Jenkins deploy also works fine. So, only appears to be an issue with the compose file. I tried a few things, but it didn't make a difference. Low priority here as I can do testing other ways :)