Skip to content
Open

Dev #824

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
233 commits
Select commit Hold shift + click to select a range
a0d6f3a
Merge pull request #773 from basedosdados/feat/mutation-remove-all-me…
AldemirLucas Feb 21, 2025
435b250
feat: data API (read from previous PRs and commits before dev reset)
rdahis Feb 22, 2025
1d739e4
Merge pull request #774 from basedosdados/feat/data_api
rdahis Feb 22, 2025
bdd3ba3
fix: search_views, list tables
rdahis Feb 22, 2025
dc7ea4f
Merge pull request #775 from basedosdados/feat/data_api
rdahis Feb 22, 2025
0fc037f
fix: allow any table and column in endpoints
rdahis Feb 22, 2025
f52dca4
Merge pull request #776 from basedosdados/feat/data_api
rdahis Feb 22, 2025
ec6adf8
chore: view return endpoint cloud table
rdahis Feb 22, 2025
2082fbb
Merge pull request #777 from basedosdados/feat/data_api
rdahis Feb 22, 2025
49598f2
chore: parameter to search for columns
rdahis Feb 22, 2025
5e51ef2
Merge pull request #778 from basedosdados/feat/data_api
rdahis Feb 22, 2025
3cc6056
chore: enlarge column search
rdahis Feb 22, 2025
e208f80
Merge pull request #779 from basedosdados/feat/data_api
rdahis Feb 22, 2025
51f6df0
chore: return endpoint parameters
rdahis Feb 22, 2025
5c65bbb
fix(Email): debbug for email activation and email reset password
isabelmeister Mar 31, 2025
4247097
Merge pull request #792 from basedosdados/fix/emails-reset-activate
isabelmeister Mar 31, 2025
ca947d0
feat: initial commit chatbot
rdahis Apr 4, 2025
13a21fb
Merge branch 'dev' into feat/data_api
rdahis Apr 7, 2025
c5f06a3
fix: all changes to fix dev environment
isabelmeister Apr 8, 2025
e791238
Merge pull request #802 from basedosdados/feat/search-ajust
AldemirLucas Apr 9, 2025
2f2acb6
WIP: Created basic chatbot backend endpoints
polvoazul Apr 10, 2025
3102f0b
Adding migrations and fixing admin
polvoazul Apr 10, 2025
5fd81b3
Fixing stuff and adding todos
polvoazul Apr 10, 2025
1e9a1cd
Bad set to 0
polvoazul Apr 11, 2025
2f9cf80
Merge pull request #805 from basedosdados/feat/mutation-remove-all-me…
AldemirLucas Apr 14, 2025
16ada87
added feedback id for tracking on langsmith
vrtornisiello Apr 14, 2025
0905a2b
renamed `ChatbotThreadListView` to `ThreadListView` and created `Chec…
vrtornisiello Apr 14, 2025
5968092
refactored views
vrtornisiello Apr 14, 2025
c058b35
added chatbot env variables
vrtornisiello Apr 14, 2025
c057bf3
temp: added a script for running django in dev mode and updated the D…
vrtornisiello Apr 14, 2025
2d2262c
updated migrations
vrtornisiello Apr 14, 2025
8ef6d45
fixed django admin
vrtornisiello Apr 14, 2025
2d2a2f5
Merge pull request #804 from basedosdados/feat/search-ajust
AldemirLucas Apr 14, 2025
be70b56
added `djangorestframework` and `djangorestframework-simplejwt` packa…
vrtornisiello Apr 15, 2025
2b9fbc5
comment could be empty/null
vrtornisiello Apr 15, 2025
f25da98
added jwt tokens urls
vrtornisiello Apr 15, 2025
f680da5
changed `chatbot` package installation path
vrtornisiello Apr 15, 2025
86048fe
created serializers and refactored views, added jwt auth, error handl…
vrtornisiello Apr 15, 2025
54b8774
Merge pull request #807 from basedosdados/feat/mutation-remove-all-me…
AldemirLucas Apr 17, 2025
11a8630
reminder for custom authentication rules
vrtornisiello Apr 22, 2025
a898fd4
updated dockerfile and compose file for chromadb
vrtornisiello Apr 22, 2025
7e63cc0
fixing a TypeError (the assistant expects `thread_id` as a string) an…
vrtornisiello Apr 22, 2025
3e5426f
Adding chatbot as submodule
polvoazul Apr 24, 2025
5a77ec0
Download submodules in ci. Also update README to alert on cloning sub…
polvoazul Apr 24, 2025
16de600
minor stuff
polvoazul Apr 24, 2025
0c5fd79
fix: adjust lint in files
isabelmeister Apr 24, 2025
014c9cf
adjusting indentation
vrtornisiello Apr 24, 2025
d49acda
updated chatbot package and using validated data
vrtornisiello Apr 24, 2025
b6add8c
updated chatbot package version to `v0.3.0`
vrtornisiello Apr 24, 2025
e244302
validated data is serialized to a UUID but `UserMessage` expects a st…
vrtornisiello Apr 25, 2025
96c786e
added healthcheck to vector database service
vrtornisiello Apr 25, 2025
01bee91
Merge pull request #808 from basedosdados/fix/populate-and-fetch_meta…
isabelmeister Apr 28, 2025
bbb44d8
created a request validation function using pydantic
vrtornisiello Apr 28, 2025
31d6ab7
just renaming views and endpoints
vrtornisiello Apr 28, 2025
2e17f2b
added authentication to the checkpoint deletion endpoint
vrtornisiello Apr 28, 2025
408857c
added docstrings
vrtornisiello Apr 28, 2025
58bfd8f
Merge branch 'main' into feat/app_chatbot
vrtornisiello Apr 28, 2025
6c36b47
update `chatbot` package to version `v0.4.0`
vrtornisiello Apr 30, 2025
2667d9d
update .gitignore
vrtornisiello Apr 30, 2025
7eb9428
updated volume mount path to match the expected path on chroma v0.6.3
vrtornisiello Apr 30, 2025
a79a868
updated `chatbot` package to `v0.4.1` and using Serializers for reque…
vrtornisiello May 6, 2025
7d3a9bb
updated chatbot package version to v0.4.2
vrtornisiello May 7, 2025
916a49b
created `ChatbotDatabase` to read metadata from the local database an…
vrtornisiello May 7, 2025
8e62c39
created chatbot authentication field on user account model
vrtornisiello May 8, 2025
3ad728b
sending feedbacks to langsmith
vrtornisiello May 9, 2025
d21a3a4
fixed authentication rule (when a user is not found in db, django sim…
vrtornisiello May 9, 2025
54e95a9
checking if the thread exists before trying to delete its checkpoints…
vrtornisiello May 9, 2025
1fa9027
added unit tests for the endpoints
vrtornisiello May 9, 2025
da53cf9
added `chatbot` dependency
vrtornisiello May 9, 2025
d3830ab
update env variables file
vrtornisiello May 9, 2025
2cbca17
optimize docker image
vrtornisiello May 13, 2025
1e65438
fix: add fields and fix order in table and dataset admin page
laura-l-amaral May 8, 2025
1167f4f
fix: clean functions
laura-l-amaral May 20, 2025
1d3db67
Merge pull request #812 from basedosdados/fix-admin-pages
laura-l-amaral May 20, 2025
959e581
feat: Adicionando POC para botão upload_columns em Tables (#813)
Winzen May 21, 2025
192d8ca
the `chatbot` package was added to the api dependencies, so it does…
vrtornisiello May 21, 2025
2201cec
add chatbot migrations
vrtornisiello May 21, 2025
2a8550e
removed unused type
vrtornisiello May 21, 2025
c13c262
edited server starting scripts
vrtornisiello May 21, 2025
5a315b1
feat: adicionando cached_property e Inline paginated (#814)
Winzen May 22, 2025
69b1fe6
update compose file
vrtornisiello May 22, 2025
098c6e6
updated chatbot admin models
vrtornisiello May 22, 2025
47ddd5a
update account admin model to show the `has_chatbot_access` flag
vrtornisiello May 22, 2025
271ba8b
remove unused pydantic import
vrtornisiello May 22, 2025
7aa1101
cloning submodules in ci workflow
vrtornisiello May 22, 2025
fa82195
update formatting
vrtornisiello May 22, 2025
5f4c1e4
add missing `Account` migrations
vrtornisiello May 22, 2025
96bc1c3
we don't need to checkout submodules in the deploy step because the d…
vrtornisiello May 23, 2025
57939a2
we need to use a deploy key to checkout the `chatbot` submodule becau…
vrtornisiello May 23, 2025
97dbcf3
we don't need to checkout submodules in this action because it only w…
vrtornisiello May 23, 2025
77e9d14
building `db_url` on runtime
vrtornisiello May 23, 2025
c04af7b
add non-sensitive env variables
vrtornisiello May 23, 2025
93728d9
Merge branch 'dev' into feat/data_api
rdahis May 27, 2025
f865310
fix: rename migration to be consistent with newer one
rdahis May 27, 2025
176e503
Merge pull request #796 from basedosdados/feat/data_api
rdahis May 27, 2025
2111338
fix: is_required, limit choices
rdahis May 27, 2025
ad4b4fb
Merge pull request #816 from basedosdados/feat/data_api
rdahis May 27, 2025
4e3adb6
fix: admin show bool endpoint table
rdahis May 27, 2025
7e5ab5f
Merge pull request #817 from basedosdados/feat/data_api
rdahis May 27, 2025
021d080
feat: change credits with hashed API key
rdahis May 27, 2025
3ed7ca1
Merge pull request #818 from basedosdados/feat/data_api
rdahis May 27, 2025
0cda833
test: fix ruff linting issues in dev branch (#820)
Winzen May 27, 2025
6008cb8
apps.py so chatbot shows in admin
rdahis May 29, 2025
9b94d61
Merge branch 'dev' into feat/app_chatbot
rdahis May 29, 2025
7c5658b
Merge pull request #794 from basedosdados/feat/app_chatbot
rdahis May 29, 2025
83eb90a
Re-trigger workflows
rdahis May 29, 2025
8d2a6b7
Empty commit
rdahis May 29, 2025
72b26d0
Merge pull request #823 from basedosdados/feat/app_chatbot
rdahis May 29, 2025
ca87828
remove chatbot related env variables
vrtornisiello May 29, 2025
1a6f1b1
Merge pull request #827 from basedosdados/fix/deploy_chatbot
rdahis May 30, 2025
ab8c168
update `chatbot` package version to `0.4.4`
vrtornisiello May 30, 2025
e01d383
enable `chatbot` package logging
vrtornisiello May 30, 2025
cda364e
update poetry lock file
vrtornisiello May 30, 2025
6b744d3
fix imports and formatting to pass linting CI workflow
vrtornisiello May 30, 2025
dddbff6
fix imports and formatting to pass linting CI workflow
vrtornisiello May 30, 2025
ce9c7a7
Merge pull request #830 from basedosdados/chore/update_chatbot_package
rdahis May 31, 2025
f5f2a23
feat: adicionar mutação para deletar assinatura Stripe imediatamente
AldemirLucas May 31, 2025
6b454d1
increase dev ram memory limit to 1.5Gi
rdahis Jun 3, 2025
6bb090b
Merge pull request #832 from basedosdados/chore/larger_ram
rdahis Jun 3, 2025
4bb0b82
build: increase dev ram memory limit to 1536Mi (k8s) (#833)
Winzen Jun 3, 2025
99d578e
fix: mudar nome do botão Ferramentas para Importar Colunas (#834)
Winzen Jun 3, 2025
e481700
feat: inline paginated tables and columns / optimizations of `@proper…
Winzen Jun 4, 2025
6a06504
feat: adding an “Importar Colunas” button (#835)
Winzen Jun 4, 2025
210dec8
remove chroma related env variables
vrtornisiello Jun 4, 2025
9c3649c
stopped installing packages needed for chromadb
vrtornisiello Jun 4, 2025
a1df927
update dependencies
vrtornisiello Jun 4, 2025
7ed5880
update chatbot views and tests
vrtornisiello Jun 4, 2025
f3b8696
update `chatbot` package to version `v0.4.5`
vrtornisiello Jun 5, 2025
4877872
add `__init__.py` file to `apps` so python sees it as a package and p…
vrtornisiello Jun 5, 2025
c52caa0
discard changes in Dockerfile
vrtornisiello Jun 5, 2025
d9f3984
setting model name directly in plain text
vrtornisiello Jun 5, 2025
e5e49e6
Merge pull request #838 from basedosdados/refactor/reduce_chatbot_ove…
vrtornisiello Jun 5, 2025
e1d5cdb
ci: sync dev and main #833, #835, #836
Winzen Jun 6, 2025
825da7f
ci: sync dev and main (#833, #835, #836) - #843
Winzen Jun 6, 2025
ed596e8
fix: table admin page
laura-l-amaral Jun 5, 2025
efaf3d7
use pgvector for similarity search
vrtornisiello Jun 16, 2025
81b8a85
use special valua `__all__` to include all model fields
vrtornisiello Jun 16, 2025
cb99c7e
remove chroma from compose files
vrtornisiello Jun 16, 2025
2feb01a
add `PGVECTOR_COLLECTION` env variable
vrtornisiello Jun 16, 2025
f3eeaf8
Merge pull request #844 from basedosdados/fix-admin-pages
laura-l-amaral Jun 17, 2025
22d1596
minor fixes
vrtornisiello Jun 17, 2025
57ec4ec
added `EMBEDDING_MODEL` env variable
vrtornisiello Jun 23, 2025
0509d78
created `populate_pgvector` command
vrtornisiello Jun 23, 2025
cb39b93
improved logging
vrtornisiello Jun 23, 2025
9b0095c
running `populate_pgvector` script in the foreground
vrtornisiello Jun 23, 2025
bade2a3
run `populate_pgvector` command when starting the server
vrtornisiello Jun 23, 2025
a7775b0
rewrite query when invoking the assistant
vrtornisiello Jun 24, 2025
3cd447e
update `chatbot` package to version `0.5.1`
vrtornisiello Jun 24, 2025
2235dca
set top-k to 5, i.e., retrieve the metadata of the 5 most relevant da…
vrtornisiello Jun 24, 2025
1cf755e
remove deploy key setup as the submodule is now public
vrtornisiello Jun 24, 2025
c9bada3
update log message
vrtornisiello Jun 24, 2025
d124ea1
improved comments
vrtornisiello Jun 24, 2025
879bd3b
Merge pull request #845 from basedosdados/feat/chatbot_similarity_search
rdahis Jun 25, 2025
70d1e0c
remove `model_provider` arg
vrtornisiello Jun 25, 2025
a765798
Merge pull request #846 from basedosdados/refactor/remove_model_provider
rdahis Jun 26, 2025
3c7e58c
support multiple chats
vrtornisiello Jul 4, 2025
d094a08
using full table ids and spaces instead of tabs for tables metadata
vrtornisiello Jul 4, 2025
e5c991f
remove default value, as `None` is the default already.
vrtornisiello Jul 7, 2025
0865d73
add optional ordering for `ThreadListView` and `MessageListView`
vrtornisiello Jul 7, 2025
b765b71
update `chatbot` app test cases
vrtornisiello Jul 7, 2025
de3456d
add a single migration file
vrtornisiello Jul 7, 2025
41f3d64
return only non-deleted threads
vrtornisiello Jul 14, 2025
46c4836
refactor thread deletion endpoint
vrtornisiello Jul 15, 2025
c33cdd6
make `title` a required field in the `Thread` model and populate old …
vrtornisiello Jul 17, 2025
0cd0fb3
Merge pull request #847 from basedosdados/feat/support-multiple-chats
vrtornisiello Jul 18, 2025
9f72df7
update `chatbot` package to version `0.5.2`
vrtornisiello Jul 18, 2025
93f7c0b
prepare `MessagePair` model for streaming
vrtornisiello Jul 18, 2025
dad3e67
add streaming support
vrtornisiello Jul 18, 2025
5849250
use `Response` and `status` from DRF
vrtornisiello Jul 21, 2025
b08fc49
update `chatbot` package to version `0.5.2`
vrtornisiello Jul 22, 2025
48c4677
fix chatbot migration comments
vrtornisiello Jul 22, 2025
2d3ea69
Merge pull request #848 from basedosdados/feat/chatbot-stream-response
vrtornisiello Jul 24, 2025
ad29477
fix: correctly parse parallel tool calls
vrtornisiello Jul 25, 2025
f627096
Merge pull request #849 from basedosdados/fix/streaming-parallel-tool…
vrtornisiello Jul 25, 2025
c333cb4
Add token bridge endpoint for chatbot authentication
rdahis Jul 30, 2025
faa15bc
fix: use poetry to install and run the project
vrtornisiello Jul 30, 2025
c3fff29
fix: make poetry venv accessible to all users
vrtornisiello Jul 30, 2025
e02b21c
add comments to `Dockerfile`
vrtornisiello Jul 30, 2025
013c8fe
Merge pull request #850 from basedosdados/fix/dockerfiles
vrtornisiello Jul 30, 2025
ace1e48
show table.version in admin
rdahis Aug 5, 2025
18b2c26
Trigger deployment - empty commit
rdahis Aug 5, 2025
a37473d
increase helm timeout period to 10 minute
rdahis Aug 5, 2025
b376674
build: increase dev ram memory (k8s) (#851)
vrtornisiello Aug 6, 2025
28a95c4
chore: sync dev and main #844, #851
Winzen Aug 6, 2025
19edc0c
Atualizar deploy-dev conflitado com main
Winzen Aug 6, 2025
2650082
chore: sync dev and main (#844, #851) #852
Winzen Aug 7, 2025
586277b
chore: update `helm upgrade` command timeout to 15 minutes (#853)
vrtornisiello Aug 7, 2025
b14c9e1
chore: sync dev and main #853
Winzen Aug 7, 2025
5209874
chore: sync dev and main (#853) & test Helm timeout #854
Winzen Aug 7, 2025
5d8dc4a
revert `Dockerfile` changes to debug deploy timeout
vrtornisiello Aug 8, 2025
e486a21
Merge pull request #855 from basedosdados/debug/revert-dockerfile-cha…
vrtornisiello Aug 8, 2025
29de638
restore `Dockerfile` updates to use Poetry
vrtornisiello Aug 8, 2025
e907f6e
Merge pull request #856 from basedosdados/revert/restore-dockerfile-u…
vrtornisiello Aug 8, 2025
ce4d77e
chore: update chatbot package (#857)
vrtornisiello Aug 20, 2025
f70a3cc
chore: update chatbot package to `v0.6.1` (#858)
vrtornisiello Aug 20, 2025
54f2a5c
fix: update chatbot v0.6.1 (#859)
vrtornisiello Aug 20, 2025
d97d78c
fix: impedir remoção de usuários administradores do Google Groups
AldemirLucas Aug 21, 2025
a0bb372
fix: validar e normalizar e-mails antes da remoção de usuários do Goo…
AldemirLucas Aug 22, 2025
c15018d
fix: aprimorar verificação de usuários administradores antes da remoç…
AldemirLucas Aug 22, 2025
88696ce
fix: tratar exceção ao buscar usuário para remoção do Google Groups
AldemirLucas Aug 22, 2025
111d659
fix: impedir remoção de e-mails internos do Google Groups
AldemirLucas Aug 22, 2025
89ac3c2
fix: comentar remoção de usuários do Google Groups para evitar exclus…
AldemirLucas Aug 22, 2025
a609e6b
fix: implementar remoção de usuários do Google Groups com tratamento …
AldemirLucas Aug 22, 2025
2f23996
Merge pull request #866 from basedosdados/feat/1343
AldemirLucas Aug 22, 2025
6306f83
Merge pull request #868 from basedosdados/feat/1343
AldemirLucas Aug 23, 2025
9349f10
feat: adicionar endpoint para estatísticas de tabelas e datasets
AldemirLucas Aug 28, 2025
2d66bbb
fix: ajustar contagem de tabelas atualizadas nos últimos 30 dias e re…
AldemirLucas Aug 29, 2025
32581c2
Merge pull request #872 from basedosdados/feat/table_stats
AldemirLucas Aug 29, 2025
1d0e649
feat: react agent (#873)
vrtornisiello Sep 1, 2025
7d32277
feat: query billing limit (#875)
vrtornisiello Sep 4, 2025
b73fb15
feat: custom react agent (#876)
vrtornisiello Sep 5, 2025
da525e9
fix: aprimorar a função de adição de usuários ao Google Groups com no…
AldemirLucas Sep 5, 2025
048358c
Merge pull request #877 from basedosdados/feat/1351
AldemirLucas Sep 5, 2025
6b3bd3f
Merge pull request #880 from basedosdados/feat/1351
AldemirLucas Sep 8, 2025
0a6e307
feat: adicionar endpoint para visualização de colunas com suporte a b…
AldemirLucas Sep 8, 2025
417f93c
feat: adicionar propriedade para cobertura temporal da tabela e aprim…
AldemirLucas Sep 9, 2025
6605f68
feat: adicionar pré-carregamento de intervalos de data para cobertura…
AldemirLucas Sep 9, 2025
6935e2b
chore: system prompt tuning + other stuff (#884)
vrtornisiello Sep 11, 2025
348bfd5
Merge pull request #885 from basedosdados/feat/stripe_cancel_immediat…
AldemirLucas Sep 16, 2025
6c44fd6
Merge pull request #886 from basedosdados/feat/columns_api
AldemirLucas Sep 16, 2025
22dbc96
fix: empty ai message (#887)
vrtornisiello Sep 16, 2025
59e18a1
refatorar: renomear mutações de assinatura Stripe para melhor clareza…
AldemirLucas Sep 22, 2025
1af3d7a
feat: implementar remoção de todos os membros de uma assinatura Strip…
AldemirLucas Sep 22, 2025
0fdc5c3
Merge pull request #888 from basedosdados/feat/renamed_mutations
AldemirLucas Sep 22, 2025
8e03440
fix: adicionar tratamento de exceção na mutação de cancelamento imedi…
AldemirLucas Sep 23, 2025
a73373e
Merge pull request #894 from basedosdados/fix/dev-graphql
AldemirLucas Sep 23, 2025
cf89973
refactor: truncate tool output (#895)
vrtornisiello Sep 23, 2025
7979bca
chore: update agent prompt (#897)
vrtornisiello Sep 25, 2025
2297688
chore: add usage guides (#903)
vrtornisiello Oct 2, 2025
a0df779
style: run ruff linter
vrtornisiello Oct 6, 2025
770d1a5
chore: add `contains=tables` parameter (#905)
vrtornisiello Oct 9, 2025
936ae92
chore: chatbot service account (#911)
vrtornisiello Oct 29, 2025
4fcafc0
feat: show sql query (#912)
vrtornisiello Oct 31, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .env.docker
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,40 @@
ADMINS="Gabriel Milan,gabriel.gazola@poli.ufrj.br"
EMAIL_HOST_USER="notifications@gmail.com"
EMAIL_HOST_PASSWORD="password"

# Django configurations
DJANGO_SECRET_KEY="some-secret"
DJANGO_SETTINGS_MODULE="backend.settings.local"

# Logger
LOGGER_LEVEL="DEBUG"
LOGGER_IGNORE="faker,haystack"
LOGGER_SERIALIZE=""

# Database
DB_HOST="database"
DB_PORT="5432"
DB_NAME="postgres"
DB_USER="postgres"
DB_PASSWORD="postgres"

# Queue
REDIS_HOST="queue"
REDIS_PORT="6379"

# Index
ELASTICSEARCH_URL=http://index:9200

# Chatbot
CHATBOT_CREDENTIALS=
BILLING_PROJECT_ID=
QUERY_PROJECT_ID=
MODEL_URI=
EMBEDDING_MODEL=
PGVECTOR_COLLECTION=
LANGCHAIN_TRACING_V2=
LANGCHAIN_API_KEY=

# Local DB populating
METABASE_USER=
METABASE_PASSWORD=
2 changes: 2 additions & 0 deletions .github/workflows/ci-python.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: recursive
- name: Set up poetry
run: pipx install poetry
- name: Set up python
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/deploy-dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -56,10 +56,10 @@ jobs:
resources:
limits:
cpu: 500m
memory: 1Gi
memory: 4096Mi
requests:
cpu: 250m
memory: 500Mi
memory: 1536Mi
envFrom:
- secretRef:
name: api-development-secrets
Expand Down Expand Up @@ -87,6 +87,7 @@ jobs:
run: |-
helm upgrade \
--wait \
--timeout 15m \
--install \
--namespace website \
--values values.yaml \
Expand Down
4 changes: 1 addition & 3 deletions .github/workflows/deploy-prod.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,7 @@ on:
jobs:
deploy-prod:
runs-on: ubuntu-latest
if: |
${{ github.event_name == 'workflow_dispatch' }}
|| ${{ github.event.workflow_run.conclusion == 'success' }}
if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }}
environment:
name: production
url: https://backend.basedosdados.org
Expand Down
4 changes: 1 addition & 3 deletions .github/workflows/deploy-staging.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,7 @@ on:
jobs:
deploy-staging:
runs-on: ubuntu-latest
if: |
${{ github.event_name == 'workflow_dispatch' }}
|| ${{ github.event.workflow_run.conclusion == 'success' }}
if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }}
environment:
name: staging
url: https://staging.backend.basedosdados.org
Expand Down
7 changes: 3 additions & 4 deletions .github/workflows/release-dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@
name: Release Image (Development)
on:
push:
branches:
- dev
branches: [dev]
jobs:
release-docker:
name: Release Image
Expand All @@ -13,6 +12,7 @@ jobs:
uses: actions/checkout@v4
with:
ref: dev
submodules: recursive
- name: Login to GitHub Container Registry
uses: docker/login-action@v2
with:
Expand All @@ -30,6 +30,5 @@ jobs:
labels: |
org.opencontainers.image.source=${{ github.event.repository.html_url }}
org.opencontainers.image.revision=${{ github.sha }}
build-args: |
build-args: |-
BUILDKIT_INLINE_CACHE=1

4 changes: 2 additions & 2 deletions .github/workflows/release-prod.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@
name: Release Image (Production)
on:
push:
branches:
- main
branches: [main]
jobs:
release-docker:
name: Release Image
Expand All @@ -13,6 +12,7 @@ jobs:
uses: actions/checkout@v4
with:
ref: main
submodules: recursive
- name: Login to GitHub Container Registry
uses: docker/login-action@v2
with:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/release-staging.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@
name: Release Image (Staging)
on:
push:
branches:
- staging
branches: [staging]
jobs:
release-docker:
name: Release Image
Expand All @@ -13,6 +12,7 @@ jobs:
uses: actions/checkout@v4
with:
ref: staging
submodules: recursive
- name: Login to GitHub Container Registry
uses: docker/login-action@v2
with:
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -256,3 +256,6 @@ fixtures*

# Version manager
.tool-versions

# fetch_metabase script
metabase_data
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "chatbot"]
path = chatbot
url = git@github.com:basedosdados/chatbot.git
3 changes: 2 additions & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,5 @@ repos:
name: yamlfix
types: [yaml]
language: system
entry: yamlfix --exclude "charts/**/*" .
entry: yamlfix
exclude: ^charts/
47 changes: 29 additions & 18 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,38 +1,49 @@
ARG PYTHON_VERSION=3.11-slim

FROM python:${PYTHON_VERSION}
FROM python:$PYTHON_VERSION

# Install virtualenv and create a virtual environment
RUN pip install --no-cache-dir -U virtualenv>=20.13.1 && virtualenv /env --python=python3.11
ENV PATH /env/bin:$PATH
# Define where Poetry virtual environments will be stored
ARG POETRY_VIRTUALENVS_PATH=/opt/pypoetry/virtualenvs

# Install pip requirements
WORKDIR /app
COPY . .
RUN /env/bin/pip install --no-cache-dir . && rm nginx.conf
# Ensure that the python output is sent straight to terminal (e.g. your container log)
# without being first buffered and that you can see the output of your application (e.g. django logs)
# in real time. Equivalent to python -u: https://docs.python.org/3/using/cmdline.html#cmdoption-u
ENV PYTHONUNBUFFERED=1

# Prevent Python from writing .pyc files to disc
# https://docs.python.org/3/using/cmdline.html#envvar-PYTHONDONTWRITEBYTECODE
ENV PYTHONDONTWRITEBYTECODE=1

# Install make, nginx and copy configuration
RUN apt-get update \
&& apt-get install -y --no-install-recommends curl make nginx \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* \
&& rm /etc/nginx/sites-enabled/default
RUN apt-get update && apt-get install -y postgresql postgresql-contrib
COPY nginx.conf /etc/nginx/nginx.conf

# Prevents Python from writing .pyc files to disc
# https://docs.python.org/3/using/cmdline.html#envvar-PYTHONDONTWRITEBYTECODE
ENV PYTHONDONTWRITEBYTECODE 1
# Install Poetry and add it to PATH so its commands can be executed
# from anywhere, without specifying the full path to its executable.
RUN curl -sSL https://install.python-poetry.org | python3 - --version 2.1.3
ENV PATH="/root/.local/bin:$PATH"

# Ensures that the python output is sent straight to terminal (e.g. your container log)
# without being first buffered and that you can see the output of your application (e.g. django logs)
# in real time. Equivalent to python -u: https://docs.python.org/3/using/cmdline.html#cmdoption-u
ENV PYTHONUNBUFFERED 1
# Create the folder where Poetry virtual environments will be stored and make it
# accessible to all users. This is needed by the 'www-data' user during server startup
RUN mkdir -p $POETRY_VIRTUALENVS_PATH && chmod 755 $POETRY_VIRTUALENVS_PATH
ENV POETRY_VIRTUALENVS_PATH=$POETRY_VIRTUALENVS_PATH

# Copy and install project
WORKDIR /app
COPY . .
RUN test -d ./chatbot || (echo "ERROR: Git submodule 'chatbot' not found. Please run 'git submodule update --init --recursive'. See backend/README.md for more information." && exit 1)
RUN poetry install --only main && rm nginx.conf

# Copy app, generate static and set permissions
RUN /env/bin/python manage.py collectstatic --no-input --settings=backend.settings.base && \
# Generate static and set permissions
RUN poetry run python manage.py collectstatic --no-input --settings=backend.settings.base && \
chown -R www-data:www-data /app

# Expose and run app
EXPOSE 80
STOPSIGNAL SIGKILL
CMD ["/app/start-server.sh"]
CMD ["poetry", "run", "/app/start-server.sh"]
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@
git clone https://github.com/basedosdados/backend.git
```

- Baixar submodulos
```
git submodule update --recursive --init
```

- Abrí-lo no seu editor de texto

- No seu ambiente de desenvolvimento, instalar [poetry](https://python-poetry.org/) para gerenciamento de dependências
Expand Down
Empty file added backend/apps/__init__.py
Empty file.
4 changes: 3 additions & 1 deletion backend/apps/account/admin.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from django.utils.translation import gettext_lazy
from faker import Faker

from backend.apps.account.models import Account, BDGroup, BDRole, Team, Role, Career, Subscription
from backend.apps.account.models import Account, BDGroup, BDRole, Career, Role, Subscription, Team
from backend.apps.account.tasks import sync_subscription_task


Expand Down Expand Up @@ -204,6 +204,7 @@ class AccountAdmin(BaseAccountAdmin):
"created_at",
"is_admin",
"is_subscriber",
"has_chatbot_access",
)
list_filter = (
SuperUserListFilter,
Expand Down Expand Up @@ -264,6 +265,7 @@ class AccountAdmin(BaseAccountAdmin):
"is_active",
"is_admin",
"is_superuser",
"has_chatbot_access",
"staff_groups",
)
},
Expand Down
4 changes: 1 addition & 3 deletions backend/apps/account/migrations/0001_initial.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,9 +84,7 @@ class Migration(migrations.Migration):
),
(
"twitter",
models.CharField(
blank=True, max_length=255, null=True, verbose_name="Twitter"
),
models.CharField(blank=True, max_length=255, null=True, verbose_name="Twitter"),
),
(
"linkedin",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# -*- coding: utf-8 -*-
# Generated by Django 4.2.18 on 2025-02-04 04:02

from django.db import migrations, models
import django.db.models.deletion
from django.db import migrations, models


class Migration(migrations.Migration):
Expand Down
3 changes: 2 additions & 1 deletion backend/apps/account/migrations/0020_career_role_new.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# -*- coding: utf-8 -*-
# Generated by Django 4.2.18 on 2025-02-04 04:04

from django.db import migrations, models
import django.db.models.deletion
from django.db import migrations, models


class Migration(migrations.Migration):
Expand Down
Original file line number Diff line number Diff line change
@@ -1,23 +1,23 @@
# -*- coding: utf-8 -*-
# Generated by Django 4.2.18 on 2025-02-05 00:43

from django.db import migrations, models
from django.db import migrations


class Migration(migrations.Migration):

dependencies = [
('account', '0020_career_role_new'),
("account", "0020_career_role_new"),
]

operations = [
migrations.RenameField(
model_name='career',
old_name='role',
new_name='role_old',
model_name="career",
old_name="role",
new_name="role_old",
),
migrations.RenameField(
model_name='career',
old_name='team',
new_name='team_old',
model_name="career",
old_name="team",
new_name="team_old",
),
]
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,19 @@


class Migration(migrations.Migration):

dependencies = [
('account', '0021_rename_role_career_role_old_and_more'),
("account", "0021_rename_role_career_role_old_and_more"),
]

operations = [
migrations.RenameField(
model_name='career',
old_name='role_new',
new_name='role',
model_name="career",
old_name="role_new",
new_name="role",
),
migrations.RenameField(
model_name='career',
old_name='team_new',
new_name='team',
model_name="career",
old_name="team_new",
new_name="team",
),
]
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,19 @@


class Migration(migrations.Migration):

dependencies = [
('account', '0022_rename_role_new_career_role_and_more'),
("account", "0022_rename_role_new_career_role_and_more"),
]

operations = [
migrations.AlterField(
model_name='career',
name='role_old',
field=models.CharField(blank=True, max_length=40, verbose_name='Role (old)'),
model_name="career",
name="role_old",
field=models.CharField(blank=True, max_length=40, verbose_name="Role (old)"),
),
migrations.AlterField(
model_name='career',
name='team_old',
field=models.CharField(blank=True, max_length=40, verbose_name='Team (old)'),
model_name="career",
name="team_old",
field=models.CharField(blank=True, max_length=40, verbose_name="Team (old)"),
),
]
Loading
Loading