The `git config -l` command stopped working after upgrading to Bitbucket 8

[gnustavo]

I'm using External Hooks version 13.1.1 on Bitbucket Data Center, running in a container based on the atlassian/bitbucket:7.21.4 Docker image.

I upgraded Bitbucket, recreating the container based on the atlassian/bitbucket:8.9.4 image. After the upgrade, my external hooks stopped working.

I investigated and noticed that the external hook scripts are being invoked in an environment in which the command git config -l does not find the repository's configuration.

I made the hook script output its environment variables and current directory both before the upgrade and after it.

Before the upgrade, the hook script is invoked in the /var/atlassian/application-data/bitbucket/shared/data/repositories/2295 directory with the following environment variables (I replaced some sensitive parts with xxxxxxxx):

APPLICATION_MODE=default
APP_NAME=bitbucket
BB_BASE_URL=https://bitbucket.xxxxxxxx.br
BB_HOOK_TRIGGER_ID=push
BB_HOOK_TYPE=PRE
BB_IS_DRY_RUN=false
BB_PROJECT_KEY=GTIC
BB_REPO_CLONE_HTTP=https://bitbucket.xxxxxxxx.br/scm/gtic/cds.git
BB_REPO_CLONE_SSH=ssh://git\@bitbucket.xxxxxxxx.br:7999/gtic/cds.git
BB_REPO_IS_FORK=false
BB_REPO_IS_PUBLIC=false
BB_REPO_SLUG=cds
BB_USER_DISPLAY_NAME=Gustavo xxxxxxxx
BB_USER_EMAIL=gustavo\@xxxxxxxx.br
BB_USER_NAME=gustavo
BB_USER_PERMISSION=SYS_ADMIN
BIN_DIR=/opt/atlassian/bitbucket/bin
BITBUCKET_HOME=/var/atlassian/application-data/bitbucket
BITBUCKET_INSTALL_DIR=/opt/atlassian/bitbucket
BITBUCKET_USER=
FEATURE_PUBLIC_ACCESS=false
GIT_ALTERNATE_OBJECT_DIRECTORIES=/var/atlassian/application-data/bitbucket/shared/data/repositories/2295/./objects
GIT_OBJECT_DIRECTORY=/var/atlassian/application-data/bitbucket/shared/data/repositories/2295/./objects/incoming-m0PJSe
GIT_QUARANTINE_PATH=/var/atlassian/application-data/bitbucket/shared/data/repositories/2295/./objects/incoming-m0PJSe
HOME=/var/atlassian/application-data/bitbucket
HOSTNAME=9ca50b5dcc0a
INST_DIR=/opt/atlassian/bitbucket
JAVA_BINARY=/opt/java/openjdk/bin/java
JAVA_HOME=/opt/java/openjdk
JAVA_VERSION=11.0.20
JDBC_DRIVER=org.postgresql.Driver
JDBC_PASSWORD=xxxxxxxx
JDBC_URL=jdbc:postgresql://postgres:5432/bitbucket?targetServerType=master
JDBC_USER=bitbucket
JMX_ENABLED=false
JRE_HOME=/opt/java/openjdk
JVM_MAXIMUM_MEMORY=2G
LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_ALL=en_US.UTF-8
LOGNAME=bitbucket
OLDPWD=/opt/atlassian/bitbucket/bin
PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PLUGIN_SEARCH_CONFIG_BASEURL=http://opensearch:9200
PWD=/var/atlassian/application-data/bitbucket/shared/data/repositories/2295
REMOTE_USER=gustavo
RUN_GID=2003
RUN_GROUP=bitbucket
RUN_UID=2003
RUN_USER=bitbucket
SEARCH_ENABLED=false
SHELL=/bin/bash
SHLVL=1
STASH_BASE_URL=https://bitbucket.xxxxxxxx.br
STASH_IS_ADMIN=false
STASH_IS_DIRECT_ADMIN=
STASH_IS_DIRECT_WRITE=
STASH_PROJECT_KEY=GTIC
STASH_PROJECT_NAME=
STASH_REPO_CLONE_HTTP=https://bitbucket.xxxxxxxx.br/scm/gtic/cds.git
STASH_REPO_CLONE_SSH=ssh://git\@bitbucket.xxxxxxxx.br:7999/gtic/cds.git
STASH_REPO_IS_FORK=false
STASH_REPO_NAME=cds
STASH_USER_DISPLAY_NAME=Gustavo xxxxxxxx
STASH_USER_EMAIL=gustavo\@xxxxxxxx.br
STASH_USER_NAME=gustavo
TMPDIR=/var/atlassian/application-data/bitbucket/tmp
TZ=America/Sao_Paulo
USER=bitbucket
_=/var/atlassian/application-data/bitbucket/shared/external-hooks/pre-receive

After the upgrade, the pre-receive script is invoked in the /var/atlassian/application-data/bitbucket/mesh/tmp/script-sandbox7711861015479551760 directory with the following environment variables:

APPLICATION_MODE=default
APP_NAME=bitbucket
BB_BASE_URL=https://bitbucket.xxxxxxxx.br
BB_HOOK_TRIGGER_ID=push
BB_HOOK_TYPE=PRE
BB_IS_DRY_RUN=false
BB_PROJECT_KEY=GTIC
BB_REPO_CLONE_HTTP=https://bitbucket.xxxxxxxx.br/scm/gtic/cds.git
BB_REPO_CLONE_SSH=ssh://git\@bitbucket.xxxxxxxx.br:7999/gtic/cds.git
BB_REPO_IS_FORK=false
BB_REPO_IS_PUBLIC=false
BB_REPO_SLUG=cds
BB_USER_DISPLAY_NAME=Gustavo xxxxxxxx
BB_USER_EMAIL=gustavo\@xxxxxxxx.br
BB_USER_NAME=gustavo
BB_USER_PERMISSION=SYS_ADMIN
BIN_DIR=/opt/atlassian/bitbucket/bin
BITBUCKET_HOME=/var/atlassian/application-data/bitbucket
BITBUCKET_INSTALL_DIR=/opt/atlassian/bitbucket
BITBUCKET_USER=
FEATURE_PUBLIC_ACCESS=false
GIT_ALTERNATE_OBJECT_DIRECTORIES=/var/atlassian/application-data/bitbucket/shared/data/repositories/2295/./objects/incoming-vezKfG
HOME=/var/atlassian/application-data/bitbucket
HOSTNAME=868e01573c1f
INST_DIR=/opt/atlassian/bitbucket
JAVA_BINARY=/opt/java/openjdk/bin/java
JAVA_HOME=/opt/java/openjdk
JAVA_VERSION=11.0.20
JDBC_DRIVER=org.postgresql.Driver
JDBC_PASSWORD=xxxxxxxx
JDBC_URL=jdbc:postgresql://postgres:5432/bitbucket?targetServerType=master
JDBC_USER=bitbucket
JMX_ENABLED=false
JRE_HOME=/opt/java/openjdk
JVM_MAXIMUM_MEMORY=2G
LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_ALL=en_US.UTF-8
LOGNAME=bitbucket
OLDPWD=/opt/atlassian/bitbucket/bin
PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PLUGIN_SEARCH_CONFIG_BASEURL=http://opensearch:9200
PWD=/var/atlassian/application-data/bitbucket/mesh/tmp/script-sandbox7711861015479551760
RUN_GID=2003
RUN_GROUP=bitbucket
RUN_UID=2003
RUN_USER=bitbucket
SEARCH_ENABLED=false
SHELL=/bin/bash
SHLVL=1
STASH_BASE_URL=https://bitbucket.xxxxxxxx.br
STASH_IS_ADMIN=false
STASH_IS_DIRECT_ADMIN=
STASH_IS_DIRECT_WRITE=
STASH_PROJECT_KEY=GTIC
STASH_PROJECT_NAME=
STASH_REPO_CLONE_HTTP=https://bitbucket.xxxxxxxx.br/scm/gtic/cds.git
STASH_REPO_CLONE_SSH=ssh://git\@bitbucket.xxxxxxxx.br:7999/gtic/cds.git
STASH_REPO_IS_FORK=false
STASH_REPO_NAME=cds
STASH_USER_DISPLAY_NAME=Gustavo xxxxxxxx
STASH_USER_EMAIL=gustavo\@xxxxxxxx.br
STASH_USER_NAME=gustavo
TMPDIR=/var/atlassian/application-data/bitbucket/mesh/tmp
TZ=America/Sao_Paulo
USER=bitbucket
_=/var/atlassian/application-data/bitbucket/shared/external-hooks/pre-receive

The command git config -l invoked by the external hook script returns just three lines:

core.repositoryformatversion=0
core.filemode=true
core.bare=true

Since the script is being invoked from a different directory, perhaps the GIT_OBJECT_DIRECTORY is missing.

I read the documentation but couldn't find anything related to this problem.

Can you help me figure this out?

Thanks!

[gnustavo]

As a workaround, I'm setting the GIT_DIR environment variable in the beginning of the hook script with code equivalent to the following shell script:

GIT_DIR=$(dirname $(dirname $(dirname $GIT_ALTERNATE_OBJECT_DIRECTORIES)))
export GIT_DIR

[gnustavo]

The workaround works in the pre-receive hook. But the post-receive hook runs without any GIT_* environment variable. There is no environment variable from which I can infer the path to the repository.

[gnustavo]

I came up with the simplest set up I can think of that should allow one to reproduce the problem using the latest atlassian/bitbucket Docker image, which is tagged 8.13 today.

Just to make sure that the problem had nothing to do with my particular environment, I created a new container like this:

$ docker volume create --name bitbucketVolume
$ docker run -v bitbucketVolume:/var/atlassian/application-data/bitbucket --name="bitbucket" -d -p 7990:7990 -p 7999:7999 atlassian/bitbucket:8.13

Then, I installed the External Hooks plugin, created a new repository, enabled the plugin in it and installed a hook like this:

$ docker exec -it -u bitbucket bitbucket bash
bitbucket@2b947c62ada2:~$ mkdir shared/external-hooks
bitbucket@2b947c62ada2:~$ cat >shared/external-hooks/pre-receive <<EOF
#!/bin/bash -x

date
echo
pwd
echo
env | sort
echo
ls -la
echo
git config -l --show-origin
exit 1
EOF
bitbucket@2b947c62ada2:~$ chmod +x shared/external-hooks/pre-receive

Then, I git-cloned the repository, created a commit and tried to git-push it. This is the result:

$ git:(master) git push
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Writing objects: 100% (3/3), 255 bytes | 255.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
remote: + date
remote: + echo
remote: + pwd
remote: + echo
remote: + env
remote: + sort
remote: + echo
remote: + ls -la
remote: + echo
remote: + git config -l --show-origin
remote: + exit 1
remote: external-pre-receive-hook declined
remote: Sat Aug 26 05:45:41 PM UTC 2023
remote: 
remote: /var/atlassian/application-data/bitbucket/mesh/tmp/script-sandbox9020684115962208243
remote: 
remote: APPLICATION_MODE=default
remote: APP_NAME=bitbucket
remote: BB_BASE_URL=http://localhost:7990
remote: BB_HOOK_TRIGGER_ID=push
remote: BB_HOOK_TYPE=PRE
remote: BB_IS_DRY_RUN=false
remote: BB_PROJECT_KEY=~GUSTAVO
remote: BB_REPO_CLONE_HTTP=http://localhost:7990/scm/~gustavo/xpto.git
remote: BB_REPO_CLONE_SSH=ssh://git@localhost:7999/~gustavo/xpto.git
remote: BB_REPO_IS_FORK=false
remote: BB_REPO_IS_PUBLIC=false
remote: BB_REPO_SLUG=xpto
remote: BB_USER_DISPLAY_NAME=Gustavo
remote: BB_USER_EMAIL=gustavo@xxxxxxxx
remote: BB_USER_NAME=gustavo
remote: BB_USER_PERMISSION=SYS_ADMIN
remote: BIN_DIR=/opt/atlassian/bitbucket/bin
remote: BITBUCKET_HOME=/var/atlassian/application-data/bitbucket
remote: BITBUCKET_INSTALL_DIR=/opt/atlassian/bitbucket
remote: BITBUCKET_USER=
remote: GIT_ALTERNATE_OBJECT_DIRECTORIES=/var/atlassian/application-data/bitbucket/shared/data/repositories/1/./objects/tmp_objdir-incoming-bjmuTb
remote: HOME=/var/atlassian/application-data/bitbucket
remote: HOSTNAME=2b947c62ada2
remote: INST_DIR=/opt/atlassian/bitbucket
remote: JAVA_BINARY=/opt/java/openjdk/bin/java
remote: JAVA_HOME=/opt/java/openjdk
remote: JAVA_VERSION=11.0.20
remote: JRE_HOME=/opt/java/openjdk
remote: LANG=en_US.UTF-8
remote: LANGUAGE=en_US:en
remote: LC_ALL=en_US.UTF-8
remote: LOGNAME=bitbucket
remote: OLDPWD=/opt/atlassian/bitbucket/bin
remote: PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
remote: PWD=/var/atlassian/application-data/bitbucket/mesh/tmp/script-sandbox9020684115962208243
remote: RUN_GID=2003
remote: RUN_GROUP=bitbucket
remote: RUN_UID=2003
remote: RUN_USER=bitbucket
remote: SEARCH_ENABLED=true
remote: SHELL=/bin/bash
remote: SHLVL=3
remote: STASH_BASE_URL=http://localhost:7990
remote: STASH_IS_ADMIN=false
remote: STASH_IS_DIRECT_ADMIN=
remote: STASH_IS_DIRECT_WRITE=
remote: STASH_PROJECT_KEY=~GUSTAVO
remote: STASH_PROJECT_NAME=
remote: STASH_REPO_CLONE_HTTP=http://localhost:7990/scm/~gustavo/xpto.git
remote: STASH_REPO_CLONE_SSH=ssh://git@localhost:7999/~gustavo/xpto.git
remote: STASH_REPO_IS_FORK=false
remote: STASH_REPO_NAME=xpto
remote: STASH_USER_DISPLAY_NAME=Gustavo
remote: STASH_USER_EMAIL=gustavo@xxxxxxxx
remote: STASH_USER_NAME=gustavo
remote: TMPDIR=/var/atlassian/application-data/bitbucket/mesh/tmp
remote: USER=bitbucket
remote: _=/usr/bin/env
remote: 
remote: total 44
remote: drwx------ 7 bitbucket bitbucket 4096 Aug 26 17:45 .
remote: drwxr-xr-x 4 bitbucket bitbucket 4096 Aug 26 17:45 ..
remote: drwxr-xr-x 2 bitbucket bitbucket 4096 Aug 26 17:45 branches
remote: -rw-r--r-- 1 bitbucket bitbucket   66 Aug 26 17:45 config
remote: -rw-r--r-- 1 bitbucket bitbucket   73 Aug 26 17:45 description
remote: -rw-r--r-- 1 bitbucket bitbucket   23 Aug 26 17:45 HEAD
remote: drwxr-xr-x 2 bitbucket bitbucket 4096 Aug 26 17:45 hooks
remote: drwxr-xr-x 2 bitbucket bitbucket 4096 Aug 26 17:45 info
remote: drwxr-xr-x 4 bitbucket bitbucket 4096 Aug 26 17:45 objects
remote: -rw-r--r-- 1 bitbucket bitbucket  105 Aug 26 17:45 packed-refs
remote: drwxr-xr-x 4 bitbucket bitbucket 4096 Aug 26 17:45 refs
remote: 
remote: file:config	core.repositoryformatversion=0
remote: file:config	core.filemode=true
remote: file:config	core.bare=true
To http://localhost:7990/scm/~gustavo/xpto.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'http://localhost:7990/scm/~gustavo/xpto.git'

Notice how the command git commit -l returns just three lines. This is different from what it should return if it was invoked in the repository's own bare clone:

bitbucket@2b947c62ada2:~$ cd ~/shared/data/repositories/1
bitbucket@2b947c62ada2:~/shared/data/repositories/1$ git config -l --show-origin
error: cannot run less: No such file or directory
file:config	include.path=../../../config/git/system-config
file:../../../config/git/system-config	core.hookspath=../../../config/git/hooks
file:../../../config/git/system-config	gc.auto=0
file:../../../config/git/system-config	gc.refs/pull-requests/**.reflogexpire=never
file:../../../config/git/system-config	gc.refs/pull-requests/**.reflogexpireunreachable=never
file:../../../config/git/system-config	gc.stash-refs/pull-requests/**.reflogexpire=never
file:../../../config/git/system-config	gc.stash-refs/pull-requests/**.reflogexpireunreachable=never
file:../../../config/git/system-config	http.receivepack=true
file:../../../config/git/system-config	maintenance.auto=false
file:../../../config/git/system-config	maintenance.strategy=none
file:../../../config/git/system-config	pack.island=(refs)/heads/
file:../../../config/git/system-config	pack.island=(refs)/tags/
file:../../../config/git/system-config	pack.island=refs/(pull-requests)/
file:../../../config/git/system-config	pack.islandcore=refs
file:../../../config/git/system-config	pack.writebitmaphashcache=true
file:../../../config/git/system-config	repack.usedeltaislands=true
file:../../../config/git/system-config	repack.writebitmaps=true
file:../../../config/git/system-config	uploadpack.allowanysha1inwant=true
file:../../../config/git/system-config	uploadpack.allowfilter=true
file:config	include.path=repository-config
file:repository-config	bitbucket.hierarchy=bb86dfa676e468a36d69
file:repository-config	bitbucket.project=~GUSTAVO
file:repository-config	bitbucket.repository=xpto
file:config	core.repositoryformatversion=0
file:config	core.filemode=true
file:config	core.bare=true

[gnustavo]

I was able to infer the path to the repository using the environment variables BB_PROJECT_KEY and BB_REPO_SLUG, which are always set, as parameters to a REST API call, like this:

repo_id=$(curl -s -f -n \
               -H 'Content-Type:application/json' \
               http://localhost:7990/rest/api/1.0/projects/"${BB_PROJECT_KEY:?}"/repos/"${BB_REPO_SLUG:?}" \
              | jq .id)

export GIR_DIR=${BITBUCKET_HOME:?}/shared/data/repositories/${repo_id}

This way I can always set the GIT_DIR environment variable before using Git.

This seems to be a general workaround, so far.