How to setup a Woodpecker (forge) addon

[mzampetakis]

I am trying to implement a Woodpecker CI forge addon as documented here: https://woodpecker-ci.org/docs/administration/addons/creating-addons

The issue is that when I try to run woodpecker server it responds the plugin was built with a different version of a package.

So my addon is like this:

package main

import (
	"context"
	"fmt"
	"net/http"

	"github.com/rs/zerolog"
	"go.woodpecker-ci.org/woodpecker/v2/server/forge"
	forge_types "go.woodpecker-ci.org/woodpecker/v2/server/forge/types"
	"go.woodpecker-ci.org/woodpecker/v2/server/model"
	addon_types "go.woodpecker-ci.org/woodpecker/v2/shared/addon/types"
)

var Type = addon_types.TypeForge

func Addon(logger zerolog.Logger, env []string) (forge.Forge, error) {
	logger.Info().Msg("hello world from addon")
	logger.Info().Msg(fmt.Sprintf("%+v"))
	return &config{l: logger}, nil
}

type config struct {
	l zerolog.Logger
}

// Name returns the string name of this driver
func (c *config) Name() string {
	c.l.Error().Msg("call Name")
	return "addon-forge"
}

// URL returns the root url of a configured forge
func (c *config) URL() string {
	c.l.Error().Msg("call URL")
	return ""
}

// Implements the rest of the Forge interface ....

And the dependencies from go.mod:

module my-plugin
go 1.21.3

require (
	github.com/rs/zerolog v1.31.0
	go.woodpecker-ci.org/woodpecker/v2 v2.2.2
)

require (...)

I compile my addon using go build -buildmode plugin -o=/tmp/bin/my-addon ./ and then I am running woodpecker server using go run ./cmd/server having this in the .env file WOODPECKER_ADDONS=/tmp/bin/my-addon.

The result is:

» go run ./cmd/server
8:32AM INF shared/logger/logger.go:101 > log level: debug
8:32AM FTL cmd/server/main.go:46 > error running server error="can't setup forge: plugin.Open(\"/tmp/bin/my-addon\"): plugin was built with a different version of package go.woodpecker-ci.org/woodpecker/v2/server/forge/types"
exit status 1

My go env is:

» go env                                                                                                                                                                                                                                                                                                                                                  [24-01-24 8:38]
GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/mzampetakis/Library/Caches/go-build'
GOENV='/Users/mzampetakis/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/mzampetakis/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/mzampetakis/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.21.3'
GCCGO='gccgo'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/v6/5x24pmfn7j91k4fqg1s3p3c40000gn/T/go-build1195100213=/tmp/go-build -gno-record-gcc-switches -fno-common'

and I am testing this under MacOs with arm64 CPU.

Also tried to add the -trimpath argument at the plugin compilation go build -trimpath -buildmode plugin -o=/tmp/bin/my-addon ./ and then when I tried to run woodpecker server I got this error:

» go run ./cmd/server
8:40AM INF shared/logger/logger.go:101 > log level: debug
8:40AM FTL cmd/server/main.go:46 > error running server error="can't setup forge: plugin.Open(\"/tmp/bin/my-addon\"): plugin was built with a different version of package internal/goarch"
exit status 1

Also tried to clean caches using go clean -modcache & go clean -cache.

Is there anything wrong with my setup or the commands? Is there any incompatibility issue?

EDIT: I have woodpecker checked out from commit a19c488a99a3be0e7502743f5db8ae4462086145 (HEAD, tag: v2.2.2)

[mzampetakis]

These are the relater PR's that added the addon support at woodpecker #2984 , #2751

So it seems that @qwerty287 your feedback here will be valuable!

[qwerty287]

Yes, this is one of the limitations.

The plugin must be built with exactly the same code version of the imported packages.
Code version means: If just one statement changes, the code version is different.
(This only applies to packages imported/compiled into both plugin and server.)

I'm currently thinking about rewriting the addon approach completely again, see #3214 (I'm the only one among the maintainers who really supports the plugin appraoch).

This will likely also solve issues like this. (It will also require a complete rewrite of the plugins, but don't worry: If you publish the source of your addon and point me to it, I'm happy to do the migration for you.)

[mzampetakis]

Code version means: If just one statement changes, the code version is different.

I realised this with the hard way 🙃

I will proceed with the implementation of my forge addon and I will try to contribute at #3214.
I will probably download the source or the binary from the release page in order to check if that resolves my issue.

It will also require a complete rewrite of the plugins

Do you mean all the plugins or just the addons?

[qwerty287]

Do you mean all the plugins or just the addons?

Addons…

I will proceed with the implementation of my forge addon and I will try to contribute at #3214.

I'm currently trying some things there, expect a POC PR in the next days.

[mzampetakis]

Addons…

So, that wont be a great issue! Please keep me in the roll for any updates! ♥️

[qwerty287]

#3268 is it.

Please don't do any reviews or similar yet, this is in very early stages.

[mzampetakis]

Downloaded sources from https://github.com/woodpecker-ci/woodpecker/releases/tag/v2.2.2 and run using my addon but the issues still presists:

» go run ./cmd/server
7:53PM INF log level: info
7:53PM FTL error running server error="can't setup forge: plugin.Open(\"/tmp/bin/woodpecker-ci-radicle-forge-addon\"): plugin was built with a different version of package go.woodpecker-ci.org/woodpecker/v2/server/forge/types"
exit status 1

Also I can see that there is no binary available fro MacOS to test this using the binary directly.

[mzampetakis]

Just pushed it. You can find it at https://app.radicle.xyz/nodes/radicle.yorgos.net.gr/rad:zeu6DziA9483mVnkUGBHh9NFfaRZ and clone the repos through: git clone https://radicle.yorgos.net.gr/zeu6DziA9483mVnkUGBHh9NFfaRZ.git woodpecker-ci-radicle-forge-addon

Thanks for the interest! 🙌🏻

[qwerty287]

I figured it out.

Let me elaborate a bit: When I started working on addon support, I thought I found an ideal way. I really liked Go plugins and the idea behind it. Now, I'm starting to hate it. (I'll never use this again in any other project.) There are various reasons for it, see #3214.

Now, I need to add another issue.

It looks like "code version" does not only mean "content of code files", but you also need to compile it with the same files.

If you move your code to the source directory of Woodpecker, remove the go.mod and go.sum and update some imports, it will work - the server and addon were compiled from the same files. Unfortunately, I never tested this when adding initial addon support, I always compiled addons directly from Woodpecker's sources.

(Probably there are workarounds for this, but I didn't try to find one.)

It also looks like this makes the whole addon system kind of unusable if you don't compile your server yourself.

I'll try to finish my work on #3214 as soon as possible so we can fix this, but my time is very limited in the next two weeks.

---

Honestly, who wrote this shit? If the Go developers are unable to have a working plugin system, they should not provide something like this. At least, they really should include a warning in the docs what you need to pay attention to, the one that's already there misses most issues that would really matter.

[mzampetakis]

It looks like "code version" does not only mean "content of code files", but you also need to compile it with the same files.

Did you find any official reference for this? If that's the case it's completely useless from my point of view.

[qwerty287]

Did you find any official reference for this?

No, but I tested it.

If that's the case it's completely useless from my point of view.

Totally agreed. I'll need to finish my addon rework to fix this I think. I'm very sorry for these inconveniences...

[mzampetakis]

There is no need for apologies @qwerty287 ! We tried that, and it does not fit our needs. So we move on and we are a bit more "wiser"!

I am still trying to figure out how this Go's "plugin" system could be somehow useful for a project. I will try to find and read some use cases.

[qwerty287]

@mzampetakis Now with #3268 the approach is different. I'll close this for now.
How to implement your addon now: https://woodpecker-ci.org/docs/next/administration/forges/addon#creating-addon-forges

[mzampetakis]

Thanks @qwerty287 ! This is great!