Debug failed 0.7.0 release

[jisantuc]

The release for 0.7.0 failed repeatedly for an invalid POM file: https://github.com/scala-exercises/sbt-exercise/actions/runs/1791462712

I'm opening this issue as a place to dump debugging info

[jisantuc]

The problem that CI reports is that developers are missing from the POM file. There are two possibilities for how that might happen. The first is that we don't have any developer metadata.

I ran developers from sbt locally after running the github task, and saw plenty of developers. Since the github task is part of the ci-publish action, this metadata should be available at publication time.

The second possibility is that while this information is available, it's not getting packaged into the pom file for reasons. I don't believe this is the case either -- I ran a publishLocal after github (not identical to ci-release, but I don't know that I can mimic ci-release locally). I got a pom file that looks like this:

<?xml version='1.0' encoding='UTF-8'?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.scala-exercises</groupId>
    <artifactId>definitions_2.12</artifactId>
    <packaging>jar</packaging>
    <description>Exercises Compiler Plugin</description>
    <url>https://github.com/scala-exercises/sbt-exercise</url>
    <version>0.6.7+142-792921bb-SNAPSHOT</version>
    <licenses>
        <license>
            <name>Apache-2.0</name>
            <url>https://api.github.com/licenses/apache-2.0</url>
            <distribution>repo</distribution>
        </license>
    </licenses>
    <name>definitions</name>
    <inceptionYear>2020</inceptionYear>
    <organization>
        <name>47 Degrees Open Source</name>
        <url>https://www.47deg.com</url>
    </organization>
    <scm>
        <url>https://github.com/scala-exercises/sbt-exercise</url>
        <connection>scm:git:https://github.com/scala-exercises/sbt-exercise.git</connection>
        <developerConnection>scm:git:git@github.com:scala-exercises/sbt-exercise.git</developerConnection>
    </scm>
    <developers>
        <developer>
            <id>47erbot</id>
            <name>47erbot</name>
            <url>https://github.com/47erbot</url>
            <email>developer@47deg.com</email>
        </developer>
        <developer>
            <id>fedefernandez</id>
            <name>Fede Fernández</name>
            <url>https://github.com/fedefernandez</url>
        </developer>
        <developer>
            <id>jisantuc</id>
            <name>James Santucci</name>
            <url>https://github.com/jisantuc</url>
            <email>james.santucci@gmail.com</email>
        </developer>
        <developer>
            <id>juanpedromoreno</id>
            <name>Juan Pedro Moreno</name>
            <url>https://github.com/juanpedromoreno</url>
        </developer>
    </developers>
    <dependencies>
        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
            <version>2.12.15</version>
        </dependency>
        <dependency>
            <groupId>org.typelevel</groupId>
            <artifactId>cats-core_2.12</artifactId>
            <version>2.7.0</version>
        </dependency>
        <dependency>
            <groupId>org.scalatest</groupId>
            <artifactId>scalatest_2.12</artifactId>
            <version>3.2.10</version>
        </dependency>
        <dependency>
            <groupId>org.scalacheck</groupId>
            <artifactId>scalacheck_2.12</artifactId>
            <version>1.15.4</version>
        </dependency>
        <dependency>
            <groupId>com.github.alexarchambault</groupId>
            <artifactId>scalacheck-shapeless_1.15_2.12</artifactId>
            <version>1.3.0</version>
        </dependency>
    </dependencies>
</project>

I downloaded the xsd file mentioned in the schema to schema.xsd and ran xmllint --schema schema.xsd definitions_2.12.pom --noout to verify that the pom file conforms to the schema, and got

$ xmllint --schema schema.xsd definitions_2.12.pom --noout
definitions_2.12.pom validates

I did the same validation to the definition_2.13.pom file and it also validates.

So I think the note about a bad pom file is a red herring maybe?

[jisantuc]

Huh, so the snapshot artifact actually doesn't have developers 🤔 :

$ http 'https://oss.sonatype.org/content/repositories/snapshots/org/scala-exercises/definitions_2.12/0.6.7+141-4773acd9-SNAPSHOT/definitions_2.12-0.6.7+141-4773acd9-SNAPSHOT.pom' > definitions_published.pom
$ cat definitions_published.pom
<?xml version='1.0' encoding='UTF-8'?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.scala-exercises</groupId>
    <artifactId>definitions_2.12</artifactId>
    <packaging>jar</packaging>
    <description>Exercises Compiler Plugin</description>
    <url>https://github.com/scala-exercises/sbt-exercise</url>
    <version>0.6.7+141-4773acd9-SNAPSHOT</version>
    <licenses>
        <license>
            <name>Apache-2.0</name>
            <url>https://api.github.com/licenses/apache-2.0</url>
            <distribution>repo</distribution>
        </license>
    </licenses>
    <name>definitions</name>
    <inceptionYear>2020</inceptionYear>
    <organization>
        <name>47 Degrees Open Source</name>
        <url>https://www.47deg.com</url>
    </organization>
    <scm>
        <url>https://github.com/scala-exercises/sbt-exercise</url>
        <connection>scm:git:https://github.com/scala-exercises/sbt-exercise.git</connection>
        <developerConnection>scm:git:git@github.com:scala-exercises/sbt-exercise.git</developerConnection>
    </scm>
    <dependencies>
        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
            <version>2.12.15</version>
        </dependency>
        <dependency>
            <groupId>org.typelevel</groupId>
            <artifactId>cats-core_2.12</artifactId>
            <version>2.7.0</version>
        </dependency>
        <dependency>
            <groupId>org.scalatest</groupId>
            <artifactId>scalatest_2.12</artifactId>
            <version>3.2.10</version>
        </dependency>
        <dependency>
            <groupId>org.scalacheck</groupId>
            <artifactId>scalacheck_2.12</artifactId>
            <version>1.15.4</version>
        </dependency>
        <dependency>
            <groupId>com.github.alexarchambault</groupId>
            <artifactId>scalacheck-shapeless_1.15_2.12</artifactId>
            <version>1.3.0</version>
        </dependency>
    </dependencies>
</project>

But without developers it still validates 🤔🤔🤔🤔🤔🤔 I think this schema might not actually specify everything that's required, which is very cool.

[jisantuc]

Yup, developers has minOccurs=0:

<xs:element minOccurs="0" name="developers">

[jisantuc]

Contrast the pom above with the pom from the last evaluator-client snapshot release in scala-exercises-runtime, which does have developers:

<?xml version='1.0' encoding='UTF-8'?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.scala-exercises</groupId>
    <artifactId>evaluator-client_2.12</artifactId>
    <packaging>jar</packaging>
    <description>Runtime library for Scala-Exercises Server</description>
    <url>https://github.com/scala-exercises/scala-exercises-runtime</url>
    <version>0.6.4+132-ab7d5c40-SNAPSHOT</version>
    <licenses>
        <license>
            <name>Apache-2.0</name>
            <url>https://api.github.com/licenses/apache-2.0</url>
            <distribution>repo</distribution>
        </license>
    </licenses>
    <name>evaluator-client</name>
    <inceptionYear>2020</inceptionYear>
    <organization>
        <name>47 Degrees Open Source</name>
        <url>https://www.47deg.com</url>
    </organization>
    <scm>
        <url>https://github.com/scala-exercises/scala-exercises-runtime</url>
        <connection>scm:git:https://github.com/scala-exercises/scala-exercises-runtime.git</connection>
        <developerConnection>scm:git:git@github.com:scala-exercises/scala-exercises-runtime.git</developerConnection>
    </scm>
    <developers>
        <developer>
            <id>alejandrohdezma</id>
            <name>Alejandro Hernández</name>
            <url>https://github.com/alejandrohdezma</url>
            <email>info@alejandrohdezma.com</email>
        </developer>
        <developer>
            <id>rafaparadela</id>
            <name>Rafa Paradela</name>
            <url>https://github.com/rafaparadela</url>
            <email>rafa.p@47deg.com</email>
        </developer>
        <developer>
            <id>raulraja</id>
            <name>Raúl Raja Martínez</name>
            <url>https://github.com/raulraja</url>
            <email>raul@47deg.com</email>
        </developer>
    </developers>
    <dependencies>
        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
            <version>2.12.15</version>
        </dependency>
        <dependency>
            <groupId>org.http4s</groupId>
            <artifactId>http4s-blaze-client_2.12</artifactId>
            <version>0.23.8</version>
        </dependency>
        <dependency>
            <groupId>org.http4s</groupId>
            <artifactId>http4s-circe_2.12</artifactId>
            <version>0.23.8</version>
        </dependency>
        <dependency>
            <groupId>io.circe</groupId>
            <artifactId>circe-core_2.12</artifactId>
            <version>0.14.1</version>
        </dependency>
        <dependency>
            <groupId>io.circe</groupId>
            <artifactId>circe-generic_2.12</artifactId>
            <version>0.14.1</version>
        </dependency>
    </dependencies>
</project>

I think for whatever reason, while everything looks like it should be working, we're actually not writing the developers to the pom, and the error is not a red herring after all.

[jisantuc]

Oh, cool, developers isn't getting set in CI for some reason:

Right before running ci-release, developers is still an empty list for all the projects:

[info] set current project to default-878[39](https://github.com/scala-exercises/sbt-exercise/runs/5071847454?check_suite_focus=true#step:9:39)e (in build file:/home/runner/work/sbt-exercise/sbt-exercise/)
[info] definitions / developers
[info] 	List()
[info] compiler / developers
[info] 	List()
[info] sbt-exercise / developers
[info] 	List()
[info] documentation / developers
[info] 	List()
[info] developers
[info] 	List()
[info] Defining ThisBuild / githubEnabled
[info] The new value will be used by ThisBuild / collaborators, ThisBuild / contributors and 2 others.
[info] 	Run `last` for details.
[info] Reapplying settings...
[info] Retrieving `scala-exercises/sbt-exercise` information from Github API
[info] Retrieving `scala-exercises/sbt-exercise` contributors from Github API
[info] Retrieving `[47](https://github.com/scala-exercises/sbt-exercise/runs/5071847454?check_suite_focus=true#step:9:47)degrees` organization from Github API
[info] Retrieving `scala-exercises/sbt-exercise` releases from Github API
[info] Retrieving `scala-exercises/sbt-exercise` collaborators from Github API
[info] set current project to default-87839e (in build file:/home/runner/work/sbt-exercise/sbt-exercise/)
[info] definitions / developers
[info] 	List()
[info] compiler / developers
[info] 	List()
[info] sbt-exercise / developers
[info] 	List()
[info] documentation / developers
[info] 	List()
[info] developers
[info] 	List()
Running ci-release.
  branch=refs/heads/test/js/release-on-this-branch

[jisantuc]

Contrast with local, where everything is fine ™️

$ sbt 'show developers;github;show developers'                                                                                                                                                  sbt-exercise
[info] welcome to sbt 1.6.1 (Azul Systems, Inc. Java 1.8.0_292)
[info] loading global plugins from /Users/james/.sbt/1.0/plugins
[info] loading settings for project sbt-exercise-build-build-build from metals.sbt ...
[info] loading project definition from /Users/james/gitdirs/sbt-exercise/project/project/project
[info] loading settings for project sbt-exercise-build-build from metals.sbt ...
[info] loading project definition from /Users/james/gitdirs/sbt-exercise/project/project
[success] Generated .bloop/sbt-exercise-build-build.json
[success] Total time: 1 s, completed Feb 4, 2022 12:58:05 PM
[info] loading settings for project sbt-exercise-build from metals.sbt,plugins.sbt ...
[info] loading project definition from /Users/james/gitdirs/sbt-exercise/project
[success] Generated .bloop/sbt-exercise-build.json
[success] Total time: 1 s, completed Feb 4, 2022 12:58:07 PM
[info] loading settings for project sbt-exercise from build.sbt ...
[info] loading settings for project default-c10130 from build.sbt ...
[info] set current project to default-c10130 (in build file:/Users/james/gitdirs/sbt-exercise/)
[info] definitions / developers
[info]  List()
[info] compiler / developers
[info]  List()
[info] sbt-exercise / developers
[info]  List()
[info] documentation / developers
[info]  List()
[info] developers
[info]  List()
[info] Defining ThisBuild / githubEnabled
[info] The new value will be used by ThisBuild / collaborators, ThisBuild / contributors and 2 others.
[info]  Run `last` for details.
[info] Reapplying settings...
[info] Retrieving `scala-exercises/sbt-exercise` information from Github API
[info] Retrieving `scala-exercises/sbt-exercise` releases from Github API
[info] Retrieving `scala-exercises/sbt-exercise` contributors from Github API
[info] Retrieving `47degrees` organization from Github API
[info] Retrieving `scala-exercises/sbt-exercise` collaborators from Github API
[info] Retrieving `fedefernandez` information from Github API
[info] Retrieving `juanpedromoreno` information from Github API
[info] Retrieving `jisantuc` information from Github API
[info] Retrieving `47erbot` information from Github API
[info] set current project to default-c10130 (in build file:/Users/james/gitdirs/sbt-exercise/)
[info] definitions / developers
[info]  List(Developer(47erbot, 47erbot, developer@47deg.com, https://github.com/47erbot), Developer(fedefernandez, Fede Fernández, , https://github.com/fedefernandez), Developer(jisantuc, James Santucci, james.santucci@gmail.com, https://github.com/jisantuc), Developer(juanpedromoreno, Juan Pedro Moreno, , https://github.com/juanpedromoreno))
[info] compiler / developers
[info]  List(Developer(47erbot, 47erbot, developer@47deg.com, https://github.com/47erbot), Developer(fedefernandez, Fede Fernández, , https://github.com/fedefernandez), Developer(jisantuc, James Santucci, james.santucci@gmail.com, https://github.com/jisantuc), Developer(juanpedromoreno, Juan Pedro Moreno, , https://github.com/juanpedromoreno))
[info] sbt-exercise / developers
[info]  List(Developer(47erbot, 47erbot, developer@47deg.com, https://github.com/47erbot), Developer(fedefernandez, Fede Fernández, , https://github.com/fedefernandez), Developer(jisantuc, James Santucci, james.santucci@gmail.com, https://github.com/jisantuc), Developer(juanpedromoreno, Juan Pedro Moreno, , https://github.com/juanpedromoreno))
[info] documentation / developers
[info]  List(Developer(47erbot, 47erbot, developer@47deg.com, https://github.com/47erbot), Developer(fedefernandez, Fede Fernández, , https://github.com/fedefernandez), Developer(jisantuc, James Santucci, james.santucci@gmail.com, https://github.com/jisantuc), Developer(juanpedromoreno, Juan Pedro Moreno, , https://github.com/juanpedromoreno))
[info] developers
[info]  List(Developer(47erbot, 47erbot, developer@47deg.com, https://github.com/47erbot), Developer(fedefernandez, Fede Fernández, , https://github.com/fedefernandez), Developer(jisantuc, James Santucci, james.santucci@gmail.com, https://github.com/jisantuc), Developer(juanpedromoreno, Juan Pedro Moreno, , https://github.com/juanpedromoreno))