Drop P2 ant tasks

[laeubi]

I just noticed that p2 build and ship some ant tasks in director.app for example (just search for jars.extra.classpath in p2).

I wonder if any of these are still useful today or if we can drop them to reduce the dependency / technology stack of p2?

org.eclipse.equinox.p2.director.app.ant.DirectorTask
org.eclipse.equinox.internal.p2.jarprocessor.ant.AntBasedProcessorExecutor
org.eclipse.equinox.internal.p2.jarprocessor.ant.AntSignCommand
org.eclipse.equinox.internal.p2.jarprocessor.ant.JarProcessorTask
org.eclipse.equinox.internal.p2.publisher.ant.FeaturesAndBundlesPublisherTask
org.eclipse.equinox.internal.p2.publisher.ant.ProductPublisherTask
org.eclipse.equinox.internal.p2.publisher.ant.PublisherTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.CompositeRepositoryTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.CreateCompositeArtifactRepositoryTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.MirrorArtifactsTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.MirrorMetadataTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.MirrorTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.ProcessRepoTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.RemoveIUTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.Repo2RunnableTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.ValidateTask

[akurtakov]

While I don't use any of the p2 tasks nor see usage of them it would be nice to ask people to vote here at cross-project-issue-dev so possible missing functionality in Tycho stack could be identified. If one votes for keeping it and is not actively helping in maintenance he/she should provide a viable reason to do so.

[merks]

Is that what's being used here?

[cdietrich]

how can one find out if they are affected?

[ewillink]

I don't think that works. Your more direct use of p2.composite.repository seems the same as my less direct use via /shared/modeling/tools/promotion/manage-composite.xml. But this is not sensibly done in Tycho since the Tycho/Maven build doesn't run on projects-storage.eclipse.org. Now that SSH access has gone I have to use a separate promoter job that does the equivalent of

ssh genie.ocl@projects-storage.eclipse.org cd /home/data/httpd/download.eclipse.org/modeling/mdt/ocl/updates/nightly ; export JAVA_HOME=/shared/common/jdk1.8.0_x64-latest ; java -version ; /shared/common/apache-ant-latest/bin/ant -f /shared/modeling/tools/promotion/manage-composite.xml remove -Dchild.repository=6.12.0

A new P2 tool, especially one with a web-interface accessed on the standard '404' EF browser page could be wonderful.

[LorenzoBettini]

We use the Ant tasks (https://github.com/eclipse/xtext/blob/main/org.eclipse.xtext.p2repository/packaging-p2composite.ant) to build the composite update site; we run those tasks through the tycho-eclipserun-plugin (https://github.com/eclipse/xtext/blob/main/org.eclipse.xtext.p2repository/pom.xml#L210). To the best of my knowledge, those p2 ant tasks are the only way to manage composite update sites (besides manually editing XML files, of course). I don't think there's anything in Tycho with that respect.

A few months ago, I started to work on a Maven plugin for managing p2 composite sites: https://github.com/LorenzoBettini/p2composite-maven-plugin.

It's functional and tested, but I still need to release it. I was also planning to propose it to be part of Tycho.

[laeubi]

A few months ago, I started to work on a Maven plugin for managing p2 composite sites: https://github.com/LorenzoBettini/p2composite-maven-plugin.
It's functional and tested, but I still need to release it. I was also planning to propose it to be part of Tycho.

That would be great, such tasks should not require to use antrun or similar...

[mickaelistria]

Actually, such "infra" work that is not really artifact-first should even not require Tycho. IMO those Ant task would fit best as Eclipse applications so they could be automated in bash; and Tycho or other could relatively easily consume them.

[HannesWell]

To the best of my knowledge, those p2 ant tasks are the only way to manage composite update sites (besides manually editing XML files, of course). I don't think there's anything in Tycho with that respect.

I wanted to mention the same. IIRC RemoveIUTask is also used somewhere.
As long as there is no Maven/Tycho or application way to manipulate Composite-Repos or in genera more advanced ways to e.g. filter repo content like it would e.g. have been required in eclipse-m2e/m2e-core#1592 I think the ANT-tasks have to stay.
Nevertheless I would appreciate if there were the mentioned alternatives, that are also used in the Eclipse Releng work.

[iloveeclipse]

p2 build and ship some ant tasks

Please provide a summary of tasks you are talking about. It is really the minimal requirement before creating a request to remove them.

[laeubi]

As mentioned any task with p2 in its name...

This is the list I can find by looking at p2 projects

org.eclipse.equinox.p2.director.app.ant.DirectorTask
org.eclipse.equinox.internal.p2.jarprocessor.ant.AntBasedProcessorExecutor
org.eclipse.equinox.internal.p2.jarprocessor.ant.AntSignCommand
org.eclipse.equinox.internal.p2.jarprocessor.ant.JarProcessorTask
org.eclipse.equinox.internal.p2.publisher.ant.FeaturesAndBundlesPublisherTask
org.eclipse.equinox.internal.p2.publisher.ant.ProductPublisherTask
org.eclipse.equinox.internal.p2.publisher.ant.PublisherTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.CompositeRepositoryTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.CreateCompositeArtifactRepositoryTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.MirrorArtifactsTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.MirrorMetadataTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.MirrorTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.ProcessRepoTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.RemoveIUTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.Repo2RunnableTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.ValidateTask

[ewillink]

In principle I agree. EMF, OCL and QVTo continue to provide some Ant tasks and it often seems that it takes more developer time to maintain them than it saves in misguided user time. If platform deprecates Ant then we can simplify.

But you are wrong to write that Tycho replaces Ant. You ignore the 'External Tools Configuration' run that has many legacy scripts. However IMHO these are much better once replaced by MWE2, so just revise your comment to write that Ant is replaced by Tycho or MWE2.

BUT in 10 years time is it more likely that Ant or Tycho will still be working? Buckminster didn't last long; I suspect Tycho will fade way quite soon too. An Eclipse without Ant seems misguided; of course work on integrating the Ant taskpath / debugging is long overdue.

[mickaelistria]

If you search <p2. in all *.xml files; you'll see a lot of usage of those tasks in various scripts used by Platform. So as long as there are users of those tasks, particularly in Platform, it's too early to consider removing them.

[laeubi]

As said any legacy stuff can simply use the ones as deployed with the current and previous or any ancient releases.... some of these scripts even look outdated as the mention pack and md5 what is already no longer supported by p2 anyways

[mickaelistria]

As said any legacy stuff can simply use the ones as deployed with the current and previous or any ancient releases....

That's too dangerous to use unmaintained bits in the release engineering process. What if one of those tasks is discovered to have a major CVE and we've dropped code? Then we can't just fix it and getting rid of the CVE would require to rework the whole existing scripts to something more modern; which would then become a lot of trouble.
As long as those are used, we must provide an easy maintenance path. Keeping the code in current master does that.

[laeubi]

As long as those are used, we must provide an easy maintenance path. Keeping the code in current master does that.

It does not, as seen with recent PDE changes, where no one really feels comfortable to maintain/fix this stuff, so without anyone being dedicated to actually maintain that code it is effectively dead and risk of preventing further improvements and maintenance...

[akurtakov]

For reference, usages I've found are:
cje-production/cleaners/cleanupNightlyRepo.sh: echo -e " <p2.composite.repository>" >>$antBuildFile
cje-production/scripts/addToComposite.xml: <p2.composite.repository>
cje-production/scripts/removeFromComposite.xml: <p2.composite.repository>
cje-production/scripts/updateGenericComposites.xml: <p2.composite.repository>
eclipse.platform.releng.tychoeclipsebuilder/eclipse/buildScripts/process-artifacts.xml: <p2.process.artifacts
eclipse.platform.releng.tychoeclipsebuilder/eclipse/buildScripts/process-artifacts.xml: <p2.remove.iu>
eclipse.platform.releng.tychoeclipsebuilder/eclipse/buildScripts/process-artifacts.xml: <p2.mirror
eclipse.platform.releng.tychoeclipsebuilder/equinox/buildConfigs/equinox-launchers/build.xml: <p2.mirror source="file://${buildRepo}">
eclipse.platform.releng.tychoeclipsebuilder/equinox/buildConfigs/equinox-launchers/build.xml: <p2.repo2runnable destination="${featureTemp2}/eclipse">
eclipse.platform.releng.tychoeclipsebuilder/repos/build.xml: <p2.mirror source="file://${buildRepo}">
eclipse.platform.releng.tychoeclipsebuilder/repos/build.xml: <p2.remove.iu>
eclipse.platform.releng.tychoeclipsebuilder/repos/build.xml: <p2.mirror source="file://${buildRepo}">
eclipse.platform.releng.tychoeclipsebuilder/repos/build.xml: <p2.remove.iu>
eclipse.platform.releng.tychoeclipsebuilder/repos/platformrepo.xml: <p2.mirror source="file://${buildRepo}">
eclipse.platform.releng.tychoeclipsebuilder/repos/platformrepo.xml: <p2.remove.iu>
eclipse.platform.releng.tychoeclipsebuilder/repos/platformrepo.xml: <p2.mirror source="file://${buildRepo}">
eclipse.platform.releng.tychoeclipsebuilder/repos/platformrepo.xml: <p2.remove.iu>
eclipse.platform.releng/bundles/org.eclipse.releng.tests/test.xml: <p2.mirror
eclipse.platform.releng/bundles/org.eclipse.releng.tests/test.xml: <p2.repo2runnable destination="${locationcompare}">
eclipse.platform.releng/bundles/org.eclipse.releng.tests/test.xml: <p2.mirror
eclipse.platform.releng/bundles/org.eclipse.releng.tests/test.xml: <p2.repo2runnable destination="${locationcomparecurrent}">

[ewillink]

/shared/modeling/tools/promotion/manage-composite.xml is the technology I continue to use via a 'shell' job to maintain composite release repos. Jenkins 'promoter' jobs also ujse it. It uses

<target name="-add" depends="-add.named" unless="composite.name"> <p2.composite.repository> <repository location="${user.dir}" /> <add> <repository location="${child.repository}" /> </add> </p2.composite.repository> </target>

so until the EF provides a web-access P2 maintainance tool or relents on the removal of ssh access, I suspect that p2.composite.repository is required.

[jonahgraham]

I don't have strong opinion on the removal (although the general sense reading these comments is that there are still many uses of it).

However I assume that removing this would follow the normal removal guidelines, i.e. deprecated & announced and wait two years before actual removal?

[laeubi]

However I assume that removing this would follow the normal removal guidelines, i.e. deprecated & announced and wait two years before actual removal?

I think so yes, even if it is not really an "API", one could consider this as user visible interface also the only reference I found is here: https://wiki.eclipse.org/Equinox/p2/Ant_Tasks what claims to be "maybe outdated" (as of 2009 (!) )....

[cdietrich]

how do other project manage their composite update sites?!?

[merks]

Like Xpect:

https://github.com/eclipse/Xpect/blob/0343856120f1cff90b55ede16d2c0f0debb2676d/org.eclipse.xpect.releng/promotion/pom.xml#L27-L111

And DataTools:

https://github.com/eclipse-datatools/datatools/blob/22b1b935815888c5092aefc202cea28bfb18ac8f/promotion/pom.xml#L20-L108

And EMF:

https://github.com/eclipse-emf/org.eclipse.emf/blob/master/releng/org.eclipse.emf.parent/promotion/pom.xml

And so on...

[cdietrich]

thx have doubts i can work on this any time soon. maybe @LorenzoBettini can

[LorenzoBettini]

@cdietrich I think we can stay with the current solution since those Ant tasks will not be removed soon (that is what I understand from other comments). Moreover, for those specific tasks, one could still use the versions from older p2 repositories (we just need to point to an old Eclipse repository in the POM). Thus, we're definitely safe ;)

However, I'll keep the above links as references for the future.

But, if we had to go with an alternative, I'd use my Maven plugin (not yet published), which might be easier to use.