Feedback thread: customDatasources
#23286
Replies: 26 comments 52 replies
-
|
I love the feature! I've already been experimenting with this and it proves to be really useful. One thing, as far as I understand the implementation (see the remaining TODO), it expects the custom registry to return JSON document. In one of my use-cases, the API returns a plain text string that is supposed to represent latest version (GET on /custom api return body "1.0.0" or "123.123.asd-123.zxcase" or even "my-custom-versioning-scheme") In that case, parsing it fails as it's not a valid JSON (I guess because the body is unquoted so it's not treated as string literal). Is that something that can be worked around at a current state? Maybe we could have some kind of configuration option as to how to translate whatever comes from API to JSON literal? |
Beta Was this translation helpful? Give feedback.
-
|
Glad that you like it! Currently this is not possible, but should be rather easy to implement. Would you be interested to implement it, if I write up an feature request issue with the requirements? |
Beta Was this translation helpful? Give feedback.
-
|
sure thing, I think I'll be able to. We could discuss it further down the road :) |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
Idea: allow to run to external binaries to fetch data. The output is then fed into the current logic with Jsonata rules. My use case here is |
Beta Was this translation helpful? Give feedback.
-
|
We can't use aws as a library instead, in a repology like way? But for self-hosted I'm open to the idea of arbitrary CLI commands for datasource lookups |
Beta Was this translation helpful? Give feedback.
-
|
Should be possible, but I'm not sure if |
Beta Was this translation helpful? Give feedback.
-
|
for self-hosted ok, but should be disabled by default like post upgrade tasks |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
I'm new to renovate, I love it, it love the docs, but I feel there's a lot of concepts being thrown at me - I think I would really benefit from this custom data source, so some documentation more specific to my type of use case would help. I feel I just need some url generator, a json parser for the content - its a bit hard to know which of the matchers and templates are relevant. |
Beta Was this translation helpful? Give feedback.
-
|
I really like it. I'm using it to get releases of Hashicorp tools using their API and update Ansible playbooks. |
Beta Was this translation helpful? Give feedback.
-
|
@Demivan great - thank you! Much appreciated! |
Beta Was this translation helpful? Give feedback.
-
|
@SirUli PRs to extend the docs are always welcome. |
Beta Was this translation helpful? Give feedback.
-
|
@Demivan i assume your .yml file then has something along these lines? In my case, renovate detects that there is "something" in the .yml as it lists the file in the dependency dashboard but it doesn't list the dependencies. So apparently i'm doing something wrong there or simply overlook something. I also tried the following (remove "custom" from the datasource): but that didn't help. Any idea? Thanks in advance! |
Beta Was this translation helpful? Give feedback.
-
|
@SirUli With your yml format Here is my yml definition: # renovate: depName=nomad
nomad_version: 1.6.0
|
Beta Was this translation helpful? Give feedback.
-
|
Yehhhaaaawww - not it worked ;) You are right, it was a bit more picky than i anticipated. I made the double quotes to be optionally there and now it went through nicely. Also i had to use loose versioning for my usecase but that is a different story. The final setup for me i In the *.yml files of ansible: # renovate: datasource=custom.nexus_generic depName=something versioning=loose
something_version: "77"In Renovate: And in each of the versiontracker.json files the following content: [
{
"version": "77",
"filelink": "https://nexus.mycompany.com/repository/software/something-77.zip"
},
{
"version": "76",
"filelink": "https://nexus.mycompany.com/repository/software/something-76.zip"
},
]Awesome work by the renovate makers! |
Beta Was this translation helpful? Give feedback.
-
|
I used the custom datasource to extract supported kubernetes versions from Gardener and that worked great. Big thanks for adding this! Especially combining this as well with hostRules for authentication did work fully as expected. Only thing, which did cost me some time and might benefit from improving is the documentation about the Here e.g. the configuration we use to track available and supported kubernetes versions in Gardener: const config = {
hostRules: [
{
matchHost: "https://api.gardener.company.com",
token: "<gardenerToken>",
}
],
customDatasources: {
"gardener-k8s-supported": {
defaultRegistryUrlTemplate: 'https://api.gardener.company.com/apis/core.gardener.cloud/v1beta1/cloudprofiles/aws',
transformTemplates: [
`{"releases": $$.spec.kubernetes.versions[classification = 'supported'].{"version": $.version, "sourceUrl":"https://github.com/kubernetes/kubernetes", "changelogUrl":$join(["https://github.com/kubernetes/kubernetes/releases/tag/",$.version])},"sourceUrl": "https://github.com/kubernetes/kubernetes","homepage": "https://github.com/kubernetes/kubernetes"}`
],
},
"gardener-k8s-available": {
defaultRegistryUrlTemplate: 'https://api.gardener.company.com/apis/core.gardener.cloud/v1beta1/cloudprofiles/aws',
transformTemplates: [
`{"releases": $$.spec.kubernetes.versions.{"version": $.version, "sourceUrl":"https://github.com/kubernetes/kubernetes", "changelogUrl":$join(["https://github.com/kubernetes/kubernetes/releases/tag/",$.version])},"sourceUrl": "https://github.com/kubernetes/kubernetes","homepage": "https://github.com/kubernetes/kubernetes"}`
],
},
}
} |
Beta Was this translation helpful? Give feedback.
-
|
Great feature. I also love that it works with host rules which does not seem to be documented. That enables i.e. fetching data from the gitlab api. |
Beta Was this translation helpful? Give feedback.
-
|
I have copy paste the the same customdatasource between the configs with no success. |
Beta Was this translation helpful? Give feedback.
-
|
Can you share a reproduction repo? |
Beta Was this translation helpful? Give feedback.
-
|
I indeed havent escaped my transformTemplate properly. Thank you very much for giving me the hint. |
Beta Was this translation helpful? Give feedback.
-
|
Not sure why you get this message. A reproduction repo would be helpful here. |
Beta Was this translation helpful? Give feedback.
-
|
I have created very basic example since i cannot share the private repository. When I use a transformTemplate that does not match I get the following error message: |
Beta Was this translation helpful? Give feedback.
-
|
This feature is useful for me to lock aws-for-fluent-bit to stable. https://github.com/aws/aws-for-fluent-bit#guidance-on-consuming-versions
{
"packageRules": [
{
"matchDepNames": ["public.ecr.aws/aws-observability/aws-for-fluent-bit"],
"matchDatasources": ["docker"],
"matchManagers": ["helm-values"],
"enabled": false
}
],
"regexManagers": [
{
"customType": "regex",
"fileMatch": [
"values\\.yaml"
],
"matchStrings": [
"public\\.ecr\\.aws\\/aws-observability\\/aws-for-fluent-bit:(?<currentValue>.*)"
],
"datasourceTemplate": "custom.aws-for-fluent-bit-stable",
"depNameTemplate": "public.ecr.aws/aws-observability/aws-for-fluent-bit"
}
],
"customDatasources": {
"aws-for-fluent-bit-stable": {
"defaultRegistryUrlTemplate": "https://raw.githubusercontent.com/aws/aws-for-fluent-bit/mainline/AWS_FOR_FLUENT_BIT_STABLE_VERSION",
"format": "plain"
}
}
} |
Beta Was this translation helpful? Give feedback.
-
|
Is there a way to debug your datasources? I tried pulling in CoreOS updates via their stream JSON APIs. I validated the transformers against the data in https://try.jsonata.org/, but still Renovate insists that the currently used version is the latest one. "regexManagers": [
{
"fileMatch": [
"template-x86_64.pkr.hcl"
],
"matchStrings": [
"coreos_release = \"(?<currentValue>\\S+)\""
],
"depNameTemplate": "coreos",
"datasourceTemplate": "custom.coreos-stable"
}
],
"customDatasources": {
"coreos-stable": {
"defaultRegistryUrlTemplate": "https://builds.coreos.fedoraproject.org/streams/stable.json",
"transformTemplates": [
"{\"releases\": [{\"version\": $$.(architectures.x86_64.artifacts.metal.release),\"changelogUrl\": \"https://fedoraproject.org/coreos/release-notes/?arch=x86_64&stream=stable\"}]}"
]
}
} |
Beta Was this translation helpful? Give feedback.
-
|
The DEBUG logs confirm that the manager is picking up the dependency, and correctly prints its current version. The repository linked above reproduces the issue, but I recreated it in a separate repo: https://github.com/mhutter/renovate-sandbox |
Beta Was this translation helpful? Give feedback.
-
|
Transformer Expression: https://try.jsonata.org/Gkb9PSPeB |
Beta Was this translation helpful? Give feedback.
-
|
Reproduced: https://github.com/renovate-reproductions/23286-1 It is not a customDatasource issue. |
Beta Was this translation helpful? Give feedback.
-
|
Aha! Would be interesting to have this in the debug logs though. As it DOES output the current version, I just assumed that this was not the issue. Thanks to your input the fix was straight-forward: I love the feature, it is just what I needed right now 👍 |
Beta Was this translation helpful? Give feedback.
-
|
Glad that you like it. A debug log for this scenarios has been added with this PR: #24514 |
Beta Was this translation helpful? Give feedback.
-
|
Very glad for this feature! Thank you! |
Beta Was this translation helpful? Give feedback.
-
|
Yes, it should work the same as with other datasources. The best way to understand your problem would be a new discussion with a reproduction repo. |
Beta Was this translation helpful? Give feedback.
-
|
Not sure if related, but I noticed our custom datasources breaking since fadf1f2, which added e.g. to the renovate/lib/util/http/auth.ts Lines 31 to 36 in 69f7240
Edit: Created #24671 as follow up |
Beta Was this translation helpful? Give feedback.
-
|
I've returned and managed to get a lot further. A few things on my wish list:
To give an example, my latest struggle: All I had was: "Failed to look up custom.X package Y". |
Beta Was this translation helpful? Give feedback.
-
|
Config to update an (ivy) dependency, using the artifactory API, hope it helps someone |
Beta Was this translation helpful? Give feedback.
-
|
Regarding docs, you can find them for the custom manager here: https://github.com/renovatebot/renovate/tree/main/lib/modules/datasource/custom Currently we have only special logging in the custom datasource for schema issues. If you can give us some specific what information would have helped you, we can consider it.
|
Beta Was this translation helpful? Give feedback.
-
An Odyssey: Trying To Use Custom Datasources To Query APT-RepositoriesSo, we have two repositories on GitLab:
Renovate works perfectly fine for a large portion of our dependencies. We can update Docker image tags in Dockerfiles, Gitlab CI files, and Helm values.yml files. We can update Helm chart versions, Pipenv dependencies, and Terraform and Terragrunt dependencies. Great experience all around. However, we also had to setup a {
customType: "regex",
fileMatch: [
"(^|/|\\.)([Dd]ocker|[Cc]ontainer)file$",
"(^|/)([Dd]ocker|[Cc]ontainer)file[^/]*$"
],
matchStrings: [
"#[ \\t]+Renovate:[ \\t]+datasource=(?<datasource>[.-_a-z]+?)[ \\t]+depName=(?<depName>\\S+?)(?:[ \\t]+(lookupName|packageName)=(?<packageName>\\S+?))?(?:[ \\t]+versioning=(?<versioning>\\S+?|regex:\\/.*?\\/))?(?:[ \\t]+extractVersion=(?<extractVersion>\\S+?))?(?:[ \\t]+registryUrl=(?<registryUrl>\\S+?))?(\\r|\\n|\\r\\n)(?:ENV|ARG)[ \\t]+.+?_VERSION[ =]\"?(?<currentValue>\\S+?)\""
],
versioningTemplate: "{{#if versioning}}{{{versioning}}}{{else}}semver{{/if}}"
}This is basically the example from the docs with some adjustments. I.e., it allows for something like # Renovate: datasource=repology depName=ubuntu_22_04/git
ENV GIT_VERSION="2.34.1"in our Dockerfile. Again, where we can use built-in datasources, this works fine. Tools from custom APT repositories - Approach 1Here's the issue: We build a Docker image that contains tools which are installed from custom APT repositories (e.g., So my first workaround was this: Renovate can use GitHub releases and Git tags as a datasource. All the tools we install are available from accessible repositories so we can examine them and use the release names / Git tags as versions. This didn't work because the names of the releases / tags don't correspond directly to the version notation used in the APT repositories. We tried to map versions (which worked but was very messy) but then you run into the issue that versions published on GitHub take a while until they show up in the APT repositories. So Repology would propose an updated Dockerfile which then couldn't be built because the version wasn't present in the APT repository.
Tools from custom APT repositories - Approach 2So, since Renovate doesn't have such a datasource, I tried using the Right, so my approach is this:
Here come the issues with this approach: The only template variable that we have access to in # Renovate: datasource=custom.local_apt_versions_file depName=google-cloud packageName=https://packages.cloud.google.com/apt/cloud-sdk/main/google-cloud-cli versioning=deb
ENV GCLOUD_VERSION="462.0.1-0"Of course, the
Now, Renovate and But lets run with it, everything after the
So I put my files into the base folder, I verified that the file URL was correct given how it was handled, ran Renovate and... nothing. The files were gone. Well, turns out that Renovate will
Tools from custom APT repositories - Approach 3So we can't use Of course, accessing these artifacts requires an authentication header. Renovate already has a {
matchHost: `${process.env.RENOVATE_ENDPOINT}/projects/${process.env.PROJECT_ID}/jobs/${process.env.APT_VERSIONS_JOB_ID}/artifacts/${process.env.APT_VERSIONS_FOLDER}`,
token: `${process.env.TOKEN}`,
authType: "Bearer"
}I run Renovate and... nothing. I get a 404 for the URL which in this case indicates an invalid authentication header. If the header had the correct format but the token was wrong, I would get 401. By the way, the trace log shows "Adding Bearer authentication for URL" but this is not done.
Finally, I tried to work around this issue by adding the authentication header myself. I added allowedHeaders: [
"Authorization",
"X-*"
]and to the headers: {
Authorization: `Bearer ${process.env.TOKEN}`
}This workaround also doesn't work. I don't know why, I stopped investigating due to time constraints at this point. I think it's because the repository we are working with is also on GitLab and Renovate internally adds hostRules to handle authentication. The
Conclusion / TLDRI'd appreciate if you would take the time to look at this @secustor and tell me where / if I went wrong here. For now, we can't use Renovate to update APT backed dependencies. I think my approach should be doable but I am blocked by the issues that I described. My proposals for remediation are quoted. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the walk through and sry for coming back this late. Happy to review a PR for such a datasource. For such usecases I can see adding a Forwarding all capture groups is from an architectural point not so desired as managers and datasources are only lightly coupled. The custom datasource is currently gated to files inside of the same repository and references are always in reference to that root. Create a separate discussion with a reproduction for this, tough the OTEL integration may help you debug this. |
Beta Was this translation helpful? Give feedback.
-
|
Hi there, Please do not unnecessarily For example, never It's OK to comment in an issue or discussion after multiple days or weeks. But please, still don't Thanks, the Renovate team |
Beta Was this translation helpful? Give feedback.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
-
|
Found an additional interesting use case for k3s https://github.com/visualon/renovate-config/pull/10/files |
Beta Was this translation helpful? Give feedback.
-
|
Custom datasources have been very helpful to us. Some general remarks:
Some of our usecases:
|
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the feature, it is great and easy to use but hard to debug : while I did success with some HTML page, I am trying to process release at sqlite (https://sqlite.org/download.html) using a customDatasources with format=html. That's would be cool if we could:
Here is the JSON I am "expecting" and the "result" I am trying to feed renovate: https://try.jsonata.org/vyDcg2vEt In this case, Renovate fails to update the version and it is hard to determine the culprit here. I also tried github-tags / releases as source and got the same kind of result for other dependency, such as libexpat. |
Beta Was this translation helpful? Give feedback.
-
This is possible, though probably only for self-hosted instances as this would require additional API calls and we have household carefully to invest those for the managed App.
On trace log level we write out the whole data block if the validation failes. You can use
That playground link has no input data for me. |
Beta Was this translation helpful? Give feedback.
-
I'll retry with new line because https://json5.org/ says that Strings may span multiple lines by escaping new line characters. I did try with new line character, but not with
I'll give it a try tomorrow, thanks 👍
In case it fails again (I did try in private mode, ...); the JSON I am expecting is something like this: {
"releases": [
{ "version": "https://sqlite.org/2024/sqlite-amalgamation-3460000.zip" },
{ "version": "https://sqlite.org/2024/sqlite-autoconf-3460000.tar.gz" },
...
]
}The JSON data is only the result of executing this javascript in developer console on https://sqlite.org/download.html (I removed links not referencing archive). console.log(JSON.stringify({releases: Array.from(document.querySelectorAll('a[href]'))
.map(anchor => anchor.href)
.filter(href => href.startsWith('http://') || href.startsWith('https://'))
.filter(href => href.match(/\.(zip|7z|tar\.(gz|bz|bz2))$/i ))
.map(href => ({version: href})) })) |
Beta Was this translation helpful? Give feedback.
-
|
Ok, no wonder it never worked: the page is using javascript to generate the links whereas a HTML comment contains the version in CSV. I'll have to resort to plain text + jsonata. |
Beta Was this translation helpful? Give feedback.
-
|
They work, but there are limitations and some UX problems. Here is what I've come up with for managing versioned files in a Nexus raw repo, which implies URLs of the form: Let's assume for the sake of simplicity running numbers for the versions; should work with anything we can write a regexp for. {
packageRules: [
{
matchDatasources: [
"custom.nexus-raw",
],
"extractVersion": "-(?<version>[0-9]+)\\.[a-z0-9.]*",
},
],
customDatasources: {
"nexus-raw": {
defaultRegistryUrlTemplate: "nexus.some.org/service/rest/repository/browse/{{ packageName }}/",
format: "html",
},
},
customManagers: [
{
customType: "regex",
fileMatch: [
"^Jenkinsfile", // just as an example
],
matchStrings: [
"nexus\\.some\\.org\\/repository\\/(?<packageName>.+?)\\/[-a-z]+-(?<currentValue>[0-9]+)\\.",
],
datasourceTemplate: "custom.nexus-raw",
versioningTemplate: "regex:^(?<patch>[0-9]+)$", // optional
}
],
}For reference, what we get from the datasource is basically a HTML file listing. Example<!DOCTYPE html>
<html lang="en">
<head>
<title>Index of /some-folder</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<link rel="icon" type="image/png" href="../../../../../../static/rapture/resources/safari-favicon-32x32.png?3.73.0-12" sizes="32x32">
<link rel="mask-icon" href="../../../../../../static/rapture/resources/favicon-white.svg?3.73.0-12" color="#00bb6c">
<link rel="icon" type="image/png" href="../../../../../../static/rapture/resources/favicon.svg?3.73.0-12" sizes="16x16">
<link rel="stylesheet" type="text/css" href="../../../../../../static/css/nexus-content.css?3.73.0-12"/>
</head>
<body class="htmlIndex">
<h1>Index of /some-folder</h1>
<table cellspacing="10">
<tr>
<th align="left">Name</th>
<th>Last Modified</th>
<th>Size</th>
<th>Description</th>
</tr>
<tr>
<td><a href="../">Parent Directory</a></td>
</tr>
<tr>
<td><a href="https://nexus.some.org/repository/my-repo/some-folder/my-artifact-123.zip">my-artifact-123.zip</a></td>
<td>
Wed Oct 30 11:15:03 Z 2024
</td>
<td align="right">
6653
</td>
<td></td>
</tr>
<tr>
<td><a href="https://nexus.some.org/repository/my-repo/some-folder/my-artifact-124.zip">my-artifact-124.zip</a></td>
<td>
Wed Oct 30 12:24:24 Z 2024
</td>
<td align="right">
6653
</td>
<td></td>
</tr>
<tr>
<td><a href="https://nexus.some.org/repository/my-repo/some-folder/my-artifact.zip">my-artifact.zip</a></td>
<td>
Wed Oct 30 12:03:20 Z 2024
</td>
<td align="right">
6653
</td>
<td></td>
</tr>
</table>
</body>
</html>The good
The bad
The ugly / confusing
SuggestionAdd |
Beta Was this translation helpful? Give feedback.
-
|
You don't need three configs. You can use |
Beta Was this translation helpful? Give feedback.
-
|
Ah, thanks for clearing that up! I was confused by Unfortunately, debugging and the (I also keep being turned around by how some patterns ( |
Beta Was this translation helpful? Give feedback.
-
|
For future reference, it's now: {
customDatasources: {
"nexus-raw": {
defaultRegistryUrlTemplate: "nexus.some.org/service/rest/repository/browse/{{ packageName }}/",
format: "html",
},
},
customManagers: [
{
customType: "regex",
fileMatch: [
"^Jenkinsfile", // just as an example
],
matchStrings: [
"nexus\\.some\\.org\\/repository\\/(?<packageName>.+?)\\/(?<fileNameBase>[-a-z]+)-(?<currentValue>[0-9]+)(?<fileEnding>\\.[a-z0-9.]*)",
],
datasourceTemplate: "custom.nexus-raw",
versioningTemplate: "regex:^(?<patch>[0-9]+)$", // optional
extractVersionTemplate: "{{{ packageName }}}/{{{ fileNameBase }}}-(?<version>[0-9]+){{{ fileEnding }}}",
},
],
} |
Beta Was this translation helpful? Give feedback.
-
|
I did have some time today to play around with custom datasources in renovate and could get it working with the Espressif ESP Component Registry. Great stuff! In case someone is interested, please find the config below. It's not yet perfect, but a good starting point. {
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"customManagers": [
{
"customType": "regex",
"fileMatch": ["idf_component.yml$"],
"datasourceTemplate": "custom.espressif",
"matchStrings": [
"(?<depName>[^\\s:]+):\\s*version:\\s*\"(?<currentValue>[^\"]+)\""
],
"versioningTemplate": "npm"
}
],
"customDatasources": {
"espressif": {
"defaultRegistryUrlTemplate": "https://components-file.espressif.com/components/{{packageName}}.json",
"transformTemplates": [
"{ \"releases\": [ versions.{ \"version\": version, \"releaseTimestamp\": created_at } ], \"homepage\": versions[0].homepage }"
]
}
}
} |
Beta Was this translation helpful? Give feedback.
-
Pagination Handling possible?This is a great feature and we've already started integrating it in one of our projects. One question remains: Can Renovate handle page pagination when accessing a As an example, when accessing Nexus' REST-API via As to Sonatype's documentation, this behavior cannot be disabled. So the question remains, if Renovate can handle this? |
Beta Was this translation helpful? Give feedback.
-
|
With this request we're determining all the release versions of a CLI program we're developing ourselves and which is hosted on our internal Nexus. We need this program in one of our Jenkins libraries and want to ensure to have always the latest version there. Unfortunately we don't have a single file hosting all releases of the CLI program so we tried to read it directly from the Nexus API which now causes the pagination problems. But maybe that's the solution: Create such a file hosting all releases and provide it for Renovate. Have to think about it. Any other ideas welcome! |
Beta Was this translation helpful? Give feedback.
-
|
My initial thoughts:
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks, @secustor ! |
Beta Was this translation helpful? Give feedback.
-
FWIW, I'd also recommend using a data source matching the repository type on Nexus (e.g.
That actually sounds like reasonable workaround. 🤔 Essentially, you'd extend a raw repo into your own registry format, adding an index file. If it's worth the redundant data and coupling with the release process. 💁 PS: 👋 😃 |
Beta Was this translation helpful? Give feedback.
-
|
Would be super nice for my application querying the Azure API. Now I can only query 100 Template Spec versions of a module (top=100 is maximum): |
Beta Was this translation helpful? Give feedback.
-
|
A risk I see in regards to custom datasources, is that they do provide shortcuts towards exploits. If Renovate ships with less-than-perfect managers, for example with arbitrary code execution vulnerabilities, the custom datasource make it a lot simpler to trick Renovate into believing there's a new version. This allows adversaries to trigger Renovate to act from repos under their control, without the need to jeopardize the real package server. |
Beta Was this translation helpful? Give feedback.
-
|
Hi is stuck to configure my renovate config to fetch Godot prerelease. I try to use The ordering looks correct. But my renovate.json do just want to update to dev5? Testing results into
Simplified versionAny help is welcome. |
Beta Was this translation helpful? Give feedback.
-
|
The order does not matter in the context of the In the worst case you have to use a |
Beta Was this translation helpful? Give feedback.
-
|
But why it is it not working? |
Beta Was this translation helpful? Give feedback.
-
|
As said above the order in which releases are returned by the datasource has not effect on the version chosen to update to. The datasource only provides the available versions without making a decision which is actually newer or Renovate should upgrade to. The code which does this is the https://docs.renovatebot.com/key-concepts/how-renovate-works/ |
Beta Was this translation helpful? Give feedback.
-
|
But why than it is not matches the latest version to So looks like the update[] is showing the latest prerelease alphabetic sorted, that was the reason to use the custom sorted datasource. |
Beta Was this translation helpful? Give feedback.
-
|
As per many of the previous comments, this is a really cool feature, with lots of powerful use cases! I had a query over the use of the use packageName/depName value in customDatasources transformTemplates. I raised a new discussion for it: #37514 but perhaps it would have been best to add here? TL/DR: Can I use the packageName or depName variable in the transformTemplates rule? |
Beta Was this translation helpful? Give feedback.
-
|
This is documented in the data source docs
|
Beta Was this translation helpful? Give feedback.
-
Answering here to not create unnecessary threads: This is covered by our tests. If you run into this than please provide a minimal reproduction repo. renovate/lib/modules/datasource/custom/index.spec.ts Lines 611 to 646 in 7cf05ac |
Beta Was this translation helpful? Give feedback.
-
|
Hi @secustor, I referenced that docs page in #37514 - but my testing has not been able to get them populated. I can see they are being populated in defaultRegistryUrlTemplate e.g. : "{{ packageName }}.json" But it's not being populated in transformTemplates e.g.: I've tried {{ packageName }} and $packageName too but no variation is being populated. What am I doing wrong? |
Beta Was this translation helpful? Give feedback.
-
|
Just spent the last couple of days wrestling with this feature and I like it. One question I'm not too sure about, if I have a datasource currently defined as {
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"description": "renovate XXX datasource",
"customDatasources": {
"XXX": {
"defaultRegistryUrlTemplate": "file://datasources/XXX/{{packageName}}.json"
}
}
}How can you differentiate between the package architecture, e.g.: {
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"description": "renovate XXX datasource",
"customDatasources": {
"XXX": {
"defaultRegistryUrlTemplate": "file://datasources/XXX/{{architecure}}/{{packageName}}.json"
}
}
}There doesn't seem to be anything in https://docs.renovatebot.com/modules/datasource/custom/ that covers this. Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
I see following options:
|
Beta Was this translation helpful? Give feedback.
and others
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Tell us more.
Tell us how you like the new
customDatasourcesexperimental feature. What works for you, what doesn't work for you? Do you see anything we could improve?Let us know what you think! 😉
Beta Was this translation helpful? Give feedback.
All reactions