Skip to content

Commit 30bf0c5

Browse files
chrisseatonchrisseaton
committed
New system for setting versions centrally
1 parent 51db1d1 commit 30bf0c5

File tree

10 files changed

+180
-102
lines changed

10 files changed

+180
-102
lines changed

‎doc/contributor/stdlib.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,8 @@ originals. At the moment, we're just storing sources in our repository and
123123
shipping the full set as MRI does, and we'll tackle issue when they come in
124124
future versions of Ruby.
125125

126+
Keep these versions in sync with `versions.json`.
127+
126128
### Default gems
127129

128130
```

‎doc/contributor/updating-ruby.md

Lines changed: 30 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -8,59 +8,46 @@ have to MRI source code while updating.
88
You can re-run these instructions at any time to compare against unmodified
99
MRI files.
1010

11-
## Create a branch for the unmodified files from MRI
11+
## Create reference branches
1212

13-
```
14-
git checkout -b mri
15-
```
13+
For both the current version of Ruby you're using, and the new version, create
14+
reference branches that include unmodified MRI sources.
1615

17-
## Import new files from MRI
16+
Check out the version of Ruby you want to create the branch for in `../ruby`.
1817

19-
Then we want to create a new commit with the new, unmodified files imported from
20-
MRI. You need MRI checked out into `../ruby` at the version you want to import.
21-
Then run `tool/import-mri-files.sh`.
22-
23-
## Create a commit with the unmodified files
18+
Then create the reference branch in the TruffleRuby repository
2419

2520
```
26-
git commit -am 'Unmodified n.n.n files'
21+
$ git checkout -b vNN
22+
$ tool/import-mri-files.sh
23+
$ git commit -am 'vNN'
2724
```
2825

29-
## View modifications
30-
31-
You can then view the modifications that we've made against MRI.
32-
33-
```
34-
git diff mri master
35-
```
36-
37-
You should get a diff of a few thousand lines.
26+
You can then compare between these two branches and yours. For example to see
27+
what changes you made on top of the old version, what's changed between the
28+
old version and the new version, and so on. Keep them around while you do the
29+
update.
3830

3931
## Update MRI with modifications
4032

41-
You can apply these modifications on top of files from a new version of MRI.
33+
In your working branch you can import MRI files again, and you can re-apply
34+
old patches using the old reference branch.
4235

4336
```
44-
git checkout -b update-mri master
4537
tool/import-mri-files.sh
46-
git diff mri master | git apply -3
38+
git diff vNN this-branch | git apply -3
4739
```
4840

41+
You'll usually get some conflicts to work out.
42+
4943
## Make other changes
5044

45+
* Update `versions.json`
5146
* Copy and paste `-h` and `--help` output to `RubyLauncher`
52-
* Copy and paste the TruffleRuby `--help` output to `doc/user/options.md`.
53-
* Update version information (version, base version, and revision) in `TruffleRuby`
47+
* Copy and paste the TruffleRuby `--help` output to `doc/user/options.md`
5448
* Update `doc/user/compatibility.md`
5549
* Update `doc/legal/legal.md`
5650
* Update `doc/contributor/stdlib.md`
57-
* Search for other instances of the old version number (there are a
58-
couple in tests). The version numbers may use `.` or `_` depending
59-
on context, and may be escaped as regular expressions so when
60-
searching it is best to use a regexp like `a\\*[._]b\\*[._]c` to try
61-
and find as many cases as possible.
62-
* The version numbers of commands like `gem` may also have changed, so
63-
perform similar searches for those.
6451

6552
## Update libraries from third-party repos
6653

@@ -70,31 +57,26 @@ then copy the original source of `flori/json` into `lib/json`.
7057
## Updating .gemspec of default gems
7158

7259
Default gems are imported from MRI files, except the .gemspec files in
73-
`lib/ruby/gems/2.4.0/specifications/default`.
60+
`lib/ruby/gems/n.n.n/specifications/default`.
7461
To update those, copy the files over from an installed MRI.
7562
```
76-
rm -rf lib/ruby/gems/2.6.0/specifications/default
77-
cp -r ~/.rubies/ruby-2.6.1/lib/ruby/gems/2.6.0/specifications/default lib/ruby/gems/2.6.0/specifications
63+
rm -rf lib/ruby/gems/n.n.n/specifications/default
64+
cp -r ~/.rubies/ruby-n.n.n/lib/ruby/gems/n.n.n/specifications/default lib/ruby/gems/n.n.n/specifications
7865
```
7966

8067
## Updating bundled gems
8168

82-
The current list of bundled gems their versions are found at
83-
https://github.com/ruby/ruby/blob/ruby_a_b/gems/bundled_gems (replace `_a_b`
84-
with the right version branch for what you are importing). See if we need to
85-
update any bundled gems.
86-
87-
```bash
88-
$ git diff vx_y_z va_b_c gems/bundled_gems # in MRI
89-
```
90-
9169
To update a bundled gem, follow these steps:
9270

93-
* Remove the current gem and gemspec from `lib/ruby/gems/a.b.c/gems` and `lib/ruby/gems/a.b.c/specifications`
94-
* Run the gem install command with the desired version. E.g. `gem install rake -v 10.4.2 --no-doc`
95-
* Update the project `.gitignore` to allow the newly install gem sources and gemspec
71+
* Remove the current gem and gemspec from `lib/ruby/gems/a.b.c/gems` and
72+
`lib/ruby/gems/a.b.c/specifications`
73+
* Run the gem install command with the desired version
74+
`gem install rake -v 10.4.2 --no-doc`
75+
* Update the project `.gitignore` to allow the newly install gem sources
76+
and gemspec
9677
* Copy from the build directory `lib` to the source `lib`
97-
* If the gem installs any executables like `rake` in `bin` ensure that the shebang has a format as follows:
78+
* If the gem installs any executables like `rake` in `bin` ensure that the
79+
shebang has a format as follows:
9880

9981
```bash
10082
#!/usr/bin/env bash

‎spec/truffle/launcher_spec.rb

Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99

1010
require_relative '../ruby/spec_helper'
1111
require 'benchmark'
12+
require 'json'
1213

1314
describe "The launcher" do
1415

@@ -22,15 +23,17 @@ def graalvm_bash_launcher?
2223
File.expand_path(File.dirname(RbConfig.ruby)).should == bindir
2324
end
2425

25-
launchers = { bundle: /^Bundler version 1\.17\.2$/,
26-
bundler: /^Bundler version 1\.17\.2$/,
27-
gem: /^3\.0\.1$/,
28-
irb: /^irb 1\.0\.0/,
29-
rake: /^rake, version 12\.3\.2/,
30-
rdoc: /^6\.1\.0$/,
31-
ri: /^ri 6\.1\.0$/,
32-
ruby: /truffleruby .* like ruby 2\.6\.1/,
33-
truffleruby: /truffleruby .* like ruby 2\.6\.1/ }
26+
versions = JSON.parse(File.read(File.join(File.dirname(__FILE__), '../../versions.json')))
27+
28+
launchers = { bundle: /^Bundler version #{versions['gems']['default']['bundler']}$/,
29+
bundler: /^Bundler version #{versions['gems']['default']['bundler']}$/,
30+
gem: /^#{versions['gems']['default']['gem']}$/,
31+
irb: /^irb #{versions['gems']['default']['irb']}/,
32+
rake: /^rake, version #{versions['gems']['default']['rake']}/,
33+
rdoc: /^#{versions['gems']['default']['rdoc']}$/,
34+
ri: /^ri #{versions['gems']['default']['rdoc']}$/,
35+
ruby: /truffleruby .* like ruby #{versions['ruby']['version']}/,
36+
truffleruby: /truffleruby .* like ruby #{versions['ruby']['version']}/ }
3437

3538
launchers.each do |launcher, (test, skip_success)|
3639
unless [:ruby, :truffleruby].include?(launcher)

‎src/main/java/org/truffleruby/RubyLanguage.java

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424
import org.truffleruby.language.LazyRubyRootNode;
2525
import org.truffleruby.language.NotProvided;
2626
import org.truffleruby.language.RubyGuards;
27+
import org.truffleruby.shared.BuildInformationImpl;
2728
import org.truffleruby.shared.TruffleRuby;
2829
import org.truffleruby.shared.Metrics;
2930
import org.truffleruby.shared.options.OptionsCatalog;
@@ -37,7 +38,7 @@
3738
name = "Ruby",
3839
id = TruffleRuby.LANGUAGE_ID,
3940
implementationName = TruffleRuby.FORMAL_NAME,
40-
version = TruffleRuby.LANGUAGE_VERSION,
41+
version = BuildInformationImpl.RUBY_VERSION,
4142
characterMimeTypes = TruffleRuby.MIME_TYPE,
4243
defaultMimeType = TruffleRuby.MIME_TYPE,
4344
dependentLanguages = TruffleRuby.LLVM_ID)

‎src/processor/java/org/truffleruby/processor/BuildInformationProcessor.java

Lines changed: 69 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525
import java.io.File;
2626
import java.io.InputStreamReader;
2727
import java.io.PrintStream;
28+
import java.util.Calendar;
2829
import java.util.HashSet;
2930
import java.util.Set;
3031

@@ -35,6 +36,24 @@ public class BuildInformationProcessor extends AbstractProcessor {
3536

3637
private final Set<String> processed = new HashSet<>();
3738

39+
private String rubyVersion;
40+
private String rubyBaseVersion;
41+
private int rubyRevision;
42+
private String revision;
43+
private String compileDate;
44+
45+
public BuildInformationProcessor() {
46+
try {
47+
rubyVersion = runCommand("../../../tool/query-versions-json.rb ruby.version");
48+
rubyBaseVersion = runCommand("../../../tool/query-versions-json.rb ruby.base");
49+
rubyRevision = Integer.parseInt(runCommand("../../../tool/query-versions-json.rb ruby.revision"));
50+
revision = runCommand("git rev-parse --short=8 HEAD");
51+
compileDate = runCommand("git log -1 --date=short --pretty=format:%cd");
52+
} catch (Exception e) {
53+
processingEnv.getMessager().printMessage(Kind.ERROR, e.getClass() + " " + e.getMessage());
54+
}
55+
}
56+
3857
@Override
3958
public SourceVersion getSupportedSourceVersion() {
4059
return SourceVersion.latest();
@@ -69,49 +88,73 @@ private void processBuildInformation(TypeElement element) throws Exception {
6988
final JavaFileObject output = processingEnv.getFiler().createSourceFile(qualifiedName + SUFFIX, element);
7089

7190
try (PrintStream stream = new PrintStream(output.openOutputStream(), true, "UTF-8")) {
91+
stream.println("/*\n" +
92+
" * Copyright (c) " + Calendar.getInstance().get(Calendar.YEAR) + " Oracle and/or its affiliates. All rights reserved. This\n" +
93+
" * code is released under a tri EPL/GPL/LGPL license. You can use it,\n" +
94+
" * redistribute it and/or modify it under the terms of the:\n" +
95+
" *\n" +
96+
" * Eclipse Public License version 1.0, or\n" +
97+
" * GNU General Public License version 2, or\n" +
98+
" * GNU Lesser General Public License version 2.1.\n" +
99+
" */");
72100
stream.println("package " + packageName + ";");
73101
stream.println();
102+
stream.println("// This file is automatically generated from versions.json");
103+
stream.println();
104+
stream.println("import javax.annotation.Generated;");
105+
stream.println();
106+
stream.println("@Generated(\"" + getClass().getName() + "\")");
74107
stream.println("public class " + element.getSimpleName() + SUFFIX + " implements " + element.getSimpleName() + " {");
75108
stream.println();
76109
stream.println(" public static final " + element.getSimpleName() + " INSTANCE = new " + element.getSimpleName() + SUFFIX + "();");
77110
stream.println();
111+
stream.println(" // This backdoor constant is just for @TruffleLanguage.Registration, which needs a constant");
112+
stream.println(" public static final String RUBY_VERSION = \"" + rubyVersion + "\";");
113+
stream.println();
78114

79115
for (Element e : element.getEnclosedElements()) {
80116
if (e instanceof ExecutableElement) {
81-
final String value;
82-
83-
switch (e.getSimpleName().toString()) {
84-
case "getRevision":
85-
value = getRevision();
86-
break;
87-
case "getCompileDate":
88-
value = getCompileDate();
89-
break;
90-
default:
91-
throw new UnsupportedOperationException(e.getSimpleName() + " method not understood");
117+
final String name = e.getSimpleName().toString();
118+
119+
if (name.equals("getRubyRevision")) {
120+
stream.println(" @Override");
121+
stream.println(" public int " + name + "() {");
122+
stream.println(" return " + rubyRevision + ";");
123+
stream.println(" }");
124+
} else {
125+
final String value;
126+
127+
switch (name) {
128+
case "getRubyVersion":
129+
value = rubyVersion;
130+
break;
131+
case "getRubyBaseVersion":
132+
value = rubyBaseVersion;
133+
break;
134+
case "getRevision":
135+
value = revision;
136+
break;
137+
case "getCompileDate":
138+
value = compileDate;
139+
break;
140+
default:
141+
throw new UnsupportedOperationException(name + " method not understood");
142+
}
143+
144+
stream.println(" @Override");
145+
stream.println(" public String " + name + "() {");
146+
stream.println(" return \"" + value + "\";");
147+
stream.println(" }");
148+
stream.println();
92149
}
93-
94-
stream.println(" @Override");
95-
stream.println(" public String " + e.getSimpleName() + "() {");
96-
stream.println(" return \"" + value + "\";");
97-
stream.println(" }");
98150
}
99151
}
100152

101-
stream.println();
102153
stream.println("}");
103154
}
104155
}
105156

106-
public String getRevision() throws Exception {
107-
return runGitCommand("git rev-parse --short=8 HEAD");
108-
}
109-
110-
public String getCompileDate() throws Exception {
111-
return runGitCommand("git log -1 --date=short --pretty=format:%cd");
112-
}
113-
114-
private String runGitCommand(String command) throws Exception {
157+
private String runCommand(String command) throws Exception {
115158
final Process git = new ProcessBuilder(command.split("\\s+"))
116159
.directory(new File(getClass().getProtectionDomain().getCodeSource().getLocation().toURI()).getParentFile())
117160
.start();

‎src/shared/java/org/truffleruby/shared/BuildInformation.java

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/*
2-
* Copyright (c) 2018 Oracle and/or its affiliates. All rights reserved. This
2+
* Copyright (c) 2018, 2019 Oracle and/or its affiliates. All rights reserved. This
33
* code is released under a tri EPL/GPL/LGPL license. You can use it,
44
* redistribute it and/or modify it under the terms of the:
55
*
@@ -14,6 +14,12 @@
1414
@PopulateBuildInformation
1515
public interface BuildInformation {
1616

17+
String getRubyVersion();
18+
19+
String getRubyBaseVersion();
20+
21+
int getRubyRevision();
22+
1723
String getRevision();
1824

1925
String getCompileDate();

‎src/shared/java/org/truffleruby/shared/TruffleRuby.java

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,9 +20,9 @@ public class TruffleRuby {
2020
public static final String EXTENSION = ".rb";
2121
public static final String LLVM_ID = "llvm";
2222
public static final String ENGINE_ID = "truffleruby";
23-
public static final String LANGUAGE_VERSION = "2.6.1";
24-
public static final String LANGUAGE_BASE_VERSION = "2.6.0"; // From RbConfig::CONFIG["ruby_version"]
25-
public static final int LANGUAGE_REVISION = 66950;
23+
public static final String LANGUAGE_VERSION = BuildInformationImpl.INSTANCE.getRubyVersion();
24+
public static final String LANGUAGE_BASE_VERSION = BuildInformationImpl.INSTANCE.getRubyBaseVersion();
25+
public static final int LANGUAGE_REVISION = BuildInformationImpl.INSTANCE.getRubyRevision();
2626
public static final String BOOT_SOURCE_NAME = "main_boot_source";
2727
public static final String RUBY_COPYRIGHT = "truffleruby - Copyright (c) 2013-2019 Oracle and/or its affiliates";
2828
public static final boolean PRE_INITIALIZE_CONTEXTS = System.getProperty("polyglot.engine.PreinitializeContexts") != null;

‎test/truffle/ecosystem/launchers.sh

Lines changed: 15 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -13,18 +13,19 @@ while [ -h "$SELF_PATH" ]; do
1313
done
1414

1515
source "$(dirname $SELF_PATH)/../common.sh.inc"
16+
QUERY="ruby $(dirname $SELF_PATH)/../../../tool/query-versions-json.rb"
1617

1718
function check_launchers() {
1819
if [ -n "$2" ]
1920
then
20-
[[ "$(${1}truffleruby --version)" =~ truffleruby\ .*\ like\ ruby\ 2.6.1 ]]
21-
[[ "$(${1}ruby --version)" =~ truffleruby\ .*\ like\ ruby\ 2.6.1 ]]
21+
[[ "$(${1}truffleruby --version)" =~ truffleruby\ .*\ like\ ruby\ `$QUERY ruby.version` ]]
22+
[[ "$(${1}ruby --version)" =~ truffleruby\ .*\ like\ ruby\ `$QUERY ruby.version` ]]
2223
fi
23-
[[ "$(${1}gem --version)" =~ ^3.0.1$ ]]
24-
[[ "$(${1}irb --version)" =~ ^irb\ 1.0.0 ]]
25-
[[ "$(${1}rake --version)" =~ ^rake,\ version\ 12.3.2 ]]
26-
[[ "$(${1}rdoc --version)" =~ ^6.1.0$ ]]
27-
[[ "$(${1}ri --version)" =~ ^ri\ 6.1.0$ ]]
24+
[[ "$(${1}gem --version)" =~ ^`$QUERY gems.default.gem`$ ]]
25+
[[ "$(${1}irb --version)" =~ ^irb\ `$QUERY gems.default.irb` ]]
26+
[[ "$(${1}rake --version)" =~ ^rake,\ version\ `$QUERY gems.default.rake` ]]
27+
[[ "$(${1}rdoc --version)" =~ ^`$QUERY gems.default.rdoc`$ ]]
28+
[[ "$(${1}ri --version)" =~ ^ri\ `$QUERY gems.default.rdoc`$ ]]
2829
}
2930

3031
function check_in_dir() {
@@ -83,13 +84,13 @@ echo '** Check bundled gems'
8384

8485
# see doc/contributor/stdlib.md
8586
bundled_gems=(
86-
"did_you_mean 1.3.0"
87-
"minitest 5.11.3"
88-
"net-telnet 0.2.0"
89-
"power_assert 1.1.3"
90-
"rake 12.3.2"
91-
"test-unit 3.2.9"
92-
"xmlrpc 0.3.0"
87+
"did_you_mean `$QUERY gems.bundled.did_you_mean`"
88+
"minitest `$QUERY gems.bundled.minitest`"
89+
"net-telnet `$QUERY gems.bundled.net-telnet`"
90+
"power_assert `$QUERY gems.bundled.power_assert`"
91+
"rake `$QUERY gems.bundled.rake`"
92+
"test-unit `$QUERY gems.bundled.test-unit`"
93+
"xmlrpc `$QUERY gems.bundled.xmlrpc`"
9394
)
9495
gem_list=$(bin/gem list)
9596

0 commit comments

Comments
 (0)