Skip to content

Commit 819817c

Browse files
drojfdrojf
committed
Add github actions build and easy build script
1 parent 34ba117 commit 819817c

File tree

6 files changed

+361
-37
lines changed

6 files changed

+361
-37
lines changed

‎.github/workflows/build.yml

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
name: Build ui-compiler.exe and optionally Build Assets
2+
3+
on:
4+
push:
5+
tags:
6+
- '*'
7+
8+
jobs:
9+
windows_build:
10+
name: Windows Build
11+
runs-on: windows-latest
12+
steps:
13+
# Download the repository
14+
- uses: actions/checkout@v2
15+
16+
# Caching for Rust
17+
- name: Cache rust builds
18+
uses: Swatinem/rust-cache@v2
19+
20+
- name: Build ui-compiler.exe
21+
run: cargo build
22+
23+
- name: Download and install dependencies, then build assets
24+
run: python build.py github_actions
25+
26+
# Publish a release (tagged commits)
27+
# For more info on options see: https://github.com/softprops/action-gh-release
28+
- name: Release (tag)
29+
uses: softprops/action-gh-release@v1
30+
if: startsWith(github.ref, 'refs/tags/') # only publish tagged commits
31+
with:
32+
files: |
33+
output/*.7z
34+
target/debug/ui-compiler.exe
35+
draft: true
36+
env:
37+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

‎.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,4 @@
22
/output
33
/target
44
**/*.rs.bk
5+
/64bit

‎README.md

Lines changed: 42 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -16,40 +16,54 @@ Please note that documentation is in two places:
1616

1717
## Usage instructions for translators and dev team
1818

19-
### Prerequisites
19+
### Setup (Windows Only)
2020

21-
1. Install Python 3
22-
2. Run `pip install numpy Pillow unitypack` to install the required Python packages
23-
3. Install Rust
24-
4. Download exactly this version of UABE https://github.com/SeriousCache/UABE/releases/tag/2.2stabled
25-
5. Download 7zip (standalone console version) from here https://www.7-zip.org/download.html
26-
6. Extract UABE and 7z and make sure **both** are on your `PATH`
27-
- On Windows, you need to restart your terminal window to update your `PATH`.
28-
- Try caling `AssetBundleExtractor.exe`
29-
- Try caling `7za.exe`
30-
7. Download the [vanilla UI archive](http://07th-mod.com/archive/vanilla.7z) and unpack it into the repository root (creating the folder `assets/vanilla`).
21+
The below instructions only work on Windows!
3122

32-
### Using the tool to generate sharedassets0.assets
23+
1. Install Python 3 from the [Python Download Page](https://www.python.org/downloads/)
24+
2. For translators, fork this repository ([Github forking instructions]( https://docs.github.com/en/get-started/quickstart/fork-a-repo))
25+
- A fork is recommended for translators as you can check in your changes to github. It also allows you to use Github for building and hosting assets
26+
3. Clone the repository (either this repository, or the one you forked) to your computer
3327

34-
To build all chapters:
35-
- On Linux open a terminal and run the below command
36-
- On Windows with "Git for Windows" installed, right click the folder and click "Git Bash Here", then use that terminal
28+
### Using the tool to generate sharedassets0.assets (Windows Only)
3729

38-
```
39-
./compileall.sh
40-
```
30+
To list the supported Higurashi chapters for this tool, run
4131

42-
To build a particular chapter/version run the following command
32+
```python build.py```
4333

44-
```
45-
cargo run <chapter> <unityversion> <system>
46-
```
34+
which should show an error message complaining about a missing `chapter` argument
4735

48-
`<chapter>` is simply `onikakushi`, `watanagashi` and so on.
36+
Then run
4937

50-
`<unityversion>` is the unity version, like `5.5.3p3` or `2017.2.5`. Note that for version `2017.2.5f1`, you just enter `2017.2.5` (currently only support the first 8 characters of the unity version)
38+
```python build.py onikakushi```
5139

52-
`<system>` is `win` or `unix`.
40+
for example, to build the sharedassets required for onikakushi. You can also run `python build.py all` to build all chapters.
41+
42+
**NOTE: If a new game is released, or a game is updated, you may need to re-download the vanilla assets. To do this, add the '--force-download' option like so:**
43+
44+
```python build.py rei --force-download```
45+
46+
You may encounter the following problems:
47+
- Windows Defender may block/delete our precompiled `ui-compiler.exe`. In this case, you can either try to unblock it, or install Rust to make the script compile it on your own computer. Contact us if you have this issue.
48+
- For any other error, likely we just need to update the build script, so please contact us.
49+
50+
### Modifying Assets
51+
52+
Assets are located in the `assets` folder. Replace any file in the `assets` folder, then run the script again, and it should be included in the generated assets files.
53+
54+
### Building assets using Github Actions
55+
56+
#### Note for forks/translators
57+
58+
Github actions might be disabled for your forks. Clicking on the 'Actions' tab should allow you to enable it. Please do this before proceeding.
59+
60+
#### Building a release
61+
62+
To use Github Actions to build a release, create a tag like `v1.0.6_onikakushi` which contains the chapter name (separated by an underscore) you want to build (or 'all' for all chapters).
63+
64+
Click on the 'Actions' tab to observe the build process.
65+
66+
Once the build is complete, go to the 'Releases' page, and a new draft release should appear. You can check everything is OK before publishing the release, or just download the files without publishing the release.
5367

5468
----
5569

@@ -71,6 +85,9 @@ You'll need to extract the 'msgothic' font files from the stock `.assets` file b
7185
2. Rename them as `msgothic_0.dat` and `msgothic_2.dat`
7286
3. Move them to `assets/vanilla/<chapter>/msgothic_0.dat` & `assets/vanilla/<chapter>/msgothic_2.dat`
7387

88+
## Building `ui-compiler.exe`
89+
90+
To build just the `ui-compiler.exe`, push any tag to the repository.
7491

7592
## Extra Notes
7693

‎build.py

Lines changed: 232 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,232 @@
1+
import re
2+
import subprocess
3+
import sys
4+
import os
5+
import pathlib
6+
import shutil
7+
import argparse
8+
from typing import List
9+
10+
# Get the github ref
11+
GIT_TAG = None
12+
GIT_REF = os.environ.get("GITHUB_REF") # Github Tag / Version info
13+
if GIT_REF is not None:
14+
GIT_TAG = GIT_REF.split("/")[-1]
15+
print(f"--- Git Ref: {GIT_REF} Git Tag: {GIT_TAG} ---")
16+
17+
# List of build variants for any given chapter
18+
#
19+
# There must be a corresponding vanilla sharedassets0.assets file located at:
20+
# assets\vanilla\{CHAPTER_NAME}[-{CRC32}]\{OS}-{UNITY_VERSION}\sharedassets0.assets
21+
# for each entry.
22+
chapter_to_build_variants = {
23+
"onikakushi": [
24+
"onikakushi 5.2.2f1 win",
25+
"onikakushi 5.2.2f1 unix",
26+
],
27+
"watanagashi": [
28+
"watanagashi 5.2.2f1 win",
29+
"watanagashi 5.2.2f1 unix"
30+
],
31+
"tatarigoroshi": [
32+
"tatarigoroshi 5.4.0f1 win",
33+
"tatarigoroshi 5.4.0f1 unix",
34+
"tatarigoroshi 5.3.5f1 win",
35+
"tatarigoroshi 5.3.4p1 win",
36+
"tatarigoroshi 5.3.4p1 unix",
37+
],
38+
"himatsubushi": [
39+
"himatsubushi 5.4.1f1 win",
40+
"himatsubushi 5.4.1f1 unix"
41+
],
42+
"meakashi": [
43+
"meakashi 5.5.3p3 win",
44+
"meakashi 5.5.3p3 unix",
45+
"meakashi 5.5.3p1 win",
46+
"meakashi 5.5.3p1 unix",
47+
],
48+
"tsumihoroboshi": [
49+
"tsumihoroboshi 5.5.3p3 win",
50+
"tsumihoroboshi 5.5.3p3 unix"
51+
# While GOG Windows is ver 5.6.7f1, we actually downgrade back to 5.5.3p3 in the installer, so we don't need this version.
52+
#'tsumihoroboshi 5.6.7f1 win'
53+
],
54+
"minagoroshi": [
55+
"minagoroshi 5.6.7f1 win",
56+
"minagoroshi 5.6.7f1 unix"
57+
# While GOG Windows is ver 5.6.7f1, we actually downgrade back to 5.5.3p3 in the installer, so we don't need this version.
58+
# 'matsuribayashi 5.6.7f1 win'
59+
# 'matsuribayashi 5.6.7f1 unix'
60+
],
61+
"matsuribayashi": [
62+
"matsuribayashi 2017.2.5 unix",
63+
# Special version for GOG/Mangagamer Linux with SHA256:
64+
# A200EC2A85349BC03B59C8E2F106B99ED0CBAAA25FC50928BB8BA2E2AA90FCE9
65+
# CRC32L 51100D6D
66+
"matsuribayashi 2017.2.5 unix 51100D6D",
67+
"matsuribayashi 2017.2.5 win",
68+
],
69+
'rei': [
70+
'rei 2019.4.3 win',
71+
'rei 2019.4.3 unix',
72+
],
73+
}
74+
75+
76+
def is_windows():
77+
return sys.platform == "win32"
78+
79+
80+
def call(args, **kwargs):
81+
print("running: {}".format(args))
82+
retcode = subprocess.call(
83+
args, shell=is_windows(), **kwargs
84+
) # use shell on windows
85+
if retcode != 0:
86+
raise Exception(f"ERROR: {args} exited with retcode: {retcode}")
87+
88+
89+
def download(url):
90+
print(f"Starting download of URL: {url}")
91+
call(["curl", "-OJLf", url])
92+
93+
94+
def seven_zip_extract(input_path, outputDir=None):
95+
args = ["7z", "x", input_path, "-y"]
96+
if outputDir:
97+
args.append("-o" + outputDir)
98+
99+
call(args)
100+
101+
102+
def get_chapter_name_from_git_tag():
103+
if GIT_TAG is None:
104+
raise Exception(
105+
"'github_actions' was selected, but environment variable GIT_REF was not set - are you sure you're running this script from Github Actions?"
106+
)
107+
else:
108+
# Look for the chapter name to build in the git tag
109+
tag_fragments = [x.lower() for x in re.split("[\W_]", GIT_REF)]
110+
111+
if "all" in tag_fragments:
112+
return "all"
113+
else:
114+
for chapter_name in chapter_to_build_variants.keys():
115+
if chapter_name.lower() in tag_fragments:
116+
return chapter_name
117+
118+
return None
119+
120+
121+
def get_build_variants(selected_chapter: str) -> List[str]:
122+
if selected_chapter == "all":
123+
commands = []
124+
for command in chapter_to_build_variants.values():
125+
commands.extend(command)
126+
return commands
127+
elif selected_chapter in chapter_to_build_variants:
128+
return chapter_to_build_variants[selected_chapter]
129+
else:
130+
raise Exception(
131+
f"Unknown Chapter {selected_chapter} - please update the build.py script"
132+
)
133+
134+
135+
# Parse command line arguments
136+
parser = argparse.ArgumentParser(
137+
description="Download and Install dependencies for ui editing scripts, then run build"
138+
)
139+
parser.add_argument(
140+
"chapter",
141+
help='The chapter to build, or "all" for all chapters',
142+
choices=["all", "github_actions"] + list(chapter_to_build_variants.keys()),
143+
)
144+
parser.add_argument("--force-download", default=False, action='store_true')
145+
args = parser.parse_args()
146+
147+
force_download = args.force_download
148+
149+
# Get chapter name from git tag if "github_actions" specified as the chapter
150+
chapter_name = args.chapter
151+
if chapter_name == "github_actions":
152+
chapter_name = get_chapter_name_from_git_tag()
153+
if chapter_name is None:
154+
print(
155+
f">>>> WARNING: No chapter name (or 'all') was found in git tag {GIT_TAG} - skipping building .assets"
156+
)
157+
exit(0)
158+
159+
# Get a list of build variants (like 'onikakushi 5.2.2f1 win') depending on commmand line arguments
160+
build_variants = get_build_variants(chapter_name)
161+
print(f"For chapter '{chapter_name}' building: {build_variants}")
162+
163+
# Install python dependencies
164+
print("Installing python dependencies")
165+
call([sys.executable, "-m", "pip", "install", "-r", "requirements.txt"])
166+
167+
# Download and extract the vanilla assets
168+
assets_path = "assets"
169+
vanilla_archive = "vanilla.7z"
170+
vanilla_folder_path = os.path.join(assets_path, "vanilla")
171+
vanilla_fully_extracted = os.path.exists(vanilla_folder_path) and not os.path.exists(vanilla_archive)
172+
if force_download or not vanilla_fully_extracted:
173+
print("Downloading and Extracting Vanilla assets")
174+
pathlib.Path(vanilla_archive).unlink(missing_ok=True)
175+
if os.path.exists(vanilla_folder_path):
176+
shutil.rmtree(vanilla_folder_path)
177+
178+
download("http://07th-mod.com/archive/vanilla.7z")
179+
seven_zip_extract(vanilla_archive)
180+
181+
# Remove the archive to indicate extraction was successful
182+
pathlib.Path(vanilla_archive).unlink(missing_ok=True)
183+
else:
184+
print("Vanilla archive already extracted - skipping")
185+
186+
# Download and extract UABE
187+
uabe_folder = "64bit"
188+
uabe_archive = "AssetsBundleExtractor_2.2stabled_64bit_with_VC2010.zip"
189+
uabe_fully_extracted = os.path.exists(uabe_folder) and not os.path.exists(uabe_archive)
190+
if force_download or not uabe_fully_extracted:
191+
print("Downloading and Extracting UABE")
192+
pathlib.Path(uabe_archive).unlink(missing_ok=True)
193+
if os.path.exists(uabe_folder):
194+
shutil.rmtree(uabe_folder)
195+
196+
# The default Windows github runner doesn't have the 2010 VC++ redistributable preventing UABE from running
197+
# This zip file bundles the required DLLs (msvcr100.dll & msvcp100.dll) so it's not required
198+
download(f"http://07th-mod.com/archive/{uabe_archive}")
199+
seven_zip_extract(uabe_archive)
200+
201+
# Remove the archive to indicate extraction was successful
202+
pathlib.Path(uabe_archive).unlink(missing_ok=True)
203+
else:
204+
print("UABE already extracted - skipping")
205+
206+
207+
# Add UABE to PATH
208+
uabe_folder = os.path.abspath(uabe_folder)
209+
os.environ["PATH"] += os.pathsep + os.pathsep.join([uabe_folder])
210+
211+
# If rust is not installed, download binary release of ui comopiler
212+
# This is mainly for users running this script on their own computer
213+
working_cargo = False
214+
try:
215+
subprocess.check_output("cargo -v")
216+
print(
217+
"Found working Rust/cargo - will compile ui-compiler.exe using repository sources"
218+
)
219+
working_cargo = True
220+
except:
221+
print("No working Rust/cargo found - download binary release of UI compiler...")
222+
download(
223+
"https://github.com/07th-mod/ui-editing-scripts/releases/latest/download/ui-compiler.exe"
224+
)
225+
226+
# Build all the requested variants
227+
for command in build_variants:
228+
print(f"Building .assets for {command}...")
229+
if working_cargo:
230+
call(f"cargo run {command}")
231+
else:
232+
call(f"ui-compiler.exe {command}")

‎requirements.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
numpy
2+
Pillow
3+
unitypack

0 commit comments

Comments
 (0)