Skip to content

Commit a49dca2

Browse files
yongtangyongtang
authored
Setup GitHub Actions for API Compatibility (#1219)
This PR setup GitHub Actions for API Compatibility (file system only due to C vs. C++ API in TensorFlow Core) on different combinations of tensorflow vs tensorflow-io. As a first step this PR adds the following API Compatibility tests: - tensorflow==2.4.0rc4 vs. tensorflow-io-nightly - tf-nightly vs. tensorflow-io-nightly In the future addtional combinations might be added. This will give us a better idea in case tensorflow updates breaks API or tensorflow-io breaks API. As we needs to test against TF's nightly build, this PR setup the GitHub Actions trigger to daily (on 8AM EST). Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
1 parent 859744b commit a49dca2

File tree

2 files changed

+121
-0
lines changed

2 files changed

+121
-0
lines changed

‎.github/workflows/api.yml

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
name: API Compatibility
2+
3+
on:
4+
schedule:
5+
- cron: "0 12 * * *"
6+
7+
env:
8+
BAZEL_OPTIMIZATION: --copt=-msse4.2 --copt=-mavx --compilation_mode=opt
9+
10+
jobs:
11+
12+
macos:
13+
name: macOS ${{ matrix.python }} + ${{ matrix.version }}
14+
runs-on: macos-latest
15+
strategy:
16+
matrix:
17+
python: ['3.8']
18+
version: ['tensorflow==2.4.0rc4:tensorflow-io-nightly', 'tf-nightly:tensorflow-io-nightly']
19+
steps:
20+
- uses: actions/checkout@v2
21+
- uses: docker-practice/actions-setup-docker@v1
22+
- uses: actions/setup-python@v1
23+
with:
24+
python-version: ${{ matrix.python }}
25+
- run: |
26+
set -x -e
27+
python -m pip install -U wheel setuptools
28+
python --version
29+
- name: Setup macOS
30+
run: |
31+
set -x -e
32+
docker version
33+
bash -x -e tests/test_azure/start_azure.sh
34+
bash -x -e tests/test_aws/aws_test.sh
35+
- name: Test macOS
36+
run: |
37+
set -x -e
38+
python --version
39+
df -h
40+
rm -rf tensorflow_io
41+
echo ${{ matrix.version }} | awk -F: '{print $1}' | xargs python -m pip install -U
42+
echo ${{ matrix.version }} | awk -F: '{print $2}' | xargs python -m pip install --no-deps -U
43+
python -m pip install pytest-benchmark boto3
44+
python -m pip freeze
45+
python -c 'import tensorflow as tf; print(tf.version.VERSION)'
46+
python -c 'import tensorflow_io as tfio; print(tfio.version.VERSION)'
47+
python -m pytest -s -v tests/test_http_eager.py
48+
python -m pytest -s -v tests/test_s3_eager.py
49+
python -m pytest -s -v tests/test_azure.py
50+
51+
linux:
52+
name: Linux ${{ matrix.python }} + ${{ matrix.version }}
53+
runs-on: ubuntu-20.04
54+
strategy:
55+
matrix:
56+
python: ['3.8']
57+
version: ['tensorflow==2.4.0rc4:tensorflow-io-nightly', 'tf-nightly:tensorflow-io-nightly']
58+
steps:
59+
- uses: actions/checkout@v2
60+
- uses: actions/setup-python@v1
61+
with:
62+
python-version: ${{ matrix.python }}
63+
- name: Setup Linux
64+
run: |
65+
set -x -e
66+
bash -x -e .github/workflows/build.space.sh
67+
bash -x -e tests/test_aws/aws_test.sh
68+
bash -x -e tests/test_azure/start_azure.sh
69+
bash -x -e tests/test_gcloud/test_gcs.sh gcs-emulator
70+
- name: Test Linux
71+
run: |
72+
set -x -e
73+
python --version
74+
df -h
75+
rm -rf tensorflow_io
76+
echo ${{ matrix.version }} | awk -F: '{print $1}' | xargs python -m pip install -U
77+
echo ${{ matrix.version }} | awk -F: '{print $2}' | xargs python -m pip install --no-deps -U
78+
python -m pip install pytest-benchmark boto3
79+
python -m pip freeze
80+
python -c 'import tensorflow as tf; print(tf.version.VERSION)'
81+
python -c 'import tensorflow_io as tfio; print(tfio.version.VERSION)'
82+
python -m pytest -s -v tests/test_http_eager.py
83+
python -m pytest -s -v tests/test_s3_eager.py
84+
python -m pytest -s -v tests/test_azure.py
85+
86+
windows:
87+
name: Windows ${{ matrix.python }} + ${{ matrix.version }}
88+
runs-on: windows-latest
89+
strategy:
90+
matrix:
91+
python: ['3.8']
92+
version: ['tensorflow==2.4.0rc4:tensorflow-io-nightly', 'tf-nightly:tensorflow-io-nightly']
93+
steps:
94+
- uses: actions/checkout@v2
95+
- uses: actions/setup-python@v1
96+
with:
97+
python-version: ${{ matrix.python }}
98+
- uses: actions/setup-node@v1
99+
with:
100+
node-version: '8.x'
101+
- name: Setup Windows
102+
shell: cmd
103+
run: |
104+
@echo on
105+
bash -x -e tests/test_azure/start_azure.sh
106+
- name: Test Windows
107+
shell: cmd
108+
run: |
109+
@echo on
110+
python --version
111+
rm -rf tensorflow_io
112+
echo ${{ matrix.version }} | awk -F: '{print $1}' | xargs python -m pip install -U
113+
echo ${{ matrix.version }} | awk -F: '{print $2}' | xargs python -m pip install --no-deps -U
114+
python -m pip install pytest-benchmark
115+
python -m pip freeze
116+
python -c 'import tensorflow as tf; print(tf.version.VERSION)'
117+
python -c 'import tensorflow_io as tfio; print(tfio.version.VERSION)'
118+
python -m pytest -s -v tests/test_http_eager.py -k remote

‎tests/test_azure.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,9 @@ class AZFSTest(tf.test.TestCase):
3030
"""
3131

3232
def __init__(self, methodName="runTest"): # pylint: disable=invalid-name
33+
34+
os.environ["TF_AZURE_USE_DEV_STORAGE"] = "1"
35+
3336
self.account = "devstoreaccount1"
3437
self.container = "aztest"
3538
self.path_root = "az://" + os.path.join(self.account, self.container)

0 commit comments

Comments
 (0)