Skip to content

Commit 8b38027

Browse files
theorytheory
committed
Add test coverage workflow.
Add a new script, `.github/ubuntu/all-apt-prereqs.sh`, to install all apt packages at once, and teach each of the engine scripts not to install apt packages when `SKIP_DEPENDS` is set. Also have the Postgres an SQLite scripts fall back on a default version to install if one is not passed. Remove the old `prereqs.sh`. And have the SQLite script install SQLite into `/opt/sqlite` to be more like the other engine installers, and have it and the Oracle script properly share the `LD_LIBRARY_PATH` variable. The new workflow, `.github/workflows/coverage.yml`, sets up all of the services and clients, as well as modules for reporting coverage to Coveralls, and runs the tests. Now that it uses a script installed in `local/bin`, teach all the workflows to cache all of `local`, not just `local/lib`. Get Oracle 21 tests working, too. Thanks to @gvenzl's help in gvenzl/oci-oracle-xe#45. Remove `.travis.yml` and references to it. Travis testing has not worked for a while, and is no more. Have `prove` run tests in parallel in 4 processes where it runs multiple tests, so that builds finish more quickly. Add build and release badges to README.md, and move most links to the bottom of the text for better plain text legibility. Most of the GitHub badges don't handle the Emoji correctly, but hopefully that will be addressed before long. Finally, add a call to `disconnect` to `t/oracle.t` to prevent a segfault that Devel::Cover seems to induce in DBD::Oracle (see perl5-dbi/DBD-Oracle#111).
1 parent f0a9ab5 commit 8b38027

27 files changed

+286
-360
lines changed

Diff for: ‎.github/ubuntu/all-apt-prereqs.sh

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
set -e
2+
3+
sudo apt-get update -qq
4+
sudo env DEBIAN_FRONTEND=noninteractive apt-get install -qq \
5+
libicu-dev gettext aspell-en software-properties-common \
6+
curl unixodbc-dev odbcinst unixodbc \
7+
default-jre \
8+
firebird-dev firebird3.0-utils \
9+
mysql-client default-libmysqlclient-dev \
10+
libarchive-tools
11+
cat t/odbc/odbcinst.ini | sudo tee -a /etc/odbcinst.ini

Diff for: ‎.github/ubuntu/exasol.sh

+5-3
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,14 @@ version=${1:-7}
66
echo $version
77

88
# Download dependencies.
9-
sudo apt-get update -qq
10-
sudo env DEBIAN_FRONTEND=noninteractive apt-get install -qq curl unixodbc-dev odbcinst unixodbc default-jre
9+
if [ -z "$SKIP_DEPENDS" ]; then
10+
sudo apt-get update -qq
11+
sudo env DEBIAN_FRONTEND=noninteractive apt-get install -qq curl unixodbc-dev odbcinst unixodbc default-jre
12+
cat t/odbc/odbcinst.ini | sudo tee -a /etc/odbcinst.ini
13+
fi
1114

1215
# Prepare the configuration.
1316
mkdir -p /opt/exasol
14-
cat t/odbc/odbcinst.ini | sudo tee -a /etc/odbcinst.ini
1517

1618
# Download and unpack Exasol ODBC Driver & EXAplus.
1719
if [[ "$version" =~ ^6 ]]; then

Diff for: ‎.github/ubuntu/firebird.sh

+9-2
Original file line numberDiff line numberDiff line change
@@ -3,5 +3,12 @@
33
set -e
44

55
# Download dependencies.
6-
sudo apt-get update -qq
7-
sudo env DEBIAN_FRONTEND=noninteractive apt-get install -qq firebird-dev firebird3.0-utils
6+
if [ -z "$SKIP_DEPENDS" ]; then
7+
sudo apt-get update -qq
8+
sudo env DEBIAN_FRONTEND=noninteractive apt-get install -qq firebird-dev firebird3.0-utils
9+
fi
10+
11+
# Tell DBD::Firebird where to find the libraries.
12+
if [[ ! -z "$GITHUB_ENV" ]]; then
13+
echo "FIREBIRD_HOME=/usr" >> $GITHUB_ENV
14+
fi

Diff for: ‎.github/ubuntu/mysql.sh

+4-3
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
set -e
44

55
# Download dependencies.
6-
sudo apt-get update -qq
7-
sudo env DEBIAN_FRONTEND=noninteractive apt-get install -qq mysql-client default-libmysqlclient-dev
8-
6+
if [ -z "$SKIP_DEPENDS" ]; then
7+
sudo apt-get update -qq
8+
sudo env DEBIAN_FRONTEND=noninteractive apt-get install -qq mysql-client default-libmysqlclient-dev
9+
fi

Diff for: ‎.github/ubuntu/oracle.sh

+9-3
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,10 @@ version=21.3.0.0.0
66
icdr=213000
77

88
# Install bsdtar, required to get --strip-components for a zip file.
9-
sudo apt-get update -qq
10-
sudo apt-get install -qq libarchive-tools
9+
if [ -z "$SKIP_DEPENDS" ]; then
10+
sudo apt-get update -qq
11+
sudo env DEBIAN_FRONTEND=noninteractive apt-get install -qq libarchive-tools
12+
fi
1113

1214
# Download Instant Client.
1315
# https://www.oracle.com/database/technologies/instant-client/downloads.html
@@ -28,5 +30,9 @@ fi
2830

2931
if [[ ! -z "$GITHUB_ENV" ]]; then
3032
echo "ORACLE_HOME=/opt/instantclient" >> $GITHUB_ENV
31-
echo "LD_LIBRARY_PATH=/opt/instantclient" >> $GITHUB_ENV
33+
if [[ -z "$LD_LIBRARY_PATH" ]]; then
34+
echo "LD_LIBRARY_PATH=/opt/instantclient" >> $GITHUB_ENV
35+
else
36+
echo "LD_LIBRARY_PATH=/opt/instantclient:$LD_LIBRARY_PATH" >> $GITHUB_ENV
37+
fi
3238
fi

Diff for: ‎.github/ubuntu/pg.sh

+1-6
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,7 @@
22

33
set -e
44

5-
PGVERSION=${PGVERSION:=$1}
6-
if [[ -z "$PGVERSION" ]]; then
7-
echo "Usage: $0 \$version"
8-
exit 32
9-
fi
10-
5+
PGVERSION=${PGVERSION:=${1:-14}}
116
[[ $PGVERSION =~ ^[0-9]$ ]] && PGVERSION+=.0
127

138
curl -O https://salsa.debian.org/postgresql/postgresql-common/-/raw/master/pgdg/apt.postgresql.org.sh

Diff for: ‎.github/ubuntu/prereqs.sh

-7
This file was deleted.

Diff for: ‎.github/ubuntu/snowflake.sh

+5-3
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,11 @@
33
set -e
44

55
# Set up Snowflake.
6-
sudo apt-get update -qq
7-
sudo apt-get install -qq unixodbc-dev odbcinst unixodbc
6+
if [ -z "$SKIP_DEPENDS" ]; then
7+
sudo apt-get update -qq
8+
sudo env DEBIAN_FRONTEND=noninteractive apt-get install -qq unixodbc-dev odbcinst unixodbc
9+
cat t/odbc/odbcinst.ini | sudo tee -a /etc/odbcinst.ini
10+
fi
811

912
# https://docs.snowflake.net/manuals/release-notes/client-change-log-snowsql.html
1013
# https://sfc-repo.snowflakecomputing.com/index.html
@@ -13,7 +16,6 @@ curl -sSLo snowdbc.tgz https://sfc-repo.snowflakecomputing.com/odbc/linux/2.24.2
1316

1417
# Install and configure ODBC.
1518
mkdir -p /opt/snowflake
16-
cat t/odbc/odbcinst.ini | sudo tee -a /etc/odbcinst.ini
1719
sudo tar --strip-components 1 -C /opt/snowflake -xzf snowdbc.tgz
1820
sudo mv /opt/snowflake/ErrorMessages/en-US /opt/snowflake/lib/
1921

Diff for: ‎.github/ubuntu/sqlite.sh

+10-12
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,7 @@
22

33
set -e
44

5-
DIR=$(pwd)
6-
7-
SQLITE=${SQLITE:=$1}
8-
if [[ -z "$SQLITE" ]]; then
9-
echo "Usage: $0 \$version"
10-
exit 32
11-
fi
5+
SQLITE=${SQLITE:=${1:-3.36.0}}
126

137
# Convert to the SQLITE_VERSION_NUMBER format https://sqlite.org/c3ref/c_source_id.html
148
SQLITE=$(perl -e 'my @v = split /[.]/, shift; printf "%d%02d%02d%02d\n", @v[0..3]' $SQLITE)
@@ -33,8 +27,8 @@ fi
3327

3428
# Download, compile, and install SQLite.
3529
curl -o sqlite.zip https://sqlite.org/$YEAR/sqlite-amalgamation-$SQLITE.zip
36-
unzip -j sqlite.zip -d sqlite
37-
cd sqlite
30+
unzip -j sqlite.zip -d /opt/sqlite
31+
cd /opt/sqlite
3832
# Build the CLI.
3933
gcc shell.c sqlite3.c -lpthread -ldl -o sqlite3
4034
# Build the shared library
@@ -47,13 +41,17 @@ URL=https://cpan.metacpan.org/authors/id/${DIST:0:1}/${DIST:0:2}/$DIST
4741
curl -o dbd.tar.gz $URL
4842
tar zxvf dbd.tar.gz --strip-components 1
4943
perl -i -pe 's/^if\s*\(\s*0\s*\)\s\{/if (1) {/' Makefile.PL
50-
perl Makefile.PL SQLITE_INC=$DIR/sqlite SQLITE_LIB=$DIR/sqlite
44+
perl Makefile.PL SQLITE_INC=/opt/sqlite SQLITE_LIB=/opt/sqlite
5145
make && make install
5246

5347
if [[ ! -z "$GITHUB_PATH" ]]; then
54-
echo "$DIR/sqlite" >> $GITHUB_PATH
48+
echo "/opt/sqlite" >> $GITHUB_PATH
5549
fi
5650

5751
if [[ ! -z "$GITHUB_ENV" ]]; then
58-
echo "LD_LIBRARY_PATH=$DIR/sqlite" >> $GITHUB_ENV
52+
if [[ -z "$LD_LIBRARY_PATH" ]]; then
53+
echo "LD_LIBRARY_PATH=/opt/sqlite" >> $GITHUB_ENV
54+
else
55+
echo "LD_LIBRARY_PATH=/opt/sqlite:$LD_LIBRARY_PATH" >> $GITHUB_ENV
56+
fi
5957
fi

Diff for: ‎.github/ubuntu/vertica.sh

+5-3
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,12 @@
22

33
set -e
44

5-
sudo apt-get update -qq
6-
sudo apt-get install -qq unixodbc-dev odbcinst unixodbc
5+
if [ -z "$SKIP_DEPENDS" ]; then
6+
sudo apt-get update -qq
7+
sudo env DEBIAN_FRONTEND=noninteractive apt-get install -qq unixodbc-dev odbcinst unixodbc
8+
cat t/odbc/odbcinst.ini | sudo tee -a /etc/odbcinst.ini
9+
fi
710

8-
cat t/odbc/odbcinst.ini | sudo tee -a /etc/odbcinst.ini
911
cat t/odbc/vertica.ini | sudo tee -a /etc/vertica.ini
1012

1113
# https://www.vertica.com/download/vertica/client-drivers/

Diff for: ‎.github/workflows/coverage.yml

+102
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
# This workflow creates the services and installs the clients in order to run
2+
# coverage tests. Each engine must be accessible for a complete coverage report.
3+
# It runs for pushes and pull requests on the main and develop branches.
4+
name: 📈 Coverage
5+
on:
6+
push:
7+
branches: [main, develop]
8+
pull_request:
9+
branches: [main, develop]
10+
jobs:
11+
Snowflake:
12+
name: 📈 Coverage
13+
runs-on: ubuntu-latest
14+
services:
15+
exasol:
16+
image: exasol/docker-db:latest
17+
ports: [ 8563 ]
18+
options: --privileged
19+
firebird:
20+
image: jacobalberty/firebird:latest
21+
ports: [ 3050 ]
22+
env:
23+
ISC_PASSWORD: nix
24+
FIREBIRD_DATABASE: sqitchtest.db
25+
mysql:
26+
image: mysql:latest
27+
env: { MYSQL_ALLOW_EMPTY_PASSWORD: yes }
28+
ports: [ 3306 ]
29+
options: --health-cmd="mysqladmin ping" --health-interval=5s --health-timeout=2s --health-retries=3
30+
oracle:
31+
image: gvenzl/oracle-xe:latest
32+
ports: [ 1521 ]
33+
env:
34+
ORACLE_PASSWORD: oracle
35+
APP_USER: sqitchtest
36+
APP_USER_PASSWORD: sqitchtest
37+
options: >-
38+
--health-cmd healthcheck.sh
39+
--health-interval 20s
40+
--health-timeout 10s
41+
--health-retries 10
42+
vertica:
43+
image: vertica/vertica-ce:latest
44+
ports: [ 5433 ]
45+
steps:
46+
- uses: actions/checkout@v2
47+
- name: Setup Clients
48+
env:
49+
SKIP_DEPENDS: true
50+
run: |
51+
.github/ubuntu/all-apt-prereqs.sh
52+
.github/ubuntu/exasol.sh
53+
.github/ubuntu/firebird.sh
54+
.github/ubuntu/mysql.sh
55+
.github/ubuntu/oracle.sh
56+
.github/ubuntu/pg.sh
57+
.github/ubuntu/snowflake.sh
58+
.github/ubuntu/vertica.sh
59+
- name: Setup Perl
60+
id: perl
61+
uses: shogo82148/actions-setup-perl@v1
62+
with: { perl-version: latest }
63+
- name: Cache CPAN Modules
64+
uses: actions/cache@v2
65+
with:
66+
path: local
67+
key: perl-${{ steps.perl.outputs.perl-hash }}
68+
- name: Download cpanfile
69+
uses: carlosperate/[email protected]
70+
with:
71+
file-url: https://fastapi.metacpan.org/source/DWHEELER/App-Sqitch-v1.1.0/dist/cpanfile
72+
- run: cpm install --verbose --show-build-log-on-failure --no-test --with-recommends --cpanfile cpanfile
73+
- run: cpm install --verbose --show-build-log-on-failure --no-test --with-recommends DBI DBD::ODBC DBD::Firebird DBD::Oracle DBD::mysql DBD::Pg Devel::Cover Devel::Cover::Report::Coveralls Algorithm::Backoff::Exponential
74+
- name: Install SQLite
75+
env: { PERL5LIB: "${{ github.workspace }}/local/lib/perl5" }
76+
run: .github/ubuntu/sqlite.sh
77+
- name: Run Tests
78+
env:
79+
PERL5LIB: "${{ github.workspace }}/local/lib/perl5"
80+
HARNESS_PERL_SWITCHES: -MDevel::Cover=-ignore,^(?:x?t|inc|bin|local)/
81+
LIVE_EXASOL_REQUIRED: true
82+
SQITCH_TEST_EXASOL_URI: db:exasol://sys:[email protected]:${{ job.services.exasol.ports[8563] }}/?Driver=Exasol;SSLCertificate=SSL_VERIFY_NONE
83+
LIVE_FIREBIRD_REQUIRED: true
84+
SQITCH_TEST_FIREBIRD_URI: db:firebird://sysdba:[email protected]:${{ job.services.firebird.ports[3050] }}//firebird/data/sqitchtest.db
85+
LIVE_MYSQL_REQUIRED: true
86+
SQITCH_TEST_MYSQL_URI: "db:mysql://[email protected]:${{ job.services.mysql.ports[3306] }}/information_schema"
87+
LIVE_ORACLE_REQUIRED: true
88+
SQITCH_TEST_ALT_ORACLE_REGISTRY: dbsnmp
89+
SQITCH_TEST_ORACLE_URI: db:oracle://system:[email protected]:${{ job.services.oracle.ports[1521] }}/XE
90+
LIVE_PG_REQUIRED: true
91+
SQITCH_TEST_PG_URI: db:pg://postgres@/postgres
92+
LIVE_SNOWFLAKE_REQUIRED: true
93+
SQITCH_TEST_SNOWFLAKE_URI: db:snowflake://${{ secrets.SNOWFLAKE_USERNAME }}:${{ secrets.SNOWFLAKE_PASSWORD }}@sra81677.us-east-1/sqitchtest?Driver=Snowflake;warehouse=compute_wh
94+
LIVE_SQLITE_REQUIRED: true
95+
LIVE_VERTICA_REQUIRED: true
96+
SQITCH_TEST_VSQL_URI: db:vertica://dbadmin@localhost:${{ job.services.vertica.ports[5433] }}/VMart?Driver=Vertica
97+
run: prove -lrj4 t
98+
- name: Report Coverage
99+
env:
100+
PERL5LIB: "${{ github.workspace }}/local/lib/perl5"
101+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
102+
run: local/bin/cover -report coveralls

Diff for: ‎.github/workflows/exasol.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ jobs:
3535
- name: Cache CPAN Modules
3636
uses: actions/cache@v2
3737
with:
38-
path: local/lib
38+
path: local
3939
key: perl-${{ steps.perl.outputs.perl-hash }}
4040
- name: Download cpanfile
4141
uses: carlosperate/[email protected]

Diff for: ‎.github/workflows/firebird.yml

+1-3
Original file line numberDiff line numberDiff line change
@@ -39,19 +39,17 @@ jobs:
3939
- name: Cache CPAN Modules
4040
uses: actions/cache@v2
4141
with:
42-
path: local/lib
42+
path: local
4343
key: perl-${{ steps.perl.outputs.perl-hash }}
4444
- name: Download cpanfile
4545
uses: carlosperate/[email protected]
4646
with:
4747
file-url: https://fastapi.metacpan.org/source/DWHEELER/App-Sqitch-v1.1.0/dist/cpanfile
4848
- run: cpm install --verbose --show-build-log-on-failure --no-test --with-recommends --cpanfile cpanfile
4949
- run: cpm install --verbose --show-build-log-on-failure --no-test --with-recommends DBD::Firebird
50-
env: { FIREBIRD_HOME: /usr }
5150
- name: prove
5251
env:
5352
PERL5LIB: "${{ github.workspace }}/local/lib/perl5"
5453
LIVE_FIREBIRD_REQUIRED: true
55-
FIREBIRD_HOME: /usr
5654
SQITCH_TEST_FIREBIRD_URI: db:firebird://sysdba:[email protected]:${{ job.services.firebird.ports[3050] }}//firebird/data/sqitchtest.db
5755
run: prove -lvr t/firebird.t

Diff for: ‎.github/workflows/mysql.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ jobs:
4545
- name: Cache CPAN Modules
4646
uses: actions/cache@v2
4747
with:
48-
path: local/lib
48+
path: local
4949
key: perl-${{ steps.perl.outputs.perl-hash }}
5050
- name: Download cpanfile
5151
uses: carlosperate/[email protected]

Diff for: ‎.github/workflows/oracle.yml

+4-4
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,12 @@ jobs:
1515
# In 11g, APP_USER "sqitchtest" is created in XE, but in more recent
1616
# versions it is created in the XEPDB1 pluggable database, which we
1717
# cannot connect to using a URI. So we use an existing user in those
18-
# versions.
18+
# versions. Uncomment code in skip_unless in t/oracle.t to find other
19+
# user schemas that will work.
1920
# * Image Source: https://github.com/gvenzl/oci-oracle-xe/
2021
# * Image Issue: https://github.com/gvenzl/oci-oracle-xe/issues/46
2122
# * DBD::Oracle Issue: https://github.com/perl5-dbi/DBD-Oracle/issues/131
22-
# - { version: 21c, tag: 21-full, service: XE, altUser: gsmuser }
23+
- { version: 21c, tag: 21, service: XE, altUser: dbsnmp }
2324
- { version: 18c, tag: 18-slim, service: XE, altUser: gsmuser }
2425
- { version: 11g, tag: 11-slim, service: XE, altuser: sqitchtest }
2526
name: "🔮 Oracle ${{ matrix.version }}"
@@ -38,7 +39,6 @@ jobs:
3839
--health-timeout 10s
3940
--health-retries 10
4041
steps:
41-
- run: "echo Filter: ${{ needs.filter.outputs.oracle }}"
4242
- uses: actions/checkout@v2
4343
- name: Setup Clients
4444
run: .github/ubuntu/oracle.sh
@@ -49,7 +49,7 @@ jobs:
4949
- name: Cache CPAN Modules
5050
uses: actions/cache@v2
5151
with:
52-
path: local/lib
52+
path: local
5353
key: perl-${{ steps.perl.outputs.perl-hash }}
5454
- name: Download cpanfile
5555
uses: carlosperate/[email protected]

Diff for: ‎.github/workflows/os.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ jobs:
2424
- name: Cache CPAN Modules
2525
uses: actions/cache@v2
2626
with:
27-
path: local/lib
27+
path: local
2828
key: perl-${{ steps.perl.outputs.perl-hash }}
2929
- name: Download cpanfile
3030
uses: carlosperate/[email protected]
@@ -37,4 +37,4 @@ jobs:
3737
- run: cpm install --verbose --show-build-log-on-failure --no-test --with-recommends Test::Spelling Test::Pod Test::Pod::Coverage
3838
- name: prove
3939
env: { PERL5LIB: "${{ github.workspace }}/local/lib/perl5" }
40-
run: prove -lr --comments --directives
40+
run: prove -lrj4

Diff for: ‎.github/workflows/perl.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ jobs:
2828
- name: Cache CPAN Modules
2929
uses: actions/cache@v2
3030
with:
31-
path: local/lib
31+
path: local
3232
key: perl-${{ steps.perl.outputs.perl-hash }}
3333
- name: Download cpanfile
3434
uses: carlosperate/[email protected]
@@ -41,4 +41,4 @@ jobs:
4141
- run: cpm install --verbose --show-build-log-on-failure --no-test --with-recommends Test::Spelling Test::Pod Test::Pod::Coverage
4242
- name: prove
4343
env: { PERL5LIB: "${{ github.workspace }}/local/lib/perl5" }
44-
run: prove -lr --comments --directives
44+
run: prove -lrj4

Diff for: ‎.github/workflows/pg.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ jobs:
2323
- name: Cache CPAN Modules
2424
uses: actions/cache@v2
2525
with:
26-
path: local/lib
26+
path: local
2727
key: perl-${{ steps.perl.outputs.perl-hash }}
2828
- name: Download cpanfile
2929
uses: carlosperate/[email protected]

0 commit comments

Comments
 (0)