Skip to content

Commit e7a6fcf

Browse files
kgrubbdanirabbit
kgrubb
authored and
danirabbit
committed
Add build system (elementary#231)
* add build system * fix vagrant build to default to daily instead of stable builds * allow version tagging and channel definitions in workflows * fix invalid syntax * Try fixing stable workflow * Update daily builder for testing * mark workflows.sh as executable * add sudo to github workflow build task * update README * get info on working directory and directory contents in pipeline * ignore build dir output * try manually building syslinux theme for now * fix invalid cd opt * fix path issue, add syslinux-common package requirement * try to upload after build completes * use boto for file uploads * set secrets in yml config * fix typo 🙊 * update daily workflow to run on branches with 'build' prefix. Update stable to include upload endpoints * use channel for upload path. rename shasum and md5 filename with build date to make visual grepping easier. * set acl for objects. run another build to test new folder/path policies * use boto3 acl setup instead of boto2 * include extra arg in upload to set public read access * Remove un-needed workaround hooks. Use git instead of bzr for seed fetching. Link initrd.lz to initrd to try and fix EFI booting. Remove unused desltop list packages. Upgrade to hwe packages. * point to initrd.lz instead of initrd in grub config * trigger another build * rename steps for badges to work. make daily builds run on merge to master
1 parent 850f898 commit e7a6fcf

31 files changed

+1510
-2
lines changed

‎.github/workflows/daily.yml

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
name: daily
2+
3+
on:
4+
push:
5+
branches:
6+
- master
7+
schedule:
8+
- cron: '0 0 * * *'
9+
10+
jobs:
11+
build:
12+
runs-on: ubuntu-latest
13+
steps:
14+
- name: Clone
15+
uses: actions/checkout@v1
16+
17+
- name: Build
18+
run: sudo ./workflows.sh daily 5.0 "${{ secrets.key }}" "${{ secrets.secret }}" "${{ secrets.endpoint }}" "${{ secrets.bucket }}"

‎.github/workflows/stable.yml

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
name: stable
2+
3+
on:
4+
push:
5+
tags:
6+
- '*'
7+
8+
jobs:
9+
build:
10+
runs-on: ubuntu-latest
11+
steps:
12+
- name: Clone
13+
uses: actions/checkout@v1
14+
15+
- name: Build
16+
run: sudo ./workflows.sh stable "$GITHUB_REF" "${{ secrets.key }}" "${{ secrets.secret }}" "${{ secrets.endpoint }}" "${{ secrets.bucket }}"
17+

‎.gitignore

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
.vagrant
2+
ubuntu-*.log
3+
.DS_Store
4+
tmp
5+
builds

‎LICENSE

+674
Large diffs are not rendered by default.

‎README.md

+49-2
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,50 @@
1-
# os
1+
<div align="center">
2+
<a href="https://elementary.io" align="center">
3+
<center align="center">
4+
<img src="https://raw.githubusercontent.com/elementary/brand/master/logomark-black.png" alt="elementary" align="center">
5+
</center>
6+
</a>
7+
<br>
8+
<h1 align="center"><center>elementary OS</center></h1>
9+
<h3 align="center"><center>Build scripts for image creation</center></h3>
10+
<br>
11+
<br>
12+
</div>
213

3-
Dummy repo to track packaging and configuration issues. If in doubt, report issues here and we can transfer them to the correct repo.
14+
<p align="center">
15+
<img src="https://github.com/elementary/os/workflows/stable/badge.svg" alt="Stable">
16+
<img src="https://github.com/elementary/os/workflows/daily/badge.svg" alt="Daily">
17+
</p>
18+
19+
---
20+
21+
## Building Locally
22+
23+
To build an elementary OS image you'll need to:
24+
25+
1) Make sure you have the following dependencies installed:
26+
27+
* `dctrl-tools`
28+
* `dpkg-dev`
29+
* `genisoimage`
30+
* `gfxboot-theme-ubuntu`
31+
* `live-build`
32+
* `squashfs-tools`
33+
* `syslinux`
34+
* `zsync`
35+
36+
2) Clone this project & `cd` into it:
37+
38+
```
39+
git clone https://github.com/elementary/os && cd os
40+
```
41+
42+
3) Configure the channel in the `etc/terraform.conf` (stable, daily).
43+
44+
4) Run the build script as a root user:
45+
46+
```
47+
./terraform.sh
48+
```
49+
50+
5) When done, your images will be in the builds folder.

‎Vagrantfile

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#!/usr/bin/env ruby
2+
3+
VAGRANTFILE_API_VERSION = '2'.freeze
4+
Vagrant.configure('2') do |config|
5+
config.vm.box = 'ubuntu/bionic64'
6+
config.disksize.size = '16GB'
7+
config.vm.define 'elementary-builder'
8+
config.vm.provision 'shell', path: 'vagrant.sh'
9+
end

‎etc/auto/clean

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
#! /bin/sh
2+
3+
set -e
4+
5+
lb clean noauto "${@}"
6+
7+
rm -f config/binary config/bootstrap config/chroot config/common config/source
8+
rm -f binary.manifest binary.manifest-desktop binary.log binary.iso.md5
9+
rm -f livecd.*
10+
rm -rf gnupg pool userdata

‎etc/auto/config

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
#! /bin/sh
2+
set -e
3+
4+
. ./terraform.conf
5+
6+
HOOKS="update-apt-file-cache update-apt-xapian-index update-mlocate-database remove-dbus-machine-id remove-openssh-server-host-keys remove-udev-persistent-rules"
7+
8+
lb config noauto \
9+
--architecture "$ARCH" \
10+
--binary-images iso-hybrid \
11+
--bootloader syslinux \
12+
--keyring-packages ubuntu-keyring \
13+
--build-with-chroot false \
14+
--distribution "$BASECODENAME" \
15+
--hooks "$HOOKS" \
16+
--iso-application "$NAME" \
17+
--iso-volume "$NAME" \
18+
--memtest none \
19+
--mode ubuntu \
20+
--parent-archive-areas "main restricted universe multiverse" \
21+
--source false \
22+
--syslinux-theme elementary-@SYSLINUX \
23+
--zsync false
24+
25+
# replace channel and suite
26+
sed -i "s/@CHANNEL/$CHANNEL/" config/archives/*.list
27+
sed -i "s/@BASECODENAME/$BASECODENAME/" config/archives/*.list
28+
29+
mkdir -p config/includes.chroot/boot/efi
30+
if [ "$ARCH" = "amd64" ]; then
31+
echo "fs0:\EFI\ubuntu\grubx64.efi" > config/includes.chroot/boot/efi/startup.nsh
32+
elif [ "$ARCH" = "i386" ]; then
33+
echo "fs0:\EFI\ubuntu\grubx32.efi" > config/includes.chroot/boot/efi/startup.nsh
34+
fi

‎etc/config/archives/appcenter.key

+30
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
-----BEGIN PGP PUBLIC KEY BLOCK-----
2+
Version: GnuPG v1
3+
4+
mQENBFcRRNIBCADrEnIR91b2vX6BrAAYqDRTKZeFdMFG7StSqFo3lZH2KFd7KBeB
5+
xYWnU3V/xJQ+1lFujo+na6z7L9qBrtikYdIn9Mc3eSyw/fli1FkMuoFmdMtj6k1F
6+
z+NEM0dSrELavBMPzXb4kjF8ZGKneR35v4Y3RP+lvZDiumZpvFzTzU0ARURkxH3N
7+
bYiX0mki75gy+EHwCReSPjBxK94+sQRoClJdrQbFaHtXDqfh2XVQMm2NnmJaf2Rb
8+
MFo75zEoxHzYbnByWdR/Xh6uhlSyDaBY0O0f1c3ycGn7UT+7CAdGka6Gu2VZUXEH
9+
GLZS4otVhtoyga+kGaiJ7SYRSQzd0S/xPPxtABEBAAG0KGVsZW1lbnRhcnkgYnVp
10+
bGRzIDxidWlsZHNAZWxlbWVudGFyeS5pbz6JATcEEwEJACEFAlcRRNICGwMFCwkI
11+
BwMFFQkKCAsFFgMCAQACHgECF4AACgkQp09z7/5wuRzfwQf/QeGtdlQiV3HJC/hF
12+
yQ4CHOxZJFrpe/Ud/tazkUiplc2Y/mrretgdGhIoJ6NpQWpDTCLFmvD5CEq9lets
13+
eRUNULw21utniuTrrDXZe1tom0FkFSdyCY8UpDDOc7LkzbQ8wKo1ASkyyEj8fVXL
14+
EoOkbsjRQJUXS/6SdoRSeViutDewjTy0YvelUMVxcX5H5TjJfXmZMGUxtGUef3Ra
15+
RDXR69kmFjf7Umv2AiiVsendqrfHRLnZQRsbblNAlNl5J+VTlKkvcwXivWewlVUa
16+
OL8HPFo9d2kxIbexG2yLwnuwmfavSFJV4NYemSPSKl6lJVbI3P+zz+U87VOMr8Kf
17+
uL0Do7kBDQRXEUTSAQgAvA3oZ7rN1iE8M3Q91HW37lBV+EtagirJUa6+m2ovPfa6
18+
TnOPsygSY8EiMPKB8rDxVRk32XnzwNl3MCmW0YjN1S2sdFv8YnPXLzXKJWYqTaCN
19+
dgJPhFz+LdgPmuW2kHH3ENZ8kOBKeVJdQ+UCxZkEdJZVhnsIP60pHpS2l8oMGZYk
20+
wtRpe6BDtJMlpgUVWK1jJrf/Uj/JA62+rpvAl27TxdZgpsmYNaOpg9gumV8aGtUk
21+
4l6X0NjSkKaka7m2BULAHotYOAK18D0z7wsslGbSfCYKNKsXKoUbX9fpxjxPUprM
22+
HQJ7NAefDgfieDImFmBpcSXEbznUzt5vgSKyHGNsuwARAQABiQEfBBgBCQAJBQJX
23+
EUTSAhsMAAoJEKdPc+/+cLkceJAH/0XPU2fz+aiph4ltt9nvdkY/iqFJ4/ng2pYj
24+
z9K9e8JWu65EBBkUn3J80cVwMTcCvZduDyMHizobPilHUyWAdwXtLXgLTY5TRfkd
25+
HQTKVhFqq/LrprUBRoAJQfTbL6XdCX8Dy7q899yaiHsBJd1IzEBiKpfTOc1RmVt/
26+
tvaG0+WhBRe/aHu5wnKkfExLdsW8mDrNUnDykTj50Cki1kfFvLWlSLz6wlXyqG1U
27+
1aB+nEM7d1/r/joFsoYft2TTscNi0HvxY8U1mNSD5H7IXK1ecGT53fSclPPkrVd5
28+
+F4YBpd9FIy7LBnLhfK0F4aBe7iTLPiYJ7TUpIez3k7ubRGGBZg=
29+
=ICrR
30+
-----END PGP PUBLIC KEY BLOCK-----

‎etc/config/archives/appcenter.list

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
#deb http://packages.elementary.io/appcenter @BASECODENAME main

‎etc/config/archives/elementary.key

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
-----BEGIN PGP PUBLIC KEY BLOCK-----
2+
Version: SKS 1.0.10
3+
4+
mI0ETocPBAEEANWiT0DBP5fpz2TB5kRG+SD2NUGVmFincQqxlHr2226gHwlObcNJKWWVt+F3
5+
3iORIMb85KXc5lihmdwu4WLRaLrmThEApMDD1uQGVgUrTA1QcJO+ma9+uph8OoLsiyw1YFOk
6+
lBRudx4RG+vBce/QKwNVPO/bKOYHVcFbuFQ74MfhABEBAAG0JExhdW5jaHBhZCBQUEEgZm9y
7+
IGVsZW1lbnRhcnkgT1MgdGVhbYi4BBMBAgAiBQJOhw8EAhsDBgsJCAcDAgYVCAIJCgsEFgID
8+
AQIeAQIXgAAKCRC/NplsTh+KWdvoA/4kAGaOfHqUzuDbzl6eEmPeskbHXUC9jFlnw7SCx6TC
9+
5/+IOZkT+71rte/R9VnLkJ4gy2bcdJATop83VTOt4u9A8FJUwLO9YGjaa0WN75r6vSHAYPEj
10+
XGgizlksadNiafQSS41B+/kGoGi93GPsXUn4lF/VCfyeHeBmzDVSZ/h37Q==
11+
=CwtG
12+
-----END PGP PUBLIC KEY BLOCK-----

‎etc/config/archives/elementary.list

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
deb http://ppa.launchpad.net/elementary-os/@CHANNEL/ubuntu @BASECODENAME main
2+
deb-src http://ppa.launchpad.net/elementary-os/@CHANNEL/ubuntu @BASECODENAME main

‎etc/config/archives/patches.key

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
-----BEGIN PGP PUBLIC KEY BLOCK-----
2+
Version: SKS 1.0.10
3+
4+
mI0ETocPBAEEANWiT0DBP5fpz2TB5kRG+SD2NUGVmFincQqxlHr2226gHwlObcNJKWWVt+F3
5+
3iORIMb85KXc5lihmdwu4WLRaLrmThEApMDD1uQGVgUrTA1QcJO+ma9+uph8OoLsiyw1YFOk
6+
lBRudx4RG+vBce/QKwNVPO/bKOYHVcFbuFQ74MfhABEBAAG0JExhdW5jaHBhZCBQUEEgZm9y
7+
IGVsZW1lbnRhcnkgT1MgdGVhbYi4BBMBAgAiBQJOhw8EAhsDBgsJCAcDAgYVCAIJCgsEFgID
8+
AQIeAQIXgAAKCRC/NplsTh+KWdvoA/4kAGaOfHqUzuDbzl6eEmPeskbHXUC9jFlnw7SCx6TC
9+
5/+IOZkT+71rte/R9VnLkJ4gy2bcdJATop83VTOt4u9A8FJUwLO9YGjaa0WN75r6vSHAYPEj
10+
XGgizlksadNiafQSS41B+/kGoGi93GPsXUn4lF/VCfyeHeBmzDVSZ/h37Q==
11+
=CwtG
12+
-----END PGP PUBLIC KEY BLOCK-----

‎etc/config/archives/patches.list

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
deb http://ppa.launchpad.net/elementary-os/os-patches/ubuntu @BASECODENAME main
2+
deb-src http://ppa.launchpad.net/elementary-os/os-patches/ubuntu @BASECODENAME main

‎etc/config/hooks/iso.binary

+50
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
#!/bin/bash
2+
3+
echo "P: Begin executing iso binary hook..."
4+
5+
source ./terraform.conf
6+
7+
# mount ubuntu's iso
8+
mkdir -p mnt
9+
mount -o loop,ro ubuntu.iso mnt
10+
11+
# copy everything over except the rootfs
12+
rsync \
13+
-a \
14+
--exclude=/.disk/info \
15+
--exclude=/autorun.inf \
16+
--exclude=/casper/* \
17+
--exclude=/dists \
18+
--exclude=/install \
19+
--exclude=/isolinux \
20+
--exclude=/md5sum.txt \
21+
--exclude=/pics \
22+
--exclude=/pool \
23+
--exclude=/preseed \
24+
--exclude=/ubuntu \
25+
--exclude=/wubi.exe \
26+
mnt/ \
27+
binary
28+
29+
# unmount ubuntu's iso and delete mount point dir
30+
umount mnt
31+
rm -rf mnt
32+
33+
# replace their init with ours
34+
rm -rf binary/casper/vmlinuz-*
35+
rm -rf binary/casper/initrd.*
36+
cp chroot/boot/$(ls -1 chroot/boot | grep ^vmlinuz | sort --version-sort | tail -n 1) binary/casper/vmlinuz
37+
cp chroot/boot/$(ls -1 chroot/boot | grep ^initrd | sort --version-sort | tail -n 1) binary/casper/initrd.lz
38+
39+
# customization
40+
echo "https://elementary.io/$CODENAME" > binary/.disk/release_notes_url
41+
sed -i "s/Ubuntu/$NAME/" binary/boot/grub/loopback.cfg
42+
sed -i "s/vmlinuz.efi/vmlinuz/" binary/boot/grub/loopback.cfg
43+
sed -i "s/\\/casper\\/initrd/\\/casper\\/initrd.lz/" binary/boot/grub/grub.cfg
44+
sed -i "s/Ubuntu/$NAME/" binary/boot/grub/grub.cfg
45+
sed -i "s/vmlinuz.efi/vmlinuz/" binary/boot/grub/grub.cfg
46+
sed -i "s/Ubuntu/$NAME/g" binary/isolinux/*.tr # fix translations
47+
sed -i "s/Ubuntu/$NAME/" binary/README.diskdefines
48+
sed -i "s/$BASEVERSION/$VERSION/" binary/README.diskdefines
49+
sed -i "s/LTS//" binary/README.diskdefines
50+
sed -i "s/\".*\"/\"$CODENAME\"/" binary/README.diskdefines

‎etc/config/hooks/remove-blacklisted-packages.chroot

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
#!/bin/sh
2+
# Description: Checkout seed branches and remove blacklisted packages
3+
4+
echo "P: Begin executing remove-blacklisted-packages chroot hook..."
5+
6+
dist="$(lsb_release -c -s -u 2>&1)"||dist="$(lsb_release -c -s)"
7+
8+
apt-get install --no-install-recommends -f -q -y git
9+
10+
git clone --depth 1 https://github.com/elementary/seeds.git --single-branch --branch "$dist"
11+
git clone --depth 1 https://github.com/elementary/platform.git --single-branch --branch "$dist"
12+
13+
for package in $(cat 'platform/blacklist' 'seeds/blacklist' | grep -v '#'); do
14+
apt-get autoremove --purge -f -q -y "$package"
15+
done
16+
17+
apt-get autoremove --purge -f -q -y git
18+
19+
rm -R ../seeds ../platform

‎etc/config/hooks/remove-extra-kernels.chroot

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
#!/bin/bash
2+
3+
echo "P: Begin executing remove-extra-kernels chroot hook..."
4+
5+
if [ $(ls -1 /boot | sort --version-sort | grep ^vmlinuz | sed -e "s/vmlinuz-//g" | sed -e "s/\.efi\.signed//g" | head -n 1) != $(ls -1 /boot | sort --version-sort | grep ^vmlinuz | sed -e "s/vmlinuz-//g" | sed -e "s/\.efi\.signed//g" | tail -n 1) ]
6+
then
7+
mv /etc/kernel/postrm.d/zz-update-grub /tmp/backup-update-grub
8+
apt-get -q -f -y autoremove --purge linux-headers-$(ls -1 /boot | sort --version-sort | grep ^vmlinuz | sed -e "s/vmlinuz-//g" | sed -e "s/\.efi\.signed//g" | head -n 1) || true
9+
mv /tmp/backup-update-grub /etc/kernel/postrm.d/zz-update-grub
10+
fi
11+
12+
if [ $(ls -1 /boot | sort --version-sort | grep ^vmlinuz | sed -e "s/vmlinuz-//g" | sed -e "s/\.efi\.signed//g" | head -n 1) != $(ls -1 /boot | sort --version-sort | grep ^vmlinuz | sed -e "s/vmlinuz-//g" | sed -e "s/\.efi\.signed//g" | tail -n 1) ]
13+
then
14+
mv /etc/kernel/postrm.d/zz-update-grub /tmp/backup-update-grub
15+
apt-get -q -f -y autoremove --purge linux-image-$(ls -1 /boot | sort --version-sort | grep ^vmlinuz | sed -e "s/vmlinuz-//g" | sed -e "s/\.efi\.signed//g" | head -n 1) || true
16+
mv /tmp/backup-update-grub /etc/kernel/postrm.d/zz-update-grub
17+
fi

‎etc/config/hooks/set-disk-info.binary

+31
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
#!/usr/bin/python2
2+
import datetime
3+
import os
4+
5+
info = 'elementary OS '
6+
7+
cf = open("terraform.conf", "r")
8+
config = cf.read()
9+
cf.close()
10+
11+
version = config.split('\nVERSION="')[1]
12+
version = version.split('"')[0]
13+
14+
codename = config.split('\nCODENAME="')[1]
15+
codename = codename.split('"')[0]
16+
codename = codename.capitalize()
17+
18+
channel = config.split('CHANNEL="')[1]
19+
channel = channel.split('"')[0]
20+
channel = channel.capitalize()
21+
22+
arch = config.split('ARCH="')[1]
23+
arch = arch.split('"')[0]
24+
25+
date = datetime.datetime.now().strftime("%Y%m%d")
26+
27+
info = info + version + " \"" + codename + "\" - " + channel + " " + arch + " (" + date + ")\n"
28+
29+
f = open("binary/.disk/info", "w")
30+
f.write(info)
31+
f.close()

‎etc/config/includes.chroot/etc/default/locale

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
LANG=en_US.UTF-8

‎etc/config/includes.chroot/etc/hosts

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
127.0.0.1 localhost.localdomain localhost
2+
::1 localhost6.localdomain6 localhost6
3+
4+
# The following lines are desirable for IPv6 capable hosts
5+
::1 localhost ip6-localhost ip6-loopback
6+
fe00::0 ip6-localnet
7+
ff02::1 ip6-allnodes
8+
ff02::2 ip6-allrouters
9+
ff02::3 ip6-allhosts

‎etc/config/includes.chroot/etc/netplan/01-network-manager-all.yaml

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# Let NetworkManager manage all devices on this system
2+
network:
3+
version: 2
4+
renderer: NetworkManager

‎etc/config/includes.chroot/etc/resolv.conf

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
nameserver 8.8.8.8

‎etc/config/package-lists/desktop.list.chroot_install

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# elementary meta packages
2+
elementary-desktop
3+
elementary-minimal
4+
elementary-standard
5+
# delete the following someday maybe
6+
linux-generic
7+
xserver-xorg
8+
xserver-xorg-core
9+
xserver-xorg-input-all
10+
xserver-xorg-input-evdev
11+
xserver-xorg-input-libinput
12+
xserver-xorg-input-wacom

‎etc/config/package-lists/desktop.list.chroot_live

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
#if ARCHITECTURES amd64
2+
linux-signed-generic-hwe-18.04
3+
linux-signed-image-generic-hwe-18.04
4+
#endif
5+
6+
bcmwl-kernel-source
7+
#distinst
8+
#libdistinst
9+
cifs-utils
10+
gparted
11+
lupin-casper
12+
ubiquity-frontend-gtk
13+
ubiquity-slideshow-elementary
14+
ubiquity-ubuntu-artwork
15+
xfsprogs

0 commit comments

Comments
 (0)