Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add new signing key #507

Merged
merged 3 commits into from
Jul 5, 2023
Merged

Add new signing key #507

merged 3 commits into from
Jul 5, 2023

Conversation

kenhys
Copy link
Contributor

@kenhys kenhys commented Jun 26, 2023 •
edited
Loading

  • signing key is generated with gpg --full-generate-key [1]
  • fix missing transitional package
    • It seems that transitional package must be placed with
      fluent-package. This is a bug of convert-artifacts-layout.sh.

[1]
RSA4096bit, Fluentd developers (Fluent Package Official Signing Key) [email protected].
Note that v5.0.0 must be singed with old key, then for next update,
we can use signing with the new key which is bundled with
fluentd-apt-source package.

NOTE: It seems that transitional package must be placed with
fluent-package. Without it, it causes the following error:

E: Failed to fetch
http://.../5/debian/bullseye/pool/contrib/f/fluent-package/td-agent_5.0.0-1_all.deb

Closes: #506

@kenhys
Copy link
Contributor Author

kenhys commented Jun 28, 2023 •
edited
Loading

Steps to verify

  1. import your new signing key
    gpg --import (decrypted txt file)

  2. build fluent-package deb packages (5.0.0)

  3. build fluentd-apt-source deb package (2023.6.29)

  4. pull current td-agent repository (e.g. td-agent-release directory)

  5. convert built apt/repositories to repository layout with convert-artifacts-layout.sh

    (cd f-p-b.repo/fluentd-apt-source/ && ./convert-artifacts-layout.sh deb)
(cd f-p-b.work/fluent-package && ./convert-artifacts-layout.sh deb)

  6. sync artifacts to repository

rsync -avz f-p-b.repo/fluentd-apt-source/artifacts/ f-p-b.work/fluent-package/artifacts/
rsync -avz f-p-b.work/fluent-package/artifacts/ td-agent-release/
  1. publish repository with manage-fluent-repositories.sh
(cd f-p-b.work/fluent-package && ./manage-fluent-repositories.sh deb /work/fluentd/fluent-package-builder/td-agent-release 5.0.0)

With new key,

  1. build fluent-package deb packages (5.0.1)

  2. convert built apt/repositories to repository layout with convert-artifacts-layout.sh

    (cd f-p-b.repo/fluent-package/ && ./convert-artifacts-layout.sh deb)

  3. sync repository

     rsync -avz f-p-b.repo/fluent-package/artifacts/ f-p-b.work/fluent-package/artifacts/
 rsync -avz f-p-b.work/fluent-package/artifacts/ td-agent-release/

  4. publish it.

    (cd f-p-b.work/fluent-package && ./manage-fluent-repositories-newkey.sh deb /work/fluentd/fluent-package-builder/td-agent-release 5.0.1)

@kenhys
Copy link
Contributor Author

kenhys commented Jun 28, 2023

exported file seems corrupted, checking why...

@kenhys
Copy link
Contributor Author

kenhys commented Jun 28, 2023

Testing repository may be broken. 🤔

Calculating upgrade... Done
The following package was automatically installed and is no longer required:
   td-agent (5.0.0-1)
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
   fluent-package (5.0.0-1)
   libncurses6 (6.2+20201114-2+deb11u1)
The following packages will be upgraded:
   fluentd-apt-source (2020.8.25-1 => 2023.6.29-1)
   td-agent (4.5.0-1 => 5.0.0-1)
2 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 10.8 MB of archives.
After this operation, 7321 kB disk space will be freed.
Do you want to continue? [Y/n] y
Err:1 http://192.168.11.2/5/debian/bullseye bullseye/contrib amd64 td-agent all 5.0.0-1
  404  Not Found [IP: 192.168.11.2 80]
Get:2 http://192.168.11.2/5/debian/bullseye bullseye/contrib amd64 fluent-package amd64 5.0.0-1 [10.7 MB]
Get:3 http://deb.debian.org/debian bullseye/main amd64 libncurses6 amd64 6.2+20201114-2+deb11u1 [102 kB]
Get:4 http://192.168.11.2/5/debian/bullseye bullseye/contrib amd64 fluentd-apt-source all 2023.6.29-1 [7796 B]
Fetched 10.8 MB in 0s (170 MB/s)             
E: Failed to fetch http://192.168.11.2/5/debian/bullseye/pool/contrib/f/fluent-package/td-agent_5.0.0-1_all.deb  404  Not Found [IP: 192.168.11.2 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

@kenhys kenhys force-pushed the add-new-signing-key branch from 90e16e0 to 316241f Compare June 28, 2023 07:18
@kenhys
Copy link
Contributor Author

kenhys commented Jun 28, 2023

Upgrade succeeds:

# apt upgrade -V
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following package was automatically installed and is no longer required:
   td-agent (5.0.0-1)
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
   fluent-package (5.0.0-1)
   libncurses6 (6.2+20201114-2+deb11u1)
The following packages will be upgraded:
   fluentd-apt-source (2020.8.25-1 => 2023.6.29-1)
   td-agent (4.5.0-1 => 5.0.0-1)
2 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 10.8 MB of archives.
After this operation, 7321 kB disk space will be freed.
Do you want to continue? [Y/n] y
Get:1 http://192.168.11.2/5/debian/bullseye bullseye/contrib amd64 td-agent all 5.0.0-1 [7268 B]
Get:2 http://192.168.11.2/5/debian/bullseye bullseye/contrib amd64 fluent-package amd64 5.0.0-1 [10.7 MB]
Get:3 http://deb.debian.org/debian bullseye/main amd64 libncurses6 amd64 6.2+20201114-2+deb11u1 [102 kB]
Get:4 http://192.168.11.2/5/debian/bullseye bullseye/contrib amd64 fluentd-apt-source all 2023.6.29-1 [7796 B]
...

Installed the new signing key:

 gpg --no-default-keyring --keyring /usr/share/keyrings/fluentd-archive-keyring.gpg --list-key
gpg: directory '/root/.gnupg' created
gpg: /root/.gnupg/trustdb.gpg: trustdb created
/usr/share/keyrings/fluentd-archive-keyring.gpg
-----------------------------------------------
pub   rsa4096 2016-12-27 [SC]
      BEE682289B2217F45AF4CC3F901F9177AB97ACBE
uid           [ unknown] Treasure Data, Inc (Treasure Agent Official Signing key) <[email protected]>
sub   rsa4096 2016-12-27 [E]

pub   rsa4096 2023-06-28 [SC]
      B40948B6A3B80E90F40E841F977D7A0943FA320E
uid           [ unknown] Fluentd developers (Fluent Package Official Signing Key) <[email protected]>
sub   rsa4096 2023-06-28 [E]

Minor upgrade succeeds with new key.

 apt upgrade -V
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following package was automatically installed and is no longer required:
   td-agent (5.0.1-1)
Use 'apt autoremove' to remove it.
The following packages will be upgraded:
   fluent-package (5.0.0-1 => 5.0.1-1)
   td-agent (5.0.0-1 => 5.0.1-1)
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 10.7 MB of archives.

@kenhys kenhys marked this pull request as ready for review June 28, 2023 07:22
@kenhys kenhys requested a review from daipom June 28, 2023 07:31
@kenhys kenhys force-pushed the add-new-signing-key branch 2 times, most recently from e3371cc to 803a49e Compare June 28, 2023 08:47
@daipom
Copy link
Contributor

daipom commented Jun 28, 2023

Note:

If the signature key is changed to a newer one in the future, it will no longer be possible to update directly to that version with the apt command.
This 5.0.0 version will make it possible to do so. (by importing the included new pub key).
If a user does not use 5.0.0 and updates directly to a future version, he/she can just run the installation script as he/she would for a new installation. (or he/she can set the key manually to the fluentd.sources).

@kenhys
Copy link
Contributor Author

kenhys commented Jun 28, 2023

TODO: fix CI.

@kenhys kenhys force-pushed the add-new-signing-key branch from 803a49e to 12ca559 Compare June 28, 2023 09:49
daipom
daipom reviewed Jun 28, 2023
View reviewed changes
Copy link
Contributor

@daipom daipom left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with this direction.

The building process of CI is failing, so we may still need to fix some points.

I will try to check the process of #507 (comment) when I have time.
If I am late, please merge this before I confirm it.

I'm a newbie in this area, so we may need to have @ashie check the direction.

@kenhys kenhys mentioned this pull request Jun 29, 2023
Merged
@kenhys kenhys force-pushed the add-new-signing-key branch 5 times, most recently from 52db64d to f17c398 Compare July 5, 2023 01:25
@kenhys
Copy link
Contributor Author

kenhys commented Jul 5, 2023

rebased with HEAD.

kenhys added 3 commits July 5, 2023 10:34
@kenhys
Add new signing key
1f673d9 
* signing key is generated with gpg --full-generate-key [1]
* fix missing transitional package
  * It seems that transitional package must be placed with
  fluent-package. This is a bug of convert-artifacts-layout.sh.

[1]
RSA4096bit, Fluentd developers (Fluent Package Official Signing Key) <[email protected]>.
Note that v5.0.0 must be singed with old key, then for next update,
we can use signing with the new key which is bundled with
fluentd-apt-source package.

NOTE: It seems that transitional package must be placed with
fluent-package. Without it, it causes the following error:

  E: Failed to fetch
  http://.../5/debian/bullseye/pool/contrib/f/fluent-package/td-agent_5.0.0-1_all.deb

Closes: fluent#506

Signed-off-by: Kentaro Hayashi <[email protected]>
@kenhys
deb: fix missing transitional package
997071d 
It seems that transitional package must be placed with
fluent-package.

It fixes the following error:

  E: Failed to fetch
  http://.../5/debian/bullseye/pool/contrib/f/fluent-package/td-agent_5.0.0-1_all.deb

Signed-off-by: Kentaro Hayashi <[email protected]>
@kenhys
deb: add v4 apt-line
4dc0c38 
Signed-off-by: Kentaro Hayashi <[email protected]>
@kenhys kenhys force-pushed the add-new-signing-key branch from f17c398 to 4dc0c38 Compare July 5, 2023 01:35
@kenhys
Copy link
Contributor Author

kenhys commented Jul 5, 2023

Updated commit message a bit.

@kenhys kenhys merged commit 32aa2ab into fluent:master Jul 5, 2023
@kenhys kenhys deleted the add-new-signing-key branch July 5, 2023 02:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@daipom daipom daipom left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Migrate package signing key
2 participants
@kenhys @daipom