migrate to new user/group name #475

kenhys · 2023-05-22T08:16:38Z

for deb, according Debian policy, dynamically allocated name should be prefixed
with an underscore.
(It may be not applicable for group name, but there is an prefixed
group name example e.g. _ssh/_cvsadmin)

See
https://www.debian.org/doc/debian-policy/ch-opersys.html#introduction

Note that adduser doesn't accept _name prefix by default. To avoid
it, we need to specify --force-badname explicitly

migration from to td-agent to _fluentd
- service, hook script, tmpfiles.d and so on.
- cleanup td-agent statoverride

For rpm, no prefixed underscore for user/group name.

migration from td-agent to fluentd
- service, hook script, tmpfiles.d, permission and so on.
- cleanup user/group on removing package

kenhys · 2023-05-22T08:17:51Z

check whether it still need to keep compat_package_dir in fixperms.

kenhys · 2023-05-23T02:01:03Z

This PR is focus on deb only.

kenhys · 2023-05-23T02:01:54Z

UID/GID migration was verified with:

++ grep -c ID=debian /etc/os-release
+ '[' 1 -eq 1 ']'
+ case $1 in
+ sudo apt install -y ./td-agent_4.5.0-1_amd64.deb
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'td-agent' instead of './td-agent_4.5.0-1_amd64.deb'
The following NEW packages will be installed:
  td-agent
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/10.7 MB of archives.
After this operation, 65.0 MB of additional disk space will be used.
Get:1 /home/vagrant/td-agent_4.5.0-1_amd64.deb td-agent amd64 4.5.0-1 [10.7 MB]
Selecting previously unselected package td-agent.
(Reading database ... 49514 files and directories currently installed.)
Preparing to unpack .../td-agent_4.5.0-1_amd64.deb ...
Unpacking td-agent (4.5.0-1) ...
Setting up td-agent (4.5.0-1) ...
Adding system user `td-agent' (UID 108) ...
Adding new group `td-agent' (GID 117) ...
Adding new user `td-agent' (UID 108) with group `td-agent' ...
Creating home directory `/var/lib/td-agent' ...
Created symlink /etc/systemd/system/multi-user.target.wants/td-agent.service → /lib/systemd/system/td-agent.service.
Processing triggers for man-db (2.9.4-2) ...
Processing triggers for libc-bin (2.31-13+deb11u5) ...
+ echo 'CHECK: td-agent/td-agent'
CHECK: td-agent/td-agent
+ id -u td-agent
108
+ id -g td-agent
117
+ systemctl is-active td-agent
active
+ '[' 0 -ne 0 ']'
+ n=1
+ set +x
./test-install.sh: line 39: 1: command not found
Try to echo with fluent-cat
Try to echo with /opt/td-agent/bin/fluent-cat
./test-install.sh: line 38: [: -lt: unary operator expected
+ sudo apt install -y ./fluent-package_5.0.0-1_amd64.deb
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'fluent-package' instead of './fluent-package_5.0.0-1_amd64.deb'
The following packages will be REMOVED:
  td-agent
The following NEW packages will be installed:
  fluent-package
0 upgraded, 1 newly installed, 1 to remove and 0 not upgraded.
Need to get 0 B/12.4 MB of archives.
After this operation, 7,581 kB of additional disk space will be used.
Get:1 /home/vagrant/fluent-package_5.0.0-1_amd64.deb fluent-package amd64 5.0.0-1 [12.4 MB]
(Reading database ... 58202 files and directories currently installed.)
Removing td-agent (4.5.0-1) ...
Selecting previously unselected package fluent-package.
(Reading database ... 49519 files and directories currently installed.)
Preparing to unpack .../fluent-package_5.0.0-1_amd64.deb ...
Unpacking fluent-package (5.0.0-1) ...
Setting up fluent-package (5.0.0-1) ...
Installing new version of config file /etc/logrotate.d/td-agent ...
Installing new version of config file /etc/td-agent/td-agent.conf ...
fluentd.conf:1: Failed to resolve user 'fluentd': No such process
fluentd.conf:2: Failed to resolve user 'fluentd': No such process
Failed to preset unit: File /etc/systemd/system/td-agent.service already exists and is a symlink to /dev/null.
/usr/bin/deb-systemd-helper: error: systemctl preset failed on fluentd.service: No such file or directory
Processing triggers for man-db (2.9.4-2) ...
Processing triggers for libc-bin (2.31-13+deb11u5) ...
+ systemctl is-active td-agent
inactive
+ systemctl is-active fluentd
active
+ echo 'CHECK: _fluentd/_fluentd'
CHECK: _fluentd/_fluentd
+ id -u _fluentd
108
+ id -g _fluentd
117

kenhys · 2023-05-23T02:21:01Z

Checking failure.

kenhys · 2023-05-23T05:00:23Z

Unexpectedly exception was raised during install fluentd-apt-source. 🤔

dpkg: unrecoverable fatal error, aborting:
 unknown system user '_fluentd' in statoverride file; the system user got removed

kenhys · 2023-05-24T06:07:57Z

Fixing migration failure with already starting service.

ashie · 2023-05-26T05:18:14Z

Still checking...

kenhys · 2023-05-29T07:41:26Z

just rebased with recent master.

fluent-package/templates/package-scripts/fluent-package/deb/postinst

fluent-package/templates/usr/lib/tmpfiles.d/fluentd.conf

fluent-package/templates/package-scripts/fluent-package/deb/postinst

kenhys · 2023-06-02T09:20:33Z

It requires #492 for Windows CI.

fluent-package/templates/package-scripts/fluent-package/deb/postinst

kenhys · 2023-06-05T06:06:30Z

Rebased because #492 was merged.

kenhys · 2023-06-05T06:45:20Z

/var/run/td-agent statoveride configuration was removed.

Note, selecting 'fluent-package' instead of './fluent-package_5.0.0-1_amd64.deb'
The following packages will be REMOVED:
  td-agent
The following NEW packages will be installed:
  fluent-package
0 upgraded, 1 newly installed, 1 to remove and 0 not upgraded.
Need to get 0 B/12.4 MB of archives.
After this operation, 7,585 kB of additional disk space will be used.
Get:1 /home/vagrant/fluent-package_5.0.0-1_amd64.deb fluent-package amd64 5.0.0-1 [12.4 MB]
(Reading database ... 65954 files and directories currently installed.)
Removing td-agent (4.5.0-1) ...
Selecting previously unselected package fluent-package.
(Reading database ... 57271 files and directories currently installed.)
Preparing to unpack .../fluent-package_5.0.0-1_amd64.deb ...
Unpacking fluent-package (5.0.0-1) ...
Setting up fluent-package (5.0.0-1) ...
Installing new version of config file /etc/logrotate.d/td-agent ...
Installing new version of config file /etc/td-agent/td-agent.conf ...
dpkg-statoverride: warning: an override for '/var/run/fluent' already exists, but --force specified so will be ignored
dpkg-statoverride: warning: an override for '/etc/td-agent' already exists, but --force specified so will be ignored
dpkg-statoverride: warning: an override for '/var/log/td-agent' already exists, but --force specified so will be ignored
Failed to preset unit: File /etc/systemd/system/td-agent.service already exists and is a symlink to /dev/null.
/usr/bin/deb-systemd-helper: error: systemctl preset failed on fluentd.service: No such file or directory
Processing triggers for man-db (2.9.4-2) ...
Processing triggers for libc-bin (2.31-13+deb11u5) ...
+ '[' 0 -ne 0 ']'
+ systemctl is-enabled td-agent
masked
+ systemctl is-enabled fluentd
enabled
+ systemctl is-active td-agent
inactive
+ systemctl is-active fluentd
active
+ echo 'CHECK: _fluentd/_fluentd'
CHECK: _fluentd/_fluentd
+ id -u _fluentd
108
+ id -g _fluentd
117
+ sudo dpkg-statoverride --list
root postdrop 2555 /usr/sbin/postqueue
postfix postdrop 2710 /var/spool/postfix/public
root crontab 2755 /usr/bin/crontab
root mlocate 2755 /usr/bin/mlocate
root ssl-cert 710 /etc/ssl/private
_fluentd _fluentd 755 /var/log/td-agent
_fluentd _fluentd 755 /var/run/fluent
root messagebus 4754 /usr/lib/dbus-1.0/dbus-daemon-launch-helper
_fluentd _fluentd 755 /etc/td-agent
root postdrop 2555 /usr/sbin/postdrop

kenhys · 2023-06-05T06:52:06Z

NOTE need to fix CI.

kenhys · 2023-06-05T08:39:21Z

Fixed remaining old statoverride for /var/run/td-agent.

ashie · 2023-06-06T05:39:08Z

For debian & ubuntu, I confirmed all issues that I found have been fixed. Thanks!
For RPM, I've found some issue:

When updating from v4, owner of /var/log/td-agent and so on are changed to root:

$ sudo LANG=C rpm -Uvh fluent-package-5.0.0-1.el7.x86_64.rpm 
Preparing...                          ################################# [100%]
Migrate group td-agent to fluentd...
failed
Migrate user td-agent to fluentd...
Updating / installing...
   1:fluent-package-5.0.0-1.el7       warning: user td-agent does not exist - using root
warning: group td-agent does not exist - using root
warning: user td-agent does not exist - using root
warning: group td-agent does not exist - using root
################################# [ 50%]
warning: user td-agent does not exist - using root
warning: group td-agent does not exist - using root
warning: user td-agent does not exist - using root
warning: group td-agent does not exist - using root
warning: user td-agent does not exist - using root
warning: group td-agent does not exist - using root
failed
prelink detected. checking /etc/prelink.conf.d/ ...
Removing prelink configuration for td-agent (/etc/prelink.conf.d/td-agent-ruby.conf) ...
Removed symlink /etc/systemd/system/multi-user.target.wants/td-agent.service.
Cleaning up / removing...
   2:td-agent-4.4.2-1.el7             ################################# [100%]

When I remove fluent-pacakge, the user fluentd is still remained.
- It might be originate from td-agent. Removing td-agent also remains td-agent user.

kenhys · 2023-06-06T05:50:53Z

When updating from v4, owner of /var/log/td-agent and so on are changed to root:

I'll check again.

for deb, according Debian policy, dynamically allocated name should be prefixed with an underscore. (It may be not applicable for group name, but there is an prefixed group name example e.g. _ssh/_cvsadmin) See https://www.debian.org/doc/debian-policy/ch-opersys.html#introduction Note that adduser doesn't accept _name prefix by default. To avoid it, we need to specify --force-badname explicitly * migration from to td-agent to _fluentd * service, hook script, tmpfiles.d and so on. * cleanup td-agent statoverride For rpm, no prefixed underscore for user/group name. * migration from td-agent to fluentd * service, hook script, tmpfiles.d, permission and so on. * cleanup user/group on removing package Signed-off-by: Kentaro Hayashi <[email protected]>

Signed-off-by: Kentaro Hayashi <[email protected]>

kenhys · 2023-06-06T08:32:48Z

Fixed remained user/group issue.
_

kenhys · 2023-06-06T08:32:55Z

waiting CI

kenhys · 2023-06-06T09:45:08Z

Upgrade from v4:

---> Package fluent-package.x86_64 0:5.0.0-1.el7 will be obsoleting
---> Package td-agent.x86_64 0:4.5.0-1.el7 will be obsoleted
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================
 Package                            Arch                       Version                           Repository                                              Size
==============================================================================================================================================================
Installing:
 fluent-package                     x86_64                     5.0.0-1.el7                       /fluent-package-5.0.0-1.el7.x86_64                      64 M
     replacing  td-agent.x86_64 4.5.0-1.el7

Transaction Summary
==============================================================================================================================================================
Install  1 Package

Total size: 64 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Migrate group td-agent to fluentd...
Migrate user td-agent to fluentd...
  Installing : fluent-package-5.0.0-1.el7.x86_64                                                                                                          1/2 
prelink detected. checking /etc/prelink.conf.d/ ...
Removing prelink configuration for td-agent (/etc/prelink.conf.d/td-agent-ruby.conf) ...
Removed symlink /etc/systemd/system/multi-user.target.wants/td-agent.service.
  Erasing    : td-agent-4.5.0-1.el7.x86_64                                                                                                                2/2 
  Verifying  : fluent-package-5.0.0-1.el7.x86_64                                                                                                          1/2 
  Verifying  : td-agent-4.5.0-1.el7.x86_64                                                                                                                2/2 

Installed:
  fluent-package.x86_64 0:5.0.0-1.el7                                                                                                                         

Replaced:
  td-agent.x86_64 0:4.5.0-1.el7

Removed:

Running transaction
  削除中                  : fluent-package-5.0.0-1.el7.x86_64                                                                                             1/1 
Removing fluentd user...
userdel: fluentd mail spool (/var/spool/mail/fluentd) not found
userdel: fluentd home directory (/var/lib/td-agent) not found
  検証中                  : fluent-package-5.0.0-1.el7.x86_64                                                                                             1/1 

削除しました:
  fluent-package.x86_64 0:5.0.0-1.el7                                                                                                                         

完了しました!
[vagrant@centos7 ~]$ getent passwd fluentd
[vagrant@centos7 ~]$ echo $?
2
[vagrant@centos7 ~]$ getent group fluentd
[vagrant@centos7 ~]$ echo $?
2

ashie · 2023-06-08T07:07:52Z

I confirmed that RPM related issues are resolved, Thanks!

Although td-agent user is remained when I remove td-agent manually before upgrading to fluent-package, it's not fluent-package's issue, it's current td-agent's issue (it doesn't remove td-agent user on uninstallation).

kenhys · 2023-06-08T07:20:40Z

Thanks!!

kenhys force-pushed the migrate-user-group branch 2 times, most recently from 8b75318 to c12c4b4 Compare May 23, 2023 02:00

kenhys requested a review from ashie May 23, 2023 02:00

kenhys marked this pull request as ready for review May 23, 2023 02:00

kenhys removed the request for review from ashie May 23, 2023 02:20

kenhys changed the title ~~deb: migrate to new user/group name~~ WIP: deb: migrate to new user/group name May 23, 2023

kenhys force-pushed the migrate-user-group branch from c12c4b4 to 47ecb58 Compare May 23, 2023 08:37

kenhys changed the title ~~WIP: deb: migrate to new user/group name~~ deb: migrate to new user/group name May 23, 2023

kenhys force-pushed the migrate-user-group branch from 47ecb58 to 1fa1d03 Compare May 24, 2023 03:08

kenhys changed the title ~~deb: migrate to new user/group name~~ migrate to new user/group name May 24, 2023

kenhys force-pushed the migrate-user-group branch 2 times, most recently from 7b910a3 to 79f04d5 Compare May 24, 2023 05:09

kenhys force-pushed the migrate-user-group branch 2 times, most recently from b3faff9 to 8f120c9 Compare May 24, 2023 06:26

ashie self-requested a review May 26, 2023 05:17

kenhys force-pushed the migrate-user-group branch from 8f120c9 to ac24426 Compare May 29, 2023 07:40

kenhys mentioned this pull request May 30, 2023

migrate configuration files #489

Merged

ashie reviewed May 31, 2023

View reviewed changes

fluent-package/templates/package-scripts/fluent-package/deb/postinst Outdated Show resolved Hide resolved

ashie reviewed May 31, 2023

View reviewed changes

fluent-package/templates/package-scripts/fluent-package/deb/postinst Outdated Show resolved Hide resolved

kenhys force-pushed the migrate-user-group branch from ac24426 to 062d452 Compare May 31, 2023 05:19

ashie reviewed May 31, 2023

View reviewed changes

fluent-package/templates/usr/lib/tmpfiles.d/fluentd.conf Show resolved Hide resolved

ashie reviewed May 31, 2023

View reviewed changes

fluent-package/templates/package-scripts/fluent-package/deb/postinst Show resolved Hide resolved

kenhys mentioned this pull request Jun 2, 2023

msi: use sdk: image for prerequisite #492

Merged

ashie reviewed Jun 5, 2023

View reviewed changes

fluent-package/templates/package-scripts/fluent-package/deb/postinst Show resolved Hide resolved

kenhys force-pushed the migrate-user-group branch from d8efa35 to 538973c Compare June 5, 2023 06:06

kenhys force-pushed the migrate-user-group branch from 538973c to 331e459 Compare June 5, 2023 06:44

kenhys force-pushed the migrate-user-group branch 2 times, most recently from 3de97a8 to 10d864c Compare June 5, 2023 08:38

kenhys force-pushed the migrate-user-group branch from 10d864c to 45b7b29 Compare June 6, 2023 08:07

kenhys added 5 commits June 6, 2023 17:26

deb: cleanup statoverride of /var/run/td-agent

f35c7f7

Signed-off-by: Kentaro Hayashi <[email protected]>

rpm: migrate to new user/group name

9156d74

Signed-off-by: Kentaro Hayashi <[email protected]>

rpm: remove user/group when rpm -e

c33f1e3

Signed-off-by: Kentaro Hayashi <[email protected]>

test: follow specific uid/gid migration on debian

2485469

Signed-off-by: Kentaro Hayashi <[email protected]>

kenhys force-pushed the migrate-user-group branch from 45b7b29 to 2485469 Compare June 6, 2023 08:31

kenhys mentioned this pull request Jun 8, 2023

dmg: migrate changed path #493

Closed

ashie approved these changes Jun 8, 2023

View reviewed changes

ashie merged commit fa532cf into fluent:master Jun 8, 2023

kenhys deleted the migrate-user-group branch June 29, 2023 02:03

daipom mentioned this pull request Nov 2, 2023

rpm: User is removed at uninstalling, which can cause error at reinstalling #597

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

migrate to new user/group name #475

migrate to new user/group name #475

kenhys commented May 22, 2023 •

edited

Loading

kenhys commented May 22, 2023

kenhys commented May 23, 2023

kenhys commented May 23, 2023 •

edited

Loading

kenhys commented May 23, 2023

kenhys commented May 23, 2023

kenhys commented May 24, 2023

ashie commented May 26, 2023

kenhys commented May 29, 2023

kenhys commented Jun 2, 2023

kenhys commented Jun 5, 2023

kenhys commented Jun 5, 2023

kenhys commented Jun 5, 2023

kenhys commented Jun 5, 2023

ashie commented Jun 6, 2023

kenhys commented Jun 6, 2023

kenhys commented Jun 6, 2023

kenhys commented Jun 6, 2023

kenhys commented Jun 6, 2023

ashie commented Jun 8, 2023 •

edited

Loading

kenhys commented Jun 8, 2023

migrate to new user/group name #475

migrate to new user/group name #475

Conversation

kenhys commented May 22, 2023 • edited Loading

kenhys commented May 22, 2023

kenhys commented May 23, 2023

kenhys commented May 23, 2023 • edited Loading

kenhys commented May 23, 2023

kenhys commented May 23, 2023

kenhys commented May 24, 2023

ashie commented May 26, 2023

kenhys commented May 29, 2023

kenhys commented Jun 2, 2023

kenhys commented Jun 5, 2023

kenhys commented Jun 5, 2023

kenhys commented Jun 5, 2023

kenhys commented Jun 5, 2023

ashie commented Jun 6, 2023

kenhys commented Jun 6, 2023

kenhys commented Jun 6, 2023

kenhys commented Jun 6, 2023

kenhys commented Jun 6, 2023

ashie commented Jun 8, 2023 • edited Loading

kenhys commented Jun 8, 2023

kenhys commented May 22, 2023 •

edited

Loading

kenhys commented May 23, 2023 •

edited

Loading

ashie commented Jun 8, 2023 •

edited

Loading