util.aug empty comments with CRLF line endings

[Majlen]

I have ran into this issue with Java .properties file with CRLF line endings.

Originally I used new properties.aug lens in augeas 0.10.0 and I had no issues. In augeas 1.2.0 I ran into issue in util.aug, which can be fixed with the included diff.

--- util.aug    2014-08-27 19:57:44.000000000 +0200
+++ util.aug.new    2014-10-07 12:02:22.728090452 +0200
@@ -81,7 +81,7 @@

   let comment_generic_seteol (r:regexp) (d:string) (eol:lens) =
     [ label "#comment" . del r d
-        . store /([^ \t\r\n].*[^ \t\r\n]|[^ \t\r\n])/ . eol ]
+        . store /([^ \t\r\n].*[^ \t\r\n]*|[^ \t\r\n]*)/ . eol ]

 (* View: comment_generic
   Map comments and set default comment sign

I do not see any issues with this change (it should be even correct, since for CRLF it is needed to select more (2) characters.

[lutter]

Unfortunately, this leads to a typecheck error (run 'augparse lenses/util.aug over the changed file)

Could you give an example of what causes CRLF errors for you ?

[timurb]

I've caught a similar issue but could fix that in lens (nor did the above patch work for me):

When a properties file have CRLF-style endings then empty lines (i.e. the lines which have no characters except CRLF in them) are not treated like those by Augeas but are transformed into paths like /files/..../my.properties/ (not quite sure right now if they end or not in \r in augeas presentation of paths).

[Majlen]

The problem happens probably only with empty comments.

With LF line ending there is no issue, whether the comment is empty with space afterwards or without.
With CRLF neither of them is working, error message is "Iterated lens matched less than it should".

timurb's issue is happening for me too, there are inaccessible nodes /files/..../my.properties/^M[1] (they are not even listable via ls command in augtool).

Example file is here: http://home.zcu.cz/~majlen/aaa.properties

[DaffyDuke]

I got same problem now with CentOS-7-4 and its cloud-init update. Cloud-init adds now in /etc/resolv.conf

; Created by cloud-init on instance boot automatically, do not edit.
;
; generated by /usr/sbin/dhclient-script
search <.....>

This secund line is a problem for puppet/augeas.
It can be fixed with upper workaround.
Is there a way to publish it with last augeas release ?

[lutter]

Can you post the full resolv.conf that augeas barfs on ? That would help troubleshooting this a lot.

[DaffyDuke]

Hi lutter, yep, thank you !

# cat /etc/resolv.conf
; Created by cloud-init on instance boot automatically, do not edit.
;
search awsqualif.net aws.eu-west-1.censured_here
nameserver myinternalip_here
nameserver myinternalip_here
options timeout:2 rotate

RPM versions / CentOS Linux release 7.4.1708 (Core)

# rpm -qa |egrep 'cloud|augeas'
augeas-1.4.0-2.el7_4.2.x86_64
cloud-init-0.7.9-9.el7.centos.2.x86_64
augeas-libs-1.4.0-2.el7_4.2.x86_64

When I play with augeas , it failes

#  augtool -e "print /files/etc/resolv.conf"
augtool> print /files/etc/resolv.conf 

When my puppet agent try to apply DNS config :

Debug: Augeas[resolvconf](provider=augeas): Opening augeas with root /, lens path , flags 32
Debug: Augeas[resolvconf](provider=augeas): Augeas version 1.8.1 is installed
Warning: Augeas[resolvconf](provider=augeas): Loading failed for one or more files, see debug for /augeas//error output
Debug: Augeas[resolvconf](provider=augeas): /augeas/files/etc/resolv.conf/error = parse_failed
Debug: Augeas[resolvconf](provider=augeas): /augeas/files/etc/resolv.conf/error/pos = 70
Debug: Augeas[resolvconf](provider=augeas): /augeas/files/etc/resolv.conf/error/line = 2
Debug: Augeas[resolvconf](provider=augeas): /augeas/files/etc/resolv.conf/error/char = 1
Debug: Augeas[resolvconf](provider=augeas): /augeas/files/etc/resolv.conf/error/lens = /opt/puppetlabs/puppet/share/augeas/lenses/dist/resolv.aug:131.10-.38:
Debug: Augeas[resolvconf](provider=augeas): /augeas/files/etc/resolv.conf/error/message = Iterated lens matched less than it should
Debug: Augeas[resolvconf](provider=augeas): Will attempt to save and only run if files changed
Debug: Augeas[resolvconf](provider=augeas): sending command 'rm' with params ["/files/etc/resolv.conf/search"]
Debug: Augeas[resolvconf](provider=augeas): sending command 'set' with params ["/files/etc/resolv.conf/search/domain[last()+1]", "censured_here"]
Debug: Augeas[resolvconf](provider=augeas): sending command 'set' with params ["/files/etc/resolv.conf/search/domain[last()+1]", "censured_here"]
Debug: Augeas[resolvconf](provider=augeas): sending command 'set' with params ["/files/etc/resolv.conf/search/domain[last()+1]", "censured_here"]
Debug: Augeas[resolvconf](provider=augeas): sending command 'set' with params ["/files/etc/resolv.conf/search/domain[last()+1]", "censured_here"]
Debug: Augeas[resolvconf](provider=augeas): sending command 'rm' with params ["/files/etc/resolv.conf/nameserver"]
Debug: Augeas[resolvconf](provider=augeas): sending command 'set' with params ["/files/etc/resolv.conf/nameserver[last()+1]", "censured_here"]
Debug: Augeas[resolvconf](provider=augeas): sending command 'set' with params ["/files/etc/resolv.conf/nameserver[last()+1]", "censured_here"]
Debug: Augeas[resolvconf](provider=augeas): sending command 'rm' with params ["/files/etc/resolv.conf/options"]
Debug: Augeas[resolvconf](provider=augeas): sending command 'set' with params ["/files/etc/resolv.conf/options/timeout", "2"]
Debug: Augeas[resolvconf](provider=augeas): sending command 'set' with params ["/files/etc/resolv.conf/options/rotate", ""]
Debug: Augeas[resolvconf](provider=augeas): Closed the augeas connection
Error: /Stage[main]/Iaas_dnsconfig/Augeas[resolvconf]: Could not evaluate: Saving failed, see debug

If I just remove line 2 which contain only the fire ; , provided by this cloud-init version, augeas do not fail.
If I downgrade cloud-init, this line do not appear any more, but it should if I understand cloud-init's source code.