Why it read one more line every time.

[samoin]

My problem is like this :
I started logstash with command "./logstash agent -f ../conf/shipper.conf --debug",and the shipper.conf file is

input {
file {
type => "syslog"
path => "/home/admarket/logstash-1.4.2/logs/*"
}
}
output {
redis {
host => "127.0.0.1"
port => 6379
data_type => "list"
key => "rawLogs"
}
}

Then i tried to create a new file "a.log",and i sited '{"mn":"a1","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}' with Vim,and then ":x" to save this file.
At this time ,it's ok, i can see it in logs and reds list.

Then i added '{"mn":"a2","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}' to file "a.log",but at this time, i got two message in logs("a1" and "a2") ,and the list size of redis increased to 3.

Then i added '{"mn":"a3","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}' to file "a.log",but at this time ,i still got two message in logs("a2" and "a3"), and the list size of redis increased to5.

 logs >
 _discover_file_glob: /home/admarket/logstash-1.4.2/logs/*: glob is: ["/home/admarket/logstash-1.4.2/logs/a.log"] {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"}
 _discover_file: /home/admarket/logstash-1.4.2/logs/*: new: /home/admarket/logstash-1.4.2/logs/a.log (exclude is []) {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"126"}
 _open_file: /home/admarket/logstash-1.4.2/logs/a.log: opening {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"98"}
 /home/admarket/logstash-1.4.2/logs/a.log: staying at position 0, no sincedb {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"145"}
 Received line {:path=>"/home/admarket/logstash-1.4.2/logs/a.log", :text=>"{\"mn\":\"a1\",\"aridx\":\"f1\",\"ipinfo\":\"a,b,\",\"interfaceid\":\"1\"}", :level=>:debug, :file=>"logstash/inputs/file.rb", :line=>"134"}
 output received {:event=>{"message"=>"{\"mn\":\"a1\",\"aridx\":\"f1\",\"ipinfo\":\"a,b,\",\"interfaceid\":\"1\"}", "@version"=>"1", "@timestamp"=>"2014-09-30T02:05:59.251Z", "type"=>"syslog", "host"=>"centos21", "path"=>"/home/admarket/logstash-1.4.2/logs/a.log"}, :level=>:debug, :file=>"(eval)", :line=>"16"}
 writing sincedb (delta since last write = 1412042759) {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"179"}
  {:level=>:debug, :host=>"127.0.0.1", :port=>6379, :timeout=>5, :db=>0, :file=>"logstash/outputs/redis.rb", :line=>"238"}
 /home/admarket/logstash-1.4.2/logs/a.log: file grew, old size 0, new size 59 {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"81"}
 _discover_file_glob: /home/admarket/logstash-1.4.2/logs/*: glob is: ["/home/admarket/logstash-1.4.2/logs/a.log"] {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"}
 /home/admarket/logstash-1.4.2/logs/a.log: old inode was [529412, 0, 64772], new is [529414, 0, 64772] {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"73"}
 :delete for /home/admarket/logstash-1.4.2/logs/a.log, deleted from @files {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"85"}
 writing sincedb (delta since last write = 27) {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"179"}

_open_file: /home/admarket/logstash-1.4.2/logs/a.log: opening {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"98"}
/home/admarket/logstash-1.4.2/logs/a.log: staying at position 0, no sincedb {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"145"}
Received line {:path=>"/home/admarket/logstash-1.4.2/logs/a.log", :text=>"{"mn":"a1","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}", :level=>:debug, :file=>"logstash/inputs/file.rb", :line=>"134"}
Received line {:path=>"/home/admarket/logstash-1.4.2/logs/a.log", :text=>"{"mn":"a2","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}", :level=>:debug, :file=>"logstash/inputs/file.rb", :line=>"134"}
output received {:event=>{"message"=>"{"mn":"a1","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}", "@Version"=>"1", "@timestamp"=>"2014-09-30T02:06:26.335Z", "type"=>"syslog", "host"=>"centos21", "path"=>"/home/admarket/logstash-1.4.2/logs/a.log"}, :level=>:debug, :file=>"(eval)", :line=>"16"}
output received {:event=>{"message"=>"{"mn":"a2","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}", "@Version"=>"1", "@timestamp"=>"2014-09-30T02:06:26.340Z", "type"=>"syslog", "host"=>"centos21", "path"=>"/home/admarket/logstash-1.4.2/logs/a.log"}, :level=>:debug, :file=>"(eval)", :line=>"16"}
discover_file_glob: /home/admarket/logstash-1.4.2/logs/: glob is: ["/home/admarket/logstash-1.4.2/logs/a.log"] {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"}
discover_file_glob: /home/admarket/logstash-1.4.2/logs/: glob is: ["/home/admarket/logstash-1.4.2/logs/a.log"] {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"}
/home/admarket/logstash-1.4.2/logs/a.log: old inode was [529414, 0, 64772], new is [529412, 0, 64772] {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"73"}
:delete for /home/admarket/logstash-1.4.2/logs/a.log, deleted from @files {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"85"}
writing sincedb (delta since last write = 25) {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"179"}
_open_file: /home/admarket/logstash-1.4.2/logs/a.log: opening {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"98"}
/home/admarket/logstash-1.4.2/logs/a.log: sincedb last value 59, cur size 177 {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"122"}
/home/admarket/logstash-1.4.2/logs/a.log: sincedb: seeking to 59 {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"124"}
Received line {:path=>"/home/admarket/logstash-1.4.2/logs/a.log", :text=>"{"mn":"a2","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}", :level=>:debug, :file=>"logstash/inputs/file.rb", :line=>"134"}
Received line {:path=>"/home/admarket/logstash-1.4.2/logs/a.log", :text=>"{"mn":"a3","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}", :level=>:debug, :file=>"logstash/inputs/file.rb", :line=>"134"}
output received {:event=>{"message"=>"{"mn":"a2","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}", "@Version"=>"1", "@timestamp"=>"2014-09-30T02:06:51.401Z", "type"=>"syslog", "host"=>"centos21", "path"=>"/home/admarket/logstash-1.4.2/logs/a.log"}, :level=>:debug, :file=>"(eval)", :line=>"16"}
output received {:event=>{"message"=>"{"mn":"a3","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}", "@Version"=>"1", "@timestamp"=>"2014-09-30T02:06:51.405Z", "type"=>"syslog", "host"=>"centos21", "path"=>"/home/admarket/logstash-1.4.2/logs/a.log"}, :level=>:debug, :file=>"(eval)", :line=>"16"}

redis list pop >
{"message":"{"mn":"a3","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}","@Version":"1","@timestamp":"2014-09-30T02:06:51.405Z","type":"syslog","host":"centos21","path":"/home/admarket/logstash-1.4.2/logs/a.log"}
{"message":"{"mn":"a2","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}","@Version":"1","@timestamp":"2014-09-30T02:06:51.401Z","type":"syslog","host":"centos21","path":"/home/admarket/logstash-1.4.2/logs/a.log"}
{"message":"{"mn":"a2","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}","@Version":"1","@timestamp":"2014-09-30T02:06:26.340Z","type":"syslog","host":"centos21","path":"/home/admarket/logstash-1.4.2/logs/a.log"}
{"message":"{"mn":"a1","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}","@Version":"1","@timestamp":"2014-09-30T02:06:26.335Z","type":"syslog","host":"centos21","path":"/home/admarket/logstash-1.4.2/logs/a.log"}
{"message":"{"mn":"a1","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}","@Version":"1","@timestamp":"2014-09-30T02:05:59.251Z","type":"syslog","host":"centos21","path":"/home/admarket/logstash-1.4.2/logs/a.log"}

I don't konw why!I need your help!Waiting your reply!Thanks a lot!

[wiibaa]

@samoin how are you adding content to your file ? From logstash logs, logstash is detecting an inode change so it will read the file from the beginning again.

[samoin]

@wiibaa I used "Vim" ,everytime ,i write "vi a.log" in command, then add the json content into this file.
So this way is not right ? How should i add content ? Use nginx log file for test?
Thanks you!

[wiibaa]

@samoin Vi is creating a new file at each save, using a simple echo '{"mn":"a2","aridx":"f1","ipinfo":"a,b,","interfaceid":"1"}' >> a.log is better to simulate appending content to a file

[samoin]

@wiibaa Thanks very much.

[wiibaa]

@samoin if you have no additional questions, could you please close the issue.

[samoin]

@wiibaa Ok,thanks .