WARNING: Cannot write latest cached revision to file

[Lw-Cui]

I made directory /var/opengrok/src /var/opengrok/data /var/opengrok/etc and put code into src. All worked well until I index a new source directory. Then I got the WARNING:

WARNING: Cannot write latest cached revision to file xxx(the name of source directroy)
java.io.FileNotFoundException: /var/opengrok/data/historycache/OpenGroklatestRev (No such file or directory)
....

Is a bug? If so, how could I fix it?
Thanks:)

[vladak]

What version are you running ?

[Lw-Cui]

0.13 rc-4. Is this fixed in 0.13 rc-5?

[vegetablemao]

I encountered the same problem in branch master

[tulinkry]

Based on the exception I assume you don't use projects with opengrok (you directly index anything which is under the /var/opengrok/src).

Could (anyone of you) provide more detailed steps how to reproduce it?

1. create the dirs src, etc, data.
2. put some sources into the src
3. index it - now the application works well and you can browse the indexed code

And now comes my confusion: what next?

4. add a new source directory - where? how? what sort of sources are inside? or "new" means that you deleted the old and populated it with new sources?
5. index it - using which command?

Is there any other exception in the logs?

I see the problem is that the directory "historycache" don't get created and therefore it can't write the latests revision file in that directory. That's the only possibility I can think of, however I can't see the circumstances.

[tulinkry]

The historycache repository doesn't get created when there is no history for any files in the repository

[mbluemer]

The issue has to do with the OpenGroklatestRev file. Can't tell you how to recreate but I'm having the same issue. Occurred when trying to re-index repos after pulling changes. I then blasted the data directory and had no luck so then I blasted both data and src directories to see if starting from scratch would help but also no luck.

[vladak]

In my case it happens when I try to index certain Tomcat SVN repo, i.e.:

1. checkout the repo:

  svn checkout http://svn.apache.org/repos/asf/tomcat/tc6.0.x/tags/TOMCAT_6_0_33

2. run the indexer with:

   -W /var/opengrok/etc/configuration.xml -c /usr/local/bin/universal-ctags -a on -s /var/opengrok/src -d /var/opengrok/data -P -H -S -U localhost:2424

For some reason (yet to be investigated) the history cache is not created for this repo. As a consequence the /var/opengrok/data/TOMCAT_6_0_33 directory does not exist. This matches @tulinkry 's observation above. The OpenGroklatestRev should not be created if history cache failed, i.e. storeLatestCachedRevision() should not be even called.

[vladak]

The reason why the above SVN repo lacks history cache is that repository prefix (2nd argument to Handler in SubversionHistoryParser.java) does not match the paths in svn log output, i.e. svn info --xml contains:

<url>http://svn.apache.org/repos/asf/tomcat/tc6.0.x/tags/TOMCAT_6_0_33</url>
<root>http://svn.apache.org/repos/asf</root>

so the tomcat/tc6.0.x/tags/TOMCAT_6_0_33 part becomes reposPath in SubversionRepository.java#setDirectoryName() however this does not match the entries in the svn log --xml --non-interactive -v output, e.g.

<path
   action="M"
   prop-mods="false"
   text-mods="true"
   kind="file">/tomcat/tc6.0.x/trunk/java/org/apache/catalina/loader/VirtualWebappLoader.java</path>

The expected prefix is /tomcat/tc6.0.x/tags/TOMCAT_6_0_33 but the prefix in the log is /tomcat/tc6.0.x/trunk/.

Hence the history entries will not match actual files under source root because of the check in SubversionHistoryParser.java#endElement:

98              } else if ("path".equals(qname)) {
99                  /*
100                   * We only want valid files in the repository, not the
101                   * top-level directory itself, hence the check for inequality.
102                   */
103                  if (s.startsWith(prefix) && !s.equals(prefix)) {

As a consequence no history cache will be created. To tell the /tomcat/tc6.0.x/trunk is the real prefix the first logentry would have to be parsed:

<path
   copyfrom-path="/tomcat/tc6.0.x/trunk"
   copyfrom-rev="1158220"
   action="A"
   prop-mods="true"
   text-mods="false"
   kind="dir">/tomcat/tc6.0.x/tags/TOMCAT_6_0_33</path>
</paths>

which I am not sure is worth the complication.

[vladak]

To sum this up: the warning message is innocuous however hints towards problem with creating history cache. There are multiple reasons why history cache creation could fail:

1. the history log command failed (e.g. because it timed out) - this should be visible in indexer logs
2. the history can be retrieved from the repo however does not match the file-system contents (as shown with the SVN repo above) - this would be visible in indexer log once the log level is bumped up to more detailed level (e.g. FINER in the case of SVN repo and it might say something like Skipping file outside repository)