OpenGrok indexes

Facts

• The cross-referencing of ATLAS code is performed with LXR indexer
• The LXR server is installed at the U.S. ATLAS Tier 1 Facility on the dedicated alxr machine
• The main feature of this indexer is the ability to jump easily to the declaration of any global identifier. Even all references to global identifiers are indexed. The indexes are stored in the MySQL database
• LXR provide the freetext search. However it is implemented using Glimpse, which is an independent tool with the regular expression search capabilities
• There are several software versions available for LXR search (to switch between versions click on the links in the top left area of the source navigation page):
  1. "nightly" (default) - contains the versions from the latest major nightlies (currently 16.X.0).
  2. "head" trunk versions from ATLAS SVN repository. It includes the same packages as nightly version but checked out from the trunk.

Updates schedule

• Daily updates are completed by 5:00 EST (11:00 CERN).
• Updates are performed for the mirrors of LXR databases. Upon the jobs completion the LXR web pages are switched to use the updated sources and databases. Therefore the LXR services are now provided 24 hours a day, 7 days a week without intterruptions.

In addition the code of some ATLAS releases can be browsed on the separate LXR pages.

Example of freetext search

Suppose the aim is to find *.cxx files with "CBNT" in the name and "CBNTAA" in the body. Then fill "Files named" with "CBNT.*\.cxx and "Containing" with "CBNTAA". Make sure that "allow perl regex" option is checked. See details below.

Important note about Glimpse search

The freetext search is powered by Glimpse that supports variety of patterns, including simple strings, strings with classes of characters, sets of strings, wild cards, and regular expressions (full description is available here). The freetext search can be started by clicking on "general search" button in the upper right corner of the main LXR page. Two fillable fields then appear in the window: "Files named" and "Containing". You can search for filenames with particular strings/patterns or for files containing particular strings/patterns. It is very easy to get confused because of varieties of Glimpse supports each of which have certain limitations. The Caveats are:

• The "File named" field can not be empty. If you wish to find all files containing abc then enter "." in the "File named" field and "abc" in the "Containing" field.
• If "Advanced (allow perl regex)" option is not checked then no symbol in the pattern is treated as a meta character (the only exception is ^ at the beginning of the pattern and $ at the end of the pattern, which are still interpreted in the usual way. Use \^ or \$ if you need them verbatim). Therefore "." means literally dot in the file name. If however "Advanced" is checked then "." matches any symbol in the file name. The syntax of regular expressions in glimpse is in general the same as that for agrep (a variation of grep developed by the author of Glimpse). The following special characters ( `$', `^', `*', `[', `^', `|', `(', `)', `!', and `\' ) as well as the following meta characters special to glimpse (and agrep): `;', `,', `#', `<', `>', `-', and `.', should be preceded by `\' if they are to be matched as regular characters
• For "Containing" field only: Glimpse supports `AND' operation denoted by the symbol `;' and `OR' operation denoted by the symbol `,', 'NOT' operation denoted by the symbol `~', or any combination. For example, you can enter "a0;z0;phi in the "Containing" field to search for files with all three patterns
• The maximum length of string in "Containing" field is 30 characters. This limitation comes from agrep ("approximate grep") tool which is the search engine for glimpse. See agrep wikipedia page for interesting details
• WARNING NOTE for simple text pattern searches in files with a certain name: at first the LXR finds all files with the indicated text pattern and then selects the files with matching names. The problem can occur because the maximum number of text search hits is set to 1000. If the limit is reached then the result of the text search of a text pattern in files with a certain name becomes incomplete (and no warning is displayed in this case). The following recipe can help to avoid confusion:
  1. search for the pattern in all files (i.e. with empty field "Files named")
  2. check if number of hits is less then 1000
  3. perform the desired search for the files with a certain name
• It was noticed that Glimpse can have problems with "<", ">" in xml files. For instance searching files containing "\<.*hep.*" provides satisfactory results while searching ".*hep.*" does not work (for xml files only; works for other files)
• Glimpse makes an effort to identify non-text files such as binary files, compressed files, uuencoded files, postscript files, binhex files, etc. Unfortunately sometimes good files are discarded, e.g. genuine text files with long strings without spaces. For instance python files with strings like

  AtlasG4Eng.G4Eng.menu_MCTruth.set_TruthStrategiesParameter("ConversionSecondaryMinEnergy",100)
  

  are discarded. To prevent this all python files (with extension .py) are included in general search index (starting Jan. 9, 2009).

Links

• LXR page in ATLAS Computing Work Book (by Vivek Jain)

Cross-Referencing of ATLAS code is provided by the U.S. ATLAS Tier 1 Center at Brookhaven National Laboratory