Merge indexer-split branch.

svn path=/trunk/; revision=2275

1 files changed, 436 insertions, 0 deletions

diff --git a/README b/README
new file mode 100644
index 000000000..b9666d50e
--- /dev/null
+++ b/README
@@ -0,0 +1,436 @@
+Table of Contents
+
+    1 Introduction
+    2 Use Cases
+    3 Features
+    4 Minimum dependencies
+        4.1 Run-time dependencies (also needed for build)
+        4.2 Additional recommended dependencies
+        4.3 For building Tracker's Deskbar-applet backend
+        4.4 Optional run-time dependency
+    5 Compilation
+        5.1 Notes on Solaris
+        5.2 Compile Options
+    6 Running Tracker
+        6.1 Setting Inotify Watch Limit
+    7 Tracker & Nautilus Search
+    8 Tracker & Deskbar applet
+    9 Tracker tools
+
+
+
+1 Introduction
+
+  Tracker is a powerful desktop-neutral first class object
+  database, tag/metadata database, search tool and indexer.
+
+  Tracker is also extremely fast and very memory-efficient
+  when compared with some other competing frameworks and has by
+  far the fastest and most memory-efficient Nautilus search and
+  Deskbar backends currently available.
+
+  It consists of a common object database that allows entities to
+  have an almost infinte number of properties, metadata (both
+  embedded/harvested as well as user definable), a comprehensive
+  database of keywords/tags and links to other entities.
+
+  It provides additional features for file-based objects
+  including context linking and audit trails for a file object.
+
+  It has the ability to index, store, harvest metadata, retrieve
+  and search all types of files and other first class objects.
+
+  Supported first class objects include:
+
+  * Files, Documents, Music, Images, Videos, Applications, Emails,
+    Conversations, History
+
+  Planned support:
+
+  * Appointments, Contacts, Projects, Tasks, Bookmarks, Playlists,
+    Notes
+
+  All discussion related to tracker happens on the Tracker
+  mailing list
+  (http://mail.gnome.org/mailman/listinfo/tracker-list) and/or
+  IRC channel #tracker on irc.gimp.net
+
+  Bugs should be filed at http://bugzilla.gnome.org.
+
+  More infomation on Tracker can be found at
+  http://tracker-project.org.
+
+
+
+2 Use Cases
+
+  Tracker is the most powerful open source metadata database and
+  indexer framework currently available and because it is built
+  around a combination indexer and sql database and not a
+  dedicated indexer, it has much more powerful use cases:
+
+  * Provide search and indexing facilities similiar to those on
+  other systems (Windows Vista and Mac OS X).
+
+  * Common database storage for all first class objects (EG a
+  common music/photo/contacts/email/bookmarks/history database)
+  complete with additional metadata and tags/keywords.
+
+  * Comprehensive one stop solution for all applications needing
+  an object database, powerful search (via RDF Query), first class
+  methods, related metadata and user-definable metadata/tags.
+
+  * Can provide a full semantic desktop with metadata everywhere.
+
+  * Can provide powerful criteria-based searching suitable for
+  creating smart file dialogs and vfolder systems.
+
+  * Can provide a more intelligent desktop using statistical
+  metadata.
+
+
+
+3 Features
+
+  * Desktop-neutral design (it's a freedesktop product built
+  around other freedesktop technologies like D-Bus and XDGMime
+  but contains no GNOME-specific dependencies besides GLib).
+
+  * Very memory efficient and non-leaking (typical RAM usage 4 - 30
+  MB). Unlike some other indexers, Tracker is designed and built
+  to run well on systems with lower memory (256MB or less). It
+  should even be efficient enough to use on some mobile devices.
+
+  * Non-bloated and written in C for maximum efficiency.
+
+  * Small size and minimal dependencies makes it easy to bundle
+  into various distros, including live cds.
+
+  * Fast indexing and unobtrusive - no need to index stuff
+  overnight. Tracker runs at nice+10 so it should have a minimal
+  impact on your system. With the addition of detection of mouse
+  and keyboard events via tracker-applet (described below), there
+  is an option to auto-pause indexing in order to improve
+  responsiveness. This is in addition to Tracker's built-in check
+  if there's heavy disk I/O in order to auto-pause, so not to
+  slow other processes.
+
+  * Provides option to disable indexing when running on battery.
+
+  * Implements the freedesktop specification for metadata
+  (http://freedesktop.org/wiki/Standards/shared-filemetadata-spec).
+
+  * Extracts embedded File, Image, Document and Audio type
+  metadata from files.
+
+  * Supports the WC3's RDF Query syntax for querying metadata
+
+  * Provides support for both free text search (like Beagle/Google)
+  as well as structured searches using RDF Query.
+
+  * Responds in real time to file system changes to keep its
+  metadata database up to date and in sync.
+
+  * Fully extensible with custom metadata - you can store,
+  retrieve, register and search via RDF Query all your own custom
+  metadata.
+
+  * Can extract a file's contents as plain text and index them.
+
+  * Provides text filters for PDF, MS Office, OpenOffice (all
+  versions), HTML and PS files.
+
+  * Can provide thumbnailing on the fly.
+
+  * It auto-pauses indexing when running low on diskspace.
+
+
+
+4 Minimum dependencies
+
+
+4.1 Run-time dependencies (also needed for build)
+
+  * SQLite 3.4
+  * libdbus 0.60
+  * dbus-glib bindings 0.60
+  * GLib 2.14
+  * zlib
+  * intltool 0.3.5
+  * GMime
+
+
+4.2 Additional recommended dependencies
+
+  * GStreamer 0.10 + plugins for audio/video file indexing
+  * xsltproc
+  * w3m
+  * wv 1.0.2
+  * poppler (pdftotext)
+  * odt2txt 0.4 (indexing OpenOffice/ODF documents)
+  * libvorbis
+  * libpng
+  * libexif
+  * libgsf
+  * GTK and GNOME stack (for GUI tools)
+  * libglade 2.5
+  * unac (accent stripper)
+  * exempi
+  * libxml2
+
+
+4.3 For building Tracker's Deskbar-applet backend
+
+  * python-dev 2.3
+  * python-gtk2-dev 2.3
+  * deskbar-applet 2.16
+
+
+4.4 Optional run-time dependency
+
+  * xdg-utils (provides some functionality needed by
+  tracker-search-tool when in a non-GNOME environment)
+
+
+
+5 Compilation
+
+  To compile and install Tracker, use the following commands :
+
+        ./configure --prefix=/usr --sysconfdir=/etc
+        make
+        sudo make install
+
+  If you install using any other prefix, you might have problems
+  with files not being installed correctly. (You may need to copy
+  and amend the dbus service file to the correct directory and/or
+  might need to update ld_conf if you install into non-standard
+  directories.)
+
+
+5.1 Notes on Solaris
+
+  To compile Tracker with GCC on Solaris uses the following
+  commands :
+
+        ./configure --prefix=/usr --sysconfdir=/etc --with-pic \
+        CFLAGS=-D_POSIX_PTHREAD_SEMANTICS
+        make
+        sudo make install
+
+  To compile Tracker with SUN Studio on Solaris uses the
+  following commands, because there are some problems to compile
+  exiv2 using SUN C++ compiler :
+
+        ./configure --prefix=/usr --sysconfdir=/etc --with-pic \
+        --disable-warnings \
+        --disable-exiv2 CFLAGS=-D_POSIX_PTHREAD_SEMANTICS
+        make
+        sudo make install
+
+
+5.2 Compile Options
+
+  Tracker has several compiler options to enable/disable certain
+  features. The following is a (hopefully complete) listing of
+  what's available:
+
+  --enable-debug-code : build with debug symbols
+
+  --enable-video-extractor=(gstreamer, xine, external, auto)
+
+  --enable-file-monitoring=(inotify, fam, polling, auto)
+
+  --disable-preferences : disables tracker-preferences capplet
+
+  --enable-deskbar-applet=(auto, handler, module) : enables
+  Deskbar-applet support; 'auto' should do since it automatically
+  chooses whether to install the 'handler' (for Deskbar-applet
+  >=2.16) or the 'module' (for Deskbar-applet >=2.19)
+
+  --with-deskbar-applet-dir=(directory where Deskbar-applet
+  should find tracker-handler; this should be automatically
+  detected, perhaps in
+  /usr/lib/deskbar-applet/{handlers,modules-2.20-compatible})
+
+  --disable-gui : disables tracker-search-tool build
+
+  --disable-pdf : disables the PDF data extractor
+
+  --disable-exif : disables the exif data extractor
+
+  --disable-libtrackergtk : disables libtracker-gtk build
+
+  --disable-gsf : disables the GSF data extractor
+
+  --disable-warnings : disables GCC warnings
+
+  --disable-unac : disables accent stripping
+
+  --disable-libxml2 : disables HTML/XML extractors (full-text
+  will still be available)
+
+  --disable-xmp : disables XMP extraction
+
+  --with-session-bus-services-dir=(path to D-Bus services
+  directory): this should be automatically detected
+
+  --enable-external-qdbm : use system qdbm instead of one included
+  in Tracker
+
+  --disable-hal : disables HAL support for ac-power detection
+
+  --disable-trackerapplet : disables Tracker's notification applet
+
+
+
+6 Running Tracker
+
+  To run Tracker, you need to manually start the Tracker daemon,
+  trackerd. By default trackerd will index your entire home
+  directory.
+
+  You can also pass a directory root to be indexed as a command
+  line parameter if you dont want your entire home directory
+  indexed. EG "trackerd -i /home/jamie/Documents" (if you want
+  your home directory indexed when explicily specifying index
+  directory roots then you must add your home directory to the
+  arguments: EG trackerd -i /home/jamie -i /mnt/share)
+
+  You can disable indexing by passing "--no-indexing"
+
+  You can enable a low memory usage mode (recommended for
+  machines with less than 256MB of RAM) by passing
+  "--enable-low-memory"
+
+  You can artificially throttle indexing by passing
+  "--throttle=VALUE" where VALUE is in the range 0-20 (with 0,
+  the default, being fastest and 20 being slowest). Default is
+  0. You should only change this value if you want to prevent
+  noisy fans or hot laptops arising from cpu intensive indexing.
+  Tracker should have a negligible impact on the system (as it
+  is scheduled) so you can safely work with it on full throttle
+  without experiencing slow downs.
+
+  You can specify directory roots to be excluded from being
+  watched or indexed by passing "--exclude=DIRECTORY" for each
+  directory root.
+
+  You can specify logging verbosity by passing "--verbosity". Valid
+  values are from 0 to 3, ranging from least to most verbose
+  respectively.
+
+  Yet another option is "--language" which allows for specifying
+  the language to use for stemmer and stop-words list.
+
+  All the above options (and more) can be set by editing Tracker
+  config file "~/.config/tracker/tracker.cfg" which is created
+  with specific defaults when non-existent (EG when trackerd is
+  ran for the first time). Ensure that you restart trackerd for
+  the changes to take effect. "tracker.cfg" also provides options
+  that allows Tracker to only index a subset of your home
+  directory as well as other folders not in your home directory
+  by setting WatchDirectoryRoots to a semicolon-delimited list of
+  directories (full path required!)
+
+  EG:
+
+  "WatchDirectoryRoots=directory1;directory2;directory3""
+
+  An additional option is the "--reindex" option which indexes
+  user data from scratch, removing the need to remove Tracker
+  database manually.
+
+  On the first run, Tracker will automatically create a new
+  database and start populating it with metadata by browsing
+  through the user's home directory and/or the root folder(s)
+  specified.
+
+  On subsequent runs, Tracker will start up much much faster and
+  will only ever incrementally index files (IE files that have
+  changed since last index).
+
+  If installed correctly, the Tracker daemon (trackerd) can also
+  be started automatically via Dbus activation (EG by running
+  tracker-search SEARCHTERM)
+
+
+6.1 Setting Inotify Watch Limit
+
+  When watching large numbers of folders, its ppossible to exceed
+  the default number of inotify watches. In order to get real time
+  updates when this value is exceeded it is necessary to increase
+  the number of allowed watches. This can be done as follows:
+
+  1. Add this line to /etc/sysctl.conf:
+     "fs.inotify.max_user_watches = (number of folders to be
+      watched; default is 8192)"
+
+  2. Reboot the system OR (on a Debian-like system) run
+     "sudo /etc/init.d/procps restart"
+
+
+
+7 Tracker & Nautilus Search
+
+  Once you have installed Tracker and have some indexed contents,
+  you should now compile Nautilus (ver 2.13.4 or higher) which
+  should auto-detect that Tracker is installed and automatically
+  compile in Tracker support. You are now ready to appreciate a
+  powerful and super efficient C-based indexer in all its
+  glory... happy hunting!
+
+  To make sure trackerd always starts when you login to GNOME,
+  you will need to add it to gnome-session (select sessions from
+  preferences menu, select startup program tab and then add
+  /usr/bin/trackerd). For non-GNOME installations, see the
+  desktop docs for how to achieve similar.
+
+
+
+8 Tracker & Deskbar applet
+
+  Tracker is also integrated in GNOME's deskbar applet. See
+  Compile Options above on how to get it built.
+
+
+
+9 Tracker tools
+
+  Tracker comes with a number of utilities that you can use:
+
+  * "tracker-applet" - notification applet which has various
+  utilities like displaying Tracker status, indexing progress,
+  and live statistics, pausing indexing, and launching both
+  tracker-search-tool and tracker-preferences
+
+  * "tracker-extract FILE" - this extracts embedded metadata from
+  FILE and prints to stdout
+
+  * "tracker-files" - returns files filtered by the mime type or
+  their category (see the manpage)
+
+  * "tracker-meta-folder" - return list of files indexed by Tracker
+  for a folder
+
+  * "tracker-preferences" - GUI tool to set Tracker preferences
+
+  * "tracker-query" - this reads an RDF Query that specifies the
+  search criteria for various fields. It prints to STDOUT all
+  matching files. You can see some example queries in the
+  RDF-Query-examples folder. You can run the examples as
+  "tracker-query < RDFFILE"
+
+  * "tracker-search SEARCHTERM" - this perfoms a google like search
+  using SEARCHTERM to retrieve all matching files where
+  SEARCHTERM appears in any searchable metadata
+
+  * "tracker-search-tool SEARCHTERM" - GUI search utility
+
+  * "tracker-stats" - this displays the current number of indexed
+  items by category
+
+  * "tracker-status" - queries status of trackerd
+
+  * "tracker-tag" - used for setting and searching tags/keywords
+