summaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* Update meson build system for libtracker-direct changeswip/sam/carlosg/direct-rewriteSam Thursfield2018-07-163-14/+5
|
* libtracker-direct: Remove unused variablesSam Thursfield2018-07-161-2/+1
|
* tracker-store: Use TrackerDirectConnection underneathCarlos Garnacho2018-07-165-424/+117
| | | | | | | | | | | | | | | Instead of the lower level TrackerDataManager object directly. The only additional thing that tracker-store does is signal emission for writeback and GraphUpdated, the internal TrackerDataManager object is still accessed to implement those features. This makes libtracker-direct the only place where queries/updates are queued, performed and dispatched. There's other indirect benefit from this, update queue handling no longer needs to hit the main thread in order to schedule the next update. Besides the very unlikely thread contention situations described in previous commits, this should maximize throughput of the updates queue.
* libtracker-data: Move notify_transaction() down to libtracker-dataCarlos Garnacho2018-07-164-19/+3
| | | | | | | | | | | Now that we don't need upper layers' information to find out the right CommitType, push this call down together with the handling of the insert/delete/rollback callbacks. One particularity is that the commit callbacks will now be called from within the update thread, just like all the other callbacks. And this is fine, with all the preparation work from the previous commits.
* tracker-store: Push TrackerData hooks down to Tracker.StoreCarlos Garnacho2018-07-165-97/+116
| | | | | | | | | | | | | | | | | | | | | | Move this out of the Resources object, which is basically a view of the internal Store object. All event accounting and signaling is now performed by the Store object, to which the Resources DBus object connects to in order to implement GraphUpdated and Writeback signals. Only one handler of these events is possible at the moment, would be nice to consider doing something marginally better on the Steroids interface at some point, at least wrt the amount of data sent through the bus. Instead of trying to schedule the timeout across threads (the TrackerData hooks run in the thread performing the updates, and we want signaling done from the main/dbus thread), the code now just sets up a timeout on the main thread that keeps running as long as there are pending updates. When the task for the last batched update returns, it will be safe for the timeout to do signaling one last time and turn itself down, all of this happening in the main thread.
* tracker-store: Protect ready writeback events with mutexCarlos Garnacho2018-07-161-0/+11
| | | | | | | | | Just like with ready GraphUpdated events, this will be potentially accessed by both the thread performing updates, and the thread doing the DBus dispatching and signaling. Just like there, the chances of contention are rather low, since emission is checked just once per second by default.
* tracker-store: Refactor writeback signal emission into separate functionCarlos Garnacho2018-07-161-21/+24
| | | | Purely cosmetic.
* tracker-store: Give ownership of writeback events on get_ready()Carlos Garnacho2018-07-163-6/+9
| | | | | | Instead of doing get_ready() and then reset_ready(), just give ownership of the events hashtable on get_ready() while resetting the internal one.
* tracker-store: Protect event batches with a mutexCarlos Garnacho2018-07-164-18/+20
| | | | | | | | | | While the pending data and event counter are only accessed by the updates thread, the ready events will be potentially accessed by both the updates and the dbus thread. That said, chances of locking will be minimal, since the get_pending() call only happens once a second (by default) or after the pending buffer grew big enough.
* tracker-store: Do immediate GraphUpdated emission checks on commit hookCarlos Garnacho2018-07-161-3/+5
| | | | | | | | | | | | | | Triggering those on insert/delete callbacks isn't right for two reasons: there could still be a rollback of the just notified data, and it's done from the wrong thread (the one performing updates instead of the main thread). To fix the first, only call this from the commit hook, we can only notify of data that was successfully stored. To fix the second, do the call on an idle that will ensure the main thread running the main loop and doing the DBus dispatching is the one handling the actual emission. At the moment the commit hook is actually executed on that same thread, but that won't stay as-is.
* libtracker-data: Move TrackerClass event maintenance to tracker-storeCarlos Garnacho2018-07-168-412/+339
| | | | | | | This is solely used by tracker-store to keep the backlog of pending GraphUpdated events. This event tracking can move to tracker-store itself, implemented atop libtracker-data's insert/delete/commit/rollback callbacks.
* libtracker-data: Drop CommitTypeCarlos Garnacho2018-07-165-76/+13
| | | | | | | | | | | This basically exists to allow deferring GraphUpdated signals while there's pending batch updates. This is arguably wrong, the priorities should totally affect the order in which updates are processed, but for the sake of interactivity once the data is in the database it makes sense to let the users know ASAP. Now all commits shall set up a timer for GraphUpdated emission is none is set yet.
* libtracker-direct: Add sync() callCarlos Garnacho2018-07-163-12/+52
| | | | This will flush all updates and trigger the WAL hook.
* libtracker-direct: Add internal function to set additional DBManager flagsCarlos Garnacho2018-07-163-1/+12
| | | | | | This will be useful for tracker-store, and the flags that make sense there don't make as much sense to add to the public TrackerSparqlConnectionFlags set.
* libtracker-data: Remove TRACKER_DB_MANAGER_REMOVE_CACHE flagCarlos Garnacho2018-07-163-8/+5
| | | | | It's unused and unhandled. As the TrackerDBManager type is internal API, just remove the flag and shuffle the numbers.
* tracker-store: Do not freeze events during TTL loadCarlos Garnacho2018-07-164-27/+1
|
* libtracker-direct: Add internal TrackerDataManager getterCarlos Garnacho2018-07-163-0/+13
| | | | | | This will make internal users able to access all the gory details that TrackerDataManager has to offer. Will help deduplicate code in tracker-store that is essentially the same than this.
* libtracker-direct: Rewrite in CCarlos Garnacho2018-07-167-453/+893
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In today's chapter of gratuituous rewrites. The instigator of this rewrite is vala's bug https://bugzilla.gnome.org/show_bug.cgi?id=789249. One might argue that bugs are bugs and eventually get fixed, but there's two things that make me think it won't happen soon: - Vala behavior of possibly iterating the main loop from the async task until the task is complete is very much deliberate in order to support the Generator pattern without a main loop, as seen at: https://wiki.gnome.org/Projects/Vala/AsyncSamples#Generator_example - OTOH, glib docs specify that a GAsyncReadyCallback must happen at a later iteration of the main context, presumably to ensure the task is not finished before the async function dispatching the task returns. This is precisely what trips Vala into starting the same task again. I don't see either changing anytime soon, and in the mean time Tracker is largely affected by it, both in perceived bugs (All those nie:url constraint warnings out of the blue had a reason, this) and in real bugs (It's sometimes attempting to insert things twice, and it may even succeed if the query does not break any cardinality/unique constraints). This affects Tracker in too fundamental ways to just shrug it away, unlike the Vala code this C/glib code works just as it looks. Now about the code... It's a pretty boring rewrite, there's a thread pool to dispatch select queries, and a single exclusive thread to handle updates. The WAL hook can possibly use an additional thread to perform non-blocking writes. All very much alike the older code. Future commits will make tracker-store use this connection implementation, so there's only this piece of code handling SPARQL updates.
* Honour TRACKER_DB_ONTOLOGIES_DIR override more logicallysam/functional-testsSam Thursfield2018-07-161-7/+12
| | | | | | Previously, we would ignore this environment variable unless the installed directory didn't exist. This meant that one of the functional tests would stop working once you ran `make install`...
* Restore Unicode quotation marks in a translatable stringPiotr Drąg2018-07-151-1/+1
| | | | | | Fixes regression in commit b458fda8fa03f5ec53f8de9edfeef1b7610aae9c. See https://developer.gnome.org/hig/stable/typography.html
* Add support to TrackerResource for outputting metadata as JSON-LDsam/resource-jsonldSam Thursfield2018-07-157-2/+207
| | | | | This also adds `tracker extract -o json` to dump resources in JSON-LD, in anticipation of corresponding updates in tracker-miners.git.
* libtracker-miner: Warn when overwriting tasks in the task poolsam/tracker-miner-fs-test-fixesSam Thursfield2018-07-151-0/+11
| | | | | | | | | The TrackerTaskPool class and its subclass TrackerSparqlBuffer are designed to contain only one task for any given GFile. If multiple tasks are pushed for the same file some of them might not be executed. This leads to issues such as: https://gitlab.gnome.org/GNOME/tracker/issues/15
* libtracker-miner/tracker-miner-fs.c: Fix "Parent not indexed yet" errorsSam Thursfield2018-07-151-34/+65
| | | | | | | | | | | | | | | | | | | | | | | | | Commit cef502e668a640 ("Add TrackerMinerFS::move-file vmethod") introduced a regression which sometimes led to errors like this: Tracker-FATAL-WARNING: Parent 'file:///tmp/tracker-miner-fs-test-77E2LZ/recursive/4' not indexed yet This was causing tracker-miner-fs-test to fail in some cases. TrackerTaskPool assumes that there is only one task in the pool per GFile. When processing item_move() operations this wasn't true because we'd create one task for removing the existing dest_file, and another task for updating the URL of source_file to point to dest_file. Both tasks would be associated with dest_file. If the SPARQL buffer was flushed after the first task was created and before the second task was created, the second task would overwrite the first task in the ->priv->tasks hash table, so when the first task completed, the second task would be removed from the task pool without ever executing. This would mean that the URL of source_file never got updated to point at dest_file, which triggered the "Parent not indexed yet" error later on.
* libtracker-miner/tracker-miner-fs.c: Rename item_move() parameterSam Thursfield2018-07-151-8/+8
|
* libtracker-miners-common: Improve error when domain rule isn't foundSam Thursfield2018-07-021-2/+8
| | | | | | | | | | | | Before: GLib-CRITICAL **: g_key_file_load_from_file: assertion 'file != NULL' failed CRITICAL **: Could not load domain ontology '(null)': Key file does not have group “DomainOntology” After: ERROR **: Unable to find default domain ontology rule /usr/local/share/tracker/domain-ontologies/default.rule
* meson: Add soversion information to the shared libraries that we installSam Thursfield2018-07-023-0/+4
| | | | This is needed so that Libtool will link against them correctly.
* meson: Expose tracker-store as a variableSam Thursfield2018-07-021-1/+1
| | | | | This allows projects that embed tracker as a subproject to access the variable.
* meson: Fix bundled FTS moduleSam Thursfield2018-06-301-1/+3
| | | | | | | The sqlite3_fts5_init() symbol was being discarded as nothing linked against it (it's loaded dynamically at runtime instead). Using link_whole instead of link_with fixes that.
* meson: Fix name of libtracker-ftsSam Thursfield2018-06-301-1/+1
| | | | The 'lib' prefix isn't necessary in the library() target name.
* libtracker-miner: Emit ::file-updated on file extension changesCarlos Garnacho2018-06-231-0/+25
| | | | | | | | | We have no access to past/current GFileInfos, so detect changes in file extensions to handle the cases where a change of filename results in the file having a different mimetype. In these cases, ::file-updated should be emitted, so it gets the right rdf:types as per its new mimetype.
* libtracker-miner: Transfer attribute updates information through the eventCarlos Garnacho2018-06-231-29/+21
| | | | | | | | | | | First, seems cleaner to do it this way, as GObject data has undefined lifetime (yes, as long as the object lives, but the TrackerFileSystem may cache those). But this also fixes an unintended side effect that "attribute only" updates take precedence over full updates, events themselves may be coalesced away, but data would remain. Actually it's the other way around, if we get a full update and an attributes-only update, we may discard the second.
* libtracker-miner: Emit TrackerDecorator::finished on 0 itemsCarlos Garnacho2018-06-231-3/+0
| | | | | If no items are found, ::finished is silenced away. Better to do this for consistency.
* libtracker-miner: Ignore root folder when crawling at placesCarlos Garnacho2018-06-211-18/+21
| | | | | | | | | | | | There's 2 situations where they may be avoided: 1) When crawling a new folder after it's been detected through TrackerMonitor, as ::file-created is emitted right away since commit 09408f1dcad. 2) When a directory is moved, as ::file-moved will be emitted for it in advance. In these situations it's wrong/undesirable to maybe emit ::file-created/-updated for those.
* libtracker-common: Drop automatic parser versioningCarlos Garnacho2018-06-219-59/+27
| | | | | | | | | | The TRACKER_PARSER_VERSION number must be bumped on every change to src/libtracker-common/tracker-parser*. This policy shall be enforced through a server-side update hook. This is a bit more cumbersome for the OTOH sparse changes to the parser, but friendlier to the meson build system, where we can't have build-generated files added to dist.
* Add missing space in tracker info outputwip/piotrdrag/missing-spacePiotr Drąg2018-06-041-1/+1
| | | | https://bugzilla.gnome.org/show_bug.cgi?id=794496
* meson: Remove duplicated lineSam Thursfield2018-06-011-1/+0
| | | | | This triggered a warning with meson 0.46 and will be treated as an error by future versions.
* libtracker-sparql: Use replacement for IOSchedulerJob.push() which is still ↵Rico Tzschichholz2018-05-301-2/+2
| | | | deprecated
* cli: Fix invalid SPARQL generated by `tracker tag --list`Sam Thursfield2018-05-191-1/+1
| | | | This was causing the command to fail.
* libtracker-data: DBInterface.create_statement() may not be abstract in valaRico Tzschichholz2018-05-161-1/+1
| | | | https://bugzilla.gnome.org/show_bug.cgi?id=796104
* meson: Install generated headers without needing a scriptSam Thursfield2018-04-223-24/+5
| | | | | | | | | | This script dates from a long time ago when Meson lacked ways to install generated headers. This fixes an issue where `ninja install` in tracker.git triggers a rebuild of lots of stuff from tracker-miners.git, which happened because the mtime of the installed generated headers would become newer than the build files in tracker-miners.git and cause ninja to rebuild them all.
* libtracker-sparql: Fix wording of function documentationSam Thursfield2018-04-201-4/+3
|
* tracker-store: Remove empty Icon= from the .desktop filePiotr Drąg2018-03-261-1/+0
| | | | It’s going to be confusing gettext when we port Tracker away from intltool.
* libtracker-data: fix compile issue when using --disable-ftsSimental Magana, Marcos2018-02-131-3/+3
| | | | | | | | | previous commit (66a6dc) refactors code, but some lines missed the refactor. This commit fixes the oversight of those lines. Signed-off-by: Simental Magana, Marcos <marcos.simental.magana@intel.com> https://bugzilla.gnome.org/show_bug.cgi?id=793282
* libtracker-data: Perform VACUUM only on large database filesCarlos Garnacho2018-02-133-1/+17
| | | | | | | | | | The agressive VACUUM on shutdown is the cause for the reported slowness in tracker-store restarts. Let's be a bit more conservative, and only trigger VACUUMing when the database file gets hideously large (4GB). There is the remote possibility that a database is still larger than 4GB after VACUUM. I'll just make it suck at the moment and do the same frequent VACUUMs we get currently.
* libtracker-miner: Emit monitor ::file-created for directories immediatelyCarlos Garnacho2018-02-131-1/+7
| | | | | | | | | | | | On monitor events on directories we used to trigger a full crawling, and only notified the directory during recursive handling. However other monitor events may happen immediately on the child, which would get queued immediately before the parent folder (which is being idly crawled). Emit ::file-created immediately here in order to ensure correct ordering in the TrackerMinerFS queue. If notifier_queue_root() ends up notifying about the file again, the event would get eventually discarded or simply handled as an update.
* libtracker-miner: Don't leak TrackerMinerFS:rootDebarshi Ray2018-02-071-0/+2
| | | | https://bugzilla.gnome.org/show_bug.cgi?id=793061
* libtracker-miner: Don't mix sync & async GFileEnumerator APIsDebarshi Ray2018-01-311-54/+39
| | | | | | | | | This enables enumeration of non-native files because GVfs doesn't like to mix asynchronous & synchronous GFileEnumerators. If an enumerator is obtained through one API variant, then it should be used via the same API variant. https://bugzilla.gnome.org/show_bug.cgi?id=792337
* libtracker-miner: Rename a variable for consistencyDebarshi Ray2018-01-311-2/+2
| | | | https://bugzilla.gnome.org/show_bug.cgi?id=792337
* vala: Bump gir_version to 2.0Ting-Wei Lan2018-01-247-7/+7
| | | | https://bugzilla.gnome.org/show_bug.cgi?id=792691
* libtracker-sparql: Be more careful about integer sizesDebarshi Ray2018-01-231-3/+3
| | | | | | | | The GVariant type string "i" refers to a signed 32-bit integer. Therefore, gint32 is a much safer bet than gint, whose size is not guaranteed across all platforms. https://bugzilla.gnome.org/show_bug.cgi?id=792301
View Lorry Controller Status