Wednesday, February 20, 2013

More Nepomuk File Watcher backends


After being in an ICPC Training Camp here in Brazil, I'm back to work on Nepomuk. I'll now work on implementing the support for more back-end options in the Nepomuk File Watcher service.

Nepomuk has a service called File Watcher, that monitors the file system, waiting for changes in files (content changed, file deleted, file moved, renamed, etc). When that happens, the changed file has to be reindexed, so the search will use its up-to-date contents.

The Linux kernel has a subsystem called Inotify that allows you to do that efficiently. You tell inotify what folders you want to monitor, it calls you when it spots an event of your interest. Currently, Nepomuk uses inotify on Linux to watch for changes. But it has it's limitations. For example, the number of watches you can create in a default installation is small, which may be a problem. Fortunately, there are some alternatives. KDE itself has a mechanism for doing that (KDirWatch), and Linux has the more recent fanotify. Each one has its advantages and disadvantages. What I'll be doing is making the File Watcher support these additional 2 back-ends, and use any subset of the three simultaneously (which will be independently enabled or disabled by the user). With a lot of help from Vishesh, of course.

That's it. Time to learn and code!

5 comments:

  1. Cool!

    One point though:
    "...which will be independently enabled or disabled by the user"

    I think ideally the end-user should not be bother with this choice at all. Nepomuk should automatically choose the best combination of whatever backends it finds installed on the system.

    ReplyDelete
  2. Agreed, to not expose this kind of settings to the user. It feels useless. Make it smart! And thank you :)

    ReplyDelete
  3. Hai Gabriel,

    in my opinion, try to get nepomuk work correctly. I read about many problems with the search results of nepomuk in some forums.
    I use kde 4.10 (kubuntu 12.10) and have disabled nepomuk, because of strange search results. nepomuk doesn't find any mp3 file, he find files in folders, that aren't include in the search folders (yes, i have cleaned the database with the nepomuk cleaner).
    So i think nepomuk is a great think, but it must work correctly to use it.

    Regards

    Volker Holthaus

    P.S.: Please excuse me bad englisch experience, i'm not a native speaker. ;-)

    ReplyDelete
  4. Have you heard abut facron. It seems to do what you want already.
    http://www.imagination-land.org/posts/2012-12-04-facron-fanotify-cron-system.html

    ReplyDelete
  5. Thanks for the comments!

    I agree that the common user shouldn't be concerned about this. The default will always be the optimal combination in most cases. We'll probably leave the "advanced configuration" just in case power users need it, or the default is causing trouble.

    It seems fanotify still doesn't suit Nepomuk's needs (needs root permissions, problems with "directed mode", etc). There's a lot of discussion running about this at the moment. We'll most likely stick with inotify and KDirNotify (not KDirWatch).

    ReplyDelete