Google Summer of Code

Hey, there!

It has been quite some time since my last post here. I'm having very little "free" time as my CS course is tasking me a lot this semester. Two cool programming assignments I'm currently working on are:

• In the Operating Systems course, we have to do some non-trivial modifications in the Linux Kernel source of our choice. I'm planning to do something related to file system monitoring, as this knowledge can be useful for the Nepomuk File Watch service. One specific thing I'll maybe do is implement recursive watches in inotify. Of course, it will probably not be production-ready after that; the most important contribution to Nepomuk will be a better understanding of the inner workings of inotify (and maybe the reason why it doesn't support recursive watches will be clear).

• In the Code Analysis and Optimization class, our first two programming assignments involve writing LLVM Passes. This has nothing to do with Nepomuk, but anyway it's still very cool to hack a real-world compiler.

Besides doing lots of programming assignments and training for programming competitions, I'm also writing my proposal for Google Summer of Code! I plan to take on the only idea related to Nepomuk in KDE's ideas page, which is to rewrite the Nepomuk Query Parser, basing the syntax for queries on a formal grammar, and add support to brand-new features. If everything goes well, in the end, you'll be able to write some very nice queries in KRunner, like:

• "music yesterday" would return the musics you heard the day before

• "(music or document) tagged classes" would search for files with the tag "Classes"

• and others I still didn't think of.

Moreover, queries can also be done internally by other applications. That means the user will possibly see improvements even when not directly doing searches using KRunner or Dolphin (although I don't know a specific use case in which an application makes a query directly).

Also, the project includes adding auto completion to queries, using both Nepomuk keywords and previous queries. That means if you always search for that loved song or work on some documents in a daily basis, Nepomuk will remember you want them quickly :)

That's it for now. I'll probably finish the first version of the proposal in the middle of this week and then get it reviewed. Hope I'll have some news soon!

Hello, Planet!

Hello, PlanetKDE! 

I'm Gabriel Poesia, a Brazilian Computer Science undergrad student at UFMG who is passionate about algorithms (I compete in programming contests, like ICPC) and free software. At the end of the last year, I started contributing to Nepomuk-KDE. I created this blog primarily to describe my first steps in the world of free software and talk about my work inside Nepomuk. Both writing here and contributing to KDE have been very interesting experiences for me, and I hope this blog can be useful and motivating to others, too. KDE is great, but still better than it is its community. There are very talented people working on it, and getting into it is a valuable opportunity to learn a lot about software development in general in real-world scenarios. The community is open and welcoming. That's easy to see while hanging on IRC!

In Nepomuk, until last month, I was mainly tackling minor issues, while understanding Nepomuk better. Today, I pushed NepomukCtl, a tool for controlling Nepomuk services, similar to akonadictl. Besides that, I'm working on making the FileWatcher service support different, independent back-ends. I'll probably participate in this year's Google Summer of Code, too! All this, of course, is not lone work. Help from fellow developers is always involved when working on KDE.

Nepomuk borrows many ideas and technologies from the Semantic Web (the name "Semantic Desktop" has a reason). Understanding the Semantic Web is very helpful when trying to grasp Nepomuk. So, I wrote a series of two posts about the Semantic Web. The first post (Oh, the Semantic Web... wait, what?) talks about its fundamental ideas, concepts and objectives. The second one (Making the Semantic Web work) goes a little deeper and shows how one would really implement those apparently abstract ideas. Many technologies cited there are indeed used in Nepomuk.

Well, that's it! I hope you enjoy my blog. You can find me on IRC, mainly in #kde-devel and #nepomuk-kde (my nickname is gpoesia there). 

More Nepomuk File Watcher backends

After being in an ICPC Training Camp here in Brazil, I'm back to work on Nepomuk. I'll now work on implementing the support for more back-end options in the Nepomuk File Watcher service.

Nepomuk has a service called File Watcher, that monitors the file system, waiting for changes in files (content changed, file deleted, file moved, renamed, etc). When that happens, the changed file has to be reindexed, so the search will use its up-to-date contents.

The Linux kernel has a subsystem called Inotify that allows you to do that efficiently. You tell inotify what folders you want to monitor, it calls you when it spots an event of your interest. Currently, Nepomuk uses inotify on Linux to watch for changes. But it has it's limitations. For example, the number of watches you can create in a default installation is small, which may be a problem. Fortunately, there are some alternatives. KDE itself has a mechanism for doing that (KDirWatch), and Linux has the more recent fanotify. Each one has its advantages and disadvantages. What I'll be doing is making the File Watcher support these additional 2 back-ends, and use any subset of the three simultaneously (which will be independently enabled or disabled by the user). With a lot of help from Vishesh, of course.

That's it. Time to learn and code!

Qt 5 released!

The wait is over: Qt 5 is now officially the latest Qt version. And the guys at Qt Studios made a very "qt" video about it.

KDE is a giant ecosystem built on and heavily relying on Qt. Many Qt developers are KDE active contributors, and many KDE contributors came up to work with Qt for a living. For example, there's KDAB. And, of course, a new Qt release affects KDE a lot. KDE's major version is the same as Qt's. We're in KDE 4 because it uses Qt 4. KDE 5 (which will still take some time) will need a whole port to Qt 5.

This may cause some fear and uncertainty to the KDE users that were there in the early stages of KDE 4. KDE 3.5 was amazingly stable. When Qt 4 was released, the obvious path of porting KDE to Qt 4 was followed. However, considering KDE's size and the many incompatibilities between Qt 3 and 4, that was terribly hard, and KDE suffered for a long time with this. Many bugs originated at the port survived until KDE 4.4 or 4.5, which is not long ago. A lot of users migrated to other desktop environments in the occasion, keeping in mind a horrible image of KDE.

Nevertheless, everything promises a much less painful port this time. From the Qt 5 Docs:

"Qt 5 is highly compatible with Qt 4. It is possible for developers of Qt 4 applications to seamlessly move on to Qt 5 with their current functionality and gradually develop new things leveraging all the great items Qt 5 makes possible."

And that doesn't look like just propaganda. The list of source incompatible changes list seems reasonable, and KDE Frameworks 5.0 is being worked on. This is a time where unit tests are most helpful. Since this year, Nepomuk has its own tests framework. As Nepomuk is split into various services (File Watcher, File Indexer, Storage) that run in different processes, it's quite hard to test some changes in it. And what can be said about testing a whole port to Qt 5 without automated tests?

By the way, in Nepomuk, I've be working on some minor tasks in the last two weeks. Briefly, I investigated an optimization in RegExpCache, a class that mainly checks if a string matches against a list of regular expressions. It turns out Qt's QRegExp class is implemented as a Nondeterministic Finite Automaton, and so matching is not done in linear time. Thus, the straightforward trial of making a RegExp that is the union of many RegExps didn't work. Now, I'm trying to fix another bug, related to the File Watcher. This made me learn a lot, and probably another post will follow just about that one :)

Developer account!

With Vishesh Handa's support, I got a KDE Developer Account! Using it, I pushed the commit that implements the refactoring described in the last post. "With great power comes great responsibility".

By the way, in January, when I'll have time, I plan to read some books on semantic technologies, like ontologies, RDF and OWL. Fortunately, many books on these subjects are available in my university's libraries. I'll go to the Faculty of Information Science library and choose two from:

1- OWL: representing information using the web ontology language
2- Programming the Semantic Web
3- Semantic Web Programming
4- Semantic Web: a guide to the future of XML, Web services and knowledge management

Who knows if knowledge won't be useful outside Nepomuk. Although Google Trends shows a steady decrease in the number of searches for "Semantic Web" and related terms, Oracle, IBM and even Google have done work and research on the subject, with some positive results. Also, Semantic Web technologies don't need to be applied to the Web - Nepomuk proves it.

Next step - refactoring

The next task I will work on will be a simple refactoring one, just like the first. I'll have to wait the end of this semester to start working harder on Nepomuk, but meanwhile I can still do something of some use. While the task itself isn't enough for much talk, the fact that it is a refactoring one is worth some attention.

Refactoring is not something an undergraduate student would care about. Why rewrite some functionality that is already working when you'll submit that programming assignment and never look at it again? This line of thought is perfectly applicable to projects of that nature - few lines of code, short life cycle, no perspective of growth.

However, when a code base starts to be measured in tens or hundreds of kilo lines of code (KLOCs), of which a good part was written years before, the situation changes drastically. Federico Quintero, known for co-founding the GNOME project, describes a state of badly written software that Refactoring practices try to thwart:

"When I was learning how to program, I noticed that I frequently hit the same problem over and over
again:  I would often write a program that worked reasonably well, and even had a good structure, but
after some time of modifying it and enhancing it, I could no longer tweak it any further.  Either its
complexity would overwhelm me, or it would be so tightly written that it allowed no room for
expansion, like a house where you cannot build up because it has a sloping roof, and you cannot build
to the sides because it has a wall all around it." [1]

This situation is perfectly conceivable. I have experienced it - like probably all programmers that started personal projects while learning to program. Continuous refactoring is a must to avoid that. I think this is even more true when the subject is a free software project. In an enterprise environment, people are paid to work, and it may simply not be an option to avoid touching that project with a messy code base and flawed architecture. In the free software  world, on the other hand, people get in and out of projects everyday and as they wish - free software, free people. It's difficult for a newcomer to engage in a project where the simplest change can break everything, and parts that were meant to be simple are in fact hard to understand and tweak. And a free software project that fails to attract new blood to it is unlikely to prosper. KDE has a lot of initiatives that help to increase its Bus Factor (like the Techbase, Season of KDE, strong participation in Google Summer of Code, mentoring programs), and it seems to work well. At least for me.

That's it for now! While refactoring is surely good for the health of the project, it will also make me learn more about Nepomuk. The two patches I wrote were against the kde-runtime repository, where the configuration manager lies, and this one will be the first inside nepomuk-core. Time to work!

[1]: Software that has Quality Without a Name (which is part of a great book, Open Advice - What We Wish We Had Known When We Started, available for free)

Starting to contribute

Last week, I did my first effective contribution to a free software project (Nepomuk-KDE). Although the commit message doesn't tell it, the change was very simple; just a small refactoring. But before getting to that, I'd like to say a little about how KDE seems to welcome new developers.

First, there is the KDE Developers Guide, a concise e-book summarizing what you need to do to begin. I chose a cool project and tried to compile it, cloning Nepomuk's git repository and trying to build it as usual (cmake + make). After solving some dependencies (needed to install soprano's development packages), I managed to build it. But then I wondered if I would have to compile a whole KDE Environment to develop and test it, because of the nature of the project. The KDE Guide tells you to ask the developers directly in this situation, and so I did. I went to its IRC Channel and said I wanted to contribute. In a while, I got an answer from jEhrichs. And no, I didn't need to compile the entire KDE :)

After a couple of days, I talked to Vishesh Handa (vHanda), one of the main Nepomuk developers, and he pointed me to how I should build Nepomuk - inside a build environment, in which I would install it so it doesn't interfere with my system's installation. The process is thoroughly explained in the Techbase. After that, I was ready.

Also, there's KDevelop. It seems to be a very neat IDE, that integrates with many tools common in KDE's development (Git, CMake, Make, even Review Board!). One nice feature it has is to create a project from a CMakeLists.txt file. The auto completion works wonders, too. I still didn't try to integrate it with my build environment, so I'm currently compiling from a terminal and only using KDevelop to edit the source code. It's hard to beat the command line interface in practicality, so I'm not very tempted to do everything from the IDE, but I'll give it a try later on.

If all that documentation and kindness from people on IRC is not enough, KDE's Bugzilla has a tag called "junior-jobs", that's used to mark considerably easier tasks - those that are appropriate for a beginner to tackle. vHanda pointed me to this task. It was very simple indeed. I provided the patch, and it was accepted and pushed! Hurray!

The next issue I tackled was to add a button to make Nepomuk check for new/changed files. KDE has a standard regarding usability, and I've already learned some from that. Choosing where to put a button was harder than I imagined, especially because programmers rarely worry too much about such decisions. Just in reading the Techbase and discussing my patch in the code review, besides learning some things, I realized how bad many user interfaces I made before were. There are some simple tips, like not using an annoying dialog to tell the user something not-that-important and force him to click OK, that I had always missed... Usability is one more thing KDE will teach me :)

However, this last patch won't be pushed soon, because the KDE Release Schedule has already closed the period in which new strings can be added. That's fair, given that the UI needs to be translated and there isn't much more time do to that before the next release, KDE SC 4.10. So, it will have to wait until 4.11.

But before that there's plenty to do! There are new issues and junior jobs in the Bugzilla waiting for patches. Doing them is a concrete way to get more used to Nepomuk's architecture, source code and tools. As soon as I get the time, I'll code some!

Summarizing: it's wasn't hard to start contributing. There is work for newcomers, and the community is open, just like the source code :)

Hello world!

Hi there! I'm Gabriel Poesia, a Brazilian Computer Science undergraduate student beginning a journey in free software and competing in programming contests. It's the first time I post to a blog. I'm starting to write this one to:

- Describe my first steps in contributing to free software, namely KDE and Nepomuk,

- Write about interesting computational problems and algorithms, mainly those related to programming contests, like ICPC and TopCoder,

- Write more! I rarely write in English when not programming, and I think it's a good opportunity to practice it.

For these three reasons, I'm sure this blog will be useful to me. I'd be happy if anyone else could benefit from it. I'll try to write here twice a week.

So, where did this idea come from? I thought it was already time to give something back to the free software community. We all use lots of free software, directly or indirectly. Just to start, even if the browser you're using to read this post isn't free software, you probably found this page using Google, which internally uses a lot of FOSS.

If you try to measure how your life is affected by the free software movement as a whole, you rapidly realize it's impossible to contribute back in the same amount. In spite of that, we can do something (little, but still something) to help. 

Gratitude is already a nice reason to contribute, but there are tons more. For instance, it's an invaluable opportunity to learn about API design, large software release cycles, real-world technologies and also, as an ICPC contestant, apply non-trivial algorithms to real problems. Additionally, you get to know very experienced and knowledgeable developers from all over the world, and make something that impacts many users.

Nepomuk, the project in which I'm starting to get involved, has all of that. It's a framework that intends to provide the whole KDE Desktop with metadata and a powerful search that goes way beyond file names. That means it interacts with many others projects, and thus needs to offer a very well designed API. Also, given the amount of information we have in our hard disks, it needs to handle large sets of information efficiently, and as seamless as possible in the user's point of view, as you don't want a slow browser because there's something indexing your files.

If that's not enough, it also utilizes many technologies that will probably be everywhere in the future: ontologies. If you've heard of the Semantic Web before, then you have a pretty good idea of what's the philosophy behind Nepomuk, but targeting the desktop. And the Semantic Web is based on ontologies and file formats like RDF. And so is Nepomuk.

That seems enough for a first post. In the next ones, I want to talk a little more about my first steps in Nepomuk. This week, I managed to get a commit pushed for the first time. I'd find it very nice if more "first commits" from other people appear after this one :)