Eugeni's blog

Working on Mandriva network tools, I looked on one of the most essential ones the network monitor (net_monitor). It was introduced a couple of releases before, and was mostly doing its job. However, it has a number of flaws and lack of features that motivated us to look closer at it.

Our old friend net_monitor, present in your favorite Mandriva distro!

The net_monitor currently used in all Mandriva versions is written in perl, is using internal drakx-net api (and is, therefore, only usable on Mandriva), and also have some issues such as memory leaks and non-usual interface. After a few thoughts and discussions we came to conclusion that it would be more adequate to project and rewrite it from scratch, turning it more modular, expansible and focused on common use cases.

Initially, I thought on using perl to write it, so it would still be part of drakx-net suite. However, after thinking on the code and the way it should work I felt that my brain was going to melt down (perl is a nice language, but it is certainly not that compatible with me). So I ended up with python, which is my language of choice (together with C). Also, I’ve received many comments saying that the net_monitor is no more relevant, as every desktop environment provides its own network monitoring tool, and it should be dropped from drakx-net. By combining those issues, we came to decision that it would be more proper to separate net_monitor into a different package – this way, it won’t depend on any drakx-net internal functionalities, and user could uninstall it if required and use his own network monitoring tool if he wants to. And, at the same time, users would still have a cute little network monitoring application on their machines.

So, as a picture says more than a thousand words, I guess I’ll just add some pictures here than additional KBs of text (EDIT: please note that the look and features of net_monitor have changed significantly in Mandriva since this post):

net_monitor monitoring a wireless connection

net_monitor monitoring a connection for which network accounting was not enabled

net_monitor displaying some statistics about your network usage (provided by vnstat)

looking at daily traffic statistics on my notebook for the past month

...and hourly statistics...

...and finding our when I killed my bandwidth..

Surely, this is just an early and preliminary version, with many missing features and such. If you want to give it a try, just install net_monitor package, and it will create /usr/bin/net_monitor executable for you. It won’t conflict with existent net_monitor from drakx-net which is installed in /usr/sbin, so both of them may coexist on your system. If you look at /usr/share/doc/net_monitor/TODO, you’ll see some of the ideas that I intend to add to it, but the idea is to keep it simple and not transform it into an emacs of network monitoring . And, of course, feel free to add your comments and suggestions (and bug reports) here!

P.S.: Just to prevent comments like ‘you should focus on fixing bugs instead of wasting time writing new things’. Net_monitor is present in Mandriva for years now, and if you look at bugzilla list it has a number of bugs and issues. So I am not creating a new app – I am bringing back from the land of the dead an old one .

P.P.S.: Answering in advance to another question – yes, it would work on any Linux distro which has python and pygtk. You’ll just have to add some tricks into your network startup scripts to enable vnstat integration, but it will work just fine even without that.

One of non-trivial tricks involved in web site scalability is the optimization of all image files. One of the sites I am helping to take care of has a front page with more than 350KB in .jpeg images. And, obviously, it takes lots of time to load and, considering the number of accesses, the bandwidth is huge. Usually, those images can be optimized in photo editor, or saved with higher compression or lower quality, but sometimes there is not much else you can do. Or you think so.

One quick trick to improve this situation is by converting some images to png with ImageMagick and running pngcrush on them. A simple script can be used to do so:

    #!/bin/bash
    totalsize=0
    for file in *jpg; do
            # file.jpg becomes file.png
            newfile=${file/jpg/png}
            # convert to png
            convert $file 1.png
            # compact with pngcrush
            pngcrush -brute 1.png $newfile > /dev/null
            # calculate old and new sizes
            newsize=$(wc -c < $newfile)
            oldsize=$(wc -c < $file)
            if [ $newsize -lt $oldsize ]; then
                    echo "$file: reduced from $oldsize to $newsize bytes"
                    # remove old jpg file
                    rm -f $file
                    # replace all references to old file everywhere
                    sed -i -e "s/$file/$newfile/g" *
                    totalsize=$[$totalsize + $oldsize - $newsize]
            else
                    # old file is smaller, remove new file
                    rm -f $newfile
            fi
    done
    echo "total reduction: $totalsize"
    # remove temporary file
    rm -f 1.png

By running it on the website in question, it managed to shrink the front page by about 200KB of image data. Considering 10000 daily accesses, it would save about 2GB of network traffic per day.

The following picture pretty much explains this post title:

Dward Fortress learning curve

If you enjoy state-of-the-art ASCII graphics, infinite gameplay, endless game possibilities and want to spend most of the rest of your life trying to understand how to play this game , you’ll certainly enjoy it!

The game page is here, and if you want a direct link to latest Linux version it is here. Ohh, and if you want some a bit more user-friendly UI, check out here.

Following yoho’s post, I thought it would be a good idea to join the Linux Planet project. My blog (well.. at least its technical part) is quite in line of the project’s goals: it is in English, it is technical, its content is all written by me, and it has the most reliable information out there about few opensource projects (for example, msec and netprofile ).

So, why not? From now on, this blog will land on Linux-Planet too!

Time has come for some msec updates.

With base on previous post, I was working (among other things) on few msec ideas. And now it looks like a good time to put them out to cooker.

First of all, I added support for exceptions into msec periodic check. I wanted to make it as flexible as possible, and I think I managed to implement everything I wanted. Right now, for each supported periodic check, it is possible to define as many exceptions as necessary. So, for example, if you run a local mandriva mirror with unsecure permissions on files, or want to exclude certain rules from firewall check, or some local users that appear unsafe to msec are safe to you, you can tell it to msec, and it will not bother you about it anymore.

For this, /etc/security/msec/exceptions file is used, and it is possible to define as many rules as necessary for each check there. The syntax is quite simple: RULE_NAME exception. To illustrate, that’s what I put into my local exception list on my machine:

CHECK_UNOWNED /home/chroot
CHECK_UNOWNED /home/images/chroot
CHECK_WRITABLE /home/chroot
CHECK_WRITABLE /home/images/chroot
CHECK_OPEN_PORT /deluge
CHECK_USER_FILES gdm
CHECK_OPEN_PORT eugeni:ircd

This way, I won’t receive msec alerts about unowned and world-writable permissions in chroots, about gdm home directory being accessible to the world, and about network ports used by deluge or connected to local ircd server. Each exception is a regexp, so the possibilities are endless.

Of course, it is possible to do it in the gui:

Showing the list of configured exceptions in msecgui

Adding a new exception

Besides that, as suggested in the last post, I also added a summary to periodic msec checks. So if you want to have a quick look on the results, you don’t have to read the entire mail.

Also, a few annoying bugs were fixed and few features were added.

But, besides that, I also contacted vdanen, the author of the rsec tool, and the sectool guys about some possible interaction between our projects. Hopefully, we’ll have some news soon.

Meanwhile, enjoy new msec and (as always) feel free to give your feedback over it.

With recent posts by vdanen and adamw, and a recent cooker mailing list thread, it became clear that msec is a very important project/package, and it should deserve much more attention and feedback.

As you probably know, msec underwent a huge redesign for Mandriva 2009.1, and it is getting a lot of attention for 2010.0. But that’s still not enough – even if it became a quite flexible and extensible package, it still has its rough edges, and I intend to solve them all. Of course, it won’t became a perfect package that would rule-them-all, but I intend to get as close to this objective as far as it is humanly possible .

So, please, if you use msec, or rsec, or sectool or any other security-concerned framework – please, speak about what you want to see in them, what are the points you are missing, and what features were left unimplemented for the time being.

As for me, I have the following items in the roadmap:

• implement skip list/exceptions for msec, for every possible test, in a similar way to mandriva bug #53307
• do my best to provide a nice common source base for both msec and rsec (I hope vdanen would be interested in that as well). Right now it is possible to configure msec to behave exactly as rsec, doing security checks and nothing besides that, but that is not that trivial to do (well.. it is for me, but not for any casual user out there ), and it should be beneficial to both projects
• provide support for sectool plugins in msec – either directly, or by converting them to msec-parseable format
• work with rsec/sectool/checksecurity/seccheck developers to provide a similar set of features for all those projects. We live in opensource world, and advances in one projects would certainly benefit all of us – specially in such critical area as system security.

So, if you have suggestions, ideas, features or any sort of comments – please, speak. We’ll hear you.

Usp finalmente colocou a minha tese na net! Uhuu!

Para quem quiser mergulhar numa aventura emocionante, conhecer as teorias, mitos e lendas do Grande Caos, comparar a sua inteligência com a artificial, e participar de uma busca dinâmica afim de e averiguar a verdadeira autonomia de computação, sintam-se convidados a se escalar nessa jornada!

http://www.teses.usp.br/teses/disponiveis/55/55134/tde-05082009-205709/