Eugeni’s blog

Defender o doutorado com camisa de Corinthians a tarde no dia que o Corinthians é campeão brasileiro a noite – não tem preço!!

Dr. Eugeni

Well, the title says it all . Yep, it is true – after 5 years, 3 journal publications, 7 conference publications and 1 book chapter, I am finally a PhD .

Update: a short description about the nature of the thesis is here

Time has come for the first msec release since Mandriva 2009.1!

This time we have several improvements, such as:

• support for audit plugins
• more msec auditing checks
• improved auditing logging
• and, of course, bugfixes.

So let me introduce some details about each one of them.

Support for audit plugins

You may remember that msec shipped with Mandriva 2009.1 introduced support for plugins infrastructure (take a look at your /usr/share/msec/plugins/ directory to see some examples). This new msec, which will be shipped with Mandriva 2010, also introduces auditing plugins.

Well, you might be asking what the ..? what is the difference between those plugins?, so let me clarify it a bit.

Msec has two main functionalities:

• Security configuration
• Security auditing

The security configuration is what you configure using msecgui or using security levels – basically, you say what settings should be used on your machine for ssh, user logins, and all kind of system configuration. The security auditing are those background checks that run daily on your machine, to determine what has changed since the last run and let you know about that.

In old msec, this security auditing was performed by security.sh, security_check.sh and diff_check.sh, so we had just three large and complex files with a lot of duplicated code. With new msec version, everything was split to reduce code duplication, improve readability and simplify plugins creation.

Let me show you a sample plugin which checks for changes in system users:

    #!/bin/bash
    # msec: check for changes in local users

    # check if we are run from main script
    if [ -z "$MSEC_TMP" -o -z "$INFOS" -o -z "$SECURITY" -o -z "$DIFF" ]; then
            # variables are set in security.sh and propagated to the subscripts
            echo "Error: this check should be run by the main msec security check!"
            echo "       do not run it directly unless you know what you are doing."
            return 1
    fi

    # files to log the list of today's and yesterday's, and difference between them
    USERS_LIST_TODAY="/var/log/security/users_list.today"
    USERS_LIST_YESTERDAY="/var/log/security/users_list.yesterday"
    USERS_LIST_DIFF="/var/log/security/users_list.diff"

    # update yesterday's list
    if [[ -f ${USERS_LIST_TODAY} ]]; then
        mv ${USERS_LIST_TODAY} ${USERS_LIST_YESTERDAY};
    fi

    # check for changes in users
    if [[ ${CHECK_USERS} == yes ]]; then
        getent passwd | cut -f 1 -d : | sort > ${USERS_LIST_TODAY}
        Diffcheck ${USERS_LIST_TODAY} ${USERS_LIST_YESTERDAY} ${USERS_LIST_DIFF} "local users"
    fi

that’s it. You just drop this file into /usr/share/msec/scripts/01_check_for_users.sh and this check will be executed every time msec security checks are run. The security log will be updated, the diff check mail will be created and mailed (along with all other checks), and it will be working automatically from now on.

More msec auditing checks

A few additional msec auditing checks were added:

• CHECK_FIREWALL — checks for changes in iptables configuration
• CHECK_USERS — checks for changes in local users (most of its code was shown above actually)
• CHECK_GROUPS — checks for changes in local groups
• FIX_OWNER — if unowned files are found on the system, this check gives the opportunity to change their ownership to nobody/nogroup, instead of blindly doing it automatically
• CHECK_RPM_PACKAGES — checks for changes in installed RPM packages
• CHECK_RPM_INTEGRITY — checks all the installed packages for changed files. Both those checks were run before under the CHECK_RPM check, but, as they are quite expensive, these two new checks were introduced instead

If you are using cooker or 2010 alpha, these options will not be added automatically to your /etc/security/msec/security.conf configuration file. The best way to experiment with them is by using msecgui, or running msec -f standard or msec -f secure to install default configuration for standard and secure levels.

Besides those items, I was thinking on an option to check for changes in PAM authentication, check for failed login attempts and support for rkhunter. And, as always, if you have any idea on some other functionality that should be interesting to have in msec, feel free to comment!

Improved auditing logging

The logging format of /var/log/security.log was changed to be compatible with syslog-based logging. This should make it easier for system applications to parse it, and for administrator to examine its contents. Now it is way easier to find information by date, kind of message and check type.

Other ideas

Among other ideas for msec I thought on the following:

• msec supports an arbitrary number of custom security levels, but msecgui only supports two basic ones (standard and secure). It could be nice to have a combobox to select a custom profile..
• gui for TOMOYO security framework, since the AppArmor project looks quite stone-cold dead. This is already a work in progress, so probably I’ll post some update on this later.
• Support for administrator-supplied rules for security and diff checks. For example, to exclude everything matching ‘/var/tmp’ from any kind of checks and reports, or excluding network ports from 3000 to 5000 from open port checks.

Besides that, there is a number of bugfixes (which are going to be backported to 2009.1 shortly).

So msec is definitely is alive and getting better and better. Stay tuned for more news!

Well, this question appeared quite frequently to me. However, I never bothered with it, as I was either on a LAN, or had a different source from which I could resume using wget, or a file was sufficiently small to redownload it again. However, this time these approaches did not work:

• The file was big (a DVD ISO)
• The only way to access it was over a SSH connection
• The only authentication method it supported was public key authentication
• The directory from where the file was downloaded was read-only
• The link was sloooow
• I already had downloaded about 70% of the file

So I started looking for solutions. Most of ideas I found on google suggested using ‘rsync –partial –rsh=ssh‘, and indeed it could work. However, rsync tried to create a temporary file on the server, and, as the directory was read-only, it failed. There probably is some option to make it work, but I don’t have plenty of rsync experience. And this approach just looked to be over complicated.

After a bit of more googling, I found out that curl supported sftp backend. And, after a few minutes trying to figure out how to make it work with public key authentication, I finally figured it out:

curl -C - --pubkey ~/.ssh/key.pub --key ~/.ssh/key \
  sftp://eugeni@somewhere/mnt/.../i586/my_precious_iso.iso \
  -o my_precious_iso.iso

To shorten it up, it is possible to write a simple wrapper function (or a script) for bash:

#!/bin/bash
function scp_resume() {
        URL="$1"
        FILE="$2"
        if [ "a$FILE" == "a" ]; then
                echo "Usage: scp_resume <sftp url> <local target>"
                return 1
        fi
        # the magic
        curl -C - $URL -o $FILE
}

function scp_resume_key() {
        URL="$1"
        FILE="$2"
        KEY="$3"
        if [ "a$FILE" == "a" ]; then
                echo "Usage: scp_resume <sftp url> <local target> <key file name>"
                return 1
        fi
        # the magic
        curl -C - --key $HOME/.ssh/$KEY --pubkey $HOME/.ssh/${KEY}.pub $URL -o $FILE
}

so it did the trick.

Just tried the about.me firefox extension with some quite fun results:

that’s all the hard work..

I noticed that this graph does not counts everything – more likely, the top-10 sites are the ones accessed using direct links, or by typing the site address manually. I access a lot of news from google reader, so they count as google.com domain. But still a nice statistics.

Together with Wordpress update, I experimented the new Wordpress theme search-and-install functionality. As you can see, I found a new theme for this blog, which is a bit lighter and cleaner than the last one. It also uses some nice javascripts all around, so the blog is a bit more ajax’ified right now.

So, after a bit more than year, old design is dead – long live new design .

Everything seems to be working nicely. Automatic update is great!

Was just playing with Google Chrome for Linux on my Mandriva installation.

In a few words – it rocks! There are still a lot of issues and non-implemented features, but even not it is:

• extremely fast
• surprisingly stable
• opens most of my favorite sites without any problems

It works pretty fine on Mandriva 2009.1/cooker, the only necessary thing is to setup the right links in /usr/lib for its libraries. But after that, it just works. I just miss the firefox extensions, like adblock, mouse gestures, stylish, twitter and delicious.com integration, but.. one cannot had it all .

Well, as the PhD is getting closer to the end, I decided to test what I learned so far in 0×1C years of existence. So I found this test:

(yes, I know, I could do better..)

As for geekery:

(that was easier.. I guessed two of the questions though)

So I am more geek than nerd. Good to know .

I still remember one of the old ones, with 500+ questions or something like that.. It took almost an hour to finish for the first time. Good old times, when computers were big and softwares small…..

I think I’ll never forget about this next time I see OMG or 0_0..

Ohh, and >:| definitely is suitable for closing bugs as RESOLVED->INVALID

I wondered why my .git directory of drakx-net was using about 70MB of disk space (I am accessing the SVN repository using git svn). Of course, I have read that periodic git repository clean could drastically save space and speed, but – what the heck – it is just a bunch of text files. So I never bothered with it.

However, after running git gc on top of drakx-net directory, the .git directory size went from 70MB down to 4MB. A 17.5x improvement! Unbelievable!

So I did the same to msec repository, with quite similar results — from 21MB down to 3MB!

So a mental note to myself – run git gc always. It rocks.

I was once again battling against the lack of free space on my hard disk (mp3 collection grows up, and I don’t have the courage to delete most of the things), so I ended up removing my Arch Linux partition. It was my default OS for about 3 years, it is one of the best distros around in my opinion, and I recommend it for anyone interesting in having a fully-customizable, dynamic, extremely fast and tunable system. But right now I am using Mandriva on all my machines, for several reasons:

• 2009.1 simply rocks .
• with the time, most of my scripts became distribution-independent (they work the same way on arch, Mandriva, Ubuntu, Fedora, and (sometimes) even FreeBSD).
• There are some things that chroots and VMs cannot do for you. So I end up using Mandriva daily anyway.
• The rolling-release style of life of Archlinux is pretty much similar to cooker (except that cooker breaks much more often). So I still feel like home .
• The latest Arch Linux updates broke my X11, and it was too boring to look deeper into it..

In some kind I feel I am back to the origins – my first Linux distributions were Slackware 2, Conectiva 3 and Mandrake 6 (…and their ’sarcastic penguin’ console ascii art which I miss a lot ).

Achei esses perdidos no meu micro

segway de pobre

So very true..

A Evolução

O primeiro celular

I am using MPD to play my small collection of mp3:

[eugeni@eugeni 16:53:31 ~] $ mpc stats
Artists:    787
Albums:     747
Songs:     9112

Everything is controlled by keybinding – for example, to run the play command, I press < Super >+Up; to run next command I press < Super >+Right, and so on. These shortcuts start a small client to mpd, which is nothing more than a shell script which:

1. Starts MPD if it is not running (I do not start it on startup to save a few seconds of boot time)
2. Runs mpdscribble in background to update the LAST.FM statistics
3. Runs all required MPD commands
4. And displays the results in a nice way

And it is multi-distro-oriented by the way .

    #!/bin/bash

    # Is mpd running
    mpc status 2> /dev/null
    ret=$?
    if [ ! "$ret" = "0" ]; then
            notify-send -i audio "Music" "Starting mpd.."
            # are we on arch linux?
            if [ -x "/etc/rc.d/mpd" ]; then
                    sudo /etc/rc.d/mpd start
            else
                    sudo su -c "service mpd start"
            fi
            # starting mpdscribble
            mpdscribble
           # starting sonata
            sonata --hidden
    fi

    mpc $*
    # Some black magick to get a bit more advanced mpd status
    # and convert it to shell variables
    eval `echo -e 'status\ncurrentsong\nclose\n' | \
        nc localhost 6600 | \
        sed -e '/OK/d' -e 's/: \(.*\)/="\1"/g'`
    # show the results
    notify-send -i audio "$Artist - $Album - $Title" \
        "$(mpc status | sed -e 's/&/and/g')"

Just yet one another script which contributes to world’s entropy .

I spent the entire week at Mandriva’s office in Curitiba (usually I work remotely from home, in São Carlos – about 500KM from there). It was a great, great week – I finally met the rest of the team (I already knew most of them because of past projects, and over the irc). And I love Curitiba city too – it is probably one of the most beautiful cities I know.

While there, I finally fixed all pending release-critical bugs which were assigned to me (including the infamous b43 bug ). And, while waiting at the airport, I got bored and added support for Brazilian VIVO cellphone provider to drakconnect.

But overall, the trip was great! Looking forward for next one.

Also, not that long ago I discovered Nexuiz (opensource 3d shooter, based on Quake engine, but HEAVILY modified). From all opensource shooters I know, this looks one of the best. However, the recently-released 2.5 version has increased the size of game data to 600MB. I built it here on my machine, and was playing quite happily against bots for some time. But I don’t know if it is a good idea to submit 600+MB over SVN to Mandriva repository. Just to think about how long will it take scares me.

And finally, I (once again) tried getting used to KDE4, with a bit of persuasion from Helio Castro. From all KDE versions I used, this one is, in my opinion, probably the best. I managed to stay with it for about two days, but then returned back to XFCE. If the development of KDE goes with the same pace as of lately, probably the next KDE release will be good enough for me . But for now, nothing like good old XFCE 4.6!

P.S.: The picture on this post was taken by bedi. I have no idea how he managed to put those effects on the photo (probably it was beer effect ), but it looks cool! hehe.

(pt) Só para deixar registrado – a partir de hoje, Eugeni->Age == 0×1c!

(en) Just to make it registered here: starting today, Eugeni->Age == 0×1c

Happy BEERthday to me. I know it is not right to spell it like this, but it is definitely much more fun .

With a few commits to s2u and msec source, msec finally got the long-announced desktop notifications support.

It is still too basic, and not-too-fancy, but it works. So with this item, I’d say that the msec rewrite for Mandriva 2009.1 is done. There are still some bugs to fix, and strings to translate, and implement the support for AppArmor as soon as updated packages are released, but all those items are quite simple to do.

Besides, msec works quite well on other distros as well. It wasn’t my main focus until now, but I was thinking on making packages of it for some other distros. And maybe writing a Fedora/CentOS/RH plugin to support selinux relabeling from msec, or add support for DPKG checks from within msec, or finally get to implement snort/ossec plugins.. the list of ideas is quite long, soon we’ll find out what will happen .

I’ve catched some terrible flu last week (around 39.5 of fever most of the week, and all sort of problems that come together with that), so development was going not that fast as I hoped. I finally got better around Friday, and managed to go to Iron Maiden’s show in São Paulo (biggest Iron Maiden show EVER – about 100000 people watching! Amazing!), and also finish some long-time pending tasks.

One of such tasks was the drakfirewall support for NFS. This one was a bit tricky, because it required modifications to nfs-server and nfs-common to enable NFS to listen on fixed ports. My initial idea was to simply put it into nfs-utils configuration files, but this approach was not well-received on cooker list . So I went to the second solution – implementing on-the-fly configuration of such ports in drakfirewall. This required some additional (transparent) steps in firewall configuration, but the resulting structure became quite flexible. Right now, drakfirewall configures NFS to use fixed ports automatically, and also detects custom user changes and adapts itself accordingly.

The funniest part of this is that it was the oldest bug in my queue (#7689), dating back to 2004. A blast from the past .

Besides this fix, there were lots of other bugfixes in drakx-net over the last few weeks (14 bugfixes at all, according to the changelog). And also some fixes for pre-historic draknetprofile bugs. The draknetprofile is still horribly broken, and not updated since about 2004 I think, so at some point it will receive the same treatment as msec had.

Speaking of msec, I also moved all pam-related code to pam plugin, continuing the modularization. I also added some ideas for msec for Mandriva’s google SoC page. There is not much left to do to implement everything I had in mind for 2009.1, and probably I’ll finish it over the next few weeks.

On a not-so-bright sight, my assigned bug count right now is of 325 (I also started maintaining iptables, kexec-tools and imagemagick). About 80% of those bugs are related to drakx-net, and about 50% of those drakx-net bugs require specific hardware to test a possible fix for them. As I don’t have the required hardware, nor know anyone who has it, I am still trying to figure out a way to work on them. The most annoying of those bugs is the support for b43-based wireless cards, which require loading two different kernel modules just to be able to see the device. If anyone has any such card, and is able to provide a remote ssh access to a cooker installation, this would be just amazing . Otherwise, I’ll do some guessing on what must be done, and pray for the Holy Penguin to be right .

A Smiley movie .

Extremely funny. hehehe.

10 Awesome Ads (For Traumatizing Children) | Cracked.com.

Um video muito MUITO bom.

World Builder from Bruce Branit on Vimeo.

Devido a maaais problemas com integração twitter com esse blog, desativei os digests do twitter temporariamente…

Assim que sair nova versão do twitter tools, e se ela funcionar, existe uma probabilidade deles voltarem aqui. Ou não .

• Autenticação por pam_blue rules! Para que usar tokens? Celular é o que há . http://tinyurl.com/dc8nvb #

Powered by Twitter Tools.

• Autenticação por pam_blue rules! Para que usar tokens? Celular é o que há . http://tinyurl.com/dc8nvb #

Powered by Twitter Tools.

• T-12 days to Iron Maiden in São Paulo. #
• Isso sim é promoção! Petra Bock, no extra São Carlos, foi de R$ 12 para R$ 1.99! Para ficar melhor é só poder comprar com visa vale! #

Powered by Twitter Tools.

• T-12 days to Iron Maiden in São Paulo. #
• Isso sim é promoção! Petra Bock, no extra São Carlos, foi de R$ 12 para R$ 1.99! Para ficar melhor é só poder comprar com visa vale! #

Powered by Twitter Tools.

• Running kde4 inside virtualbox feels like running doom1 on 386 with 2MB RAM. It works and looks nice, but it is slow as hell. #

Powered by Twitter Tools.

One of most wanted features for new MSEC in Mandriva 2009.1 was the support for plugins. It is an interesting idea, as it allows to convert MSEC from a tool with fixed-and-controlled-by-msec-gods list of features into a utility that can be extended by anyone, adding their own functionality, or just implementing something specific to their organization, class, or environment.

Well, starting today, it is possible to do so . I just committed into cooker a new version of msec which supports plugins, and fully integrates them with other msec modules. I added a “sample” plugin (which will someday become the AppArmor plugin, but.. as it is still not working in cooker, I just added it to use as a proof-of-concept).

Plugins are automatically loaded by libmsec on startup, checking for all entries in config.PLUGINS_DIR configuration variable (which defaults to /usr/share/msec/plugins). For each file there, it is parsed as a python script, the plugin name is determined, and a plugin class is initialized with current msec settings (such as logging backend, chroot’ed configuration and list of modified system files). After that, the plugin is automatically added into the config.SETTINGS array, which correlates msec variables (such as ENABLE_APPARMOR) with corresponding callback function (libmsec.somefunction or, in our case, apparmor.enable_apparmor) and list of valid parameters (in our case, yes and no).

After that, everything continues normally. Msec processes the configuration parameters as usual, until it gets to the ENABLE_APPARMOR parameter. At this point, it detects that this functionality is provided by the apparmor plugin, and it is handled by enable_apparmor function. So it simply calls this function, in exactly the same way as any other libmsec element.

To show how simple a msec plugin can be, the following is the complete code of (not yet functional) AppArmor plugin:

#!/usr/bin/python
"""AppArmor plugin for msec """

# main plugin class name
PLUGIN = "apparmor"

# msec configuration
import config

class apparmor:
    def __init__(self, log=None, configfiles=None, root=None):
        """Initializes AppArmor plugin"""
        # initializing plugin with libmsec data
        self.log = log
        self.configfiles = configfiles
        self.root = root

        # configuring entry in global settings
        param = 'ENABLE_APPARMOR'
        callback = "%s.enable_apparmor" % PLUGIN
        valid_values = ['yes', 'no']
        config.SETTINGS[param] = (callback, valid_values)

        # insert entry into system security settings
        config.SETTINGS_SYSTEM.append(param)

    def enable_apparmor(self, params):
        """Enable AppArmor security framework on boot"""
        if self.log:
            #self.log.info("AppArmor plugin: not implemented yet!")
            pass

That’s it. Any python functionality can be added to the enable_apparmor function afterwards.

This is more like a proof-of-concept than complete plugin, but the remaining pieces will be polished soon. Keep visiting here for news .

Combining this with the possibility of creating custom msec frontends (right now we have command line frontend (msec) and a graphical one (msecgui)), the possibilities are endless. You could create a WEB frontend with just a few lines of python code (for example, using web.py or django), add plugins which enforce settings for your organization (for example, configure all user home directories to start with “user_” prefix, check periodically for changes into /usr/local/big_project/* files, synchronize ldap databases for offices, and so on).

• por algum motivo a integração twitter/blog quebrou de vez esses dias… #

Powered by Twitter Tools.

• 18:47 T-13 (and counting) para o carnaval! #

Automatically shipped by LoudTwitter