Paul’s Journal

The module formally known as mod_wombat was renamed mod_lua, and has pulled into the Apache HTTP Server trunk, and will be part of the future 2.4 stable release.

For an example of why it is cool, lets look at replacing a common task with mod_rewrite: Blocking Image Theft.

The HTTPD wiki even has an example of how to do this for us:

RewriteEngine on
RewriteCond %{HTTP_REFERER} !=""
RewriteCond %{HTTP_REFERER} !example\.com [NC]
RewriteRule \.(jpe?g|gif|png)$ - [F,NC]

With the new mod_lua, you can do this using real if statements and functions.

For example:

<LuaHookTranslateName imagetheft>
function is_image(path)
    -- You could put complicated regular expressions here.
    if path:match("%a+.png") then
        return true
    end
    return false
end

function imagetheft(r)
    if not is_image(r.uri) then
        return apache2.DECLINED
    end

    referer = r:headers_in("Referer")
    if referer then
        if referer:find('example.com') then
            return apache2.DECLINED
        else
            r:err("Forbidden for Image Theft! uri=".. r.uri)
            return 403
        end
    end
    return apache2.DECLINED
end
</LuaHookTranslateName>

While this example comes out signifigantly longer, I do believe in the long run it will let people write more maintainable configurations, espcially for things like complicated RewriteRules — since basic things that have long be obsecured into RewriteCond can now be easily accessed and evaluated with an If statement in Lua.

Much of the API is still in flux, but hopefully we will come up with some shortcut builtins that make many common taskes easier and shorter.

After using Rhino for server side javascript at work, I can say I somewhat like server side javascript.  Others like Steve were already convinced a long time ago.

However, I don’t really like being tied into the whole Java world because of it.

When Google released their v8 Javascript Engine earlier this year, I always wanted to build an Apache Module for it.

This afternoon I had some time, and so I created mod_v8.

It doesn’t do much beyond a Hello World right now, but it is as simple as this:

ap.write("Hello World!");

I’m not sure if I will spend time making it a proper project, I really want to spend more time on making httpd 2.4 before getting too distracted with shiny things….

My todo list from before thanksgiving:

• Improve the Simple MPM.
• Integrate Lua and mod_wombat into httpd trunk.
• Improve the FastCGI Support in httpd turnk.
• Finish my little infrastructure stats & graphs project.

Well, I did get something working on my Infrastructure Stats Project.  You can see the work in progress here.  It uses flot and jquery to draw graphs of the mail and code commits at the ASF since the Apache Project started in 1996.

I didn’t get around to most of my httpd goals, since every time I work on the Simple MPM, the fact that KQueue causes a kernel panic on OSX kinda discourages me.  I was hoping that the latest kernels from Apple would fix the problem, but they don’t.

I did manage to commit 4 new modules to httpd trunk however:

• mod_heartbeat - Generates Multicast heartbeat messages containing the status of the server.
• mod_heartmonitor - Collects these Multicast heartbeats for other modules to use.
• mod_ratelimit - Bandwidth Rate Limiting for Clients.
• mod_lbmethod_heartbeat - Module to Load Balance mod_proxy_balancer workers using the data from the heartbeats.

mod_heartbeat, mod_heartmonitor, and mod_ratelimit were all originally written at Joost, and my employer was nice enough to let us contribute them back to Apache.

I’ve also announced the intent to roll the first 2.3.0-alpha release next weekend.  Hopefully this means within a few months, a new stable 2.4.x branch will come out.

Yesterday, Apple approved the Joost Application for the IPhone.

Apple Store Link.

The Joost Application is something I helped prototype, and wrote most of the server side support, although I was busy with other things and haven’t worked on it recently, and I’m glad it is now released to the public.

It isn’t perfect, as some of the reviews have said, we are doing our best with the APIs that Apple provides, but the Video Player has some pretty big bugs we do our best to work around, but I hope Apple significantly improves MPMoviePlayerController with another firmware release.

Radar Bugs: (what are rdar links?)

• 6352281 - Problems with the Headphone getting inserted causing video to pause
• 6299534 - Problems with the Next Button on the default Apple UI.
• 5888718 - Video player does not detect when it is upside down.
• 6407138 - Audio Sync Issues

(open source) things to do over thanksgiving weekend:

• Improve the Simple MPM.
• Integrate Lua and mod_wombat into httpd trunk.
• Improve the FastCGI Support in httpd turnk.
• Finish my little infrastructure stats & graphs project.

I’ll be happy if I make progress on any one of them…..

Star Trek: The Original Series - The Ultimate Computer:

Yay!

I’m on a new feathercast episode talking about Apache 3.0.

ps, New Orleans and ApacheCon is fun.

pss. Last day is Today.

I installed the wprdpress iPhone app while on a train.

It told me that my WP version was too old, so using pterm I sshed into my server, and upgraded from 2.3 to 2.6.

And now I’m writing a blog post on my iPhone.

The iphone is not perfect, but nothing else is even close.

Earlier this week, I committed a new ‘Simple’ MPM to Apache httpd trunk.

More info on the mailing lists.

I should write more about the Simple MPM, but ApacheCon New Orleans is  coming up next week, so I want to try to get more of the code in before then, since traditionally coding during the conference never works out, and I’ll try to write up a blog post about why its cool sometime next week during the conference.

Apache HTTP Server 2.2.10 was released more than a week ago.

One of the new features I don’t think anyone has mentioned much is that we now have built in support for chroot.  Just add ChrootDir “/srv/my_root”, and all IO in Apache after the initial startup will be inside the chroot.

I don’t often write about politics, or anything related to it, on my journal.

However, the SEVEN HUNDRED BILLION DOLLAR buyout deal that Henry Paulson, the Secretary of the United States Treasury, is trying to get done, is insane.

This isn’t Capitalism.

If I didn’t believe in Capitalism, I wouldn’t be working at a startup.

If Joost fails, I don’t expect anyone to help me.  I expect it to hurt. But it is my risk to take, why do giant financial companies, who have done apparently stupid things, get treated differently?

If Joost is successful, I expect rewards, and this is the basis of any capitalistic system,  but these buyouts are creating a system in which stupid companies can do badly — and their employees still get rewarded.

I didn’t live during the Great Depression.  I don’t know how bad it can get. I have lived an ‘easy’ life in modern America.

But I would rather see another depression than this massive buyout.

Whatever is going on, it is not capitalism, and I am sad to see America going in this direction.

I know, I know, I haven’t written a blog post in months.

Forgive me

Anyways, figured some people might be interested in what I commited last night for the Apache HTTP Server….

First is the ability to ’suspend’ an HTTP Request from a Module.

Normally in Apache, there is a one to one relationship between the running thread, and a client connection that it is servicing.  In the Event MPM previously, we had broken it out so between different HTTP requests in the same TCP connection, aka Keep Alives, could be serviced by different threads.

Now with r697357 in httpd trunk, you can return SUSPENDED from a content handler.  The core  will assume you have taken over handling of the request, and get out of your way.

This is insanely important, because in the long run it lets you write very cool modules, like a full async HTTP Proxy, aka something like Varnish, where you could easily handle tens of thousands of client connections, using only a hundred odd threads, where today you would need a huge amount of RAM, and a single thread or process for every client to do this with Apache.

Secondly, as a demonstration that it works, I wrote mod_dialup. It is a module that sends static content at a bandwidth rate limit, defined by the various old modem standards. So, you can browse your site with a 56k V.92 modem, by adding something like this:

<Location /mysite>

ModemStandard V.92

</Location>

Previously to do bandwidth rate limiting modules would have to block an entire thread, for each client, and insert sleeps to slow the bandwidth down.  Using the new suspend feature, a handler can get callback N milliseconds in the future, and it will be invoked by the Event MPM on a different thread, once the timer hits.  From there the handler can continue to send data to the client.

To the other few people in Spokane with the the Internet: I’m in back in Spokane until Sunday May 25th.

The last week has seen many new features and improvments made to httpd. Many of them have been accelerated by people at the ApacheCon EU Hackathon this week.

• mod_session

  On Friday Graham Leggett introduced a series of modules to support generation of sessions from HTTPD. Included is mod_session_crytpo, which encrypts the data using AES. This is the first time ‘form based’ authentication has had real support in the Apache Core.
  [docs: mod_session]
  [thread: Apache support for form authentication]
  

• mod_socache

  On Tuesday Joe Orton commited the new Small Object Cache modules, which have been under discussions for a couple months now. The mod_sslsession cache has been changed to use this. Currently suported cache backends are DBM, memcached, and Shared Memory. I expect many other modules will changed to use this cache API as time goes on.
  [svn: ap_socache.h]
  [thread: [PATCH] ap_socache.h & mod_socache_*]
  

• If/Else blocks added

  Nick Kew ported the expression parser from mod_includes, and has used this to add If and Else blocks to the core.This provides a viable alternative to mod_rewrite and RewriteCond, and letsyou set any modules configuration values.
  [docs: if]
  [thread: Dynamic configuration for the hackathon?]
  [commit: r644253]
  

• Turkish Documentation

  Nilgün Belma Bugüner contributed a complete translation of the Apache HTTP Server documentation in Turkish.
  [docs: Turkish]
  [thread: New Turkish Documents]
  [commit: r645667]
  

• Serf Bucket Discussions

  Discussion at the Hackathon covered how Serf Buckets use a “pull” method, for both input and output, unlike the current filter stack in httpd, which is Pull for input filters, but push for output filters. There was general agreement that the expieriment of mod_serf should be expanded up the filter stack.
  [svn: mod_serf.c]
  

• Simple MPM created

  Paul Querna started work on a new MPM at the Hackathon. The MPM hopes to run on both Unix and Win32 platforms, and keep the same behavoirs on both.
  [svn: SIMPLE.README]