Catalyst, nginx and FastCGI – my first bloody nose

0

Always a glutton for punishment, a recent project to redevelop a website involved changing pretty much every layer of the stack – Apache became nginx, mod_perl became FastCGI, and the custom code was moved into a Catalyst project. Having never seriously worked with any of these before, my first deployment (today!) was an interesting affair. (more…)

Creating a blog in Catalyst

2

It seems like everyone is doing it – I know of at least two other codebases for a blog controller – which makes me wonder: is there a repository of drop-in components for the Catalyst framework, or are we all doomed to re-invent the wheel? I did ask on #catalyst about it, but having not thought it all through properly I asked if there was a CMS system built on Catalyst (which is something completely different) (more…)

Back from the wilderness (and musings about Perl)

0

I’ve spent the last week away, where I’ve had absolutely no Internet access at all. No WWW, no RSS, no email – not even any Twitter (although I did tweet a couple of times from my phone). It’s surprising (or not) how much “connected” time I’ve got used to – it was like a week out of time, not even knowing the news (the only TV I get to watch these days is CBeebies…)

I was just catching up on my email/RSS backlog when a thread (you’ll probably have to join the group to read it) on the Perl Mongers LinkedIn group about Perl caught my eye – especially as the participants were talking about Perl CGI/FastCGI/mod_perl.

Bottom of the heap came Perl/CGI – somewhat unfairly, IMHO. For me, there is a time and a place for Perl/CGI – although it needs the entire Perl process set up and the script interpreted for each call, it’s perfect for the occasional hack and/or stuff that’s still in active (prototype) development stage. Not having to restart/reload servers to make changes visible can be a big plus when you’re sat there with the client/end-user – you get the feedback, hack the code, hit refresh. Of course, once you’re done (and especially if the script is going to be public facing and get some hits) you should probably redevelop it into something more “solid”, but for the one-off script it’s the quickest and easiest path.

FastCGI seemed to be well received as a solution. Now, I’ve only just started looking into it, and I didn’t get much done last week, but my (current) understanding is the the “script” is interpreted and runs as a daemon, accepting the requests at a handler subroutine. Because the interpreter is persistent, there’s no setup or interpretation delay for each call, which makes it quicker than Perl/CGI. The number of child handlers (and so resource usage) can be controlled by the process, which makes resource allocation easier (for the reason that…) As the Perl FastCGI “server” runs separately from the HTTP server process (communicating using sockets), potentially as a completely different user, security (and resource allocation, as per the previous point) is easier to manage. You can use TCP/IP sockets for the FastCGI “server”, which can allow you to split the stack even further into “application servers” (running the FastCGI app) and “front-end HTTP servers” (which can just proxy to the application servers). With a mix of static and dynamic content, you can easily see where you need to beef up your servers – static stuff will hit the HTTP servers, dynamic stuff will hit the application servers. Throw in checks by the front-ends to make sure they’re getting the data (potentially trying all of the applications servers in turn until it can get one that can actually server the request) and you’ve got the recipe for a site with pretty good availability. The only fly in the ointment here is that the FastCGI “server” needs to be reloaded each time the code changes – not so bad if you have got the front-ends doing availability checks.

I’ve left mod_perl to last, for a couple of reasons. I’ve been using mod_perl on extremely busy sites for several years now, I don’t consider myself to be any kind of expert with it, but it’s served me (extremely) well. It has its pros and cons, and so far the balance has tipped in mod_perl’s favour – I guess I’m just used to the caveats. mod_perl, like FastCGI, only interprets the Perl code once per worker (thread/fork), so you get the same speed increase that FastCGI affords. The mod_perl stuff though, as its name implies, is kept within each worker, so the memory usage of each thread/fork does go up, and everything runs as the same user as the Apache HTTP daemon. mod_perl code can also be tightly coupled to the Apache HTTP daemon, making it harder to migrate in the future, but it does allow for some pretty low-level (in terms of protocol) access. Changing the code does require a reload of the Apache HTTP daemon (although a “graceful” one works fine, and is not so intrusive to the visitors) but you can pre-load a module that automatically checks for changes and issues the reload for you (useful for “staging” servers between development and live).

Now, the real fun comes/came when someone said:

If you evaluate mod_perl, you’d be evaluating something from an ancient world that is really an inferior solution. Most large scale Perl systems are using FastCGI.

“something from an ancient world” – well, mod_perl is (AFAIK) still being developed, and I still see it appearing in HTTP responses, so I wouldn’t consider it ancient.

“inferior solution” – guessing this is compared to FastCGI. There are some obvious benefits of FastCGI over mod_perl, there’s no doubt, but mod_perl’s tight integration with the Apache HTTP daemon allows it to do stuff that FastCGI couldn’t. So, claiming one solution is inferior to the other isn’t really fair.

And the “Most large scale Perl systems are using FastCGI” claim just screams for a “[citation needed]”. No evidence was provided for this claim, and it’s a pretty bold one to make – even qualifying it with an “IMHO” would make it better.

Other people in the thread say that:

I’d argue that mod_perl is not so much antiquated as just “out of vogue”.

Now that amused me quite a bit. Perl itself is a bit “out of vogue”, hence stuff like the Planet Perl Iron Man challenge to raise its profile a bit, so for mod_perl to be described as “out of vogue” (even amongst the Perl community) is a bit of a kick for it. :)

Original post “hit back” with:

mod_perl certainly isn’t dead, and it is a very viable solution if you need an API into the webserver itself (which ties you to apache). If you don’t need that, mod_perl is a really poor solution.

It’s fallen out of grace because of its own demerits, and hasn’t kept up with modern technologies.

More bold claims. I haven’t experienced any of that myself (apart from the tie to the Apache HTTP daemon), and I fail to see how mod_perl hasn’t been able to leverage CPAN to keep up with “modern technologies”. Maybe I’m missing someone more fundamental in that claim (like the internal coding of mod_perl)…

The fact that Catalyst already has support for running as FastCGI, and that I’m looking to refactor some code into a something that’s more manageable, my “move” to FastCGI should be quite interesting (not least of all because I’ll have performance metrics from the mod_perl versions to compare against). As part of the (greater) system also has PHP code, it’ll be good to have a single “acceleration” platform for both (rather than running both mod_perl/APC).

So, where do I stand? I’m using mod_perl, I’ll probably carry on using mod_perl for some time, but I can also see that the FastCGI route also has some advantages that I’m interested in exploring. I’m also going to keep hacking Perl/CGI scripts as well, probably just out of spite. :D

New Year, new look, new stuff

0

So, it’s 2010 (however you like to pronounce it), it’s out with the old and in with the new! I must confess to having lived in a bit of a bubble for a while, not venturing out of my comfort zone, and so things got a little stale. No longer. (more…)

I’ve been working in Flash – is that ironic, Alanis?

0

People who know me are familiar with my dislike for Flash, so for me to be doing some work in it might come as a surprise.
(more…)

A poem about archiving

0

(Found this in my home directory – it came up on an IRC channel I was on, and I was bored)

rar, tar, zip, arc, dmg
ar, shar, cab, arj, rle

:)

Repost from my Facebook page

1

I think this should be a matter of public record: (taken from my Facebook profile)

Last night, while I was out with Becki (whom I’ve barely seen in the last week because of work) I was removed from the Birmingham Freecycle group – the Group I’ve been a moderator of for the last 5 years.

There was no discussion on the local moderator support list, no warning, and no consultation with the members of the Group – the Group was simply snatched.

When you’ve been dealing with the same people for that long, you’d hope there would be some kind of insight and common respect, but it appears not...

Still, at least it means that Birmingham Freecycle is now off of Yahoo!Groups:

http://groups.freecycle.org/birmingham_freecycle

Tell your friends :)


(name removed)
The msg I saw on (what is now) Freegle is :

“The main reason for our move is simple. While Freecycle is a GREAT idea, all our UK groups were basically forced to work under a system devised in and for the United States. Leaders of Freecycle in the UK spent more than two years talking with the main Freecycle Network trying to get the freedom to run things in a way more suited to our own country and our own laws. Nothing changed.

In July four leading members of our National UK Freecycle team resigned. Moderators around the country then formed an Independent Association of Freecycle Moderators and again tried talks with The Freecycle Network. But again to no avail.

Since then an increasing number of Group Owners and Moderators have been SUMMARILY DISMISSED from Freecycle including the former UK Director of Freecycle UK.

So we have decided to go our own way.”

Yeah, that’s the standard ADMIN that Freegle have kindly provided.

The truth (and they so love to claim they know/report it) is that:

“the system devised in and for the United States” is actually devised in my conservatory and is for all Groups worldwide.

The ‘leaders of Freecycle in the UK’ basically have done sod all for the last two years. Neil Morris, the previous UK Director, was telling me *last year* that he was planning on stepping down and concentrating on his PhD – 12 months later, he hadn’t even bothered to create a job description for his replacement. He was also tasked to build a UK Leadership Team, to manage the work of The Freecycle Network in the UK – 6 months later, at the point of his public resignation, still nothing.

In July, four people did resign – Neil Morris and three of his sidekicks. Remember the month – in July, Neil Morris publicly resigned.

An Independent Association was formed – or rather, a vote was taken (I don’t think the results were ever actually published) and some moderators assumed control of this group. Attempting “talks with The Freecycle Network” is quite amusing – don’t forget, at this time there was *no* UK Director, hence no-one to talk *to*.

Now, back to the UK Director role РNeil Morris publicly tendered his resignation in July, but by the end of September had completely blocked any attempts to call a meeting of the UK charity trustees to accept his resignation. In the end, the meeting had to be called in his absence, and a special article in the terms of association invoked to finally be able accept his resignation and install a new UK Director.

Within *hours* of the new UK Director being announced, the Independent Association declared that they could no longer “work with The Freecycle Network” and would be creating a new umbrella organisation for Groups to move to. Once again, not exactly making an effort there, are they? When they finally have someone official to deal with, instead they decide to create split.

The “increasing number of Group Owners and Moderators” that were “SUMMARILY DISMISSED” were, in fact, the leaders of the Independent Association, who were asked to step down from running Freecycle Groups *after* they had publicly announced their intention to create a split. Not hard to see the conflict of interest there, is it?

And my favourite bit: “including the former UK Director of Freecycle UK”. Let’s make this perfectly clear:

THE GUY PUBLICLY RESIGNED THREE MONTHS PREVIOUSLY

Exactly how can you dismiss someone who has resigned? My personal belief is that Neil Morris was pissed off that The Freecycle Network actually accepted his resignation.

The real fact is that this has been a power grab that’s been a long time planned, based on FUD generated by the Independent Association (now the leaders of the new network). They’re now sniping Groups from The Freecycle Network, based on the self-fulfilling prophecy that they themselves created.

In some ways, painful though this has been, this has actually been a positive thing for The Freecycle Network. The UK now has a new Director (based in Wolverhampton, actually :)) who’s actually getting things done, and the people who were holding the UK back with their inaction and conspiracy theories have left. Trying to take a positive view of the past weeks events, a massive thorn has been pulled from the side of The Freecycle Network, and though there’s been a lot of healthy flesh taken with it, The Freecycle Network will survive, and be stronger for it.

Without wishing to spread rumours, I hear the the Edinburgh Freecycle Group, similarly hijacked, is starting to revolt against their move, which was also made without notice and consultation from the Group members. It would appear that the new network themselves exhibit the control freak behaviour that they’ve ascribed to The Freecycle Network – the more things change, the more they stay the same.


Another week in the code mines (almost) complete

0

It’s Friday again, which draws another week to a close.

This week, I have been mostly hacking away at Freecycle stuff – I made the mistake of finessing my complexity estimates for the current Sprint down, so the total amount was reasonable. As such, I’ve had to do more work than I probably continue to maintain, but in short bursts it’s actually quite nice to stretch the metaphorical muscles a bit. A rewrite/code cleanup turned out to be a lot more work than I expected, and this really put a strain on the deadline. I would have had to do it sometime, and clearing technical debt rarely gets any attention when users are requesting features, so having to get almost half of that section done this fortnight is good progress. I feel guilty at promising to deliver certain things at certain times (especially as I effectively set my own deadlines) and failing to achieve that – I let the users down, and I let myself down.

So, with all the Freecycle fun, I haven’t managed to make any progress on any of my own projects. Never mind.

Was good to go swimming this week with the kids – Alfie’s really taken to the water now, which was a bit of a worry in the past. It’s approaching harvest time at the garden farm – carrots and tomatoes, onions look almost ready. Something keeps munching Alfie’s pumpkins before they can grow, which is a bit upsetting for him. :(

Got my parents coming up tomorrow – off for an all-you-can-eat buffet and spend some time with the (grand)kids. Hopefully Sunday will be a quiet one. :)

And, of course, I’m sure I’ll be doing yet more Freecycle work – just keep getting sucked in whenever I go online. :)

Increase hard disk capacity – just add a sticker!

0

While I was waiting for a compile/unit test to finish, I noticed that one of the failed hard disks (a Maxtor MaxLine Plus II 250Gb ATA/133 HDD) I was using as a paperweight had two stickers on it. Having nothing better to do, I pulled the top one off, just to see what the changes were. (more…)

Fire!

0

Excitement outside the house tonight…

Anyone got any marshmallows?

Go to Top