Page 13 of 28
Perl 5.10.0
Sure, it’s already slathered across every news site known to man but just like everyone else I have different reasons for being excited about Perl 5.10.0.
Like many others, I often choose Perl for its seamless regex support so the faster regex engine in this new release is definitely welcome. This version also fixes an erroneous piece of missing syntax for subscript slices. I was always confused why the interpreter forced me to use -> instead of the non-arrowed form. The big one, not because it was hugely lacking but simply a seemingly odd oversight, is the switch statement, which is called given/when in Perl:
given ($foo) {
when (/^abc/) { $abc = 1; }
when (/^def/) { $def = 1; }
when (/^xyz/) { $xyz = 1; }
default { $nothing = 1; }
}
Sure, switch statements don’t really do anything special, but they are often quite a bit cleaner than if/elif chains and the fact that Perl’s implementation can handle regex is awesome.
Web Browsing With Python
Python provides a stateful web browsing module called mechanize, named after Perl’s mature and featureful WWW::Mechanize. Though it isn’t as powerful as the Perl version, mechanize provides an easy-to-use framework for browsing web pages including interacting with forms and accessing SSL content. The documentation for mechanize on the web is sparse, but viewing the source file (/usr/lib/python2.5/site-packages/mechanize/_mechanize.py on my Ubuntu machine) provides some needed insight. Here’s a quick overview of the operation of mechanize:
#!/usr/bin/python
import re
from mechanize import Browser
br = Browser()
# Ignore robots.txt
br.set_handle_robots( False )
# Google demands a user-agent that isn't a robot
br.addheaders = [('User-agent', 'Firefox')]
# Retrieve the Google home page, saving the response
br.open( "http://google.com" )
# Select the search box and search for 'foo'
br.select_form( 'f' )
br.form[ 'q' ] = 'foo'
# Get the search results
br.submit()
# Find the link to foofighters.com; why did we run a search?
resp = None
for link in br.links():
siteMatch = re.compile( 'www.foofighters.com' ).search( link.url )
if siteMatch:
resp = br.follow_link( link )
break
# Print the site
content = resp.get_data()
print content
That’s a pretty straightforward and simple usage. The get_data() method gives you the HTML content of the pages, which I often find suitable to run a .split('\n') and then do some regex on line by line.
Random Futurama
I like Futurama. All of it. Though I occasionally have a hankering for a specific episode, I often just want /any/ episode, so I wrote a quick Perl script to play one at random. I have a directory filled with all of Futurama, so this is an easy proposition:
#!/usr/bin/perl use strict; use warnings; my @files = split /\n/, `ls *avi`; my $num = rand(scalar(@files)); system "mplayer "$files[$num]"";
Custom Weather Radar Images
If you know the slightest bit about weather and the geography of the area which you are observing, you can, in the short term, forecast more accurately than what a weather website provides. I often want to be able to bring a radar image up quickly and clicking around websites takes time. Sure I could bookmark the weather page directly, but this was an interesting exercise and will prove useful when I get an iPhone.
The National Weather Service doesn’t have a simple image of the weather on their radar page, rather, it is an image that is dynamically generated from a set of standard overlays using Javascript. This allows them to more efficiently distribute radar images, since the terrain and maps that are the background never change. The organization they use is well documented, making it easy to find the overlays for you local radar. In my case, I wanted the topographical map, counties and highways in the image. I found the URL for each of these components using the aforementioned documentation and grabbed the appropriate images with wget.
wget http://radar.weather.gov/ridge/Legend/N0R/LWXN0RLegend0.gif\ http://radar.weather.gov/ridge/Overlays/County/Short/LWXCountyShort.gif\ http://radar.weather.gov/ridge/Overlays/Highways/Short/LWXHighwaysShort.gif\ http://radar.weather.gov/ridge/Overlays/Topo/Short/LWXTopoShort.jpg\ http://radar.weather.gov/ridge/RadarImg/N0R/LWXN0R_0.gif
The only layer that will change will be the last one, the actual radar data, so in my script that is the only one that will be wgetd on subsequent runs. To build the image, I used the composite program that is part of the ImageMagick package:
composite -compose atop LWXN0R0.gif LWXTopoShort.jpg basereflectivity.jpg composite -compose atop LWXHighwaysShort.gif basereflectivity.jpg basereflectivity.jpg composite -compose atop LWXCountyShort.gif basereflectivity.jpg basereflectivity.jpg composite -compose atop LWXN0RLegend0.gif basereflectivity.jpg basereflectivity.jpg
I want the counties, highways and legend to show up on top of the weather data which itself is pasted atop the topographical map. To achieve this, I first toss the radar data onto the topographical image creating a new image, base_reflectivity.jpg. Then, I add each of the other layers to the base_reflectivity.jpg image in sequence.
Connecting to Oracle Without a Password on Windows
If you have forgotten, were never given or otherwise don’t have the password to an Oracle database, never fear, there is a method to accessing the database. From the local machine you must be a user in the group “ora_dba”. Run “sqlplus” (the command line version) with the option “/nolog”, which tells SQL*Plus not to login. At the “SQL>” prompt, type “connect / as sysdba” which ought to log you in. At that point, you can change the password for any account (sys would be a good one to change, since apparently you don’t know it) using the command alter user <username> identified by "<password>";. Make sure to commit; after doing that.
Disconnecting: Corrupted MAC on input.
I woke up to find the entitled error in the rsnapshot logs on one of my machines, which occurred as it was trying to remotely backup, via SSH, another. A quick search and reading through a number of threads revealed this message stating that this problem was encountered with a Linksys router involved. Lo and behold, my home network was just switched to a Linksys BEFSR41 when the problem began.
I think the router may also be the culprit for the occasional “error establishing encrypted connection code -12192” that Firefox has been giving me since the switch.
Perl, Python and Ruby
After reading this article comparing the new Ruby 1.9 to it’s older version and Python, I thought it would be interesting to see how my preferred language, Perl, compares. First, because my hardware is certainly different from the author of that post, I re-ran the Python and Ruby 1.9 tests:
Python: real 0m43.415s user 0m41.455s sys 0m0.132s Ruby: real 0m35.379s user 0m33.722s sys 0m0.112s
I threw together some quick Perl:
sub fib {
my $n = shift;
if ( $n == 0 or $n == 1 ) {
return $n;
} else {
return fib($n - 1) + fib($n - 2);
}
}
for (0..35) {
print "n=$_ => " . fib($_) . "\n";
}
and ran it:
Perl: real 1m15.329s user 1m9.188s sys 0m0.364s
That’s impressive; Perl sucks at this recursive implementation of computing Fibonacci numbers. Since I had the makings of a little suite of benchmarks, I implemented this test in C and Java:
C code:
include
int fib(int n) { if ( (n == 0) || (n == 1) ) return n; else return (fib(n-1) + fib(n-2)); } //fib
int main() { int x; for (x = 0; x < 36; x++) { printf( "n = %d => %d\n", x, fib(x) ); } //for
}
C timing:
real 0m0.605s user 0m0.564s sys 0m0.000s
Java code:
class Fib {
static int fib(int n) {
if ( (n == 0) || (n == 1) )
return n;
else
return (fib(n-1) + fib(n-2));
} //fib
1
2
3
4
5
public static void main(String args[]) {
for (int x = 0; x < 36; x++) {
System.out.println( "n = " + x + " => " + fib(x) );
} //for
} //main
} //Fib
Java timing:
real 0m0.455s user 0m0.412s sys 0m0.012s
It’s very interesting to see the huge speed disparities in these languages, but remember, this was a single, very limited test; don’t take these benchmarks as having any significant meaning. Really, don’t take them to mean much of anything, unless all you do is write lots of simple recursive code.
Quick Tip: Python's noop Equivalent
Python can’t deal with empty code blocks, which may arise if you want to stub out a block but leave the implementation for later. If you have one in your code, the interpreter will give the following error:
File "foo.py", line 1
if (foo == bar):
^
IndentationError: expected an indented block
In this case, you can use the pass operator which will do nothing but take up a line, resulting in the following code:
if (foo == bar):
pass #TODO
else:
print "nothing to do!"
mdadm Isn't Someone You Want to Hear From
A server of mine, Claudius, sent me a very polite, though unpleasant email early this morning
from: mdadm monitoring root@dinomite.net to: root@dinomite.net, date: Nov 28, 2007 6:25 AM subject: Fail event on /dev/md0:claudiusThis is an automatically generated mail message from mdadm running on claudius
A Fail event had been detected on md device /dev/md0.
Faithfully yours, etc.
Ugh; that means one of the drives in RAID set on Claudius has failed. I only run RAID0 on the machine, so data loss isn’t any issue unless the second drive gives up, too. I try not to actively administer machines; I just get them setup in a solid state and then make use of them, no reason to waste time fiddling around. I hadn’t done anything of note with the machine in question in some time, so I wasn’t even sure what the drive setup was, though df clears that up:
claudius:/home/dinomite/public_html# df -h Filesystem Size Used Avail Use% Mounted on /dev/md0 5.5G 1.7G 3.6G 32% / varrun 506M 116K 506M 1% /var/run varlock 506M 4.0K 506M 1% /var/lock udev 506M 88K 506M 1% /dev devshm 506M 0 506M 0% /dev/shm /dev/md1 68G 13G 54G 19% /home /dev/hdd1 19G 13G 5.1G 72% /backup tmpfs 506M 18M 488M 4% /lib/modules/2.6.15-29-k7/volatile
So, md0 is the root (/) filesystem and Claudius also has a second RAID device, md1, which supports the /home filesystem. I used mdadm --detail to get some more information about md0:
claudius:/home/dinomite/public_html# mdadm --detail /dev/md0
/dev/md0:
Version : 00.90.03
Creation Time : Sat Aug 26 11:50:00 2006
Raid Level : raid1
Array Size : 5855552 (5.58 GiB 6.00 GB)
Device Size : 5855552 (5.58 GiB 6.00 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Wed Nov 28 11:35:45 2007
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 1
Spare Devices : 0
UUID : 16c480c4:6bbc2b89:189b2529:3af755b2
Events : 0.16665869
Number Major Minor RaidDevice State
0 0 0 - removed
1 3 65 1 active sync /dev/hdb1
2 3 1 - faulty /dev/hda1
So it thinks there’s a problem with /dev/hda1, one of the two partitions that makes up this RAID device. At this point, I recall that Claudius has three disks in it, a 20G (/backup) and two 80G, the latter two support both of the systems RAID devices, md0 and md1. The notice that mdadm sent me was only concerning md0 which is odd, because md1 spans both of those physical disks as well:
claudius:/home/dinomite/public_html# mdadm --detail /dev/md1
/dev/md1:
Version : 00.90.03
Creation Time : Sat Aug 26 11:50:07 2006
Raid Level : raid1
Array Size : 71778304 (68.45 GiB 73.50 GB)
Device Size : 71778304 (68.45 GiB 73.50 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 1
Persistence : Superblock is persistent
Update Time : Wed Nov 28 11:38:25 2007
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
UUID : 41d43030:23b3662d:c1c06634:4132ebc3
Events : 0.2117050
Number Major Minor RaidDevice State
0 3 2 0 active sync /dev/hda2
1 3 66 1 active sync /dev/hdb2
and it doesn’t show any problems. What does this mean? Well, I’m not exactly sure, but it does mean that the disk /dev/hda which holds the failed partition (hda1) of md0 isn’t completely bad; it’s still spinning away and working. That probably means that the filesystem on hda1 somehow got muddled up or that there’s a physical error on the portion of the disk support hda1. Either way, the first step for me is to simly rebuild the array as it was before. This is as simple as removing the faulty partition and adding it back to the array:
claudius:/home/dinomite/public_html# mdadm /dev/md0 -r /dev/hda1 mdadm: hot removed /dev/hda1 claudius:/home/dinomite/public_html# mdadm /dev/md0 -a /dev/hda1 mdadm: hot added /dev/hda1
The RAID device will automatically begin rebuilding this partition and keep you informed of the status.
claudius:/home/dinomite/public_html# mdadm --detail /dev/md0
/dev/md0:
Version : 00.90.03
Creation Time : Sat Aug 26 11:50:00 2006
Raid Level : raid1
Array Size : 5855552 (5.58 GiB 6.00 GB)
Device Size : 5855552 (5.58 GiB 6.00 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Wed Nov 28 12:03:36 2007
State : clean, degraded, recovering
Active Devices : 1
Working Devices : 2
Failed Devices : 0
Spare Devices : 1
Rebuild Status : 3% complete
UUID : 16c480c4:6bbc2b89:189b2529:3af755b2
Events : 0.16666531
Number Major Minor RaidDevice State
0 0 0 - removed
1 3 65 1 active sync /dev/hdb1
2 3 1 0 spare rebuilding /dev/hda1
Though a failed RAID device isn’t the best way to start your morning, managing them in Linux is a fairly straightforward operation. If the device actually had been bad, I would simply have to shut down the machine, replace the bad drive, partition it like the other one (fdisk -l helps) and then add it to the array as shown above. In most cases, this wouldn’t be much harder than what I had to do, though my case the server is a few hundred miles away, so that’d be quite a hike.
Tab History Add-on for Firefox
There are literally hundreds of lists of the best Firefox extensions yet few mention what I deem to be the most important: Tab History. A feature that I first became addicted to in Galeon, Tab History causes newly created tabs created by middle-clicking a link or clicking “Open Link in New Tab” to inherit the history of their parent.
That is, I can run a search, open a whole bunch of the windows in tabs and then close the windows with reckless abandon once I decide one of the tabs fulfills my needs. Later, if I find that the tab I’m reading doesn’t have the information I was searching for, I don’t need to ctrl+shift+t and sift through the tabs that I had prematurely closed searching for the web search that I had run. Instead, because the tab in question was spawned from said search, I simply hit ‘back’ and head right back to the original page.
Similarly, if I’m reading an article opening links in new tabs as I go, I will usually close that article’s tab before perusing the links that it contained. If I happen upon one of those new tabs and have forgotten what the context for it was in the article, I can simply hit the back button to find the answer.
Tab History really is the most important Firefox extension to me.