One other little known command

Do you know about "join"? It's a nice little utility that comes from the "coreutils" package, which means that it's there by default on most, if not all Unix/Linux distributions. What this does is join (duh) two files, based on the values in fields.

For example, the other day I had to correlate Apache requests from the logs on one Apache server acting as a proxy, with the logs of another Apache server used as a backend. Because of missing configuration, the proxy Apache had public IP addresses, but didn't have domain names (virtual hosts), and the backend had domain names but only the private IP addresses from the proxy systems. So, I took copies of the two Apache's combined_logs, and awk'ed them to only keep the IP, URI requested, and domain name requested (this was necessary because it's better to have clear field numbers with join, and Apache can, if configured to do it, log the UserAgent of a client, which spans multiple fields if you field separator is space).

With the logs doctored as required to be easy to handle with join, you can just run the command:

join -1 2 -2 2 -o "1.1 0 2.3" log_proxy.txt log_backend.txt

What this will do, is tell join to use the second field from the first file (-1 2, aka log_proxy.txt), the second field from the second file (-2 2, aka log_backend), and join the data together to form a new file, outputted to standard output, following the format stated in -o: first file's first field (the public IP), the join field (0, or the URI in this case, which appears in both files), and the third field in the second file (the virtual host domain). You can obviously adjust the field numbers if necessary, or change various settings such as the field separator (default is space), which is all very clearly documented in the man page.

On other news, I seem to have because fairly well know in my workplace to be the Ubuntu wizard... Maybe the Ubuntu laptop bag helps? Anyway... Friday I was asked by Marc about installing Ubuntu on a Lenovo T60, and yesterday I was asked about my thoughts and experiences with Intrepid by Richard. I've already managed to get one Ubuntu machine in use as a server, and hopefully there will be more to come in the future.

Planet Ubuntu Meme (aka I am so unoriginal)

My computers are named with the names of Greek/Roman dieties, and more specifically some that I've also seen in the songs of my favorite band, The Cruxshadows.

So far, I have:

• eurydice : my openwrt router
• orpheus: my main desktop computer (currently offline because of some issues though)
• icarus: my work laptop. It also obviously has a name for the office.
• daedelus: used to be my palm pilot.
  
• cerberus: my EEEPC.
• persephone: my jailbroken iPod Touch.

Hopefully there will be more to come.

Dropbox was released today

Dropbox was released in Public Beta today. Yay!

As soon as I got to my email with the invite, I installed the 8.04 version of the Dropbox package from the website on my primary system (running Intrepid). It worked perfectly, installed without a glitch, and I can now happily synchronize files between machines.

Oh, and I subscribed to identi.ca today. I'll attempt to regularly post something there too... I just need to find a good way of doing these posts easily and without always logging in to the website.

More network-related tools: Network Discovery

I've found out about this other really nifty tool for networks. It's called NeDi (for NEtwork DIscovery), and is used to inventory devices connected to a network, the ports available on network devices, routes, modules, and other kinds of details that are just fun to have, or really useful to refer to from a central repository. It's nice too because it installs extremely well on Ubuntu; a wiki entry even exists to give a quick how-to on the installation and configuration process: https://help.ubuntu.com/community/NediHowTo.

Once you've gone through the steps described in the wiki entry, and run your first nedi.pl -cob (and waited the a minutes if you've happily asked it to map your whole corporate networks (oops!)), you can access the web interface and view all the information that was gathered, such as devices routes, firmware versions, models, serial numbers, and what end-user devices are connected where.

NeDi is also apparently part of the GroundWork OpenSource products, which is a pretty interesting suite of software if you want to roll out systems monitoring in your location.

Two useful network-related commands

mtrudel@laptopl-mtrudel:~$ sudo ethtool eth0
[sudo] password for mtrudel:
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 100Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbag
Wake-on: g
Current message level: 0x00000001 (1)
Link detected: yes

ethtool is a command that gives you information on the capabilities of your network card, and the network that it is connected to. The most interesting things are the Speed and Duplex lines, which are somewhat useful when diagnosing some very specific problems. Some people may already know of mii-tool which is also available on default installs of ubuntu, however, you should know that mii-tool doesn't support Gigabit ethernet, and as such may give you weird results in that case. ethtool, on the other hand, works fine on gig.

Another cool command I found out about recently, is an implementation of a CDP listener on linux: cdpr. cdpr is available with a simple sudo apt-get install cdpr. cdpr listens for CDP advertisements coming from Cisco hardware, which will let you know what switch you are connected to, some additional information about the switch, and more importantly, which port you are connected in. That's pretty neat for large offices, especially if you're not sure where all the wired are going, and which is which when in a wiring cabinet. To use cdpr:

mtrudel@laptopl-mtrudel:~$ sudo cdpr
cdpr - Cisco Discovery Protocol Reporter
Version 2.2.1
Copyright (c) 2002-2006 - MonkeyMental.com

1. eth0 (No description available)
2. tun0 (No description available)
3. any (Pseudo-device that captures on all interfaces)
4. lo (No description available)
Enter the interface number (1-4):1
Using Device: eth0
Warning opening device (arptype 65534 not supported by libpcap - falling back to cooked socket)
Waiting for CDP advertisement:
(default config is to transmit CDP packets every 60 seconds)
Device ID
value: switch4529
Addresses
value: 192.168.45.29
Port ID
value: FastEthernet0/33

Now I'd just need to walk over to that switch to do whatever I may need to do directly on it... or telnet, ssh, or point a browser to it if I needed to change the configuration.

An education issue? + Yay for the Intrepid user switcher and shutdown/logoff window

Talking to my father tonight, helping him with some computer tasks and advocating free software in the process, I've come for myself to the realization of what many may already know, but anyway... the use of free and open-source software is an education issue. What I mean by that is that if we want FOSS to be adopted in a more widespread fashion, if we want projects to succeed (and FOSS-focused companies to succeed), we need to make sure that people know that alternatives exist. We need to let people know too, that the alternatives are legit -- because I think that this may actually be a problem too. Maybe people are accustomed to paying for software that has very specific capabilities, or lots of features, or for operating systems, and from that habit they are inferring that whatever is not being charged for is like the today sadly common pirated software? Is anybody else noticing this kind of thing?

As such, I think that is really is important to support projects like Fabian Rodriguez' Ubuntu In Libraries, that promote availability of open-source software, education of potential users on the fact that legit, free, and working software actually exists, and that it can replace other software with fewer features that you could potentially pay a lot more for.

Extending this, I also think that Ubuntu should be advocated in a more overt manner in schools. I'm often somewhat shocked at the fact that teachers will encourage students to procure themselves extremely specialized and pricey software like Adobe Photoshop or Illustrator (which tend to be somewhat out of most students' budgets, especially living on their own... and usually also out of their parents' budgets!), when viable alternatives like GIMP or Xara Xtreme, which could not only do the same work in many cases, but benefit from the additional user base while helping these new users by adding features. If there is demand for something, it has more of a chance to be added to a software project. At the same time, from my experience with the commercial and free alternatives of graphics tools, I see that a lot of things are done exactly the same way, with very similar menus or icons for the same tasks. I'm no expert, but wouldn't that still be good if students are taught the theory basis, and can then transport, transfer that knowledge to whatever different tool they use to do the job? At the very least, I say thanks to Ubuntu for making some of the finer of these alternative programs easy to install or even available by default in new installs (I'm thinking OpenOffice.org).