For a while now, I've wanted to see what interesting things I could do with a microcontroller by teaching it how to send and receive D-RATS packets.  I bought an Arduino Diecimila a while ago and played with it a little bit.  It's a really cool device and makes quick work of getting a working microcontroller programmed and functional.  I highly recommend it.

When I first got it almost six months ago, I got stuck on level conversion to RS232, which I needed in order to interface with a D-STAR radio.  I had a MAX232 chip and four properly-sized capacitors hooked up according to the spec sheet but it just wasn't happening.  I got frustrated and put it on the shelf.

Recently I decided to give it another go, and sought some advice from someone with more hardware skills than I.  Eric boosted my confidence by sanity checking my plan and suggested that perhaps I had a bad capacitor from Radio Shack.  I was a little skeptical, but ordered some new ones from a more reputable source.  The new ones arrived a couple days ago and I'm pleased to report that they fixed me right up.  Wahoo.

Along with the new caps, I also ordered some additional components I needed to create a semi-permanent setup.  While shopping around, I found the Arduino Pro Mini, which is an unbelievably small unit with nothing but labled solder tabs.  The size was just too attractive to pass up, so I got one of those too.

After getting the MAX232 prototyped on a breadboard with the new caps and proving that it worked, I set out to make a little board that had the Arduino on it (using headers so I could remove it for reprogramming), the MAX232 and associated capacitors, and a substantial 5V regulator so I could power it off of 15V (the Pro Mini has an upper limit of 12V which is too low for operation in a vehicle with a good alternator).  Below is what I ended up with:

It's a little hard to see, but the Arduino is in the foreground sitting atop the headers.  Under it is the 5V regulator and in the background is the MAX232.  The wires off to the left go to a male DB9, suitable for attaching to a D-STAR radio's data cable.  In all, there is less than $30 on the board.  For a size reference, here is the unit next to a normal SD card:

It's extremely small so it should be easy to mount in an enclosure and tuck in the car near my radio.

Right now the firmware has just a ring buffer large enough to capture a D-RATS ping request.  It will reply to KK7DS E or a CQCQCQ ping.  Even in this basic form, it's pretty neat to be able to ping a moving vehicle and get a response without requiring a running PC in the car.  However, I have a lot of ideas for neat things to add:

• I have a serial 16x2 LCD display that could be used to display chat and status messages
• A keyboard or keypad could be added to allow two-way messaging with a very small device
• Adding a GPS would allow the unit to respond to postition report requests as well as replace the ability to do GPS beaconing
• Creating some new D-RATS RPC commands would allow a remote station to control the DIO pins to illuminate LEDs, throw relays, or control just about anything
• Adding a temperature (or other type of) sensor could provide status readings from a remote station with minimal hardware

The "Nifty! Ham Accessories" company publishes a large number of quick reference guides and manuals for a variety of ham hardware and software.  I think the most popular items are the nifty guides which are pocket-sized cheat sheets for today's complex radios.  Recently, the "Nifty E-Z Guide to D-STAR operation" was released as a small paperback book.  It covers a wide range of D-STAR topics such as programming the radios, using repeaters and gateways, as well as operation of the DV dongle.  I was happy to hear that D-RATS was mentioned in Chapter 8, titled "DV Mode Slow-speed Data".

The author covers the basic steps to get a D-RATS station configured and on the air, as well as the highlights of various bits of functionality, including beacons, file transfers, and networked operation.  If you're a hard-core D-RATS user and are plugged into the new 0.3 beta versions then the screenshots will look a little dated since the book (obviously) covers the currently-released version.  However, if you're an average user looking for a good reference guide to keep by your rig, this will cover all the basics and could prove quite valuable as a last-minute D-STAR/D-RATS refresher.

Check it out!

The weekend before last was the 2009 Dayton Hamvention.  I'm a week late crafting this write-up because of some work-related travel that came up unexpectedly last week.  It was a good show, but it seemed clear that the poor economy had taken its toll on the attendance.  Luckily the level of interest in D-STAR has increased since last year, so there was a higher relative concentration of people interested in D-STAR and D-RATS than before.  As a result, I still had a fairly large amount of traffic and questions.

D-RATS had a much larger presence this year than last, with a large booth on the arena floor:

There were four computers, one on each end connected to a large display screen and two in the middle for hands-on demonstration.  The right-most machine was connected to the internet and the ratflector for most of the show, which allowed people to chat with foreign stations and see their positions on the map:

I was also really glad to have a few visitors from back home, some of which manned the booth for me so I could get out to see the rest of the show:

From left to right: Rob N1KEZ, John KX7YT,myself, and John N7AAM.

Update 27-May-2009: I forgot to post a link to the D-RATS flyer that was available at the show.  You can get the PDF version here.

I've used GNUS to read my mail for a long time now.  Every once in a while I try to move to something else because of simplicity or ease of use, but I always come back.  The unparalleled ability to customize and script the environment is just really hard to beat.

The "resting state" in GNUS is the group buffer, which shows you (by default) only groups that have unread messages in them.  I like this a lot as it provides a sort of dashboard to the mail items that need attention.  However, if I need to recall an already-read message, I have to hit L to show all groups, and then search (because I have a lot) for the appropriate group and then enter it to find the message.  This isn't a huge hassle, but sometimes I get annoyed at the process.

Since 90% of the time, I'm recalling messages from my inbox, I decided to write a function that would make this common case quicker.  I came up with the following bit of elisp:

(defun dan/select-inbox ()
  "Select inbox"
  (interactive)
  (save-excursion
    (gnus-group-list-all-groups)
    (if (search-forward-regexp "mail.*misc" (point-max) t)
    (gnus-group-select-group))))

Assuming I'm in the group buffer, I can run this function and it asks me right away how many messages I want to view from my inbox.  Naturally, I want a quick key sequence to trigger this function, so I chose the unused "G i"keys for "Group...Inbox".  I bound it thusly:

 (define-key gnus-group-group-map "i" 'dan/select-inbox)

Now from inside the group buffer, I can hit "G i", type "10" (or so) and quickly recall recent messages.

This past weekend, I was up in the Seattle area (again) for the 2009 Communications Academy conference.  This was my first time attending, and I found it to be quite good.  There was a lot of good information on various emergency communications topics and definitely a good concentration of knowledgeable folks.  I gave basically the same talk on D-RATS that I did at MicroHams a couple of weeks ago, but augmented with changes that have occurred since then.

Digital communications was a particularly prominent theme, which shouldn't come as much of a surprise.  There was a lot of talk of D-STAR as well as P25.  I attended a talk the first morning comparing the two at a relatively low level, which I found quite informative.  The D-STAR stuff was all review, of course, but the P25 information was new to me.  In the afternoon of the second day I attended the "Digital Radio Possibilities for Amateur Radio" talk.  While there was no attempt to hide the fact that it was focused on P25, I felt that I had been lured into the room by the promise of a discussion (and perceived benefit) of interoperability between public safety P25 networks and potential amateur ones.  Unfortunately, it seemed to avoid that particular issue and promote P25 as the best digital voice standard for Amateur radio just because the public safety folks had chosen that route.  I was disappointed that the capabilities and benefits of D-STAR were completely ignored.

From what I learned in the various talks, I'm really not sure why P25 has much place in the amateur realm, at least any more than any other radio technology that can be pulled onto the amateur spectrum for experimentation.  The prices of used Phase-I gear are significantly higher than new D-STAR radios, and provide significantly less functionality.  The speaker also explained that Phase-II gear is almost categorically uninteresting to the amateur service, which seems to put a clear end date on production and availability of hardware.  Lots of people complain about D-STAR being a potentially dead-end technology with a high price of entry, given that there is only one manufacturer that could pull the plug and make the investment worthless.  That may happen, but with P25 there's already a date for the funeral.

Finally, many P25 proponets in the amateur community seem to toss the word "interoperability" around quite a bit.  The only time that was addressed in any of the talks I heard this weekend was in the context of using a P25 radio to monitor public safety frequencies. Given such a silly argument, I still don't know how interoperability is supposed to be a benefit of P25.  They can't talk on our frequencies and we can't talk on theirs. Does it matter if we use the same type of radios?

I certainly don't think D-STAR is the perfect digital voice technology for amateur radio, but from what I learned this weekend, it's definitely the best option at the moment.

This past Saturday, I presented at the Microhams 2009 Digital Communications conference.  I covered a basic demo of D-RATS functionality and showed the new 0.3.x UI I'm working on.  Response was quite good and the questions I received were refreshingly technical and well thought out.  Some new ideas came out of the discussions I had during and after the presentation, which I'm looking forward to working on.  Overall I think it was the most interesting conference I've been to in a while, with just about every presentation keeping me interested the entire time.

In case you don't know, the Microhams group is closely related to Microsoft, and (as I understand it) was founded and is largely populated by Microsoft employees.  As such, the conference was held on the Microsoft corporate campus in Redmond, WA.  I thoroughly enjoyed taking my Linux machine up to the podium (which sported a large brass-colored Microsoft logo) and plugging it into the projector.  I got a couple of decent jabs at Microsoft worked into the material, and the audience was receptive to the fun.

As I was walking out to the parking area and waiting for Taylor to pick me up, I snapped the following shot of the posters that adorn the small-town-feel of the bit of the campus I was on:

I was amused :)

I think that one of the hardest things to do as a programmer is anticipate the ways in which your product will be used.  If it's not very powerful to start with, then it's not likely that people will try to push the envelope of its capabilities.  However, if it's above a certain level, users will attempt things that the creator didn't anticipate.  Not knowing the details of the implementation, the users will expect to achieve a certain goal that exceeds capabilities.

I'm not very good at thinking outside the box in this way, and thus usually don't come up with very creative test cases for my work.  It's unfortunate, but I think that it is (in some cases) a common problem of being too close to the code.  I know what it's supposed to do, I test to verify that it does that, but I have a hard time forcing myself to try to do something that wasn't intended.  I need to work on that.

However, in the case of D-RATS at least, I have a healthy contingent of people constantly working to break stuff.  They don't realize that they're trying (well, some do), but without a manual, box, or guide telling them what is in and out of scope, they tend to push on it until it breaks.  This has, undoubtedly, led to some of the better features and more robustness than I would have been able to accomplish alone.

A good example of this came about recently when one of my users told me that his map in D-RATS was getting slower and slower.  I had just added a few significant performance enhancements to the map code, so I wasn't sure if I had broken something in the process or what.  After seeing one of his error messages, I asked him to send me his overlay files.  The overlays are a function I added that allow users to put static points on the map of things that don't move, which are overlaid with the actual radio stations that the map is intended to track.  I have about ten such points on my map, and it's quite useful.  He sent me his overlays and I put them on my system and brought up the map to see if I could reproduce the performance issues he was experiencing.  Here is what I saw:

Clearly, he had a different level of functionality in mind than I did.  The phrase "pushing the envelope" applies quite well here.  This, of course, led me to do quite a bit of work on the map code around the bits that draw the overlays.  Hopefully this will improve things quite a bit.

Bravo, Dave.  Bravo.

When I started full-time with IBM, I was working on the Open Virtualization team, specifically on some core bits of the Xen hypervisor.  I really enjoyed that work, as it was close to what I wanted to be doing.  I enjoy the low-level stuff a lot, but team plans took me back and forth between that and higher-level things.  Recently, I had a long streak of very high level work with the libvirt-cim project.  It was nice to propose, implement, and lead something so large, especially given its relative success.  However, lately I have been itching to move back to something lower down in the stack.

Given the recent economic climate, not many people are getting a chance to change jobs or be picky about what they work on.  I was lucky enough to have the opportunity to move to one of the kernel teams in the LTC recently, so I jumped at the chance.  I have actually been working on the new team since the beginning of the year, but this week my transition was finally recognized as official in the company org charts.  Not wanting to put the cart before the horse, I waited until something concrete showed up in writing before I announced it to the world.

So, I say goodbye to the libvirt-cim project and the people working on it with me and hello to the LTC checkpoint/restart team.  I'll be (and have recently been) working on checkpoint and restart support of running tasks in the Linux kernel.  It's a challenging project that I'm excited to work on, with plenty of opportunities for learning.  Plenty.

This past weekend I met three other hams at the Synopsys site in Hillsboro to install our group's D-STAR repeater.  We have had the components of this stack for a while now and have been testing it at a low-level site (my house).  Late last year, a member of our group approached the Synopsys facilities supervisor about potentially locating some or all of our D-STAR repeater equipment on their roof.  They graciously agreed and we closed on that this weekend.

We installed a Comet GP-98 antenna on a ten-foot pole with a very short run of 9913 coax through the wall into the roof stairwell enclosure.  Inside, we put a 7' fully-enclosed rack cabinet with our D-STAR repeater controller, UHF RF module, and our UHF duplexer cans from TX/RX Systems.  We got everything installed and set out to test things to make sure they worked properly.

Initially, we were getting almost no power output from the repeater past the duplexer, but full output into a dummy load.  Not looking to prolong the installation day too far, we decided to tune up and install a small mobile notch duplexer that I happened to have with me so that we could at least get the machine on the air and then go home and diagnose the issue with the large duplexer offline.  Performance with the notch was not good at all, and I was worried that there were other issues at play.  Anything but the strongest signals into the repeater were seriously garbled and absolutely no data traffic was making it through the unit.

At home, we were able to determine that while the duplexer cans were tuned perfectly for our transmit frequency, they were presenting a relatively high SWR to the repeater PA, which caused it to scale back its power dramatically (from about 20W to 2.5W or so).  We retuned the duplexer a bit to give an additional 1db of loss but with very low SWR.  I tested it on a ground-level repeater module and confirmed that it was putting out reasonable power again.

On Wednesday, I returned to the Synopsys site to reinstall the better duplexer and do a little bit of testing.  With a wattmeter, I was able to confirm that the repeater was putting out a reasonable amount of power after the duplexer.  After calling on the air for testers, it was immediately apparant that it was performing significantly better with the cans than the notch filter.  More testing that day and the day after was extremely positive.  Mobile stations were getting into the repeater quite well, and almost all fixed stations could hit it easily at 5W, even with a handheld on a rubber duck.  Wednesday night, I was able to have a QSO *and* transmit GPS position reports clearly from PDX Airport some 18 miles east.  Thursday morning, another station was able to get into the repeater easily while mobile from the intersection of Highway 6 and Highway 8, about 20 miles west.  Quite impressive for a repeater that is only six stories off the ground and putting out about 15 watts after losses!

The machine is KD7RFI_B on 440.550+MHz.  There is no gateway on it (and we're not sure if we will try to install one at this site or not) at the moment.  If you're within range, jump on and say hi!

In order to present D-RATS as a nice and neat App package on MacOS X, I have been using Py2App.  When it works, this utility generates a App package that contains everything D-RATS needs aside from the base operating system to run.  To do this, I install MacPorts and then build a full Python 2.5 stack with things like pygtk, libxml2, libxslt, etc.  When I run Py2App, it captures all of the necessary libraries and support files from the MacPorts installation and relocates them into the App package to form a nice, neat, and easy-to-run package for Mac users.

Each time I set out to create this environment, I was presented with the same problem.  Since it took me several attempts spanning many months to figure out the proper way to solve it, I decided to document it here so that there would be a single place that explained how to get out of this particular jam.

The problem was that when running Py2App, I would inevitably get the following error message:

ValueError: New Mach-O header is too large to relocate

In order to figure out what it was complaining about, I would type print self in the debugger, which would show me the path of the library it was trying to relocate.  I think that the first one I would aways hit with this problem was  libpangoxft-1.0.0.dylib.  After hacking up a solution, I believe there were a couple more in the stack that I needed that would present the same problem.

It turns out that the problem is one of space.  When the dynamic libraries for MacPorts are created, they encode the full path of the library into the header, which is something like /opt/local/lib/libpangoxft-1.0.0.2203.1.dylib.  When Py2App goes to relocate the library into the package, it needs to change this to a relative path which points to its location in the package relative to some other object.   It would appear that if the length of the original string is shorter than the one it needs to become in the package, the relocation logic used by Py2App is stuck and thus dies with the above error message.

The Py2App author indicates that the only solution to the problem is to re-link the library with an option that will instruct the linker to create the maximum amount of space for the library name in the header, despite what is requires at link time.  This ensures that the relocated library path name will fit in the space allocated.  It turns out that in addition to the prescribed option, the -Xlinker option is also required to make some component in the chain obey the request to pad the header.  This bit took me quite a while to figure out.

I figured that the safest (and easiest to reproduce) option would be to convince MacPorts to build everything with this option instead of trying to insert it into the build environment of only the affected libraries.  To do this, I modified the port command's library file where the default configure options are held.  For me, this was /opt/local/share/macports/Tcl/port1.0/portconfigure.tcl.  I edited the line starting with default configure.ldflags to look like this:

default configure.ldflags   {"-L${prefix}/lib -Xlinker -headerpad_max_install_names"}

After doing that on a fresh install of MacPorts, I proceeded to build the required stack.  In the end, Py2App agreed to relocate everything into the app package and it worked well.

Since I've got this going, I will point out an additional issue that I ran into in the whole process, which took some actual work to figure out.

The problem was with the pango configuration files.  These normally live in /etc/pango, or /opt/local/etc/pango in the MacPorts installation.  These must be in place in the App package or the pango library will not properly render fonts.  Moreover, the configuration files must properly point to the relocated objects within the package.

In order to capture the act of copying and modifying the pango configuration scripts, I wrote the following script which can be run after the App build is complete:

#!/bin/bash

make_pango_modules() {
        local src=$1
        local dst=$2
        local sf=${src}/etc/pango/pango.modules
        local df=${dst}/etc/pango/pango.modules

        cat $sf | sed 's/\/opt\/.*\/lib/..\/Resources/' > $df
}

make_pango_rc() {
        local src=$1
        local dst=$2
        local sf=${src}/etc/pango/pangorc
        local df=${dst}/etc/pango/pangorc

        cat $sf | sed 's/\/opt\/.*\/etc/.\/etc/' > $df
}

make_pangox_aliases() {
        local src=$1
        local dst=$2

        cp ${src}/etc/pango/pangox.aliases ${dst}/etc/pango
}

usage() {
        echo 'Usage: make_pango.sh [PATH_TO_MACPORTS] [PATH_TO_APP]'
        echo 'Example:'
        echo '  make_pango.sh /opt/local dist/d-rats.app'
}

if [ -z "$1" ]; then
        usage
        exit 1
fi

if [ -z "$2" ]; then
        usage
        exit 1
fi

base=$1
app="$2/Contents/Resources"

mkdir -p ${app}/etc/pango

make_pango_modules $base $app
make_pango_rc $base $app
make_pangox_aliases $base $app

By running this script with something like:

./make_pango.sh /opt/local dist/myprogram.app

You'll get a properly configured pango environment.