Tiny Python Panadapter (QST 4/2014)

[This posting refers to my QST magazine article, "A Tiny Python Panadapter", April, 2014, pp 33-38.  For latest information, see my website. The code repository and a support mailing list are available at SourceForge.net.]

Message from Gary KM5TY:

I'm interesting in putting together my own panadapter, however am not gelling with the idea of the low bandwidth around a signal / amount of spectrum visible with just a sound card. I am thinking that a dedicated sampling board taking in 500kHz to 1.5MHz possibly more would be much more interesting. What do you think of adding such a piece of hardware to a Raspi or Beaglebone? Would the python code be a bottle neck for the data?

Of course, more bandwidth would be very nice.  Unfortunately, to go beyond the simple soundcards for the Tiny Python Panadapter would require considerable new work or expense.  You'd have to consider the following

• Limited CPU power of the tiny cards.  The RPi is already marginal with a 48 kHz sample rate (equivalent to 192 kBytes/sec). [Note added: We've recently made a big improvement for the Pi by downgrading to USB1.1.  The default USB 2.0 can't handle our continuous stream.] You could get to higher rates by starting over with hand-tailored C/C++ code, but there would still be a limit.
• Cost.  The USB soundcards and the RPi/BBB are commodity items that let us do interesting things in the sub-$100 range.  Fast ADC boards are going to be more expensive, if they are available for small platforms -- or you can design & build your own out of chips.
• Proportionality.  The systems you're suggesting are on the market already - like the QS1R and the Flex 6000 line.  These are great products, but expensive.  They are built with high performance FPGA and/or PC-level computing power.  It would be interesting to develop and open-source a free alternative project.  (Some are already out there, like openHPSDR.) But that's well beyond TPP's scope.

On the one hand, it's fun to try to squeeze all the performance you can from a $35 or $45 computer board.  On the other hand, be realistic!  It's so much easier to work with a modern full-size PC.  They are incredibly fast, even the budget models.  How many hours of programming time is it worth to cram the functionality into a tiny board, while saving only a few hundred dollars?  (My time is worth something, isn't yours? :)  In the end, cheap hardware and miniaturization are good, but they're not everything.

It's economic thinking like this that led me to see what could be done with Python on the tiny boards.  At least for me, Python is much "cheaper" to develop with than the C/C++ alternatives.  The added benefit was that it should be easier for "newbies" to pick up and modify.

Raspberry/Beagle/Core Benchmarks

Measured the other day using iPython:

iPython's "timeit" function is hardly the last word in benchmarking, but it can be very useful if you're thinking about Python applications.  Raspberry Pi is very powerful compared to Arduino, say, but very slow compared to modern desktop machines.   No surprises, but I haven't seen relative benchmarks before.

Raspberry Pi - 30X more time for coffee

alliedelec.com

My Raspberry Pi is benchmarking about 30X slower than one thread (out of 8) on my desktop Core i7 processor. It's an interesting challenge to do real-time Python / wxWidgets programming on the little guy, but we're making progress.

KX3 IQ & Python SDR

It's a work in progress, but here's a shot of my Python / wxPython / Numpy "panadapter" display for the Electraft KX3.  The KX3 provides a very nice IQ output (wideband, quadrature audio), covering +/- 24 kHz around the VFO tuning frequency.

I have long been interested in building an SDR software backend using Python et al.   Two principles: (1) You don't understand it if you haven't coded it, and (2) Python is the quickest and often best way to put together complicated software.  It's most at home on Linux systems, but it is not hard to port most applications to Windows.  (Not quite so easy to port: hardware-oriented stuff like audio and Ethernet.)

Like most "friendly" apps, you find that 95% of the work goes into the display and user interface.  The numerical part is almost trivial, using Numpy:

    data = np.array(iqdata[::2] + iqdata[1::2]*1j)
    z = np.roll( fft.fft(data), SIZEC/2)  # place center freq in center
    pwrwork = pwrwork + np.square( np.absolute(z) )

The total code size is about 590 lines at this point.

This application (kx3iq.py) is really a remote control app.  I "beam" the IQ data from the KX3 via my Beagleboard XM and a UDP Ethernet stream to the Python app across the room, with Hamlib rigctld provide rig control.  Using 16 bit 48 kHz sampling, that's a data rate of ~200 kB/s without any compression -- doable over most ISP's these days, if you want a long distance remote.

Yet another clock program

So I was grumping to myself that my nice, big, cheap Timex digital clock wasn't doing the best job for the ham station. I had modified it for 24 hour time, but the 10 hours digit could only provide a squiggle and not a proper "2" for 20-23 hours. That, and it is physically in the wrong place (away from my computer screen) and it doesn't show civil (local) time along with UTC. I've searched for a good hardware solution with no luck.

Then I looked at some of the many Linux clock options out there. None of those was exactly right for me. I wanted digits I could read across the room, but that would fit onto my crowded Linux desktop. Anyway, it's always more fun to use software you build yourself. I pulled out my Python and wxWidgets (wxPython) references and set to it. As with designing and building your own hardware, half the reward is the stuff you learn along the way.

So here is the product. If it looks like it might be useful to you, it should run on any recent Linux system with no hassles. I also checked it on Windows XP and Windows 7 for good measure. It works there, but you will have to download Python and wxPython from the web. (Linux systems like Ubuntu provide these as part of their repository system.) MacOS will likely also support this program.

This is free (as in beer) and open source software, distributed under the GNU General Public License. That means you can download it and modify it to your taste, as long as you are willing to make your improved source available to the community if and when you distribute your new version.

In a future version, it might be worth working on desktop space efficiency. You notice that nearly half the window area is wasted on Gnome's frame decorations and menus. That could be reduced by using "shaped" frames at the cost of complicating the code and creating non-standard window behavior.

HAUS Software

The OMS utility monitor project is now called HAUS (Home Automatic Utility System). Going for the big time, here!

The Python software is available now on my "static" site:

http://aa6e.net/software/HAUS/index.html

More information about operating experience and preliminary conclusions for the project will be coming.

Original blog post: http://blog.aa6e.net/2008/10/more-important-than-amateur-radio.html

More important than Amateur Radio?

Yes, my oil usage, my carbon footprint, and ways to evade the financial meltdown. All these have taken me away from ham radio for the moment.

All these, plus one other. I won a little applications contest being held by a company called Plat'Home. They have a line of tiny Linux servers called the OpenMicroServer (OMS 400). These are headless, diskless Linux platforms, based on the RMI Alchemy au1550 chip -- a MIPS architecture device that runs at 400 MHz. A Compact Flash chip can be installed to serve as a hard drive. I use a 2 GB device. Plat'Home supplies a minimal Linux distribution called SSD/Linux, but you can also install Debian. (I haven't gone to Debian yet, but may do so.)

My application was a "Home Utility Support System" - a combination of hardware and software that would monitor operation of the home heating and hot water systems with an eye toward analyzing operations and reducing heating costs.

A prelimary report is available here as a PDF. The installation is shown in the photo at left. (Click on photos for high res.) Physical I/O is through Ethernet connected to a WiFi bridge that hooks into the household LAN, and through the USB to an inexpensive DLP-IO8-G 8-channel data acquisition and control module from DLP Design.

I built a number of interface devices that monitor the state (on/off) of the oil burner, the hot water circulator, and our three heating zone circulators. (Our hot water is supplied indirectly through a heat exchanger that receives hot water from the furnace "boiler".) The DLP interface also supports DS18B20 digital temperature sensors. I use two of these to monitor outside ambient temperature and hot water outlet (pipe) temperature.


The software is all developed in Python to log data every 30 seconds and to do analysis and data plots once a day. A sample run is shown here, as produced by gnuplot software running on the OMS400. The computer also runs the standard Apache web server to make data available to any authorized user anywhere on the Internet.

The work is still in progress, but I have a preliminary interesting, if depressing, result. The idle cycle of boiler (when no heat or hot water is demanded) is about one burst of several minutes every 3 hours. This just maintains the boiler's water temperature in its operating range. At a nominal 1.65 gallons per minute burn rate and with the current price of heating oil, that amounts to some $3.00 a day every day of the year!

Furnace people tell us never to let the system cool down, because of the thermal/mechanical cycling and because of possible corrosion problems, but paying ~$1,000 per year to idle is excessive. (I do dial down the operating temperature in the summer months, but we can't go too far or the domestic hot water will not come to full temperature.)

So there is some incentive to continue this project, potentially saving money (for ham work, of course), saving the environment, and having a little fun by the way.

Many thanks to Plat'Home for their support.

QRZPY - A program for QRZ.com access

Here's an old-fashioned (CLI-based!) but useful Python program that lets you create mailing labels and examine and dump records from the QRZ.com XML database. (Online.qrz.com account required.)

The program is developed for Linux and other Unix-like environments, but could readily be adapted for other operating systems.

For mailing labels, qrzpy supports 3 x 10 standard stick-on label stock for your LaserWriter or other printer. Alas, the QRZ.com address data may sometimes overflow the space available (see photo), but such exceptions are reported to the user.

Like most of my Python adventures, it's about the journey of discovering how to do things and perhaps to help others figure out Python programming -- as much as it is the final product.

See my software page www.aa6e.net/software for further details.

Z100 -> P100!

After a little discussion with Jack Smith about the inner workings of his Z100 Tuning Aid (last post), one thing led to another.

The result is another little Python utility that does much of what the Z100 does, but in software under Linux.

The scoop and download are available here.

Bandpass Controls for HF Digimodes

When working PSK31 or other "digimodes" on HF (high frequency) radio, we commonly use computer soundcards to analyze and transmit data. I like to use the Linux program fldigi for this, along with my Ten-Tec Orion transceiver.

The soundcard takes in the entire audio output from the Orion, typically 100 - 3,000 Hz. Fldigi (and similar software) allows you to point to a transmission of interest on a "waterfall" spectral display. This automatically sets the decoder to analyze data in a small band around the cursor using digital filtering.

This works well when the bands are relatively quiet, but when you have a band crowded with strong signals, the signal of interest can be strongly affected by "out of band" signals (but still in the 3 kHz audio region) that key the receiver's AGC (automatic gain control).

In this case, we need narrower IF (intermediate frequency) filtering than 3 kHz. Fortunately, a DSP (digital signal processing) rig like the Orion provides "infinitely" adjustable IF bandpass characteristics, so it is possible to "zoom in" on the signal of interest, largely rejecting any signals outside the small IF passband.

I have written a small Python/wxPython application "oFilter.py" that puts up a panel to allow Orion bandpass control from the same screen as fldigi. (Since fldigi communicates with the Orion via Hamlib at the same time as oFilter, there is a small potential for I/O conflict, but this is not a serious problem.)

Here are a few screenshots to show what is going on. Ultimately, it would be great to integrate the oFilter functions into fldigi or a similar program, using convenient mouse controls.

The PSK31 band at 14.070 MHz with many Europeans

coming in the 3 kHz default bandpass.

"Zooming in" the IF bandpass to 200 Hz,
more-or-less centered on the signal of interest.

Zooming in to a 100 Hz bandpass (minimum available).

The oFilter.py application.

New Rigserve Project on Sourceforge

Some of you know that I've been working on "Rigserve", which is meant to be a much streamlined server-style application providing much of the functionality of Hamlib. We avoid most of the cross-platform problems by defining our API over an IP connection, which is human-readable and even testable over Telnet. Rigserve is implemented in object-oriented style using Python, which should allow it to run on many platforms. I am not sorry to jettison low-level C, the GNU Automake stuff, SWIG, and all that!

We have talked about the relationship of this development to Hamlib. Should we think of it as a candidate for "V2 Hamlib"? Well, Rigserve is not a library, and there is no backwards compatibility. Rigserve does share some philosophy with Hamlib, but that's about it. I have concluded that it should stand on its own, but we should give full credit to the many folks who have brought us Hamlib as we have it today.

[There are some alternate approaches, too, such as XML rigCAT descriptions at http://w1hkj.com/xmlarchives.html . These may be useful to both Hamlib and Rigserve down the road.]

There is now a project at http://sourceforge.net/projects/rigserve with a slightly updated version 0.21 available for download. The files are managed in the Subversion (SVN) repository.

I would welcome anyone who wants to contribute to rigserve to join this project. There shouldn't be a conflict of interest here, because the intersection of hotshot C and Python programmers is probably limited. Though I am neither(!), I will continue to support the TenTec Orion for Hamlib.

It has been interesting to start a Sourceforge project and to learn Subversion and the other tools. Frustrating, too, because SF's shell server and compile farm chose this week to go into meltdown. The project web page is at rigserve.sf.net.

73, Martin AA6E

Rigserve

Rigserve is a new approach to local and remote control of ham rigs, inspired by work on Hamlib. Rigserve is an IP network server, programmed in Python, that provides a simple text-based interface to control an arbitrarily large number of rigs. The code is compatible with Linux-like OSs and Windows. Rig backends are provided initially for the Ten-Tec Orion (I and II) and the Icom R8500.

See hamlib-developer.blogspot.com and www.aa6e.net/aa6e/software for more information.

A Python tuner for digital modes on Ten-Tec Orion

I've been using this little control panel (opsk) for some time now. Maybe it's time to release it for any other Orion/Linux/Python/PSK addicts out there. This app lets you select the HF band you want to use, tune the antenna if necessary, and control the passband to zero in on a PSK signal you see on your waterfall display.

It uses TKINTER and pySerial technology.

Python Tone Generator

A new Python-based Tone Generator application is available from the AA6E software library at www.aa6e.net/aa6e/software/tone. It was a learning exercise for me to get acquainted with the wxWidgets cross-platform library for software development. We are also using the wxPython library that provides a Python "wrapper".

The present version is for Linux and Unix only, since it makes use of the OSSaudiodev module for sound generation.

Tone will work with dual soundcards, and will provide single or dual tones with sine-, square-, or triangular-waveforms. It can be useful for many kinds of system tests, including intermodulation distortion.