1.
Introduction
xdemorse is a X/GTK+ application for decoding Morse code signals
into text. xdemorse detects the "dihs" and "dahs" that make a Morse
code character via the computer's sound card, which can be
connected to a radio receiver tuned to a CW Morse code transmission
or to a tone generator. The input signal is processed by a Goertzel
tone detector algorithm which produces "mark" or "space" (signal/no
signal) outputs and the resulting stream of Morse code "elements"
is decoded into an ASCII character for printing to the Text
viewer.
Goertzel tone
detector
xdemorse was designed from the beginning to be simple and
efficient, decoding Morse code using (almost) only integer
arithmetic and simple processing methods. It uses the computer's
sound card to read the "dit" and "dah" elements that make the Morse
coded characters by implementing an integer Goertzel tone detector
algorithm, which measures the level of sound signals from a radio
receiver or tone generator. This Goertzel tone detector is centered
on a nominal frequency of 500 Hz, which is lower than the more
common 700-800 Hz BFO setting, because my PSK31 applications have
to work with a 500 Hz audio tone due to various limitations. I have
chosen to run my Morse decoding applications on the same BFO
setting, to avoid regular changes to my receiver.
The Mark/Space (key-down - key-up) condition is detected by saving the output of the Goertzel tone detector into a ring buffer, for a period of two Morse code Unit Elements, e.g. for the duration of two "dits". The change from Mark (tone) to Space (no tone) and vice-versa, is decided by summing the saved tone detector outputs, over one Unit Element, into a "leading edge" sum and then summing the following outputs into a "trailing edge" sum, again over one Unit Element. The ratio of the leading edge to the trailing edge sums rises and falls, depending on whether the leading edge or trailing edge sum is the sum of Mark tone or Space tone outputs from the Goertzel detector. This ratio is compared to a reference threshold to determine the Mark or Space condition of the tone from the Receiver. The sum of the leading and trailing edge sums is also used as an indication of signal strength and is used as a kind of "squelch" to minimize spurious character decodes.
Error and noise
tolerance
xdemorse has a certain level of tolerance towards operator errors
(bad "fist") regarding deviation from the standard duration of the
various elements that make up the Morse code. Below is a table of
the basic Morse code elements and the range of durations that
xdemorse can deal with:
Morse Element Duration xdemorse
-------------------------------------------------
dit 1 units 1/2-2 units
dah 3 units 2-4 units
Inter-elem space 1 units 1/2-2 units
Inter-char space 3 units 2-4 units
Inter-word space > 5 units > 4 units
The Morse code characters, punctuation marks and special signals
that xdemorse can currently recognize are in the file
doc/Morse-code.txt. Extra characters etc can be added to the
collection by inserting the hex equivalent of the decoded Morse
code character in the xdemorse.h header file before compilation.
Please see detailed explanation in that file.
"Waterfall" display with
CAT
xdemorse has a "Waterfall" (audio spectrum) display derived from an
integer-arithmetic FFT of the receiver's audio output. A vertical
white line indicates the center frequency of the tone detector.
This can be disabled with the "Center Line" check button if
desired.. When tuning a signal manually, its trace should be
centered on this line for best results. Clicking with the center
button on the display reverses the background/foreground colors of
the display.
xdemorse has built-in CAT capability but only for the Yaesu FT847 or FT857 and Elecraft K2 or K3. This is an unfortunate limitation but an attempt to convert xdemorse to using the "hamlib" library was unsuccessful for various reasons. The waterfall display can be used to tune in a signal by clicking with the left mouse button near its trace. xdemorse will scan a small section of the display either side of the mouse pointer, looking for the strongest trace and then tune the receiver so that the signal is centered on the red line. Since the FT847/FT857 has only a 10 Hz resolution when tuned by the CAT, there can be an error of +-5 Hz in the tuning of a signal. Other errors resulting from the finite resolution of the FFT can increase the tuning discrepancy and since xdemorse has a sharp tone detector, it may be necessary to fine-tune the receiver manually for best results. The K3 vfo has a resolution of 1Hz so that tuning in this case is more accurate.
Sound-card
set-up
xdemorse automatically sets up the sound card and tries to set the
input capture level to the level specified in the .xdemorserc
config file in the user's home directory. If it fails it will give
an appropriate warning and the mixer level for capture must be set
up manually using a mixer application like 'aumix'.
For proper operation the level of the input audio signal must be adjusted from the receiver or the tone generator, using the 'scope' display at the bottom left of the main window. If the audio signal is taken from an outlet that provides a fixed audio level, then the Capture level entry in .xdemorserc will need to be edited by trial and error, until the signal plotted in the scope display is about 80% of the height of the scope frame.
3.
Compilation
Please note that I use Void Linux
AMD64 which is a "bleeding edge" type distribution, so there
may be compilation and/or run time difficulties if you are using a
relatively old distro. This is mostly true of the basic
dependencies like GTK+ 2 and Glade-2, and there can also be sound
card incompatibility problems at run time.
To compile the package, it may be preferable to first run the included "autogen.sh" script in the package's top directory, to produce a fresh build environment. Then the "configure" script can be run with optional parameters to override the default settings and compiler flags.
Running "make" in the package's top directory should produce the executable binary in src/. Running "make install" will install the binary into /usr/local/bin by default or under the specified prefix. The desktop file is installed in /usr/share/applications/ and the menu icon in /usr/share/pixmaps/. Documentation is installed in /usr/share/doc/xdemorse/ and the runtime config file in /usr/share/doc/xdemorse/examples/xdemorserc.example. This will have to be copied to the user's home directiry and edited by the user as required.
If you have problems identifying your sound card mixer devices (capture source, capture volume etc), running the "amixer" tool from the alsa-utils package will produce a detailed list of all available devices and functions of the sound card. There is also this hypertext documentation file which you can copy to a location of your choice.
xdemorse [-hv] -h: Print this usage information and exit. -v: Print version number and exit.
5. Operation
Before running xdemorse, it may be necessary to customize some
entries in the .xdemorserc configuration text file in your home
directory. There are seven items that may need editing: The ALSA
sound card name (default is hw:0), the DSP sampling rate (default
is 48000), the ALSA "channel" name (default is FRONT_LEFT), the
capture source, the capture volume control, the capture volume
setting (default is 60%). If -- is set in the Capture volume entry,
then xdemorse will not try to set the capture volume.
Stereo mode (e.g. specifying a "channel" name other than MONO) allows the computer's sound card to be connected to two audio sources, without the need to move plugs or switch audio. In my case this allows me to feed audio from my ham radio receiver and a weather satellite receiver to the sound card separately.
Before xdemorse is used to decode Morse signals, the input level to the sound card must be set up correctly. For this, the radio receiver or tone generator must be connected to the "line" input of the sound card with a suitable cable and connectors, with either the left or right channel connected to the receiver's o/p if in stereo mode, or if in mono mode normally the left channel input. The receiver's output level is adjusted so that the maximum signal input level, as shown in the 'scope' display at the bottom left of the main window, is about 80% of the scope's window height. The receiver should be tuned to a strong, clean CW transmission, with the AGC setting in the fast position and IF/AF bandwidth in the narrowest setting. Please note that the receiver should be tuned accurately to the incoming signal so that the BFO note is 500Hz. This can be done by slowly tuning the receiver, with a tuning step of 1Hz if available, so that the INPUT SIGNAL display in the scope is at a maximum. The waterfall display is also very useful for tuning, since a signal trace tuned to be exactly under the red middle line of the waterfall will result in a 500 Hz BFO frequency.
By default xdemorse starts with Morse code speed set to 20 wpm and the detector thresholds set at 50 for the Level ("squelch") and at 2.0 for the Ratio (mark/space detector). These are shown in the three spin-wheel widgets at the right of the scope window. As xdemorse begins to decode Morse coded signals, it estimates the sending speed and adapts to it, indicating the actual speed in the 'Speed' spin-wheel, if the "Auto" check button is enabled. If the Morse transmission is a lot faster or slower than the indicated speed, it may not be decoded correctly and manual changing of the speed setting will be needed to fix the problem. Auto adaptation to incoming Morse speed can be disabled with the "Auto" check button.
To decode Morse code from a communications receiver, set the BFO so that the tone of the CW signal is near to 500 Hz and then start xdemorse. Assuming that keying is not too bad (operator's 'fist' is reasonably good) then decoded characters should appear in the 'Receive Window' of xdemorse. Some changes to the speed and/or squelch and ratio settings may be helpful and experience will show the best setting for the latter. A maximum tuning step of 1 Hz should be selected on digital receivers and tuning should be slow. A narrow CW filter is an advantage although very narrow (less than ~200 Hz) IF or dsp/AF filters tend to ring and cause spurious characters, mainly E's to be printed on the screen.
Please note that the 'scope' display of xdemorse has three functions, it can display the input signal waveform, the output from the Mark/Space (signal/no-signal) Ratio detector or the Level (squelch) value. The first function is the default and this and the other displays can be selected from the four radio buttons at the right of the Scope display.
Generally xdemorse has a fairly wide tolerance of sending errors (bad fists or noise) but in practice it has proved to be difficult to correctly decode CW signals on the amateur bands, with the main problem appearing to be bad keying habits. Sometimes inter-character spacing is too small so two or more characters are taken as one and therefore cannot be decoded correctly. Characters not recognized by xdemorse are indicated by an asterisk * on the screen. Word spacing also seems to be inconsistent with many hams resulting in fragmented or fused words.
Finally a list of all characters, punctuation marks and special signals recognized by xdemorse are listed in the doc/Morse-code.txt file. Additional characters can be entered in two tables in the xdemorse.h header file before compilation, following the instructions in there.
6. Bugs and annoyances
I have fixed whatever bugs I came across testing xdemorse but there
may be some hiding, waiting for the right conditions to appear.
The Morse code decoding algorithm may need further improvements to make it more tolerant to errors and noise and the mark/space detection process may need to be improved to reduce spurious output due to noise interference.
Version 0.1: First beta release of this basic Morse decoding X/GTK+ application, which is based on the console Morse decoding tool 'demorse'.
Version 0.2: Added a simple "Oscilloscope" display of the incoming waveform or the Detector output.
Version 0.3: Added a simple FFT-based "Waterfall" display of the receiver's audio spectrum. The Waterfall display incorporates CAT functions for the FT847 or FT857 transceiver allowing netting of the receiver by clicking on the trace of a signal.
Version 0.4: Replaced the original software synchronous tone detector with a Goertzel tone detector, since it has slightly better performance.
Version 0.5: Changed Mark/Space tone detection from thresholding the signal level to detecting the mark->space and space->mark transition edge. Seems to give more reliable decoding.
Version 0.6: Added a display function for the output of the Goertzel signal detector. This is displayed int the "scope" window at the left lower corner of the xdemorse window.
Version 0.7: Changed the detector scheme to a Mark/Space transition (edge) detector which works by counting the number of consecutive signal samples (steps) that change by more than 5% in the same direction (up or down). If the number of steps is more than a given threshold (default 60% of the Morse element duration) then this is taken to indicate a rising or falling edge. This seems to work better than the previous slope detection scheme.
Version 0.8 Added a parser to read a runtime configuration file from the user's home directory. This file (~/.xdemorse) must be edited to match the user's setup. The command line options for specifying the various parameters xdemorse needs have been removed. Also its no longer necessary to make any changes to the xdemorse.h header.
Version 0.9 After a bug report from Juha Vierinen regarding seg faulting of xnec2c, my graphical adaptation of NEC2, I changed all "sprintf" commands to "snprintf" to avoid buffer overruns. Following on the above changes, I revised all similar situations in xdemorse source code and changed all "sprintf" commands to "snprintf" just in case. While going through the xdemorse source code, I also fixed some minor bugs like typos and tidied error messages and other aspects of the GUI.
Version 1.0 After a bug report from Pino Zollo ZP4KFX, I modified the waterfall display function to avoid a divide by zero condition that caused a SIGFPE crash.
Version 1.1 After a bug report from Pino Zollo ZP4KFX regarding GUI problems with my Hellschreiber program xfhell, I made some changes to the GUI code in most of my GTK2 applications.
Version 1.2 After a bug report from Pino Zollo ZP4KFX,
regarding failure of "xdemorse" to start the Morse decoding loop
after reading its configuration file, I have modified the functions
that read this file so that more detailed error messages are
printed if entries are malformed.
I added 3 new entries to the ~/.xdemorserc config file to allow a
wider range of max and min Morse speeds and to define the initial
Morse speed (WPM) at start-up. I also added a "Receive" button in
the GUI to start and stop xdemorse as needed.
Version 1.3 Made the page size of spin buttons 0 to make setting of spin button values compatible with GTK 2.4.
Version 2.0-beta After many reports of difficulties and/or failure to set up and run xdemorse, due to poor or non-existent support for some sound cards in ALSA's OSS compatibility layer, I carried out major changes in xdemorse to use the ALSA sound API instead of the original OSS API. Because of very sparse or lacking documentation for the ALSA sound API, I had great difficulty getting xdemorse to work under ALSA, especially the Mixer interface. I am releasing xdemorse as version 2.0-beta for public testing, hoping it will be more successful with modern sound cards. Please don't blame me if it is not, even simple sound card programming with ALSA can be very difficult and/or tricky and buggy, since there are few if any tutorials available, especially for the Mixer interface.
I have also made several changes to the transceiver CAT code to deal with error conditions better and to be enabled by the "Receive" button, so that CAT is enabled only when xdemorse is actually receiving Morse signals. Please note that all the above changes inevitably resulted in regression bugs which I fixed, together with a few that were already there from the previous version. However, it is unlikely that I was able to catch all the bugs, so please do not hold me responsible for any problems xdemorse may create while in use on your system.
Version 2.1 Increased the height of the 'scope' and waterfall displays to improve resolution. Also separated the 'Auto Speed' check-button into its own frame. Again changed the Mark/Space detection scheme to measuring the average value of the Goertzel detector's slope.
Version 2.2 Added CAT support for the Elecraft K3 transceiver. It is likely that this will work with the K2 as well, although I am not able to verify this. Improved the Integer FFT functions to improve the detection of signals, as well as to present a color coded waterfall display. Also modified the "scope" display, to present a colored signal display.
Version 2.3: Replaced the GDK drawing primitives with the
equivalents from the Cairo drawing library (e.g. gdk_draw_line()
with cairo_line_to()). This gives a nicer anti-aliased display of
the signal detector.
Changed the waterfall display style so that signals are shown with
a color coding on a black background. Added a check button to
control the drawing of the white center line in the waterfall. The
signal "scope" display is now in bright green on dark green
background.
Version 2.4: I have updated the basic files of the GNU Autotools build system, to be compatible with the current version of these tools at the time of writing (February 2013).
Version 2.5: I have fixed a number of defects reported by the source code auditing tools of the Coverity Scan website. These were mainly related to programming practices and did not affect the operation of this application.
Version 2.6: I have made changes to the Mark/Space detector in the hope of improving the low signal/noise ratio performance of xdemorse. I believe xdemorse is now at least a little more effective than earlier versions in weak signal reception.
Version 2.7: I have modified the installation script to comply with Debian practices, as requested by João Eriberto. Documentation is installed in /usr/share/doc/xdemorse/ and the runtime config file in /usr/share/doc/xdemorse/examples/xdemorserc.examples. The desktop file is installed in /usr/share/applications/ and the menu icon in /usr/share/pixmaps/.
8. Copying This software package is released under the GNU Public License. Please see the COPYING file for more details.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details:
August 16 2002.Last modified: Mon Mar 31 16:13:49 EEST 2003