One of the reasons I enjoy Amateur Radio is being able to talk with people in different parts of the world. At the moment I'm living in an apartment where I'm not able to put up a permanent antenna, and I don't really like the idea of setting up a temporary antenna every time I want to work HF, so for the most part my radio activities are limited to local repeaters, where I'm talking to the same set of people all the time. Not that I don't enjoy talking with the locals, but it's also nice to talk with new people as well.
D-Star is a system which uses digital signals to carry voice and data signals over the air. It uses special digital radios which connect to digital repeaters, most of which are connected to a world-wide network of repeaters. D-Star users are able to send commands to their local repeater which cause it to connect to other repeaters around the world. While connected, any transmissions into one of the connected repeaters are heard on the RF output of BOTH repeaters, making it possible to connect your local repeater to some other repeater, and hold a conversation with somebody who may be literally on the other side of the planet.
There are also computer systems called "reflectors" which are connected to the network, and allow multiple repeaters to connect together at the same time, passing audio signals between all of the connected repeaters. Reflectors have made it possible for me, in Florida, to hold a single conversation with people from Texas, Ohio, Scotland, and two different provinces in Australia, at the same time, while driving to work in the morning.
There is a device called the DV Dongle which allows a computer to connect to the same network used by repeaters and reflectors around the world, and talk to other D-Star users without using a radio at all. This make it possible for people who may not live near a D-Star repeater to still participate in D-Star and talk to others around the world.
The dongle actually contains the "vocoder" chip which translates between a digital audio stream, and a stream of numbers suitable for transmission over the air, or over the internet. This chip is currently the only legal way to do this translation, since the ABME algorithm is patented and the owners don't want to allow anybody to develop their own implementation of it.
I have two problems with this- one with the owners of the algorithm, for not allowing others to develop their own implementations, even under "non-commercial only" terms, and one with the JAL, who developed the D-Star protocol, for choosing an encumbered algorithm to begin with. However, what's done is done, and until somebody comes up with an alternative, we are unfortunately stuck with it.
Robin Cutshaw, AA4RC, who designed the dongle and sells them through amateur radio dealers (such as Amateur Electronic Supply) makes a free Java program called DVTool which allows you to connect to a repeater or reflector, and talk with other amateur operators through your computer, without using a radio.
Because the program is written in Java, it can be executed on any system with a Java runtime. However, Java doesn't have universal support for sound input, and requires an OS-specific module to handle the sound coming from a microphone. For this reason, it can only be run under Linux, Mac OS X, or Windows.
I normally use DVTool on Mac OS X, although I have also used it on a Linux netbook as well. Robin's directions for the Mac tell you to just double-click on the DVTool.jar file, and that does work. However, you can also run it from a command line, like so:
$ java -jar DVTool.jar
When you do this, it prints diagnostic information to the terminal session where you ran the command. Part of the information it writes, at the beginning of each incoming transmission, is the call sign sent by the radio which is transmitting (every packet sent by a D-Star radio has the sender's call sign embedded in the header.)
I have written a program which works as a "wrapper" around the Java tool. It runs the program, reads the diagnostic output, and prints it to the terminal window (so it looks just like running the program directly.) However, when it sees the line which identifies the transmitting call sign, it uses QRZ.com to look up the name and address associated with that call sign, and prints that information to the terminal window as well. I think it's rather cool to see the person's name and location before they even start speaking.
When the program looks up a call sign on QRZ, it remembers the data in memory, so it doesn't overload QRZ's servers with requests. It also writes that information to a text file on your system, and when it starts it reads that text file back into memory, so in theory it should never have to look up anybody's call sign more than once.
This is a sample of what the program looks like when it's running. This was taken during the Ozark Mountain D-Star Net on Reflector 1C, on 2009-10-04. The timestamps are local time on the computer itself, which in this case was EDT. (Pardon the background image, I use semi-transparent terminal windows and the Mac OS X screen capture widget captured the background along with it.)
![[dvwatcher3.png]](dvwatcher3.png)
If you are interested in playing around with it, here's the most recent version of the script. Enjoy.
| File: | dvwatcher |
| Size: | 5,711 bytes |
| Date: | 2009-10-05 04:28:49 +0000 |
| MD5: | 10944a69b4721d35ee1cc24c660a8faf |
| SHA-1: | 42a91c58d33dd0bde59abeaceda1a922c3f248e9 |
| RIPEMD-160: | dd360ddb223e1441d0419a09ff7626699b1d0db0 |
| PGP Signature: | dvwatcher.asc |
Some of my friends here in town are running D-Star gateways, which are the Linux computers that connect a set of D-Star repeaters to the internet. When they built the first machine a few years ago, they asked me to write a script to back the machine up, so I set up something similar to what is described on this page on my regular web site, which essentially mirrors the entire machine to a remote server. I had a script on the gateway machine which used pg_dump to make a static copy of the database, and when the backup machine pulled its mirror, this static copy was part of what got backed up.
Recently they asked me to take another look at the script, because the machine which had been pulling the backups had physically died, so even though the PostgreSQL backup was happening, the files weren't being copied off-site anymore.
When I logged in to look at my existing script, I found that the PostgreSQL backups also weren't happening, because I had made a basic mistake by not explicitly setting the PATH variable at the beginning of the script. The pg_dump command is installed in a non-standard location, so the script couldn't find the command, and therefore no backups were made.
I am a Linux system administrator for a living. The "standard" locations on a D-Star gateway machine are not the same as the standard locations used by normal CentOS machines. On a normal CentOS machine, PostgreSQL is installed using an operating system package, and pg_dump and the other commands are installed in /usr/bin, which is one of the standard locations in the PATH of a script which doesn't explicitly set its own path.
While I was debugging the problem, I got an email from one of the other local guys, with an RTF file describing how to use a backup script written by Brian NJ6N, on the dstar.info site. I didn't really need the script, but I figured I'd take a look at it anyway, just to see if maybe he caught something I had missed.
His script works, but if you read it, it's not the easiest thing in the world to understand, especially for somebody who may not be used to writing shell scripts. Part of my "day jobs" for the past seventeen years has been writing and debugging scripts, and one of the things I've been working on, particularly over the past few years, is making my scripts easier for other people to understand. I'm doing this for two reasons:
Somebody is going to have to look at, work with, and/or maintain the script at some point in the future. If they have trouble understanding it, they're going to remember you (or whoever's name is at the top of the script) as somebody who writes bad scripts, even if they do work.
That somebody might be yourself, two years down the line, trying to debug an obscure problem with a script that you barely remember writing, and cursing your own name into the dirt for writing such sloppy code. (If it's not obvious, I say this from direct personal experience.)
So, in the interest of having a script that my friends will be able to understand and maintain, that I'll be able to understand three years from now after not having looked at it, and in case anybody else is interested in such a thing, I am putting my script here.
| File: | dstargw-backup.sh.txt |
| Size: | 7,520 bytes |
| Date: | 2012-07-26 23:39:33 +0000 |
| MD5: | 321a27e73de7ef49e5190e9385557e7a |
| SHA-1: | 90eb61d139f5690ab36a994f7167289ef6fcdcfc |
| RIPEMD-160: | c37444c862b982dd39bb50ad31cf5c2205fe72b2 |
| PGP Signature: | dstargw-backup.sh.txt.asc |
Note that the file is called "dstargw-backup.sh.txt" on the web site. I did this so that the web server would show the contents in the browser, so you can look at it without having to download it. You may need to right-click the link and choose an option like "Save link as..." in order to save a copy on your system... or just use wget from the gateway machine directory, as shown below.
Here's a quick walk-through of how to install the script on a gateway machine. You will need to SSH into the machine, and you need to be root.
[root@w4xyz ~]# cd
[root@w4xyz ~]# mkdir -p .ssh bin
[root@w4xyz ~]# chmod 0700 .ssh
[root@w4xyz ~]# chmod 0755 bin
[root@w4xyz ~]# cd bin
[root@w4xyz bin]# wget http://www.kg4zow.us/dstar/dstargw-backup.sh.txt
--2012-07-26 12:08:41-- http://www.kg4zow.us/dstar/dstargw-backup.sh.txt
Resolving www.kg4zow.us... 69.28.72.69
Connecting to www.kg4zow.us|69.28.72.69|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7044 (6.9K) [text/plain]
Saving to: `dstargw-backup.sh.txt'
100%[======================================>] 7,044
--.-K/s in 0.03s
2012-07-26 12:08:41 (202 KB/s) - `dstargw-backup.sh.txt' saved [7044/7044]
[root@w4xyz bin]# mv dstargw-backup.sh.txt dstargw-backup.sh
[root@w4xyz bin]# chmod 0755 dstargw-backup.sh
I normally install these kinds of scripts in root's $HOME/bin directory, however if you'd like to install it somewhere else, feel free. Just remember to adjust the path when setting up the cron job (below) so that it runs from wherever you have installed it.
As root, you can now run "dstargw-backup.sh", and it will make a backup file. However, remembering to manually log into the machine and run the command is a pain the ... so you probably want the machine to run the script by itself at a certain time every day. Here's how to do it.
As root, do the following:
[root@w4xyz ~]# cd /etc/cron.d
[root@w4xyz cron.d]# echo 'MAILTO="user@domain"' >
dstargw-backup
(Substitute your own email address here)
[root@w4xyz cron.d]# echo '27 2 * * * root
/root/bin/dstargw-backup.sh' >> dstargw-backup
Whatever email address you enter on the MAILTO= line will receive an email every time the script is automatically executed. This will allow you to make sure it's running without any error messages. You can skip this line if you like, but if you do, you won't have any way to tell if anything is going wrong with the script, unless you manually log in and check it by hand.
In addition, the numbers at the beginning of the last command control when the script executes. The first two numbers are the minute and the hour, so this example tells the script to run at 2:27am every day. If you want it to run at (for example) 7:13pm, you would start the line with "13 19" instead. I try to avoid times which are exactly on the hour, or on a half-hour or quarter-hour, beause in many cases there are other automated jobs happening on the machine at those times, and I don't want to spike the load on the machine.
After creating this file, you will need to restart the crond service, to be 100% sure that it knows about the new file.
[root@w4xyz ~]# service crond restart Stopping crond: [ OK ] Starting crond: [ OK ] [root@w4xyz ~]#
Making a backup file is nice, but it doesn't do you a whole lot of good if the machine physically breaks, goes missing, catches fire, or otherwise becomes unusable. It helps to send the data off-site as soon as the backup file is created.
The script offers two ways of doing this. One is to email the file to one or more email addresses, and the other is to scp (secure copy, part of the ssh suite of commands) the file to a remote server. My friends here in Orlando are doing both of these things.
Both of these off-site options require you to edit the script, which means you will need to know how to use a text editor, such as nano (which is my preference) or vi, both of which are pre-installed on Icom's image for the gateway machines.
This web page isn't going to try and teach you how to use an editor, I'm going to assume that you already know how.
Edit the script. Near the top you will see a section like this:
######################################## # If the backup file should be emailed, specify the email address that the # message(s) should be sent FROM and TO. If you need to send the file to # multiple addresses, separate the addresses with spaces, like this: # BACKUPS_TO="user1@domain user2@domain user3@domain" # # Both variables must be present in order for any emails to be sent. #BACKUPS_FROM="dstargw@domain.xyz" #BACKUPS_TO="user@domain.xyz"
In order to make the script email the file, you need to un-comment the two lines at the bottom of the section (i.e. remove the "#" from the beginning of the lines) and you need to fill in the email addresses. These are:
BACKUPS_FROM should be set to the email address FROM WHICH you want the emails to be sent. If there are problems delivering the messages, this email address will receive the "bounce" messages. You probably want to have this point to the person who's responsible for maintaining the scripts on the machine (which may be yourself, since you're the one reading this page.)
Something else to be aware of... If the domain name of this email address has an SPF record (a way for the owner of a domain to identify the IP addresses from which legitimate email should originate, so that people who run mail servers around the world can reject forgeries because they come from an IP address which isn't listed) you will probably want to make sure that the gateway's publicly-accessible IP is listed in the SPF record... or use an email address from a domain whose SPF record isn't exhaustive (i.e. it doesn't contain a "-all" clause.) I use my gmail address for this, because all of my own domains have exhaustive SPF records. If some joker in China, Russia, or Iowa wants to send out email claiming to be from my domains, I want other server owners to know that those messages can safely be dropped, because they aren't really coming from me. (Nothing against Iowa, I just don't live there.)
BACKUPS_TO should be set to the email address (or addresses) TO WHICH you want the emails to be sent. As it shows in the comments in the script, you can specify multiple email addresses by separating them with spaces, all within the quotes, like so:
BACKUPS_TO="user1@domain user2@domain user3@domain"
If you already have a server on the internet (maybe a web server) which has some free disk space and you have SSH access, you can configure the script to automatically upload the backup file to that server. This is a bit more complicated than sending email, but it's definitely worth it.
This section assumes that you are using SSH to access both machines, from a laptop or desktop machine, with an SSH terminal program which offers the ability to copy and paste pieces of text from one window to another.
To start with, you need to configure the other server to allow logins using SSH keys to authenticate. I've already written directions on how to do this, see the "Allowing me remote SSH access" section of my "Information for my clients" page. Check and possibly do the steps in the first part of this section (i.e. possibly make the changes to your /etc/ssh/sshd_config file) but don't add my SSH public key unless you're going to have me log into your machine for some reason.
Once the server is configured, you will need to possibly create a userid for the gateway to use when uploading files. This is done the same way you would create a normal user, however you should set a really long and ugly password for the user, so that nobody can log into it using a password. There are many random password generators out there, and a lot of commands which will generate long sequences of pseudo-random characters which work very well as hard-to-break passwords. One of these (shown below) is the base64 command, reading bytes from the kernel's random number generator.
In this example, I'm creating a userid called "backups" on a CentOS machine. The rest of this section will assume that the userid is called backups, if you choose a different userid then you will need to adjust accordingly.
# dd if=/dev/urandom bs=256 count=1 | base64 -w 64
1+0 records in
1+0 records out
256 bytes (256 B) copied, 0.000194 seconds, 1.3 MB/s
/fiUKaKY6vGTzjzm3JF8GfBBh1r0dUjruXfvCd9mEKe8LNB/uhO9ic8W6rmC3Nca
2Ddbd7YHNu+hw65NBnYUTZeEgyXQ7H66hySCC7EV/CKEuaTJmzAGld22chofdSw7
+h3eCOvrEt1nK1tGGY4RQmBx4wQa7DqTPJj92XdbGaseW4bg5jEEw1yY/LPzGNzf
jLOJ215iixZq/4WiSOlf5AbV/3NQSQ9Tk/LACOBY1JFoUJ0WejAZDTjOKmZM+JTr
DuF/593nqkPai0jtCXe43cttuu52T/PEJTwAVrjsRjuR4bvCK0S/b8Q5QTQ5IrRt
Uq1cJ4juZ8kVEQ4MveXXZA==
(Highlight a long string of the random characters from one of the lines
and do a COPY.)
# useradd backups
# passwd backups
Changing password for user backups.
New UNIX password: (do a PASTE.)
Retype new UNIX password: (do another PASTE.)
passwd: all authentication tokens updated successfully.
Once the password is set, you'll never need it again, so don't bother writing it down or anything.
For the record, this is NOT a real password that I'm using on anything. I ran the command simply to illustrate how it works. The random password I used on the real backup server in this case was generated using a procedure similar to this, but using a slightly different command.
The other thing you'll need is to decide where the backup files will be stored. The easiest option is to just store them in the backups user's home directory. If you choose some other location, you'll have to set the permissions on that directory so that the backups user is able to write files there.
The next step is to generate an SSH key pair. This is a pair of keys, one which can be freely shared with the world, and one which must remain private. The keys are "matched pair", and work based on an algorithm where a message which is ENcrypted using one of the keys, can only be DEcrypted using the other key.
Creating the key pair is done ON THE GATEWAY MACHINE. We will be creating the key pair, and then copying the PUBLIC key to the backup server.
[root@w4xyz ~]# ssh-keygen -f /root/.ssh/id_rsa_backups -C 'W4XYZ backups'
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase): (Just hit ENTER here.)
Enter same passphrase again: (Just hit ENTER here.)
Your identification has been saved in /root/.ssh/id_rsa_backups.
Your public key has been saved in /root/.ssh/id_rsa_backups.pub.
The key fingerprint is:
54:de:7d:be:84:dd:95:a0:c3:4d:bf:3f:33:2a:00:9c W4XYZ backups
[root@w4xyz ~]# cat /root/.ssh/id_rsa_backups.pub
(DO NOT FORGET the ".pub" at the end of this filename!)
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA7zQPXEqSAVwRJf1l4aEc7+2rvAF10Gbd7IDggdhIL6+l
vSv+xA8BK/KpYChZ6kiWnPw7FTNCcZbipa06npvYpxEV80zWtDjX4wpLhlKQBzC2pyAPEu/zfiGEHEJ+
/+UUXdhiQZ3+Cs6PfRU3ShePbDb+FpNKFG/YT2TRl7CjBFbCvhQTh+W1Rc/YId8W1E4AC08yJQNmEfxO
8KvdFjVx1ct85aEdod3pyFRskbx93nQpYB14LRzpZZCqoMYLlrD5SLZEnhWxdHYx0ugjePLfl16tc6gs
2wLYNVynUHQzF6ODn+qR8Wk3oZFtmygxYvJzXYPiKQ5wi2LBX5uhr1iupw== W4XYZ backups
[root@w4xyz ~]#
You should see a single long line of text, starting with "ssh-rsa", and ending with the comment you use in the "-C" option of your ssh-keygen command (as shown above.) If you see a block of text which ends with "-----END RSA PRIVATE KEY-----", you forgot the ".pub" at the end of the filename, and what you're looking at is the private key. Try the "cat" command again, with the correct filename.
Highlight the entire public key, starting with "ssh-rsa" and ending with the comment at the end (the same block of text which is highlighted in green above.) Do a COPY.
Then, on the backup server, do this:
# cd ~backups
# mkdir -p .ssh
# cd .ssh
# echo '(Do a PASTE, the long block of text should appear.)'
> id_rsa_w4xyz.pub
# cat id_rsa_w4xyz.pub
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA7zQPXEqSAVwRJf1l4aEc7+2rvAF10Gbd7IDggdhIL6+l
vSv+xA8BK/KpYChZ6kiWnPw7FTNCcZbipa06npvYpxEV80zWtDjX4wpLhlKQBzC2pyAPEu/zfiGEHEJ+
/+UUXdhiQZ3+Cs6PfRU3ShePbDb+FpNKFG/YT2TRl7CjBFbCvhQTh+W1Rc/YId8W1E4AC08yJQNmEfxO
8KvdFjVx1ct85aEdod3pyFRskbx93nQpYB14LRzpZZCqoMYLlrD5SLZEnhWxdHYx0ugjePLfl16tc6gs
2wLYNVynUHQzF6ODn+qR8Wk3oZFtmygxYvJzXYPiKQ5wi2LBX5uhr1iupw== W4XYZ backups
#
The block of text you see here should be identical to the block of text you saw on the gateway machine. If not, try the whole copy/paste thing again.
Once you are sure you have a good copy of the public key, you should add it to the authorized_keys file, like so:
# cat id_rsa_w4xyz.pub >> authorized_keys
You should also check and fix the permissions on the .ssh directory and the authorized_keys file:
# cd ~backups/.ssh
# chown -R backups .
# cdmod -R go= .
Once this is done, we can make the appropriate configuration changes to the dstargw-backup.sh script on the gateway machine. Find the block near the top of the script which looks like this:
######################################## # SSH private key file and target for scp copy. # # The SCP_TARGET value should look like one of these: # SCP_TARGET="userid@hostname:/directory/" # SCP_TARGET="-P port userid@hostname:/directory/" # # Both variables must be present, and the SCP_KEY file must exist, # in order for the copy to be done. #SCP_KEY="/root/.ssh/id_rsa_backup" #SCP_TARGET="userid@hostname:/directory/"
You will need to un-comment both of the lines at the end (i.e. remove the "#" from the beginning of the line. Then you need to set the values of the two variables:
SCP_KEY needs to be the full pathname of the SSH PRIVATE KEY file (the one which does NOT have ".pub" at the end of it.) If you used the filename shown in the example above, this may already be correct, but check it just to be sure.
SCP_TARGET tells the script where to to copy the file. This value will contain the userid and hostname for the backup server. If you are storing the backups in the user's home directory, this value should end with a ":" character. If you are storing them somewhere else, this value should have a ":" at the end of the hostname, followed by the path to where the files should be stored, and ending with a "/".) And if the backup server's SSH service is listening on a port number other than 22, you may need to include "-P port " at the beginning of the string.
Because this isn't the simplest thing in the world, here are some examples of what this value might look like. We're going to assume that the userid is "backups", and the backup server's hostname is "badger.domain.xyz".
| Value | Description |
| The server's sshd is listening on port 22, and the backups
are written to the |
|
| The server's sshd is listening on port 22, and the backups are written to /var/backups/. You will need to make sure that the backups user has permission to create files in this directory. | |
| The server's sshd is listening on port 12345, and the
backups are written to the |
|
| The server's sshd is listening on port 12345, and the backups are written to /var/backups/. You will need to make sure that the backups user has permission to create files in this directory. |
After changing the script, you should run it by hand and make sure it works the way you expect it:
# /root/bin/dstargw-backup.sh ===== Removing old SQL backup files removed `/dstar/backups/W4XYZ-20120726T022701-0400.sql' ===== Backing up PostgreSQL database to /dstar/backups/W4XYZ-20120726T172747-0400.sql ===== Building the list of files to be backed up ===== Backing up files to /dstar/backups/W4XYZ-20120726T172747-0400.tar.gz tar: Removing leading `/' from member names (This can be ignored.) tar: /etc/sysconfig/iptables: Cannot stat: No such file or directory (This file is on the list to be backed up, but doesn't exist on this particular machine. This message can be ignored.) tar: Error exit delayed from previous errors (If there were no other errors, this can be ignored as well.) ===== Emailing backup file to kg4zow@kg4zow.us ===== Copying backup file to backups@badger.domain.xyz: W4XYZ-20120726T172747-0400.tar.gz 100% 12MB 3.9MB/s 00:03 ===== Finished
As long as everything goes well, and you have set up the cron job correctly,