Software for DIY off-road navigation

Grimmevil, sxc.hu

Grimmevil, sxc.hu

In the Wild

Anyone doing off-road navigation needs special maps. You can create these maps in no time at all.

Moving a vehicle over unpaved roads requires a robust operating system to run the software. Linux qualifies as such, and Ubuntu is among the most popular.

In my example, Ubuntu runs on two computers: the home PC for preparing maps and the netbook that takes over the display and navigation per GPS receiver in your vehicle. The software to be installed consists of the following:

  • QLandkarteGT [1] to test the maps
  • Mkgmap and Splitter [4] to convert the map material from OSM format to the Garmin one
  • Merkaartor [5] to create your own OSM maps

You only need to install QLandkarteGT on your netbook. The software together with the GPS receiver serve to display the maps, calculate routes and tracks, locate your position, and record the route.

Be warned: A lengthy off-road tour preparation has to take place on your home PC. The maps and tracks prepared on it before your trip get transferred via USB memory device to the netbook or a copied to disk for the software to read from.

Preparation

Installing Ubuntu on a PC usually presents no problem. Distributions other than Ubuntu usually do as well. What's important is using a current version of the aforementioned software for the distribution. For QLandkarteGT, use version 1.4.0 or later; for Mkgmap, you need version r2815 or later (with the corresponding Splitter at version r311); and Merkaartor requires at least version 0.17.2. Both Mkgmap and Splitter require Java version 7.

You can create cards directly from the OpenStreetMap server in a readable format (Garmin or GPX) for QLandkarteGT, although considerable computing resources are involved. For my tests, I used a computer with six CPU cores, 8GB RAM, and 21GB swap space. Even with such a powerful setup, the last successful attempt at creating a map took about three hours.

My attempt at converting an OSM map of Germany with Mkgmap into Garmin format crashed at the end of three days. Later in this article, I'll show how to make the conversion a success. For now, just be aware that processing maps requires good hardware and lots of patience.

If you've already decided at this point not to calculate your own maps, you won't need to install Mkgmap, Splitter, or Merkaartor. An easier way is to convert maps from OpenStreetMap to a format usable by QLandkarteGT for Garmin devices, which means that you need the programs GMapTool [2] and cGPSmapper [3].

QLandkarteGT

A central part of the project is the QLandkarteGT software. Although you can find it in most distribution repositories, including Ubuntu, it's usually woefully outdated.

With Ubuntu, you have the advantage of integrating already compiled newer versions from PPAs in the Software Center. You can then get the current repositories for installation instead of the outdated versions.

QLandkarteGT requires two additional repositories, one for the application itself and another for the GDAL library. These are both added in a terminal through the following commands.

$ sudo add-apt-repository ppa:ubuntugis/ppa
$ sudo add-apt-repository ppa:mms-prodeia/qlandkarte

Once you've integrated both repositories as such, the software list will be at its latest state.

The easiest way is through the package manager. First, load the list of programs anew and find the corresponding entries. If the software was already installed, the system will bring it to its latest version.

Start QLandkarteGT after the installation and verify that it's version 1.4.0 or later. With QLandkarteGT, you can view maps on the home PC, enter waypoints, calculate routes, and create tracks from them. You can save this geodata on a USB stick and read it from the netbook later on.

Working with QLandkarteGT requires Internet access, because the software accesses maps from OpenStreetMap. Select the OpenStreetMap server in the toolbox from the Stream subtab of the Maps tab (Figure 1).

Figure 1: The OpenStreetMap server selection is in the toolbox under the Maps tab and the Stream subtab and forms the basis for the project.

You can also choose Google Maps as an alternative, although OpenStreetMap is preferred in that it explicitly permits private use of maps.

Setting Waypoints

A main component of off-road navigation is calculating a route. To create one you need two or more waypoints. You can add a waypoint to the map by clicking the Waypoints tab (Figure 2) in the toolbox and then New Waypoint (or press F5). Then, click the map to create the point. Alternatively, you can click the Waypoint menu and the entry New Waypoint .

Figure 2: Select the Waypoints tab settings to create a new track point.

After creating the point a dialog opens (Figure 3) that already has the coordinates. All that's missing is a descriptive name, the symbol used on the map, and the option of making the waypoint permanent.

Figure 3: The waypoint dialog allows setting a permanent point. The coordinates are already indicated, with just a few more values to enter.

Set all the relevant waypoints before beginning your trip. Then, save the geodata in QLandkarteGT format by clicking the File menu and Save Geo Data . Be sure to select the QLB (QLandkarteGT Binary format) or else the application uses the GPX conversion format.

QLandkarteGT doesn't save the map itself, but rather the geodata contained therein – that is, waypoints, tracks, and routes. QLandkarteGT gets the map itself off the Internet from the OpenStreetMap streaming server. I'll cover creating a map on the USB storage media a bit later in this article.

Calculating Routes

Once you've entered the waypoints, you create a route from them. Note that QLandkarteGT creates a route according to the default from the open route server. For now, this calculates routes in Europe only – rather limiting for off-roaders seeking exotic corners of the world.

Destinations outside Europe therefore require an alternate route planner. MapQuest is one that is available for QLandkarteGT. The actual planning can then take place easily on the maps from OpenStreetMap or Google.

Figure 4 shows a route calculated from the OSM streaming map on the stored vector map of a Garmin device. Calculating routes on the home PC is extremely important for off-road use, because you rarely have Internet access on your netbook while on the road.

Figure 4: Although Open Route Service can only calculate routes for Europe, MapQuest can calculate routes for the whole world.

You create the route in three steps. First, mark all the waypoints that you want to evaluate with check marks. Second, right-click the waypoints and select Create Route from the context menu..The software then creates a direct route (straight line) between each waypoint. The program shows the selected points, and you can remove incorrect ones and give the route a name.

What's important is the direction of the waypoints. The software processes them from top to bottom; therefore, the starting point is the furthest up and the endpoint is the furthest down (Figure 5).

Figure 5: Use the horizontal arrows to remove waypoints from the route and use the vertical arrows to move the waypoints up and down to change the direction of the route.

The third step is to use the routing server (on the Routes tab) to find the exact place on the map for the route. Do this by marking the polygon-shaped route and, with a right-click, choose Calculate Route from the context menu.

As a result, the software shows an icon on the route to indicate that it's being processed, then it loads the calculated route from the server. On the map, you'll see the route superimposed on the road, with the specified direction indicated (Figure 6).

Figure 6: The finished route is matched to the paths and shows its direction.

You need to use a robust netbook without any movable components, such as DVD players or hard drives, for the actual off-road trip – the netbook performance will end up being correspondingly slow. The installation is somewhat more complicated than that on the home PC.

Creating Boot Media

The Ubuntu homepage used to have a bootable USB image, but not anymore. Thus, you need a bootable CD or DVD with the current version. For the workshop, I used Ubuntu 12.04 LTS. If you already have installed Ubuntu on your home PC, use the Startup Disk Creator opened from the Start menu.

Otherwise, you can download the CD or DVD image or a bootable Ubuntu version and burn it to disk. Then, start the computer from the disk and create a bootable USB stick with the program. The USB stick should have a minimum capacity of 8GB.

The USB stick contains only the system for the actual installation to the SD card. If you no longer need the system on the built-in SSD hard drive, this can be a destination as well. In the case of the SD card, it's advisable to have a model with at least 16GB disk space.

Boot the EeePC and press F2 while starting it, which enters the system BIOS, where you select the USB stick as start media. Clicking the first entry has the BIOS show you the order of the listed drives. From this menu, be sure that the USB is at the top of the list. If not, choose the item and use the plus sign to move it to the top. Press F10 to save your settings, exit the menu, and reboot.

The system will now boot from the USB stick and provide Ubuntu 12.04 LTS, which you then install on the SD or SSD card. Indicate where you want to install GRUB, then you can start from the SD or SSD card without having to do so manually from BIOS.

Completion

After the installation restart the computer. At this point, call up the BIOS one last time, remove the USB stick before booting, and press F2 again. Then, reset the boot order so that the start media (SD or SSD) is at the top, then reboot.

Your netbook is now a full-fledged (if somewhat slower) Ubuntu computer where you can install the newest version of QLandkarteGT, as described above for the home PC. The netbook doesn't need any additional software. If you're using an 16GB SD card, there's plenty of free space.

All the work you've done on maps, waypoints, routes, and tracks on the home PC is best saved on a separate USB stick and used on the netbook. The resources on the netbook are still scarce. Using a USB stick shouldn't present any problems, even with respect to mechanical stability when bumping along a dirt trail, as long as it's a micro-USB stick.

You now need the maps installed on the netbook, seeing that no Internet access will be available on your off-road trip. Sources for these maps come from the streaming servers from OpenStreetMap, Google, or elsewhere.

So, how do you get the maps, waypoints, routes, and tracks for QLandkarteGT onto the netbook? Figure 8 shows an overview of the project. The central part is your home PC (at the left) with QLandkarteGT. On it, you create waypoints, routes, and tracks using the OpenStreetMap server and MapQuest route server.

Figure 8: An overview of the entire navigation project.

Load the maps either in Garmin or OSM format and then convert them to QLandkarteGT format. Finally, save the maps, waypoints, routes, and tracks on a USB stick and read them into the netbook in your vehicle. You can then go on your way guided by GPS.

Waypoints, Routes, Tracks

As a basis, you indicate the waypoints for your map on your home PC in QLandkarteGT with help from your favorite map server. From your waypoints, you calculate the routes based on the Open Route Service (Europe only) or MapQuest. From these, you create tracks with altitude data with help from the Geonames.org server. These steps were described in my previous article in the series.

This is where the input comes in, or storage on the USB stick. Unfortunately, the OpenStreetMap project delivers only small map sections in OSM format, not entire countries or continents.

The database for the entire world, planet.osm , for example, takes 32GB of compressed data after unpacking, almost half a hard drive. And, QLandkarteGT can't read OSM maps – only those in uncompressed Garmin format.

As Figure 8 shows, you have three different ways to get to usable maps for the off-road trip. Two of them require loading completed maps, either in Garmin or OSM format. The third takes more work, using Merkaartor to assemble your own OSM map from parts.

Free Maps in Garmin Format

To avoid misunderstanding, the term "Garmin maps" in this article refers to free OSM maps in Garmin format. Inquisitive users have already partially reverse-engineered the undocumented Garmin format [6], as described in some detail on the project website on SourceForge [7].Volunteers are calculating the format from OpenStreetMap sources and providing them for download, thus the material is neither produced nor supported by Garmin itself.

The maps are actually from the OpenStreetMap project, but in Garmin format, and the authors explicitly allow free use of them. Please read the authors' terms of use – some request payment or apply certain restrictions based on payment.

Editing these maps isn't recommended, because it is a hard and tedious task (more on this later). If you still want to modify them based on errors you detect, use Merkaartor together with the OpenStreetMap server (also more on this later).

Your changes will hopefully make it back sometime to the OpenStreetMap map in Garmin format. Critical handling of OpenStreetMap maps is especially needed if you want to traverse unknown or particularly dangerous territory. Additionally, you should bring along a good map from a well-known geographic publishing house and a compass.

The free Garmin format maps are distributed across multiple websites. The OpenStreetMap wiki [8] provides a comprehensive list of these sites. You usually download them as compressed files, with sizes in the gigabyte range, based on the region, country, and continent. Not only does the download take a while, but the unpacking does, too: In my test, a OSM_egeneric took the AMD six-core processor with 8GB RAM almost two minutes to unpack.

Different Archives

Unpacking the map data results in either a single IMG file or a directory with a TDB file and an IMG file of the same name. The latter case is a bit easier: No conversion is necessary, and you can read the map directly into QLandkarteGT.

To do so, click the File menu in QLandkarteGT, then Load Map . Next, open the TDB file and immediately afterward the IMG file of the same name. Both bring up the new map that will appear on the Maps tab under Vector . Based on your zoom level, you might see only a white area; by zooming in, you'll eventually see the map details.

Some archives have multiple versions of the TDB files. Then, it's only a matter of finding out which one works in QLandkarteGT without crashing. Now's a good time to get it right, because you'll have other work to do on your trip.

Saving the directory with all the files on a USB stick lets you load them on your travel netbook. The USB stick then must be plugged in all the time, or you can copy the directory onto the netbook's SSD.

If the map is a single IMG file, you'll need to extract the individual grids (as IMG files) and the TDD file from it.

To do so, copy the programs GMapTool and cGPSmapper-static in the directory and start a terminal window. In it, change to the directory where the map and both programs are and execute the following commands:

$ ./gmt -S <file>.img
$ ./cgpsmapper-static mapset.mp

For file , use the name of the map file that you get after unpacking; mapset.mp is a fixed name.

Whereas GMapTool chops up the IMG into grids, cGPSmapper-static creates the mapset.img and mapset.tdb files, which you then load with QLandkarteGT as described earlier. Again, you copy the whole directory to the USB stick to load it on the netbook.

Downloading OSM Maps

A few sources provide ready-made maps in OSM format – the original format of the OpenStreetMap project. An especially good source is the GeoFabrik server that includes numerous OSM maps that are updated daily.

I tested the reasonably-sized OSM map of Portugal. The corresponding portugal-latest.osm.bz2 file had a size of 110MB after downloading. Upon decompression, the portugal-latest.osm file took up about 1.5GB space.

The first attempt at converting the map with Mkgmap into a format readable by QLandkarteGT returned an error that the map was too big and, therefore, needed to be broken up.

The Mkgmap homepage recommends the Splitter [10] program for this purpose and provides the matching download link. After setting up the small Java program (see the "Setting up Splitter" box) change to the OSM map's directory. There you enter the command for splitting up the file (Listing 1, line 1).

Listing 1

Splitting the File

01 $ /usr/lib/jvm/java-7-openjdk-amd64/bin/java \
  -jar path-to-splitter-version/splitter.jar ./portugal-latest.osm
02 $ /usr/lib/jvm/java-7-openjdk-amd64/bin/java -jar path-to-mkgmap-version/mkgmap.jar \
  --route --add-pois-to-areas --series-name="name-of-map-in-register"\
  --index --tdbfile 6324.osm.pbf

Setting Up Splitter

Both Mkgmap and Splitter are Java programs that you can download from the Mkgmap project [8]. The project is quite active and the versions change often. Obtain both program as packed (TAR.BZ or ZIP) JAR archives that you can unpack in your favorite directory without further installation.

To determine whether the JRE is already installed and which version, look in your /usr/lib/jvm directory. The directory contains the various Java versions. The one you need for Mkgmap and Splitter is the Java 7 OpenJDK for your architecture.

If /usr/lib/jvm doesn't include a matching Java 7 version, install the OpenJDK Java 7 JRE either from the Ubuntu Software Center or directly from the Oracle [9].

The portugal-latest.osm Portugal map took a good 11 minutes to process. The map directory is then filled with many map segments that you process further with Mkgmap. Next, execute the second command on the next line in Listing 1.

The --tdbfile option serves to read the resulting map directly into QLandkarteGT. The --series-name option specifies the name under which the map should appear in QLandkarteGT's map register.

In the Portugal example, I had data collection readable by QLandkarteGT after about three and a half minutes. Even with a comparatively smaller map like Portugal's the map preparation process can be resource-consuming, so you should never do this in your netbook.

Creating a Few Maps

If you want to create your own maps or – at some point – upload the result of your trip as tracks to OpenStreetMap, then get ready for quite lot of work. The best tool for this task is Merkaartor. You can also find the tool in the Ubuntu repository, where you can easily access it through your package manager.

Merkaartor proves to be a rather resource-hungry program. The minimum RAM required is 8GB and significant swap space for larger maps. A faster processor would reduce the waiting time during processing, and a wide monitor would help because of the program's window distribution.

Starting Merkaartor

After starting, Merkaartor shows three screen regions. On the left is the (still empty) selection of loaded maps. In the middle is a pixelated area that will eventually display the map segments. You can change this configuration, although it's outside the scope of this article to explain how. In the region on the right is the status data of the selected objects (Figure 9).

Figure 9: Merkaartor shows three screen regions: on the left is the map list, in the middle are the actual maps – as yet still empty – and on the right is the status of the selected map details.

From the File | Download menu (or by clicking the button by the same name), you download the map data from the OSM server. This opens an entry window where you can highlight the map regions in different ways. The best one is From the map below… (Figure 10).

Figure 10: Loading from the OpenStreetMap server is probably the easiest using From the map below….

When you click OK , Merkaartor starts the download. At first, you may end up choosing a too large map section, whereby the OSM server returns an error. Try zooming in on the map a bit more until the OSM server stops complaining. When you're done, the middle region of the Merkaartor window displays a lot of lines and points – that is, the raw OSM map data (Figure 11). The map becomes more readable as soon as you apply a background. Right-clicking the Map item on the left provides a selection of backgrounds (Figure 12).

Figure 11: The raw OSM map data can be cryptic. Easily recognizable, however, is the downloaded map region (the little red points).
Figure 12: The map background can be from various server data. The one shown is from OSM Mapnik.

Clicking the box in front of the map field displays the background and makes it easier to interpret the data. The background, however, is not part of the stored map.However, you're still short of a usable map at this point. The map segment is far too small for effective Merkaartor processing and further map snippets need to be added to cover the required area.

This is where Merkaartor shines. When you call up the Download menu again, you see the last chosen map segment. Because it wasn't overly large, simply moving the map over to another segment yields an extended one of reasonable size. Repeating the process (which can take a few minutes) eventually leads to a map segment that covers the entire geographic region. Save this file in Merkaartor format, which can again take a few minutes and even an hour or more for a large map. The process is also heavy on memory and CPU usage.

Saving OSM Files

Although Merkaartor saves files in its own format, to be able to work with it outside of Merkaartor, you will have to export it to OSM at some point. This process can take a long time, perhaps an hour or more. I recommend converting the files, as described earlier, into a pair of TDB and IMG files that QLandkarteGT can read. Figure 7 shows the resulting conversion loaded into QLandkarteGT for the map in Figure 13.

Figure 7: If possible, the tool integrates information about the altitude from the geonames.org server when calculating tracks from a route.
Figure 13: Worth the wait: The outskirts of Cadiz downloaded by Merkaartor from the OSM server, saved as an OSM file, and made readable by Splitter and Mkgmap for QLandkarteGT.

Vehicle Inspection and Traffic Regulations

I'll end with a few comments about road traffic – vehicular traffic isn't just about operating the vehicle. This is, after all, off-road navigation – where the normal rules don't apply.

In the African savannah, there's really no sense to be talking about speed limits or the such. Other countries have other requirements, traffic regulations, and so on. Hence, you should be aware of whether the equipment installation in your vehicle is legal, especially according to local laws.

To provide at least some assurance, be sure that the equipment installation in your vehicle is done by a reputable workshop. Automobile clubs can give you some advice about local regulations. For example, some netbook mounts can be easily installed and uninstalled, which can lead to trouble in some countries where this isn't allowed, and you might have to go without off-road navigation.

In any case, make sure that the vehicle's security functions aren't inhibited by the mounts. A netbook in front of an airbag wouldn't work well in an accident.

In daily use, the usual on-road navigation systems are superior to the off-road ones in this article. They can talk to the driver and allow hands on the steering wheel.

These DIY navigators don't have this capability. Therefore, a passenger should be readily on hand to check the screen and the surrounding landscape while the driver pays attention to the driving task. On-road and off-road are very different scenarios.

Conclusion

The first part of this series dealt with the required hardware on the basis of an EeePC. It's left to your imagination how you adapt your own old hardware correspondingly. You can also pick a distribution different from Ubuntu 12.04 LTS.

In this second and final part of the series, I have presented the necessary software such as QLandkarteGT, Mkgmap, Merkaartor, GMapTool, and cGPSmapper.

Now you have all the tools and knowledge at hand to obtain OSM maps from the Internet to create your off-road tour and traverse the landscape over waypoints, routes, and tracks. Creating a map with Merkaartor is time-consuming and is only necessary when no other usable maps for a region are available.

With all the trust in your netbook and reliability of Ubuntu, you should keep two further things on hand in the wilderness: a good magnetic compass and a detailed printed map of the region. Neither a compass nor a geological survey map require a reboot – and both keep working when the battery on your computer is dead. l