Integrating network storage through the file manager

If you need to access a web server from Windows, you usually have to download additional applications off the web, such as the open source programs Filezilla [1] or WinSCP [2]. Both of these application support FTP/FTPS, and WinSCP also offers SSH/SCP.

On Linux, however, you do not have to look far for a network-capable file manager: The Thunar and Dolphin file managers, which are built into the major desktop environments, get on perfectly with all popular file servers on the Internet. Even the terminal offers a variety of tools that facilitate the synchronization of data between your computer and a server on the Internet.

Gnome and Files

What used to be called Epiphany, File Roller, or Nautilus in Gnome, are now simply called Web or Internet , Archive manager and Files .

The descriptive name would suggest that the programs are simpler to use, but there can be no doubt that the developers tend to overdo things at times. This is particularly true of the File Manager: Convenient features such as the two-pane mode, which used to be there, have disappeared from Files .

On a brighter note, Files continues to support integration via FTP, SSH, SFTP, WebDAV, and SMB using the Gnome Virtual File System (GVFS, for short).

GVFS integrates network shares transparently into the filesystem so that remote files look as if they are stored locally. This means you can create your blog directly on the web with Files or use Gedit to edit the files of the CMS you use, without first copying the files to your local PC.

In the App menu under Connect to Server… , Files also offers a connection manager. First, type the network address of the web server you are using as the URL in Server address (see Figure 1).

Figure 1: Gnome Files (formerly Nautilus) comes with a simple network drive manager.

Files remembers the recently used servers in the lower, Recent Servers field. To shorten this process, just type Ctrl+L to enable the address bar in the file manager.

Once integrated, you can use the network drive and the files stored there like ordinary local directories. This means that you can edit the PHP or CSS files of your website with your favorite editor.

You can open photos and image files directly in Files with GIMP, without having to copy the data to your computer beforehand.

Slave Driver

Whereas Gnome uses GVFS, KDE SC relies on its I/O slaves or KIO slaves for short. KIO slaves implement an asynchronous virtual filesystem that gives applications access to remote network drives via various protocols. The KIO slaves use plugins to support FTP, FTPS, SSH, WebDAV, and SMB. Additionally, they rely on calls such as audiocd:/ (access to audio CDs), http:/ (complete web browser), man:/ (read manpages), or settings:/ (access the settings in the KDE control center).

You enter these command lines in the address bar, like in the Gnome file manager. To activate the address bar, choose View from the menu and then Address | Change Address menu (Figure 2) or press Ctrl+L. The syntax is the same.

Figure 2: Two panels and access to the network via KIO slaves transform Dolphin into a full-fledged FTP client.

In other words, you can use FTP, with the short or long URL form already introduced in the Gnome section. For SSH access, however, you need to type:

fish://<example>.com.

If you need to juggle with files and directories between different computers, a two-pane mode is available. Enable it in View | Split or by pressing F3.

For frequently used network drives, your best approach is to create a bookmark. To do so, you simply drag a folder from the server into the left sidebar of the file manager.

GVFS with Thunar

XFCE combines lean applications such as Mousepad, XNoise, Abiword, or Gnumeric to create a lightweight desktop environment. The Thunar file manager used by XFCE can also handle network drives. It relies on the GVFS back end, developed by Gnome, to do this.

By default, Thunar generally uses a directly editable address bar. If you just see breadcrumb navigation with buttons for each directory in the path instead of this, enable the address bar by selecting View | Address bar | Toolbar style (Figure 3).

Figure 3: Lean and but still network-capable: The Thunar file manager from the XFCE desktop environment.

Alternatively, you can press Ctrl+L to access a dialog from which you can open arbitrary locations.

Midnight Commander

The flexible command-line file manager Midnight Commander has extensive configuration options, and – among other things – lets you set up a hotlist.

Through this list, you can directly access predefined local or remote locations. This means you can move data back and forth between local directories and a server in an instant. All entries for the hotlist go into the hotlist file in the .config/mc/ folder.

The first line of Listing 1 shows an example for a computer that uses SSH to communicate. Directly after the ENTRY keyword, you need to enter a description that will appear in the hotlist menu. Pressing \ accesses the menu.

Listing 1

Connections

ENTRY "Webserver" URL "/sh://user@host/directory"
ENTRY "FTP" URL "/ftp://user@host"

The URL keyword is followed by the parameter specification in a somewhat unusual notation: The /sh.:// string identifies the connection as SSH. This is followed by user, host, and directory information for the remote machine.

The second line of Listing 1 shows an example of an FTP connection, which only differs from the first example in terms of the prefix. If you use a ~/.netrc file, then the login at the other end works without you entering your password. The same thing applies if you are using a SSH key without a password. However, this approach compromises secure practices.

After setting up the connections, you can work as usual in two-window mode, using operations such as copy, move, or delete. For simple file synchronization, this setup is perfectly okay. You also can use Midnight Commander's built-in functions to quickly compare files and directories (Figure 4).

Figure 4: With the built-in functions in Midnight Commander, you can track down changes to files, locally or remotely, quickly and reliably.

Fully Synchronized

Special programs, such as Sitecopy [3], monitor files and compare the versions on the local computer to those on the server on demand.

If you add new files at one location, Sitecopy pushes them to the remote site. It then deletes unneeded files and updates modified files. The only condition is that you must have an addressable FTP or WebDAV server.

On Ubuntu, you just need to look for Sitecopy in the package manager and install the archive. The install is similar in other distributions. If, however, you fail to find Sitecopy in your distro's repositories, your only option is to build from the source code.

The tool expects to find its configuration in the hidden ~/.sitecopyrc file, which contains the server information (Listing 2). To synchronize more sites, add a new entry with a new name at the end of the file.

Listing 2

Server Information

site alicecat
  server www.alice.de
  protocol ftp
  username alice
  password topsecret
  local /home/alice/website/cats
  remote ~/cats

Any lines with a hashtag (#) are ignored. Otherwise, each line contains the name of a setting, followed by a space and the actual value. Line 2 contains the server address (or alternatively its IP address). The next line defines the protocol used in the data exchange. This is followed by the access credentials.

If an error occurs during the file transfer, or you lose the connection, Fetch mode helps you recover. The --fetch option first checks the server status, and --update eliminates any inconsistencies. For an overview of all the important commands, see Table 1.

Table 1

Sitecopy at a Glance

sitecopy --list <site name> Creates a list with all the differences between the local files and the files on the server.
sitecopy --fetch <site name> Fetches a list of the files stored on the server.
sitecopy --update <site name> Updates the files on the server.
sitecopy --synchronize <site name> Downloads files from the server to the local directory.
sitecopy --initialize <site name> Sitecopy assumes a previously unused server.
sitecopy --catchup <site name> Sitecopy assumes that the local copy is consistent with the one on the server.
sitecopy --allsites Updates all servers specified in the configuration file.

Because of a bug, syncing aborts with an error message if the remote site is nested several levels deep. You can easily call the command several times in succession, until the program has created all the levels correctly and begins to copy the files.

In addition to the configuration file, you can create a special directory with the mkdir ~/.sitecopy command. The tool uses it to save the most recent state of the files on the server. The documentation refers to this as the "Storage Directory."

Make sure that only you have access to the configuration file and the Storage Directory; otherwise, the program refuses to work.

To set the appropriate privileges, you can run the following commands:

$ chmod 600 .sitecopyrc
$ chmod 700 .sitecopy

Data transfer via FTP is unencrypted. If you need to transfer sensitive data, you will probably want to use an external WebDAV server that supports SSL.

To do so, just add the http secure line to your ~/.sitecopyrc file.

Conclusions

None of the mainstream Linux file managers today makes the mistake of ignoring standard network protocols. Dolphin on KDE and Thunar and Nautilus/Files on Xfce or Gnome provide a very convenient approach to working with data on remote systems, treating it as if it were stored locally on your computer. Gnome's GVFS proves particularly smart and useful. It transparently integrates network storage with your filesystem, so access is not limited to graphical tools and editors, but all your CLI tools can also process files on FTP and SSH servers.

Tip

GVFS transparently mounts web shares. You can go to /run/user/<uid>/gvfs/ftp:host=<example>.com/<path> to find your online files in a terminal window, too. This means you can use CLI tools such as ImageMagick, Exiftool, or the like directly on your network-stored data.

To jump directly to this directory, install the nautilus-open-terminal package, open the network drive, and then right-click on a free area in the file manager. The Open in terminal menu item then starts the terminal in the appropriate directory. More current versions of the Gnome file manager include this feature out the box.