There are now reasonably-priced Android devices with plenty of storage — storage that can be used for something other than the manufacturer's hard-to-remove bloatware or many little Android apps, each of which shows advertising and does almost nothing.

This project describes how to install a small Linux distribution on your Android device and thereby create a platform for all those free desktop Linux programs, programs that differ from those available for Android — they don't usually cost anything or show advertising.

In many cases a program, free on desktop Linux, has a price on Android, or bombards you with advertising, or both. This is easy to explain — because of the history and traditions behind Linux, its developers are likely to be predisposed toward idealism and philanthropy (or are likely to see themselves that way). Android has a different history, and a typical Android developer, apart from being younger, may have an outlook dramatically different from that among Linux developers.

One of the reasons for this contrast is that Linux began as an alternative to an extremely profit-oriented, not to say rapacious, operating system called Windows. A side effect of this early history is that Linux developers were perpetually aware that their OS wasn't Windows, and wasn't even like Windows, in any sense of the word.

By contrast, Android began without lurking in the shadow of something it wasn't (except possibly iOS), as well as being a commercial project from the beginning. Google expected to make money from Android, and as developers signed on to create Android applications, many of them also expected to make money, by charging for their creations, or by placing advertising within them, or both.

But the presence of a Linux kernel within Android makes it a chimera — a pastiche of open-source Linux, open-source Java and a number of sophisticated hardware elements meant to maximize the usefulness and attractiveness of a small, lightweight technical wonder. This has the result that the Android platform represents openness in contrast to iOS — both the Linux core and the Java elements are open-source and flexible.

All these facts mean that an Android device is ripe for modification and experiments — enhancements — customizations meant to increase its usefulness to the owner, who in modern times may otherwise be the lowest priority in the design of a consumer product. The irony is that most Android owners never take advantage of this openess, or even grasp its full extent. This article should give a measure of Android's flexibility, as well as allow it to accommodate many excellent Linux desktop applications.

So ... let's get started.

This guide is meant for relatively knowledgeable Android device owners. If you still can't figure out how to make your Android contacts list stay synchronized with the contacts list on your desktop machine, this project might be too far advanced. If this is so, I hope you will read on anyway — learn what remarkable things an Android device can do that its manufacturer didn't anticipate or intend.

Here's an outline, starting with a basic Android device and ending with one that hosts a Linux desktop distribution alongside Android (without replacing Android or its applcations):

1. First, before making any changes to your Android device, find out if your device has enough free space for this project.

   

   Figure 2: Android storage dialog showing sufficient free space

   • Open Settings ... Storage. Look for the item "Available". With any luck you will see something like Figure 2.
   • If you don't have 6 gigabytes or more of free space, consider using a different Android device, or free up space by deleting lots of applications and data.
2. Having confirmed that you have adequate storage, root your device if it's not already rooted. There are too many variations in this procedure, and too many differences between device manufacturers, to be covered here, but there are detailed instructions on the Web.
3. A word about rooting Android devices. In a deep sense, rooting your device means taking it over from a possibly paternalistic manufacturer. It means accepting ownership and responsibility. It means the training wheels come off. It also means your warranty may be invalidated, and that with a few ill-considered actions you might turn your device into an inert doorstop. In a more positive sense rooting your device makes it like a desktop or laptop computer, machines that arrive already rooted.
4. Having rooted your device, next install a key application named Linux Deploy from the Google Play Store, which will do most of the heavy lifting ahead. Linux Deploy is free and doesn't have advertising.
5. Parenthetically, I should add that there's another Linux-installing app available at the Google Play Store called Complete Linux Installer, but my advice is to avoid it. It hasn't been updated in a while and it has a number of serious defects that prevent it from being a reliable way to install or run Linux.
6. Turning again to the app I recommend, Linux Deploy:


Figure 3: Linux Deploy

• Figure 3 shows what Linux Deploy looks like when you first run it.
• To begin configuring your Linux installation, press the down-arrow icon at the upper right.
• Now focus your attention on the items under the "Deploy" category — don't select any of the items under "Action" just yet.
7. Here's a list of Linux Deploy configuration settings you may want to change from their defaults:
   • Under the "Distribution" menu item, choose a Linux distribution that you prefer, from a long list provided by Linux Deploy's author. If you don't have any specific preference and aren't a specialist, Ubuntu is a reasonable choice.
   • Under the "Distribution suite" menu item, choose a version of your Linux choice. Newer versions tend to have more features and assume the presence of a newer Linux kernel, but have a correspondingly larger installation size.
   • The "Architecture" menu item is automatically configured based on your Android device's processor and shouldn't be changed.
   • In most cases the default "Installation type" (file) should be accepted.
   • The "Installation path" defaults to the root directory of /sdcard/, but you may prefer to place the Linux image file in a directory of its own to avoid inadvertent deletion.
   • The "Image size (MB)" menu item allows the user to select an image size larger than the default of 512 MB, but don't try to enter a size greater than 4095 MB — the Android filesystem can't manage a file larger than this. I recommend the largest size consistent with available storage, and I think the 512 MB default is way too small for most practical uses.
   • If you decide to let the provided SSH server run (generally a good idea), you will need to use menu item "SSH Settings" to choose a port other than the default of 22. As I've discovered the hard way, even a rooted Android device can only use port numbers above 1024. A typical choice for an SSH server is 2222, although this choice might conflict with other apps on your device that offer SSH services.
   • The menu item "GUI Settings" is used to tune the Linux virtual machine's display properties, but unlike some of the other menu choices, this option can be deferred until later.
   • To be able to access the Android filesystem from within your virtual machine, enable "Custom mounts" and use menu item "Mount points" to choose the mounts you're interested in. This is an important choice because it makes data available to the Linux virtual machine without occupying any of the storage set aside for it.
   • NOTE: It is reported that, in a number of Linux virtual machine installations, some hosting Android versions will not allow write access to its primary storage device — the internal device known colloquially as sdcard. To remedy this, go to menu item "Mount Points", select the default storage mount (which defaults to /storage/emulated/0), use the menu at the upper right (three vertical dots) to select "Edit", and change the path to /data/media/0. And remember that, within your Linux virtual machine, this mount point will be identified as /mnt/0. Only do this if you have problems writing to the Android device's storage areas.
8. Having configured your virtual Linux machine as you like, it's time to build it. Under menu category "Action", choose "Install". Depending on your device's speed, the installation might take as much as 25 - 30 minutes (much downloading from network archives is required).
9. The installer should create an image file at the location you specified, which will contain a working Linux virtual machine.
10. Assuming there are no show-stoppers during the installation, it should be possible to return to the main Linux Deploy display and click "Start", which will activate your Linux machine.
11. The default display for your Linux machine is an app called a "VNC viewer". Having tested a few, I recommend the ingeniously titled "VNC Viewer" app from the Google Play Store.
• Once you have a VNC viewer installed and running, open a VNC transaction on your Android device using "localhost" or "127.0.0.1" as the address. This will make a connection with your running Linux machine. The username is "android" and the default password is "changeme" (and do change it).
• You also have the option of running VNC viewers from your desktop or elsewhere on your network. Although the goal is to run Linux on Android with a local display, some setup tasks are easier with a desktop keyboard and mouse available.
• To run a VNC viewer from a local network machine, enter your Android device's IP address, available at Settings ... WiFi ... menu item Advanced ... IP Address.

Figure 4: Android/Linux system layout

Figure 4 diagrams the relationship between your Android device's resources and those offered by your newly installed Linux virtual machine. Here are some notes:

• The reference to CHROOT tells us that the Linux virtual machine has its own operating environment and resources, but it's running under the same Linux kernel as Android. Because Android uses an up-to-date kernel, this won't be likely to cause any compatibility difficulties.
• The reference to "Linux Deploy mount points" shows an easy and important way to simplify data transfers between Android and the Linux machine — see the above configuration section to set them up.
• The reference to "bidirectional network protocols" means the Linux machine's communication abilities extend far beyond what Figure 4 shows. If the user chooses to run any kind of server or client available to desktop Linux, chances are it will run the same way on this Linux virtual machine and be available to both Android and your local network.
• By the same token, the Android side of the system can offer network data to the Linux side. For example, using an app called BlueNMEA, by means of the shared network I provide my Linux virtual machine with a GPS data stream able to support a nautical navigation program that normally can only be operated on a desktop machine. There are a number of similar apps at the Google Play Store, this one is perfectly adequate and shows no ads.

It's important to emphasize that Figure 4 is just a cursory overview of the possibilities offered by this arrangement. Any imaginable application that can (a) fit in the small Linux virtual machine (maximum size 4 GB) and (b) can be productively controlled and used by way of a VNC client terminal or shell session, is an obvious candidate for installation on the Linux machine.

As just one example, if during installation you have taken the precaution of assigning the SSH server a port that it can actually use (i.e. above port 1024), and enabled suitable mount points, then very simple and reliable file transfers can be made using SSH-based protocols, from any local network machine, via the Linux virtual machine, to the Android main storage area. Here's a step-by-step example:

• First, from a local-network laptop or desktop, let's establish that we can log onto the Linux virtual machine, using the hosting Android device's IP address (in this example, using the port I chose for SSH communications):

  $ ssh android@[Android IP] -p 2223
  android@pl-nexus7's password: ****************
  Welcome to Ubuntu 15.04 (GNU/Linux 3.4.0-g8aa6344 armv7l)
  
   * Documentation:  https://help.ubuntu.com/
  Ubuntu 15.04 [running on Android via Linux Deploy]
  Last login: Fri Oct  2 14:03:48 2015 from pl-alpha.com
  android@localhost:~$ sudo su
  root@localhost:/home/android# |
  

• Next, I want to test rsync, my favorite way to efficiently synchronize two filesystems. Let's see if we have the server-side rsync application installed. From the Linux virtual machine shell session:

  # which rsync
  # 
  

• Apparently there's no rsync installed by default on the Ubuntu distribution I've selected (something I can't understand — rsync is a fantastic command-line tool). That problem is easily remedied, just as on a desktop Linux machine:

  # apt-get install rsync
  [long list of actions culminating in rsync being installed]
  # which rsync
  /usr/bin/rsync
  

• Just like desktop Linux. Now let's log out of the virtual Linux shell session and from a desktop machine shell, try a file transfer using rsync, routed by way of the Linux virtual machine, to the Android filesystem for which we have configured mount points. Remember this shell session is on the desktop:

  $ rsync -e 'ssh -p 2223' -av (filenames) android@[Android IP]:/mnt/0/temp/
  

  This example worked perfectly for me, for a predefined Android directory located at /sdcard/temp. (If it doesn't work for you, read this for a likely fix.) This example required me to specify the desired port in a peculiar way, but there are much better ways to specify both the Android device's identity and the configured port, an issue explained in greater depth here.

Opportunities

The opportunities offered by this setup are legion. You can run any kind of Linux server, just as though you were running it from a desktop environment. It's important to remember that, even though Android has a Linux kernel, it's not Linux as that term is generally understood — it has a very peculiar environment that's an endless source of headaches for those accustomed to normal Linux. Running a real Linux ecosystem in a chrooted process neatly resolves these issues. With a sufficiently large virtual machine one can install any number of familiar desktop tools and applications, and (for those needing a graphic display) interact with them using a VNC client.

Figure 5: Navigating my boat through Alaska
(note the nautical chart display at upper left)

Figure 6: OpenCPN running on Android

Figure 7: JTides running on Android

One reason I personally embarked on this project was so I could have a backup of my primary source of navigational information while boating, a program called OpenCPN. When I travel to Alaska each summer, I have a few laptops available, each set up to run this program and with a full set of electronic nautical charts installed (which are now free for downloading, if you haven't heard the good news). But I've always worried about what would happen if the laptops died, for example if in an emergency salt water intruded into the boat's cockpit (a common occurrence during my around-the-world solo sail).

As it happens, this past sailing season no salt water made its way into my boat's cockpit, but for various reasons two of my three laptops died and I became aware of how close I came to not having a reliable source of navigational information. So I resolved to engineer a way to make OpenCPN run on Android — or, as a last resort, write my own Android nautical chart display app.

Imagine my surprise when I discovered there's now a relatively easy way to run a Linux distribution on Android, and then install any desktop apps one cares to have available in a small, lightweight (and almost waterproof) package. So, on each of my Android devices I've installed a Linux distribution using the instructions given here, then I installed the Linux version of OpenCPN, and finally, for a source of GPS data, I installed BlueNMEA, a simple way to provide real-time GPS data on the Android device's local network, easily acquired by OpenCPN.

(As it happens, there's now an Android version of OpenCPN, but, in keeping with typical Android thinking, this app, the outcome of thousands of public-spirited volunteer programming and field-testing hours, is priced at US$10. I'm sure those behind this sudden commercialization of a free, open-source project, first acquired the enthusiastic agreement of all those who contributed their time and energy toward it over the years.)

The other Android handicap, surprisingly enough, is that it can't run Java applications. When people first hear this, they're likely to say, "Wait, what?", because Android is mostly Java — but in spite of this, it doesn't have the Java virtual machine required to run ordinary Java programs. This problem is also solved by installing a Linux distribution, then installing Java within it:

# apt-get install default-jre 

Simple, right? Now that I have Java available, alongside OpenCPN I can run my favorite tide prediction program JTides (which I wrote, and which is free).

This project makes many kinds of tools into realistic prospects, for developers as well as end users. For example, Wireshark can easily be installed to monitor network activity on the Android device, providing information otherwise difficult to acquire. Another approach, perhaps more practical, is to install and run tcpdump on the Linux virtual machine and pipe the results to a desktop machine running Wireshark:

# ssh -p 2223 android@[Android IP] tcpdump -U -iany -w - not port 2223 \
    | wireshark -i - -k

Problems

An earlier described problem with Linux image sizes has been solved. I wanted to have a Linux virtual machine larger than 4 GB in size, but most user-accessible Android storage partitions are formatted with the outdated Windows disk protocol of FAT32, which has a maximum file size of 4 GB (why? Because of 32-bit addressing: 2³² = 4,294,967,296 = 4GB).

To solve this problem, I create a partition on an external storage card using the ext4 format (which some newer Android versions accept and read), a format that doesn't have the size limit, and once it's in place, a special Android path is required to properly access it and allow writing to subdirectories (I had already defeated the write restrictions on the external storage device present in newer Android versions, a topic covered here). On two of my Android devices the special path is /mnt/media_rw/sdcard1, but to discover the specific path name on your device, simply issue this command from a shell outside the Linux virtual machine (for example by using an ADB shell, or Terminal Emulator):

# df

The above command lists all the filesystem access points, one of which will be prefixed by /mnt/media_rw/. Use this path in the Linux Deploy installation procedure and you should be able to create a Linux virtual machine image larger than 4 GB.

Be patient!

One more thing about the installation process in Linux Deploy — don't jump to conclusions. If during installation you see this sequence —

>>> install
Checking installation path ... done 
Making new disk image (12000 MB) ... dd: /mnt/media_rw/sdcard1/linux/linux.img: Invalid argument

— you may think the process has failed, but until and unless you see the word "fail" and an installation process closure symbol, like this:

fail
<< install

, then wait, be patient. Eventually, in spite of the error message, you may see this continuation:

12000+0 records in
12000+0 records out
12582912000 bytes (11.7GB) copied, 645.251866 seconds, 18.6MB/s
done 
Making file system (ext4) ... ext2fs_check_if_mount: Can't check if filesystem is mounted
due to missing mtab file while determining whether /mnt/media_rw/sdcard1/linux/linux.img
is mounted.
done 
Mounting partitions:  
/ ... done

Past the above log entry, with its own error messages, the main installation commences — package downloading and configuration, and so forth. Because Android external storage devices tend to have slow read/write times, the time required to create a 12 GB image is rather spectacular, and the temptation exists to assume the installation has failed, when in fact it's still churning along (the above example required 645 seconds, or 10 minutes 45 seconds, just to create the image).

This project represents a meaningful and useful way to exploit the openness of the Android operating system. If a particular virtual machine turns out well, copies can be made for use on other Android devices (assuming they have the same processor and a similar kernel). Also, the Linux virtual machine that's created can be deleted without any effect on Android, because no changes to Android are required for it to work. This means, unlike many Android changes, the results of this project are completely reversible — one may simply delete the Linux virtual machine image, uninstall Linux Deploy, and the device is returned to its prior state. I personally can't imagine doing that — I find having a Linux presence on my Android device to be a definite advantage.