Android

Extending the Kernel with Built-in Kernel Headers

Joel Fernandes — Wed, 24 Jul 2019 11:30:00 +0000

Note: this article is a followup to Zack Brown's "Android Low Memory Killer—In or Out?"

Linux kernel headers are the unstable, constantly-changing, internal API of the kernel. This includes internal kernel structures (for example, task_struct) as well as helper macros and functions. Unlike the UAPI headers used to build userspace programs that are stable and backward-compatible, the internal kernel headers can change at any time and any release. While this allows the kernel unlimited flexibility to evolve and change, it presents some difficulties for code that needs to be loaded into the kernel at runtime and executed in kernel context.

Kernel modules are a prime example of such code code. They execute in kernel context and depend on this same unstable API that can change at any time. A module has to be built for the kernel it is running on and may not load on another because of an internal API change could break it. Another example is eBPF tracing programs. These programs are dynamically compiled from C to eBPF, loaded into the kernel and execute in kernel space in an in-kernel BPF virtual machine. Since these programs trace the kernel, they need to use the in kernel API at times, and they have the same challenges as kernel modules as far as internal API changes go. They may need to understand what data structures in the kernel look like or call kernel helper functions.

Kernel headers are usually unavailable on the target where these BPF tracing programs need to be dynamically compiled and run. That is certainly the case with Android, which runs on billions of devices. It is not practical to ship custom kernel headers for every device. My solution to the problem is to embed the kernel headers within the kernel image itself and make it available through the sysfs virtual filesystem (usually mounted at /sys) as a compressed archive file (/sys/kernel/kheaders.tar.xz). This archive can be uncompressed as needed to a temporary directory. This simple change guarantees that the headers are always shipped with the running kernel.

Several kernel developers disagreed with the solution; however, kernel maintainer Greg Kroah-Hartman was supportive of the solution as were many others. Greg argued that the solution is simple and just works as did other kernel developers. Linus pulled the patches in v5.2 of the kernel release.

To enable the embedded kernel headers, build your kernel with CONFIG_KHEADERS=y kernel option, or =m if you want to save some memory.

The rest of this article looks at challenges with kernel headers, solutions and the limitations.

Challenges with Kernel Headers

Filesystem or Archive?

Go to Full Article

Lessons in Vendor Lock-in: Google and Huawei

Kyle Rankin — Fri, 05 Jul 2019 11:00:00 +0000

by Kyle Rankin

What happens when you're locked in to a vendor that's too big to fail, but is on the opposite end of a trade war?

The story of Google no longer giving Huawei access to Android updates is still developing, so by the time you read this, the situation may have changed. At the moment, Google has granted Huawei a 90-day window whereby it will have access to Android OS updates, the Google Play store and other Google-owned Android assets. After that point, due to trade negotiations between the US and China, Huawei no longer will have that access.

Whether or not this new policy between Google and Huawei is still in place when this article is published, this article isn't about trade policy or politics. Instead, I'm going to examine this as a new lesson in vendor lock-in that I don't think many have considered before: what happens when the vendor you rely on is forced by its government to stop you from being a customer?

Too Big to Fail

Vendor lock-in isn't new, but until the last decade or so, it generally was thought of by engineers as a bad thing. Companies would take advantage the fact that you used one of their products that was legitimately good to use the rest of their products that may or may not be as good as those from their competitors. People felt the pain of being stuck with inferior products and rebelled.

These days, a lot of engineers have entered the industry in a world where the new giants of lock-in are still growing and have only flexed their lock-in powers a bit. Many engineers shrug off worries about choosing a solution that requires you to use only products from one vendor, in particular if that vendor is a large enough company. There is an assumption that those companies are too big ever to fail, so why would it matter that you rely on them (as many companies in the cloud do) for every aspect of their technology stack?

Many people who justify lock-in with companies who are too big to fail point to all of the even more important companies who use that vendor who would have even bigger problems should that vendor have a major bug, outage or go out of business. It would take so much effort to use cross-platform technologies, the thinking goes, when the risk of going all-in with a single vendor seems so small.

Huawei also probably figured (rightly) that Google and Android were too big to fail. Why worry about the risks of being beholden to a single vendor for your OS when that vendor was used by other large companies and would have even bigger problems if the vendor went away?

Go to Full Article

Android Low-Memory Killer--In or Out?

Zack Brown — Tue, 18 Jun 2019 12:00:00 +0000

by Zack Brown

One of the jobs of the Linux kernel—and all operating system kernels—is to manage the resources available to the system. When those resources get used up, what should it do? If the resource is RAM, there's not much choice. It's not feasible to take over the behavior of any piece of user software, understand what that software does, and make it more memory-efficient. Instead, the kernel has very little choice but to try to identify the software that is most responsible for using up the system's RAM and kill that process.

The official kernel does this with its OOM (out-of-memory) killer. But, Linux descendants like Android want a little more—they want to perform a similar form of garbage collection, but while the system is still fully responsive. They want a low-memory killer that doesn't wait until the last possible moment to terminate an app. The unspoken assumption is that phone apps are not so likely to run crucial systems like heart-lung machines or nuclear fusion reactors, so one running process (more or less) doesn't really matter on an Android machine.

A low-memory killer did exist in the Linux source tree until recently. It was removed, partly because of the overlap with the existing OOM code, and partly because the same functionality could be provided by a userspace process. And, one element of Linux kernel development is that if something can be done just as well in userspace, it should be done there.

Sultan Alsawaf recently threw open his window, thrust his head out, and shouted, "I'm mad as hell, and I'm not gonna take this anymore!" And, he re-implemented a low-memory killer for the Android kernel. He felt the userspace version was terrible and needed to be ditched. Among other things, he said, it killed too many processes and was too slow. He felt that the technical justification of migrating to the userspace dæmon had not been made clear, and an in-kernel solution was really the way to go.

In Sultan's implementation, the algorithm was simple—if a memory request failed, then the process was killed—no fuss, no muss and no rough stuff.

There was a unified wall of opposition to this patch. So much so that it became clear that Sultan's main purpose was not to submit the patch successfully, but to light a fire under the asses of the people maintaining the userspace version, in hopes that they might implement some of the improvements he wanted.

Michal Hocko articulated his opposition to Sultan's patch very clearly—the Linux kernel would not have two separate OOM killers sitting side by side. The proper OOM killer would be implemented as well as could be, and any low-memory killers and other memory finaglers would have to exist in userspace for particular projects like Android.

Go to Full Article

Now Is the Time to Start Planning for the Post-Android World

Glyn Moody — Mon, 08 Oct 2018 11:30:00 +0000

by Glyn Moody

We need a free software mobile operating system. Is it eelo?

Remember Windows? It was an operating system that was quite popular in the old days of computing. However, its global market share has been in decline for some time, and last year, the Age of Windows ended, and the Age of Android began.

Android—and thus Linux—is now everywhere. We take it for granted that Android is used on more than two billion devices, which come in just about every form factor—smartphones, tablets, wearables, Internet of Things, in-car systems and so on. Now, in the Open Source world, we just assume that Android always will hold around 90% of the smartphone sector, whatever the brand name on the device, and that we always will live in an Android world.

Except—we won't. Just as Windows took over from DOS, and Android took over from Windows, something will take over from Android. Some might say "yes, but not yet". While Android goes from strength to strength, and Apple is content to make huge profits from its smaller, tightly controlled market, there's no reason for Android to lose its dominance. After all, there are no obvious challengers and no obvious need for something new.

However, what if the key event in the decline and fall of Android has already taken place, but was something quite different from what we were expecting? Perhaps it won't be a frontal attack by another platform, but more of a subtle fracture deep within the Android ecosystem, caused by some external shock. Something like this, perhaps:

Today, the Commission has decided to fine Google 4.34 billion euros for breaching EU antitrust rules. Google has engaged in illegal practices to cement its dominant market position in internet search. It must put an effective end to this conduct within 90 days or face penalty payments.

What's striking is not so much the monetary aspect, impressive though that is, but the following: "our decision stops Google from controlling which search and browser apps manufacturers can pre-install on Android devices, or which Android operating system they can adopt."

Go to Full Article

The Chromebook Grows Up

Philip Raymond — Fri, 17 Aug 2018 11:30:00 +0000

by Philip Raymond

Android apps meet the desktop in the Chromebook.

What started out as a project to provide a cheap, functional, secure and fast laptop experience has become so much more. Chromebooks in general have suffered from a lack of street-cred acceptance. Yes, they did a great job of doing the everyday basics—web browsing and...well, that was about it. Today, with the integration of Android apps, all new and recently built Chrome OS devices do much more offline—nearly as much as a conventional laptop or desktop, be it video editing, photo editing or a way to switch to a Linux desktop for developers or those who just like to do that sort of thing.

Figure 1. Pixelbook in the Dark

Before I go further, let me briefly describe the Linux road I've traveled, driven by my curiosity to learn and see for myself how much could be done in an Open Source world. I've used Linux and have been a Linux enthusiast ever since I first loaded SUSE in 2003. About three years later, I switched to Ubuntu, then Xubuntu, then Lubuntu, then back to Ubuntu (I actually liked Unity, even though I was fine with GNOME too). I have dual-booted Linux on several Gateway desktops and Dell laptops, with Windows on the other partition. I also have owned a Zareason laptop and most recently, a System 76 laptop—both exclusively Ubuntu, and both very sound, well-built laptops.

Then, since I was due for a new laptop, I decided to try a Chromebook, now that Android apps would greatly increase the chances of having a good experience, and I was right. Chrome OS is wicked fast, and it's never crashed in my first six months of using it. I mention this only to provide some background as to why I think Chrome OS is, in my opinion, the Linux desktop for the masses that's been predicted for as long as I've used Linux. Granted, it has a huge corporate behemoth in the form of Google behind it, but that's also why it has advanced in public acceptance as far as it has. This article's main purpose is to report on how far it has come along and what to expect in the future—it's a bright one!

Chromebooks now have access to Microsoft Office tools, which is a must for those whose employers run only MS Office products. Although Google Docs does a good job with basic document creation and conversion, and although you can create a slide presentation with it, it won't do things like watch or create a PowerPoint presentation. That's where the Microsoft PowerPoint Android app comes in handy. If you need to watch one, simply download the PowerPoint file and open it with PowerPoint (you can do this without paying for Microsoft office). However, if you want to create or edit one, you'll have to pay for a yearly subscription or use your company's subscription.

Go to Full Article

A Look at Google's Project Fi

Shawn Powers — Tue, 17 Jul 2018 12:15:15 +0000

by Shawn Powers

Google's Project Fi is a great cell-phone service, but the data-only SIMs make it incredible for network projects!

I have a lot of cell phones. I have iPhones (old and new), Android phones (old, new, very old and funny-shaped), and I have a few legacy phones that aren't either Android or iPhone. Remember Maemo? Yeah, and I still have one of those old Nokia phones somewhere too. Admittedly, part of the reason I have such a collection is that I tend to hoard nostalgic technology, but part of it is practical too.

I've used phones as IP cameras for BirdTopia (my recorded and streamed bird-feeder collection). I've created WiFi-only audiobook devices that I use when I'm out and about. I've used old phones as SONOS remotes, Plex players, Chromecast initiators and countless other tasks that tiny little computers are perfect for doing. One of the frustrating things about using old cell phones for projects like that though is they only have WiFi access, because adding multiple devices to a cell plan becomes expensive quickly. That's not the case anymore, however, thanks to Google's Project Fi.

Most people love Project Fi because of the tower-hopping features or because of the fair pricing. I like those features too, but the real bonus for me is the "data only" SIM option. Like most people, I rarely make phone calls anymore, and there are so many chat apps, texting isn't very important either. With most cell-phone plans, there's an "access" fee per line. With Project Fi, additional devices don't cost anything more! (But, more about that later.) The Project Fi experience is worth investigating.

What's the Deal?

Project Fi is a play on the term "WiFi" and is pronounced "Project Fye", as opposed to "Project Fee", which is what I called it at first. Several features set Project Fi apart from other cell-phone plans.

First, Project Fi uses towers from three carriers: T-Mobile, US Cellular and Sprint. When using supported hardware, Project Fi constantly monitors signal strength and seamlessly transitions between the various towers. Depending on where you live, this can mean constant access to the fastest network or a better chance of having any coverage at all. (I'm in the latter group, as I live in a rural area.)

The second standout feature of Project Fi is the pricing model. Every phone pays a $20/month fee for unlimited calls and texts. On top of that, all phones and devices share a data pool that costs $10/GB. The data cost isn't remarkably low, but Google handles it very well. I recently discovered that it's not billed in full $10 increments (Figure 1). If you use 10.01GB of data, you pay $10.01, not $20.

Go to Full Article

UserLAnd, a Turnkey Linux in Your Pocket

Petros Koutoupis — Mon, 09 Jul 2018 12:07:07 +0000

by Petros Koutoupis

There comes a time when having a full-fledged Linux distribution within reach is necessary or just plain useful. And, what could be more within reach than having that same distribution on a computing device most people have with them at all times? Yes, I'm talking about a smartphone—specifically, an Android-powered smartphone. Enter UserLAnd.

UserLAnd offers a quick and easy way to run an entire Linux distribution, or even just a Linux application or game, from your pocket. It installs as an Android app and is available for download from the Android Google Play Store. The best part is that because it operates from a typical chroot environment, you don't need to root your device.

I was fortunate enough to have a chance to spin up one of the early beta builds of UserLAnd. This beta build was limited only to SSH and VNC local connections from my Android mobile device, but it was more than enough to establish a sound sense of how things are and where things will progress.

To handle the SSH connection, UserLAnd leverages ConnectBot while using bVNC for anything graphical. The beta build I used supported only TWM. Future updates will add additional window managers and a desktop environment. Both ConnectBot and bVNC are installed when you create and launch your session (see below).

Immediately after installation and upon launching the application, you are greeted with a clean environment—that is, no root filesystems and no sessions defined.

Figure 1. A Fresh and Clean Installation of UserLAnd

There isn't much to do here until you create a base root filesystem to use in one or more connected sessions. Now, because this was a beta build, my option was limited to Debian.

Figure 2. Creating a Root Filesystem

Once the root filesystem is created, you can create your session, which includes connection type and user name. For connection types, in my case, the drop-down menu listed ConnectBot for the command-line interface and bVNC for a graphical environment. Future releases will add more options.

Figure 3. Creating Your Connection Sessions

Go to Full Article

Looking Back: What Was Happening Ten Years Ago?

Glyn Moody — Tue, 27 Feb 2018 15:48:52 +0000

by Glyn Moody

That was then, this is now: what's next for the Open Source world?

A decade passes so quickly. And yet, ten years for open source is half its life. How have things changed in those ten years? So much has happened in this fast-moving and exciting world, it's hard to remember. But we're in luck. The continuing availability of Linux Journal's past issues and website means we have a kind of time capsule that shows us how things were, and how we saw them.

Ten years ago, I was writing a regular column for Linux Journal, much like this one. Looking through the 80 or so posts from that time reveals a world very different from the one we inhabit today. The biggest change from then to now can be summed up in a word: Microsoft. A decade back, Microsoft towered over the world of computing like no other company. More important, it (rightly) saw open source as a threat and took continuing, wide-ranging action to weaken it in every way it could.

Its general strategy was to spread FUD (fear, uncertainty and doubt). At every turn, it sought to question the capability and viability of open source. It even tried to convince the world that we no longer needed to talk about free software and open source—anyone remember "mixed source"?

Alongside general mud-flinging, Microsoft's weapon of choice to undermine and thwart open source was a claim of massive patent infringement across the entire ecosystem. The company asserted that the Linux kernel violated 42 of its patents; free software graphical interfaces another 65; the OpenOffice.org suite of programs, 45; and assorted other free software 83 more. The strategy was two-fold: first to squeeze licensing fees from companies that were using open source, and second, perhaps even more important, to paint open source as little more than a pale imitation of Microsoft's original and brilliant ideas.

The patent battle rumbled on for years. And although it did generate considerable revenues for the company, it failed dismally in its aim to discredit free software.

Go to Full Article

Introducing the Alarmy Android App

Shawn Powers — Tue, 23 Jan 2018 19:56:42 +0000

by Shawn Powers

Shawn takes a quick look at "The World's Most Annoying Alarm Clock App".

I realized a while back that I've started setting my alarm an hour early so I can snooze over and over before waking up. Intellectually, I know that's silly, but there's just something fulfilling about hitting snooze and snuggling back into bed. But since I end up losing an hour of good sleep, I decided I needed a change. Change is hard, and so I opted for an app to help me. Meet Alarmy.

Alarmy is an Android app that calls itself "The World's Most Annoying Alarm Clock App", and after using it, I think I agree. Like other apps designed to make you actually wake up before snoozing, Alarmy supports things like math problems that must be solved or violently shaking your phone in order to turn off the alarm. But Alarmy doesn't stop there. The most popular (and heinous) mode has you register a place in your home that must be visited and photographed in order to stop the alarm.

When you're fully awake, it's easy to figure out what sort of place makes the most sense to register. The Alarmy folks recommend the bathroom sink, but I personally recommend the coffee pot. If I trot out to the kitchen, and take a photo of the coffee pot, you can bet I'm also going to make a cup of coffee while I'm there. It's just too tempting. I'll probably visit the bathroom while my Keurig brews a cup, but by that point, I'm awake and craving coffee, so I'm unlikely to go back to bed.

Is it cruel? Yes. Does it work? Absolutely. Alarmy is free and offers ad-elimination for a small fee. Ads usually don't bother me in an app like this, but if I like an app, I usually pay for it anyway to support the developer—in this case, the sadistic, cruel developer! Check out Alarmy at the Google Play Store.

Go to Full Article

Sleep Fast, Sleep Hard with the Pzizz Android App

Shawn Powers — Wed, 10 Jan 2018 16:22:48 +0000

by Shawn Powers

Introducing the Pzizz Android app.

The older I get, the more I can truly appreciate a good nap. No really, there's just something about it—taking a nap mid-afternoon is an amazing experience. Unfortunately, with a busy work schedule, I find it difficult to take a nap. It's not that I can't afford the 20-minute break; it's just that I can't ever get to sleep—that is, until I discovered Pzizz.

Pzizz is an Android app that generates a custom "nap narrative" that helps ease you off to dreamland and wakes you up when it's over. I was very skeptical about how restful a 20-minute forced nap could be, and at first, I doubted I'd fall asleep at all. Thankfully, I was very wrong.

Pzizz (which my spell check really wants to change to "pizza") generates an eerie, 3D-sound soundtrack, and it provides verbal prompting that helps lull you off to sleep. And, it works. It works well. The music is hard to describe. There are chimes, strings and ocean sounds, and they all blend into a "moving" 3D auditory experience that knocks me right out. I'm often worried the eerie sounds will give me nightmares, but quite the opposite seems to happen. I always fall asleep, and I always feel rested when the app wakes me up 20 minutes later. (The amount time can be adjusted to suit your nap preferences.)

So far, I've used Pzizz only for naps, but there's also a module to aid with sleeping through the night. I can't wear earphones or earbuds that long, so I've never tried it overnight. I think if I lived alone, I might try connecting my phone to speakers and play Pzizz out loud. If you have a bed mate, it's the sort of thing you'd have to discuss in advance, and I don't think my wife would be keen on the eerie-sounding music and British man's voice all night!

The Pzizz app is free in the Google Play Store, and I can't recommend it enough. Pzizz does exactly what it claims to do while never trying to push an expensive "upgrade". Give it a try this afternoon—unless you're a bus driver, in which case, wait until after work to give it a try.

Go to Full Article