IOT

Linux IoT Development: Adjusting from a Binary OS to the Yocto Project Workflow

Mirza Krak — Thu, 11 Jul 2019 11:30:00 +0000

Introducing the Yocto Project and the benefits of using it in embedded Linux development.

In embedded Linux development, there are two approaches when it comes to what operating system to run on your device. You either build your own distribution (with tools such as Yocto/OpenEmbedded-Core, Buildroot and so on), or you use a binary distribution where Debian and derivatives are common.

It's common to start out with a binary distribution. This is a natural approach, because it's a familiar environment for most people who have used Linux on a PC. All the commodities are in place, and someone else has created the distribution image for you to download. There normally are custom vendor images for specific hardware that contain optimizations to make it easy to get started to utilize your hardware fully.

Any package imaginable is an apt install command away. This, of course, makes it suitable for prototyping and evaluation, giving you a head start in developing your application and your product. In some cases, you even might ship pre-series devices using this setup to evaluate your idea and product further. This is referred to as the "golden image" approach and involves the following steps:

Flash the downloaded Debian image to an SD card.
Boot the SD card, log in and make any modifications needed (for example, installing custom applications). Once all the modifications are complete, this becomes your golden image.
Duplicate the SD card into an image on your workstation (for example, using dd).
Flash the "golden image" to a fleet of devices.

And every time you need to make a change, you just repeat steps 2–4, with one change—that is, you boot the already saved "golden image" in step 2 instead of the "vanilla" image.

At a certain point, the approach of downloading a pre-built distribution image and applying changes to it manually will become a problem, as it does not scale well and is error-prone due to the amount of manual labor that can lead to inconsistent output. The optimization would be to find ways to automate this, generating distribution images that contain your applications and your configuration in a reproducible way.

This is a crossroad where you decide either to stick with a binary distribution or move your idea and the result of the evaluation and prototyping phase to a tool that's able to generate custom distributions and images in a reproducible and automated way.

Go to Full Article

FOSS Project Spotlight: Mender.io, an Open-Source Over-the-Air Software Update Manager for IoT Devices

Ralph Nguyen — Tue, 29 Jan 2019 13:00:00 +0000

by Ralph Nguyen

Mender is an open-source (Apache 2.0) project to address over-the-air (OTA) software update management for Linux-based IoT devices. When we researched this five years ago, there were no open-source end-to-end (device-to-server) options to manage the lifecycle of OTA updates for connected devices. Some open-source options were available, but they either had a proprietary management server, or they were client-only and required integration with another back-end server.

In short, the options available to IoT device-makers either had vendor lock-in or simply were too kludgy. Thus, we created Mender, which has two components: the runtime client integrated into the device and the management server with an intuitive user interface to manage updates at scale for large fleets.

Figure 1. The Mender Server's User Interface

We found in our initial research phase that many embedded systems developers created their own remote update mechanism, which usually took risky shortcuts around security and robustness. Embedded development traditionally has been a very diverse space, and the lack of technology standardization generates a lot of custom work for device-makers. Unlike web development and accepted standards, such as the LAMP stack, device-makers had to create much of their stack. This includes the fundamental capability of remote updates. And, most developers had no other choice but to build their own, given how exotic hardware and OS combinations could be for connected devices. We created a community repository called Mender Hub to allow developers to create and reuse tested and validated integrations to enable OTA updates for any combination of hardware and OS.

A consequence of the growth of IoT devices is the increase of easy targets for malicious actors, evident in the proliferation of malware targeting poorly secured IoT devices. There have been an increasing number of malware attacks infecting poorly secured connected devices. The 2016 Dyn DDoS attack was one of the clearest examples of the ramifications of poorly secured IoT devices, which was executed through the Mirai malware infecting a large number of IoT devices and enslaved them into a botnet. The IoT botnet attack caused major outages across internet platforms and services, including Amazon, GitHub and Netflix.

The increasing connectivity of cars, medical devices and more is making IoT security a serious public health issue. We created Mender to help with baseline security-hardening, and security patching is fundamental. But remote updates is quite challenging and has a lot of nuances to consider to establish a secure and robust OTA process.

Go to Full Article

Five Trends Influencing Linux's Growth at the Endpoint

Jeff Kalberg — Wed, 26 Dec 2018 13:00:00 +0000

by Jeff Kalberg

A recent IDC InfoBrief identified Linux as the only endpoint operating system growing globally. While Windows market share remains flat, at 39% in 2015 and 2017, Linux has grown from 30% in 2015 to 35% in 2017, worldwide. And the trend is accelerating.

Considering everywhere that systems built around the Linux kernel are used, we quickly realize that Linux is the most dominating operating system in the comparatively brief history of computer technology. Information systems have changed dramatically since August 1991 when Linus Torvalds announced, “I'm doing a (free) operating system (just a hobby, won't be big like gnu) for 386(486) AT clones.” With all due respect to the icons of open source, Linus is without question the Nicolas Tesla of information technology.

The influence of Linux boggles the mind—smartphones, televisions, digital video recorders, airline entertainment systems, automobile control systems, digital signage, routers, switches and, of course, the desktop operating system for the one percent, which in this case are those of us who run a Linux distro as their core OS.

Why Linux?

Although the “Which is the better operating system: Microsoft Windows or a Linux-based OS?” debate is as popular as ever these days, in truth, Linux has won the war. If there is any doubt, consider the influence of the Linux-based Android operating system (and its UNIX-based Apple brethren) to that of Microsoft Windows. Windows still has a place in our lives, but only because of the large volume of core applications that require a Windows OS. This will not always be the case, and to Microsoft’s credit, it has seen the future, and the future is Linux.

Over the past ten years, Microsoft has been enabling Linux and open-source technology. In July 2009, Microsoft quietly contributed 22,000 lines of source code to the Linux kernel under the GPLv2 license. Without a doubt, Microsoft’s motives were self-serving; it needed to ensure that Windows and Linux would interoperate well into the future. Microsoft achieved this goal when its code was accepted by the Linux kernel developers.

In more recent years, Microsoft has continued to embrace Linux. Microsoft supports Linux-based operating systems running on its Hyper-V hypervisor and Microsoft Azure, which uses Linux-based components and supports Linux OS guests. Microsoft ported SQL Server to Linux, albeit for internal use, and has made it available publicly. And by developing the Windows Subsystem for Linux, Microsoft made it possible to run Linux application workloads on Windows Server.

Go to Full Article

Empowering Linux Developers for the New Wave of Innovation

Evan Dandrea — Fri, 13 Jul 2018 12:15:15 +0000

by Evan Dandrea

New businesses with software at their core are being created every day. Developers are the lifeblood of so much of what is being built and of technological innovation, and they are ever more vital to operations across the entire business. So why wouldn't we empower them?

Machine learning and IoT in particular offer huge opportunities for developers, especially those facing the crowded markets of other platforms, to engage with a sizeable untapped audience.

That Linux is open source makes it an amazing breeding ground for innovation. Developers aren’t constrained by closed ecosystems, meaning that Linux has long been the operating system of choice for developers. So by engaging with Linux, businesses can attract the best available developer skills.

The Linux ecosystem has always strived for a high degree of quality. Historically it was the Linux community taking sole responsibility for packaging software, gating each application update with careful review to ensure it worked as advertised on each distribution of Linux. This proved difficult for all sides.

Broad access to the code was needed, and open-source software could be offered through the app store. User support requests and bugs were channelled through the Linux distributions, and there was such a volume of reporting, it became difficult to feed information back to the appropriate software authors.

As the number of applications and Linux distributions grew, it became increasingly clear this model would not scale much further. Software authors took matters into their own hands, often picking a single Linux distribution to support and skipping the app store entirely. Because of this, they lost app discoverability and gained the complexity of running duplicative infrastructure.

This placed increased responsibility on developers at a time when the expectations of their role was already expanding. They are no longer just makers, they now bear the risk of breaking robotic arms with their code or bringing down MRI machines with a patch.

As an industry we acknowledge this problem—you can potentially have a bad update and software isn’t an exact science—but we then ask these developers to roll the dice. Do you risk compromise or self-inflicted harm?

Meanwhile the surface area increases. The industry continues a steady march of automation, creating ever more software components to plug together and layer solutions on. Not only do developers face the update question for their own code, they also must trust all developers facing that same decision in all the code beneath their own.

Go to Full Article

Building a Voice-Controlled Front End to IoT Devices

Michael J. Hammel — Mon, 25 Jun 2018 13:44:37 +0000

by Michael J. Hammel

Apple, Google and Amazon are taking voice control to the next level. But can voice control be a DIY project? Turns out, it can. And, it isn't as hard as you might think.

Siri, Alexa and Google Home can all translate voice commands into basic activities, especially if those activities involve nothing more than sharing digital files like music and movies. Integration with home automation is also possible, though perhaps not as simply as users might desire—at least, not yet.

Still, the idea of converting voice commands into actions is intriguing to the maker world. The offerings from the big three seem like magic in a box, but we all know it's just software and hardware. No magic here. If that's the case, one might ask how anyone could build magic boxes?

It turns out that, using only one online API and a number of freely available libraries, the process is not as complex as it might seem. This article covers the Jarvis project, a Java application for capturing audio, translating to text, extracting and executing commands and vocally responding to the user. It also explores the programming issues related to integrating these components for programmed results. That means there is no machine learning or neural networks involved. The end goal is to have a selection of key words cause a specific method to be called to perform an action.

APIs and Messaging

Jarvis started life several years ago as an experiment to see if voice control was possible in a DIY project. The first step was to determine what open-source support already existed. A couple weeks of research uncovered a number of possible projects in a variety of languages. This research is documented in a text document included in the docs/notes.txt file in the source repository. The final choice of a programming language was based on the selection of both a speech-to-text API and a natural language processor library.

Since Jarvis was experimental (it has since graduated to a tool in the IronMan project), it started with a requirement that it be as easy as possible to get working. Audio acquisition in Java is very straightforward and a bit simpler to use than in C or other languages. More important, once audio is collected, an API for converting it to text would be needed. The easiest API found for this was Google's Cloud Speech REST API. Since both audio collection and REST interfaces are fairly easy to handle in Java, it seemed that would be the likely choice of programming language for the project.

Go to Full Article

Open Hardware: Good for Your Brand, Good for Your Bottom Line

VM Brasseur — Wed, 20 Jun 2018 12:15:15 +0000

by VM Brasseur

With the rise of IoT, we're inside a short window where "open" is a strong differentiator for hardware products. Is your company ready to take advantage of it?

I don't know how to put this, but Hardware is kind of a Big Deal, and thanks to the Internet of Things (aka IoT), it's getting bigger every year. The analyst firm IDC expects spending on IoT to reach nearly $800 billion USD by the end of 2018. A study by Intel shows that by 2025, the global worth of IoT technology might be as high as more than $6 trillion USD; whereas Forbes reports that the global market could be nearly $9 trillion USD in 2020.

These statistics are based on the traditional model of closed design and development of the chips, boards and objects that will make these devices a reality. However, what if hardware developers were to learn from and leverage the popularity of free and open-source software (aka FOSS)? What if the future of IoT were Open? It's my belief that the device developers who apply the lessons of FOSS to hardware development will be those best positioned to become the powerhouses of that $9 trillion market. Similarly to software, open hardware will be seen first as a differentiator (rather than an eccentricity) and later, as the industry matures, as the default operating mode for hardware development.

Go to Full Article

Why You Should Do It Yourself

Kyle Rankin — Thu, 31 May 2018 12:15:15 +0000

by Kyle Rankin

Bring back the DIY movement and start with your own Linux servers.

It wasn't very long ago that we lived in a society where it was a given that average people would do things themselves. There was a built-in assumption that you would perform basic repairs on household items, do general maintenance and repairs on your car, mow your lawn, cook your food and patch your clothes. The items around you reflected this assumption with visible and easy-to-access screws, spare buttons sewn on the bottom of shirts and user-replaceable parts.

Through the years though, culture has changed toward one more focused on convenience. The microeconomic idea of "opportunity cost" (an idea that you can assign value to each course of action and weigh it against alternative actions you didn't take) has resulted in many people who earn a reasonable wage concluding that they should do almost nothing themselves.

The typical thinking goes like this: if my hourly wage is higher than the hourly cost of a landscaping service, even though that landscaping service costs me money, it's still cheaper than if I mowed my own lawn, because I could somehow be earning my hourly wage doing something else. This same calculation ends up justifying oil-change and landscaping services, microwave TV dinners and replacing items when they break instead of repairing them yourself. The result has been a switch to a service-oriented economy, with the advent of cheaper, more disposable items that hide their screws and vehicles that are all but hermetically sealed under the hood.

This same convenience culture has found its way into technology, with entrepreneurs in Silicon Valley wracking their brains to think of some new service they could invent to do some new task for you. Linux and the Open Source movement overall is one of the few places where you can still find this do-it-yourself ethos in place.

When referring to proprietary software, Linux users used to say "You wouldn't buy a car with the hood welded shut!" With Linux, you can poke under the hood and see exactly how the system is running. The metaphorical screws are exposed, and you can take the software apart and repair it yourself if you are so inclined. Yet to be honest, so many people these days would buy a car with the hood welded shut. They also are fine with buying computers and software that are metaphorically welded shut all justified by convenience and opportunity cost.

Go to Full Article

Smart-Home Lightning Hacks

Shawn Powers — Mon, 14 May 2018 11:30:00 +0000

by Shawn Powers

Home automation should make life simpler, not more complex!

Kyle Rankin occasionally uses the "lightning hacks" format for his Hack and / LJ column when he has a bunch of neat topics to cover that wouldn't be enough for a complete article on their own. Thinking along those lines for this article, I figured it would be great to cover various home-automation stuff I do. Not only is it fun to share ideas, but if I make a list of all the cool things I'm currently doing, it will make it easier to compare the functionality of open-source options I'd like to explore next. If you haven't been dipping your toes into the world of home automation, maybe some of these hacks will change your mind.

My Setup

Most home-automation ideas can be implemented in multiple ways. In fact, I'm counting on that as I look into less-proprietary methods in the near future. But right now, I'm using a Samsung SmartThings hub. Yes, it is proprietary, but Samsung really has opened up the API and allowed developers to create device drivers and apps to customize the platform. I think SmartThings is the most feature-complete solution for home automation right now, but it does have a few frustrations. The most annoying is that it requires a constant connection to the internet in order to function. Most folks are frustrated with the inherent privacy concerns of home automation taking place in the cloud, and that's a big problem. For me, the more frustrating aspect is the effect shoddy internet service has on a home. If the internet goes down, so does 90% of my house! I have a few workarounds, but I know that a solid (not fast) internet connection is vital if your solution is cloud-based like SmartThings.

Anyway, my setup consists of the following:

Samsung SmartThings Hub v2.
Amazon Echo devices all over the house.
Google Home devices all over the house.
Sonos speakers in most rooms.
Various lights, switches, sensors and so on.

Having both Amazon Echo and Google Home isn't something I encourage; it's just that I have a habit of trying new technology, and they are both so amazing, I haven't yet chosen one over the other. Thankfully, they're pretty happy to function together.

Go to Full Article

Custom Embedded Linux Distributions

Michael J. Hammel — Thu, 01 Feb 2018 15:55:42 +0000

by Michael J. Hammel

The proliferation of inexpensive IoT boards means the time has come to gain control not only of applications but also the entire software platform. So, how do you build a custom distribution with cross-compiled applications targeted for a specific purpose? As Michael J. Hammel explains here, it's not as hard as you might think.

Why Go Custom?

In the past, many embedded projects used off-the-shelf distributions and stripped them down to bare essentials for a number of reasons. First, removing unused packages reduced storage requirements. Embedded systems are typically shy of large amounts of storage at boot time, and the storage available, in non-volatile memory, can require copying large amounts of the OS to memory to run. Second, removing unused packages reduced possible attack vectors. There is no sense hanging on to potentially vulnerable packages if you don't need them. Finally, removing unused packages reduced distribution management overhead. Having dependencies between packages means keeping them in sync if any one package requires an update from the upstream distribution. That can be a validation nightmare.

Yet, starting with an existing distribution and removing packages isn't as easy as it sounds. Removing one package might break dependencies held by a variety of other packages, and dependencies can change in the upstream distribution management. Additionally, some packages simply cannot be removed without great pain due to their integrated nature within the boot or runtime process. All of this takes control of the platform outside the project and can lead to unexpected delays in development.

A popular alternative is to build a custom distribution using build tools available from an upstream distribution provider. Both Gentoo and Debian provide options for this type of bottom-up build. The most popular of these is probably the Debian debootstrap utility. It retrieves prebuilt core components and allows users to cherry-pick the packages of interest in building their platforms. But, debootstrap originally was only for x86 platforms. Although there are ARM (and possibly other) options now, debootstrap and Gentoo's catalyst still take dependency management away from the local project.

Some people will argue that letting someone else manage the platform software (like Android) is much easier than doing it yourself. But, those distributions are general-purpose, and when you're sitting on a lightweight, resource-limited IoT device, you may think twice about any any advantage that is taken out of your hands.

Go to Full Article

InfluxData

James Gray — Fri, 27 Oct 2017 17:03:50 +0000

by James Gray

What is ephemeral data, you ask? InfluxData can supply the answer, because handling it is the business of the company's InfluxData open-source platform that is custom-built for metrics and events. Ephemeral data is transitory, existing only briefly, and is becoming vital for modern applications built where containers, microservices and sensors can come and go and are intermittently connected. The updated InfluxData 1.3 Platform can handle a billion (yes, with a "b"!) unique time series, making it easier to handle ephemeral data coming from containers or adding and removing sensors in IoT-tracking systems. InfluxData addresses the explosion of data points and sources, monitoring and controls requiring nanosecond precision coming from sensors and microservices.

The InfluxData platform provides a comprehensive set of tools and services to accumulate metrics and events data, analyze the data and act on the data via powerful visualizations and notifications. New features in release 1.3 include time-series indexing, high-availability anomaly detection, query language improvements and automatic cluster rebalancing. InfluxData calls the new release "one of the most significant technical advancements in the platform to date".

Go to Full Article